<%BANNER%>

Stimulus Functions in Token-Reinforcement Schedules

HIDE
 Title Page
 Acknowledgement
 Table of Contents
 List of Tables
 List of Figures
 Abstract
 Introduction
 Experiment 1
 Experiment 2
 Experiment 3
 Experiment 4
 Discussion
 References
 Biographical sketch
 

PAGE 1

STIMULUS FUNCTIONS IN TOKEN-REINFORCEMENT SCHEDULES By CHRISTOPHER E. BULLOCK A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2006

PAGE 2

Copyright 2006 by Christopher E. Bullock

PAGE 3

ACKNOWLEDGMENTS I would like to thank my advisor, Dr. Timothy D. Hackenberg, for his guidance and mentoring during the research and writing portions of this project. I would also like to thank my family and lab mates for support throughout the course of this project. iii

PAGE 4

TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................................................................................iii LIST OF TABLES .............................................................................................................vi LIST OF FIGURES ..........................................................................................................vii ABSTRACT .......................................................................................................................ix CHAPTER 1 GENERAL INTRODUCTION....................................................................................1 Purpose of the Present Research...................................................................................6 Schedules as Tools for Investigating Stimulus Function..............................................7 2 EXPERIMENT 1........................................................................................................13 Method........................................................................................................................17 Subjects................................................................................................................17 Apparatus.............................................................................................................17 Procedure.............................................................................................................19 Results.........................................................................................................................21 Discussion...................................................................................................................29 3 EXPERIMENT 2........................................................................................................31 Method........................................................................................................................33 Subjects................................................................................................................33 Apparatus.............................................................................................................34 Procedure.............................................................................................................34 Results.........................................................................................................................36 Discussion...................................................................................................................40 4 EXPERIMENT 3........................................................................................................43 Method........................................................................................................................46 Subjects and Apparatus.......................................................................................46 Procedure.............................................................................................................46 iv

PAGE 5

Results.........................................................................................................................48 Discussion...................................................................................................................52 5 EXPERIMENT 4........................................................................................................54 Method........................................................................................................................56 Subjects and Apparatus.......................................................................................56 Procedure.............................................................................................................56 Results.........................................................................................................................57 Discussion...................................................................................................................62 6 GENERAL DISCUSSION.........................................................................................64 LIST OF REFERENCES...................................................................................................72 BIOGRAPHICAL SKETCH.............................................................................................75 v

PAGE 6

LIST OF TABLES Table page 2-1. Order of conditions and number of sessions per condition for Experiment 1...........21 3-1. Order of conditions and number of sessions per condition for Experiment 2...........35 4-1. Order of conditions and number of sessions per condition for Experiment 3...........48 5-1. Order of conditions and number of sessions per condition for Experiment 4...........57 vi

PAGE 7

LIST OF FIGURES Figure page 2-1. Mean responses per minute (not including pre-ratio pause) plotted as a function of exchange ratio......................................................................................................22 2-2. Mean pre-ratio pausing plotted as a function of exchange ratio...............................23 2-3. Mean within ratio responses per minute (not including pre-ratio pause) plotted as a function of token production segment for subjects run primarily in the small token box..................................................................................................................25 2-4. Mean within ratio responses per minute (not including pre-ratio pause) plotted as a function of token production segment for subjects run primarily in the large token box..................................................................................................................26 2-5. Mean pre-ratio pausing plotted as a function of token production segment for subjects run primarily in the small token box..........................................................27 2-6. Mean pre-ratio pausing plotted as a function of token production segment for subjects run primarily in the large token box...........................................................28 3-1. Mean responses per minute (not including pre-ratio pause) for each condition......37 3-2. Mean pre-ratio pause for each condition..................................................................38 3-3. Mean responses per minute (not including pre-ratio pause) plotted as a function segment.....................................................................................................................39 3-4. Mean pre-ratio pausing plotted as a function of segment.........................................40 4-1. Mean responses per minute (not including pre-ratio pause) for each condition......49 4-2. Mean pre-ratio pause for each condition..................................................................50 4-3. Mean responses per minute (not including pre-ratio pause) plotted as a function segment.....................................................................................................................51 4-4. Mean pre-ratio pausing plotted as a function of segment.........................................52 5-1. Mean responses per minute (not including pre-ratio pause) for each condition......58 vii

PAGE 8

5-2. Mean pre-ratio pausing for each condition...............................................................59 5-3. Mean responses per minute (not including pre-ratio pause) plotted as a function of segment................................................................................................................60 5-4. Mean pre-ratio pausing and plotted as a function of segment..................................61 5-5. Mean time between the last response of a segment and the token production plotted as a function of segment...............................................................................62 viii

PAGE 9

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy STIMULUS FUNCTIONS IN TOKEN-REINFORCEMENT SCHEDULES By Christopher E. Bullock May 2006 Chair: Timothy D. Hackenberg Major Department: Psychology The present study examined pigeons responding on tokenreinforcement schedules using a two-component multiple schedule with a token-reinforcement schedule in one component and one of several other schedule types in the other. In Experiment 1 responding under a token-reinforcement schedule was compared to that under an equivalent tandem schedule. It was found that response rates under the tandem schedule were higher than under the token and that response patterning in the token-reinforcement schedule was more graded than under the tandem schedule. In Experiment 2 responding under a token-reinforcement schedule was compared to that under a series of brief-stimulus schedule variants. Response rates under brief-stimulus arrangements were higher than under token arrangements, resembling responding in the tandem components from Experiment 1. In Experiment 3, responding under a token-reinforcement schedule was compared to that maintained under several extended-chained schedule variants. Responding was sensitive to reinforcement magnitude, in that response rates were higher under token-schedules than under comparable extended-chained schedules with a single ix

PAGE 10

reinforcer. Weakening the correlation between number of tokens and temporal proximity to reinforcement attenuated the discriminative functions of the tokens. In Experiment 4 responding under a token-reinforcement schedule was compared to that under a procedure that arranged for response-independent tokens and exchange periods, yoked to their occurrence in the previous token component. Response rates were reduced but not eliminated under yoked response-independent token delivery. Only when tokens were removed entirely was responding eliminated. On the whole, the results from all experiments suggest that the tokens may serve a variety of stimulus functionsconditioned reinforcing, discriminative, and elicitingdepending on the contingencies. Further, the data suggest several points of contact between token-reinforcement, extended-chain, and other second-order schedules. x

PAGE 11

CHAPTER 1 GENERAL INTRODUCTION In a token reinforcement procedure, a token (e.g., a coin, a gold star, a check on a list) is provided contingent on a particular response. Tokens are then later exchanged for other reinforcers (e.g., food, access to preferred activities). For example, a child may earn a gold star for every 5 math problems completed, and at the end of the day can exchange the stars at a store for candy or toys; a rat may earn a marble for every 20 responses, and when 10 marbles have been produced can exchange the marbles for food. The tokens have been conceptualized as conditioned (acquired) reinforcers, thought to gain reinforcing value due to their correlation with primary reinforcers. They may also serve important antecedent (discriminative and eliciting) functions, signaling temporal proximity to primary reinforcers. Identifying the conditions under which tokens serve signaling and/or conditioned reinforcing functions is important in a complete account of responding under token-reinforcement schedules. Additionally, understanding token-reinforced behavior is important in that these procedures are often utilized to promote and maintain prosocial behavior in a variety of academic and clinical settings (Kazdin and Bootzin, 1972). Thus, understanding the determinants of behavior in token systems is of theoretical as well as practical importance. In the first laboratory investigation of token reinforcement, Wolfe (1936) found that chimpanzees would work for tokens exchangeable for food. He initially established discrimination between tokens with and without value by arranging exchange opportunities for one type token (white poker chip) but not for another (brass poker chip). 1

PAGE 12

2 The chimpanzees were then exposed to a schedule in which a token or food was produced by a response (lifting a weight). Two variants of this schedule were used, one in which the weight lifted was constant and the other where the amount of weight increased following each response. It was found that for both the constant weight and increasing weight conditions, contingent token delivery maintained behavior in much the same way as contingent food delivery, suggesting that the tokens were serving as conditioned reinforcers, acquired through a history of relations with other stimuli. Cowles (1937) extended the work of Wolfe (1936) by showing that tokens could maintain behavior under conditions with delayed primary reinforcement in which groups of tokens were required to produce exchanges for food. Initially, a single token could be exchanged for food reinforcement. The number of tokens required before an exchange opportunity became available was gradually increased, until long pauses in responding occurred. Responding was consistently maintained under such conditions, with 10-30 tokens per exchange, providing the first demonstration of token reinforcement under intermittent reinforcement schedules. The results showed that tokens could maintain behavior under conditions in which primary reinforcement was temporally distant. Following two decades of inactivity, token reinforcement procedures were revived by Kellehers work in the 1950s (Kelleher, 1956, 1957, 1958). In one study along these lines, Kelleher (1958) exposed chimpanzees to a schedule in which tokens (poker chips) were delivered upon the completion of a fixed-ratio (FR) schedule (FR schedules require a fixed number of responses for completion). Following the delivery of a fixed number of poker chips, the animals were given the opportunity to exchange the chips for primary reinforcement (food). Through the course of the experiment manipulations were made of

PAGE 13

3 the number of responses required to produce a token (token-production schedule) and the number of tokens required before an exchange opportunity (exchange-production schedule). The exchange schedule was held constant at an FR 60 except under the highest token-production schedule value, in which it was reduced to an FR 50. As the token-production schedule was increased from an FR 30 to an FR 125, response rates decreased and pausing increased, during the initial portions of a cycle (when no or only a few tokens had been earned.) Interestingly, when the chimpanzees were given 50 poker chips at the start of a session long pre-ratio pauses ceased, suggesting a potential discriminative function of the tokens. That is, a discriminative function was demonstrated by showing that altering the number of tokens present produced behavior early in a cycle that was typically seen later in a cycle. Later studies maintained this emphasis on conditioned reinforcement and temporal organization of behavior. Malagodi (1967) examined rats responding on FR 20 token production schedules, with marbles as token reinforcers. During exchanges, tokens could be deposited in a receptacle with each deposit producing a food pellet. Fixed numbers of tokens were required to produce exchange periods, with this number varying from 1 to 8 across groups of sessions. Similar to the token-production effects reported by Kelleher (1958), as the exchange requirement was increased, response rates decreased and pausing increased. Waddell, Leander, Webbe, and Malagodi (1972) examined rats responding on a schedule in which tokens (marbles) were produced according to an FR 20 and exchanged according to a fixed-interval (FI) schedule that varied across conditions. (Fixed-interval schedules arrange for a consequence to occur following the first response after a set time

PAGE 14

4 interval has expired). Response patterning within the FR 20 token-production schedules were similar to that seen under simple FR schedules, a pre-ratio pause followed by rapid rates of responding. Response rates across token-production segments within the exchange cycle were similar to those seen under simple FI schedules: the rate of FR token-production sequences increased in proximity to food. Also similar to simple FI schedules, overall response rates decreased as a function of FI duration. In a similar vein, Webbe and Malagodi (1978) examined rats responding on FR 20 token reinforcement schedules with FR or variable ratio (VR) schedule of exchange production. (Variable ratio schedules deliver a consequence after a number of responses have been emitted, with the number of responses required varying around some preset average). Across a series of conditions, the exchange schedule alternated between a VR 6 and FR 6. They found that response rates were higher, and pre-ratio pauses were lower, under the VR exchange schedule when compared to the FR exchange schedule. These results are in accord with behavior maintained under simple FR and VR schedules. However, only one VR and FR exchange value and only one token production value were used. Foster, Hackenberg, and Vaidya (2001) extended the work of Webbe and Malagodi (1978) by examining the influence of exchange-production schedule type and value on token-reinforced behavior. They also extended prior work by using a different species (pigeons) and different token reinforcers (lights mounted in a horizontal row above the response keys in an otherwise standard conditioning chamber). (Such non-manipulable tokens have a number of advantages with respect to examining stimulus functions of tokens, as described in more detail below.) In their experiment, the value of the FR and

PAGE 15

5 VR exchange schedule was varied across conditions from 1 to 8 while the token-production schedule was held constant at FR 50. Response rates decreased and pausing increased, as a function of ratio size under both schedules. Consistent with the results of Webbe and Malagodi (1978), response rates under the VR exchange were less affected by changes in ratio value than those under the FR schedule. This effect, combined with results reported in the studies discussed above, provide strong evidence that in token-reinforcement schedules the schedules in place for token production and exchange each influence behavior in a manner similar to that of analogous simple schedules in isolation. Bullock and Hackenberg (2006) examined the role of both the token-production and exchange-production schedules in a token reinforcement procedure with pigeons. Prior research has typically assessed the effect of manipulating either the token production or exchange schedule in the context of a fixed value of the other. Bullock and Hackenberg (2006), however, examined the token production and exchange schedules with both varied across a range of values. Comparable to FR performance under simple schedules, response rates decreased as the token-production FR was increased within a given exchange value. Further, decreases in response rates under the larger FR token-production schedules were even more pronounced under higher exchange schedules. Similarly, within a given token-production value, increases in the exchange schedule produced decreases in response rates, particularly under higher token-production values. Under the higher exchange-schedule values, responding within an exchange cycle was graded, with low responses rates early in the cycle increasing as more tokens were earned. These results indicate that the effects of token production and exchange-schedule manipulations vary depending on the value of the other schedule. Under lower token

PAGE 16

6 production schedules, increases in exchange-schedule value had less of an effect than under higher token-production schedules. However, because the study was not designed to assess stimulus functions of the tokens directly, more precise statements about the function of the tokens were limited. Purpose of the Present Research Although the studies discussed above demonstrated several important determinants of responding under token-reinforcement procedures, they failed to isolate the stimulus functions of the tokens. There are at least three potential functions of stimuli in token reinforcement schedules: discriminative, reinforcing, and eliciting. Previously neutral stimuli can gain a discriminative function via their temporal correlation with primary reinforcement. In other cases stimuli can serve as conditioned reinforcers due to their correlation with a reduction in delay to or increase in magnitude of primary reinforcement (Gollub, 1970). A third function is suggested by research on serial autoshaping in which stimuli correlated with the presentation of food delivered under some response-independent time-based schedule can elicit responding (Ricci, 1973). The present research investigates the potential controlling variables discussed above. The basic methodological strategy was to compare and contrast token schedules with other reinforcement schedules which have proven useful in revealing stimulus function. For example, in order to examine if tokens have an effect on behavior one could compare behavior maintained by a token-reinforcement schedule to a similar schedule in which they were absent. In the present research, stimulus functions were examined by using schedules which were similar to token-reinforcement schedules but had particular features useful for comparison purposes. The following section provides a brief overview of these schedules and their relation to the present issues.

PAGE 17

7 Schedules as Tools for Investigating Stimulus Function Token schedules have been conceptualized as second-order schedules of reinforcement, or schedules of schedules with the behavior that produces tokens considered a unitary response that is itself reinforced according to some other schedule (Kelleher, 1966). For example, under a second order FR 4 (FR 10) schedule, the first-order schedule required 10 responses to produce a token and the second-order schedule required 4 tokens before an opportunity to exchange these tokens for food was presented; the FR 10 token production schedule can be conceptualized as a unitary response that is itself reinforced according to an FR 4, hereafter termed an FR 4 (FR 10) token-reinforcement schedule. Several types of second-order schedules have stimulus arrangements potentially useful for revealing the function of tokens. The conventional second-order schedule involves brief-stimulus presentation (e.g., a flash of light, a tone) contingent on the completion of a first-order schedule requirement and primary reinforcement contingent on completion of the second-order schedule requirement (Gollub, 1970). A token reinforcement schedule differs from this more conventional arrangement in that completion of the first-order schedule requirement produces a stimulus that remains present throughout the duration of the second-order schedule. In addition, the number of stimulus presentations in a token-reinforcement schedule is directly correlated with the magnitude of reinforcement available upon completion of the second-order schedule. Second-order schedules are part of a larger family of sequence schedules that also includes chained schedules. Chained schedules are schedules that arrange for the presentation of a stimulus following the completion of some simple schedule (link), with primary reinforcement delivered after the completion of a number of links. Unlike token

PAGE 18

8 reinforcement schedules, in chained schedules each successive link may have a different response requirement and a distinct stimulus is present during each link. Further, similar to brief-stimulus schedules, in chained schedules there is no correlation between number of simple schedules completed and magnitude of reinforcement. However, when chained schedules have the same simple schedule in each link, the simple schedule becomes analogous to the first-order schedule in token-reinforcement and brief-stimulus schedules. Comparing and contrasting behavior maintained under token reinforcement schedules to that maintained under brief-stimulus and chained schedules allows for an examination of the effects of stimulus duration, stimulus accumulation, and a correlation with number of stimulus presentations and primary reinforcer magnitude. According to Kelleher and Gollub (1962), the effects of stimuli in chained or second-order schedules on responding are difficult to assess. First, the subject may not be attending to the stimuli presented in the second-order schedule (in this case tokens), in which case behavior would be a function of the contingency between responding and reinforcement. Further, responding in the presence of a segment of a token-reinforcement procedure could be due to the temporal proximity of that segment to primary reinforcement, the conditioned reinforcing value of the token that responding has produced in the past, or both. Finally, response rates could be due to some property of the stimuli themselves, apart from that gained via operant contingencies. Given these possibilities, statements about the functions of tokens in token reinforcement procedures require appropriate control procedures. One method for assessing whether the stimuli in second-order schedules are having an effect is to employ an equivalent tandem schedule. A tandem schedule, when used as

PAGE 19

9 a control condition, typically arranges for the segment response requirements and schedules to be identical to those of the second-order schedule of interest with the exception that there are no discriminative stimuli denoting the completion of a given segment (Kelleher & Gollub, 1962). For example, Kelleher (1966) examined pigeons responding under a second-order schedule in which 30 consecutive FI 2-min schedules were required for food reinforcement. In this procedure, following the completion of each FI 2-min schedule, a white key light was briefly illuminated. In order to assess the effects of the key-light flashes, a tandem procedure was employed in a separate condition in which the schedule requirement and the total reinforcement were identical, but no stimuli were presented following the completion of the FI schedules. If the stimuli have some effect on behavior one would expect differences in performance between the second-order and tandem procedures. In Experiment 1 of the present investigation, we compared performance from a token-reinforcement schedule to that of a tandem schedule equivalent in all respects except for the tokens. Any difference in responding between the token and tandem schedules would demonstrate that the presence of the tokens have an effect on behavior. As mentioned earlier, the stimuli in brief-stimulus schedules differ from those in token-reinforcement schedules with respect to the duration of presentation, accumulation, and correlation between stimulus number and position with responses and time before primary reinforcement. A modified token-reinforcement schedule, modeled after a brief-stimulus schedule, can thus serve as a basis of comparison to assess the importance of the procedural differences between token-reinforcement and brief-stimulus schedules in determining stimulus function. In one condition the briefly-presented token schedule was

PAGE 20

10 identical to that of the token schedule with the exception of presentation duration. In a second condition the tokens did not accumulate, rather a single token was presented with its position changing. In a third condition, performance on a token-reinforcement schedule was compared to that under one that simply flashed all four tokens. Differences in responding between the 3 briefly-presented token-schedule variants would indicate the importance of manner of stimulus presentation while differences between the standard and briefly-presented token schedules would indicate the importance of stimulus duration. Another method of assessing the functions of stimuli in second-order schedules involves altering the order of presentation (Kelleher & Gollub, 1962). Varying the order by which stimuli are presented in a token-reinforcement schedule may provide information concerning the importance of a correlation between number of tokens and temporal proximity to food as a response determinant. For instance an FR 4 (FR 50) schedule of token reinforcement was used as a standard for comparison in Experiments 2, 3, and 4 of the present study. Under this schedule every 50 responses would illuminate a token, with the illumination of the fourth token preceding reinforcement. In Experiment 3 of the present investigation, two of the conditions involved an alteration in the manner of token presentation, the results of which were compared to the standard token schedule. In one condition the stimulus order was reversed such that a cycle began with four tokens illuminated that darkened in reverse order as each segment was completed. If the absence of tokens had an effect on responding in link one, then the reversed order condition should produce a change in behavior early in the cycle. Another condition in Experiment three involved tokens illuminating according to a VR 50 schedule but with

PAGE 21

11 the token exchange still occurring after 200 responses and still delivering four food reinforcers. The VR contingency was designed to weaken the correlation between number of tokens and proximity to reinforcement. If the contingency was an important determinant of the effects of the tokens, then the VR token production condition should produce responding more like that under the tandem conditions. Altering the contingencies by which stimuli are presented in second-order schedules allows assessment of the degree to which these stimuli, in addition to their response-dependent presentation, might affect behavior. If the tokens are presented response independently, yoked to their temporal occurrence under the regular procedure, the role of any possible respondent (eliciting) functions of the tokens could be assessed. For instance, if while under an FR 4 (FR 25) schedule of token reinforcement, a pigeon earned the first token after 60s, the second after another 45s, the third after another 30s, and the last after another 15s, then under the yoked procedure, the tokens would be presented at these temporal intervals, irrespective of responding. The fourth experiment of this study uses just such an arrangement, with the first and third components of a session comprised of the standard token-reinforcement schedule, while the second and fourth components had a schedule in place in which the tokens and exchange periods were presented independently, yoked to their temporal occurrence in the previous component. In this case it may be that responses were elicited due to the temporal relationship between token presentation and food (Ricci, 1973). Lastly, a method generally employed in the present series of experiment to assess stimulus function in second-order schedules involved the use of a multiple schedule (Kelleher and Gollub, 1962). A multiple schedule involves some manner of alternation

PAGE 22

12 between two schedules, each presented independently with a distinct discriminative stimulus. Throughout the present study a two-component multiple schedule was employed, allowing for the comparison of a token-reinforcement procedure with some other schedule within each condition. Taken together the experiments reported here were designed to examine under what conditions functions of tokens in token-reinforcement schedules can have conditioned-reinforcing, discriminative, and eliciting functions. In particular, findings from experiments involving added-stimulus schedules, extended-chain schedules, token-reinforcement schedules, and serial autoshaping procedures suggest that the functions of these stimuli may vary depending on how they are related to the schedule of primary reinforcement. The present studies were designed not only to examine stimulus function in token-reinforcement schedules, but to also allow for points of contact between and give a broader account of the functions of stimuli in second-order and extended-chained schedules. The results of the present experiments thus lend themselves both to a better understanding of token-reinforcement schedules and to a broader conceptualization of how determinants of behavior under token-reinforcement schedules relate to those of other forms of second-order schedules.

PAGE 23

CHAPTER 2 EXPERIMENT 1 Previous research on schedules of token reinforcement has shown that response rates vary inversely with the value of FR token production and exchange schedules (Foster et al., 2001; Webbe and Malagodi, 1978). Bullock and Hackenberg (2006) showed that the relationship between response rates and FR exchange-schedule value varies depending on the value of the FR tokenproduction schedule. Foster et al., (2001) found schedule-typical patterns under token production and exchange schedules, suggestive that the tokens had some function. However, tandem-control conditions would allow for a more precise characterization of that function (or functions). An experiment investigating extended-chain schedules by Jwaideh (1973) serves as an example of utilizing tandem-control conditions to examine stimulus functions and serves as a potential point of contact between token reinforcement and extended-chain procedures. Pigeons were exposed to a series of chained schedules, each with an accompanying equivalent tandem schedules (same response requirement as a chained schedule but with no stimuli delineating transitions between schedule components, or links). The number of links in the chain was varied from 1 to 5 with FR schedules in each link. The total response requirement to complete all links was varied from 12 to 240. Two additional conditions were conducted, one in which the order of the chain sequence was reversed and the other in which the terminal stimulus in the chain was also used in the initial link. Performance under the chained schedule was compared to that of the equivalent tandem schedule to assess any potential functions of the stimuli. That is, 13

PAGE 24

14 response requirements were identical in the two schedules; the only difference was whether each link was (chained) or was not (tandem) correlated with a distinct stimulus. Overall response rates decreased and pre-ratio pausing increased, both as a direct function of the number of components and the number of responses to reinforcement. Further, response rates under tandem conditions were generally higher than equivalent chained conditions. Reversing the order of the stimuli in the chain resulted in initial increases in response rates that soon returned to those seen previously under the normal chained schedule. When the same stimulus was used for the first and last link of the chain, however, pre-ratio pausing decreased and remained shorter than under the regular chain procedure. The author suggested that differences in performance under the tandem and chained schedules demonstrated a function of the stimuli. It was suggested that stimuli early in the chain came to produce low rates of responding due to their correlation with long delays to reinforcement, while stimuli in the later links of the chain produced higher rates of responding due to their correlation with short delays to food. The experiment by Jwaideh (1973) had the same FR schedule for each link in the chain and a fixed number of links in the chain per condition. Because the requirement for each link was constant, each link could be conceptualized as analogous to a token-production segment. Further, the fixed number of links required to produce reinforcement is analogous to an FR exchange schedule. The main differences between the procedure used by Jwaideh (1973) and a token schedule are that in the latter procedure (a) the number of tokens earned is correlated with the magnitude of reinforcement (number of food deliveries during an exchange period), and (b) the tokens accumulate in continuous fashion rather than having a distinct stimulus accompany each link of a chain schedule.

PAGE 25

15 The accumulation of tokens in token reinforcement procedure resembles stimulus presentation of another form of second-order schedule, an added-stimulus schedule (Zimmerman & Ferster, 1963, 1964), which provides a further point of contact with token-reinforcement schedules. Zimmerman and Ferster (1964) examined responding under an added-stimulus schedule where pigeons responding on the left of two keys resulted in a houselight flash and a voltmeter (a gauge that could be displaced from zero to maximum of an 80 degree arc) incrementing towards a terminal position, with stimulus changes reinforced according to a variable-interval (VI) schedule (VI schedules arrange for a consequence to occur contingent on the first response following some period of time that varies around a preset average). Once the voltmeter had been fully displaced, the right key became operative, and each subsequent peck produced food (one per voltmeter increment). The VI schedule by which the voltmeter incremented (VI 1 min and VI 3 min), the number of increments required to reach the maximum (FR 10 and FR 20), and the presence/absence of the voltmeter and houselight stimuli were varied systematically across conditions. Response rates were initially low and accelerated with the number of voltmeter increments or temporal proximity to primary reinforcement. Increasing the number of steps for the voltmeter from FR 10 to FR 20 decreased response rates across both the VI 1 min and VI 3 min increment schedules, an effect similar to that of increasing the exchange schedule in token-reinforcement schedules (Bullock and Hackenberg, 2006; Foster et al, 2001). The removal of the voltmeter advance and houselight flash following the completion of each VI (tandem schedule) resulted in a more constant rate of responding across the cycle. These results suggest that the added stimuli served a discriminative function similar to the link stimuli used in extended chained schedules (Jwaideh, 1973).

PAGE 26

16 The present experiment used a token-reinforcement procedure similar to that of Bullock and Hackenberg (2006) and Foster et al. (2001). In keeping with the suggestions of Kelleher and Gollub (1962) that a multiple schedule can serve as part of a control procedure to investigate the functions of stimuli in second-order schedules, the present procedure utilized a two-component multiple schedule. One component of the multiple schedule was comprised of a token-reinforcement schedule and the other component a tandem schedule with otherwise identical contingencies. The token-production schedule remained constant at FR 50, a value at which graded patterns of responding across successive token-production segments have been seen in prior research. This graded pattern is an important indicator of discriminative functions of added stimuli such as tokens. The exchange-production FR was varied across conditions in a manner consistent with prior research in our laboratory (Bullock & Hackenberg, 2006; Foster et al., 2001). The present experiment is thus a systematic replication (Sidman, 1960) of our prior work, but with tandem-control conditions to assess the stimulus functions of the tokens. Half the pigeons in this experiment were exposed to small tokens (light emitting diodes, or LEDs) while the other half were exposed to larger tokens (jeweled stimulus lights, 1.5 cm in diameter). Prior published work on token-reinforcement schedules in our laboratory used the small-token preparation (Foster et al., 2001; Foster & Hackenberg, 2004; Hackenberg & Vaidya, 2003; Jackson & Hackenberg, 1996), but we had suggestive evidence of more pronounced and systematic effects from the larger tokens. A comparison of the effect of token size on performance under token-reinforcement schedules allowed for an assessment of whether the physical properties of the tokens themselves would produce differential discriminative effects (see Gollub, 1970, for a

PAGE 27

17 discussion of the effects of stimulus properties in second-order schedules of briefly-presented stimuli). In sum, the purpose of Experiment 1 was to (a) assess stimulus functions of the tokens by comparing directly performance under token and equivalent tandem-control conditions, (b) replicate previous findings of exchange-production FR manipulations, and (c) evaluate the effects of token size/salience on token-reinforced behavior. Method Subjects Six White Carneau pigeons (Columba livia) (numbered 907, 83, 832, 999, 910, 47) served as subjects. Pigeon 832 had prior experience with token-reinforcement schedules. Pigeons were individually housed under a 16.5 hr / 7.5 hr light:dark cycle and had constant access to water and health grit in home cages. Pigeons were maintained at 80% 20 g of their free-feeding weights with supplemental post-session feeding. Apparatus Two standard three-key pigeon chambers with a modified stimulus panel served as the experimental apparatus. The first chamber (large token chamber) was 35 cm high by 31 cm long by 34.5 cm wide, and had a stimulus panel with three response keys centered horizontally 10 cm from the ceiling to the key center and 8 cm from the adjacent key(s) (center to center). Further, a row of 12 evenly spaced stimulus lights with red caps, approximately 1.5 cm in diameter, was centered 7.5 cm above the response keys (center to center) and protruded 1.3 cm into the enclosure. The stimulus lights were always illuminated left to right, in sequential order, and served as tokens in this arrangement. Food was delivered through an opening centered 10.5 cm under the center key

PAGE 28

18 (approximately 5.5 cm wide and 5 cm tall). This box was also equipped with a Sonalert that provided an auditory stimulus (0.1 s tone) that accompanied token onset and offset. The second chamber (small token chamber) was 36 cm high by 50 cm long by 36 cm wide and the intelligence panel had 3 response keys centered vertically 11.5 cm from the ceiling to the key center and 9 cm from each other (center to center). For this chamber, a stimulus array of 34 red, evenly spaced, light-emitting diodes (LEDs), 0.4 cm in diameter, were centered 5 cm above the keys and 1.25 cm apart from each other (center to center) and protruded 0.3 cm into the enclosure. The LEDs were always illuminated left to right, in sequential order. An electromechanical stepping switch (Lehigh Valley Electronics, Model 1427) located on top of the chamber controlled LED illumination, the operation of which also provided auditory feedback each time a token was presented or removed. A food hopper opening was centered 11.5 cm below the left key (approximately 5.5 cm wide and 5 cm tall). Both chambers had a houselight centered above the token array that provided diffuse illumination. When operative, side keys were illuminated green or yellow, and the center key red. Pecks with a force between approximately 0.11-0.14 N (small token box) and 0.13 N (large token box) were counted. A solenoid-operated hopper could be raised into the food opening, allowing access to mixed grain. A white light inside the hopper illuminated during the food presentation. A photo-beam recorded head entry into the hopper. Continuous white noise and ventilation fans were active during experimental sessions to mask extraneous sounds. In a separate room a computer equipped with Med-PC software controlled experimental events and collected data.

PAGE 29

19 Procedure Preliminary Training. All pigeons were exposed to a series of training conditions (data not shown) prior to Experiment 1. Naive pigeons were initially adapted to the experimental chamber with the houselight illuminated and trained to eat food from the grain hopper. For birds with no history of key pecking, pecks to the center key were shaped via reinforcing successive approximations. All birds were then exposed to an FR 1 schedule in which pecks on the red illuminated center key produced food access. This was followed by sessions in which the left side key illuminated; a peck on this key would darken the side key and illuminate the center key, a peck on the center key would darken the center key and produce food reinforcement. This training arrangement lasted until birds were reliably pecking the side and center keys. This was follow by exposure to a multiple schedule with an FR 100 in effect during both components. Each bird was then exposed to several days of token-food pairings. These sessions consisted of the alternating illumination of the left side key within a session (randomly yellow or green). After the side key was pecked (FR 1) a token was illuminated and a tone was sounded, after which the side key darkened and the center key illuminated. A peck on the center key resulted in the darkening of one token and 1.5 s of food (timed from head into hopper). These sessions lasted for 64 reinforcers and were in effect for 3-4 days. Standard Procedure. Each session consisted of a 2-component multiple-schedule with two exposures to each component. Components occurred in a pseudo-random order, with a component remaining in effect for 16, 1.5-s food deliveries. Components were separated by 30 s blackouts, or intercomponent intervals. Sessions began with the illumination of the white houselight and left key (either green or yellow depending on component type with colors counterbalance across birds). Conditions lasted for at least 14

PAGE 30

20 sessions and until response rates were deemed stable via visual inspection of overall responses per minute for each component. Data were generally considered stable when no monotonically increasing or decreasing trends and the highest or lowest points were not present in the last 5 sessions of a condition. During the token components, tokens were earned according to an FR 50 token-production schedule (i.e., 50 responses produced one token) and exchanged according to an FR exchange-production schedule that varied systematically from FR 2 to 8 across conditions. Tokens were illuminated left to right. Completing the exchange ratio requirement produced an exchange period, during which the left key darkened and the center key illuminated red. A single response on this key darkened the rightmost token and raised the food hopper for 1.5-s. This exchange period remained in effect until all tokens earned that cycle had been exchanged for food. The period was followed by an immediate return to the token-production cycle (left key illuminated) or the inter-component blackout. After the ratio was completed and the exchange period initiated for the tandem component, a number of tokens equal to that in the token component was illuminated. The response requirement and token-exchange stimulus conditions were otherwise identical for the tandem and token components with the exception that no stimulus change occurred within the ratio under the tandem schedule (i.e., a fixed-ratio schedule). Table 2-1 lists the order of conditions and number of sessions per condition. Key colors were reversed under replications.

PAGE 31

21 Table 2-1. Order of conditions and number of sessions per condition for Experiment 1. Listed are the token schedule, but each condition also included a tandem schedule. The superscript a denotes a color reversal, the superscript b denotes conditions conducted in large token box, while an denotes that a condition was not completed. Number of sessions per condition is listed in parentheses. Pigeon Small Tokens Large Tokens 832 910 999 47 83 907 FR 2[50] (17) FR 4[50] (22) FR 8[50] (37) FR 4[50]a (28) FR 4[50]ab (53) FR 2[50](22) FR 4[50](70) FR 8[50](16)* FR 4[50] a (39) FR 2[50](19) FR 4[50](56) FR 8[50](61) FR 4[50] a (15) FR 4[50] ab (34) FR 2[50] (34) FR 4[50] (31) FR 8[50] (33) FR 4[50] a (19) FR 2[50] (35) FR 4[50] (14) FR 8[50] (30) FR 4[50] a (46) FR 2[50] (42) FR 4[50] (21) FR 8[50] (26) FR 4[50] a (49) Results Figures 2-1 and 2-2 show for each pigeon the means and standard deviations of the running response rates (response rates factoring out pre-ratio pausing) and pre-ratio pausing, respectively, as a function of exchange-production ratio across the final 5 sessions of each condition. Graphs in the left and right columns show data for pigeons typically exposed to the smaller and larger tokens, respectively. Filled points represent data from token components, open points data from tandem components, with unconnected points denoting replications. The final conditions for several birds were replications across chamber type. Thus, for 1 condition 907 was run in the small token chamber while for 1 condition 999 and 832 were run in the large token chamber. Data from these conditions are denoted by squares. Response rates varied inversely, and pre-ratio pausing varied directly, with the value of the exchange-schedule ratio for both token and tandem components. Further, for 4 out of 6 pigeons response rates in the tandem components were generally higher and pre-ratio pausing lower than rates in the token components (the exceptions being 999 and

PAGE 32

22 910). Differences in performance under the tandem and token components were greater for birds exposed to the larger tokens. 832 50100150200250300 Token Tandem 47 910 RESPONSES PER MINUTE 50100150200250300 83 999CONDITION FR 2 [50]FR 4 [50]FR 8 [50] 50100150200250300 907 FR 2 [50]FR 4 [50]FR 8 [50] SMALL TOKENSBIG TOKENS Figure 2-1. Mean responses per minute (not including pre-ratio pause) and standard deviations plotted as a function of exchange ratio from the last 5 sessions of a condition. Left panels show the data from subjects primarily run in the small token box while right panels show data for subjects primarily run in the large token box. Open symbols represent data from tandem components, closed from token components; disconnected symbols represent replications, and while squares represent data from replications across different chambers and token sizes.

PAGE 33

23 832 110100 Token Tandem 47 910 PRE-RATIO PAUSE (SEC) 110100 83 999CONDITION FR 2 [50]FR 4 [50]FR 8 [50] 110100 907 FR 2 [50]FR 4 [50]FR 8 [50] SMALL TOKENSBIG TOKENS Figure 2-2. Mean pre-ratio pausing and standard deviations plotted as a function of exchange ratio from the last 5 sessions of a condition. Note that the y-axis is logarithmic. Left panels show the data from subjects primarily run in the small token box while right panels show data for subjects primarily run in the large token box. Open symbols represent data from tandem components, closed from token components, disconnected symbols represent replications, while squares represent data from replications across token type.

PAGE 34

24 Figures 2-3 and 2-4 show response rates as a function of segment (the ordinal position within the exchange-production cycle) for pigeons exposed to the small and large tokens, respectively. Figures 2-5 and 2-6, organized similar to Figures 2-3 and 2-4, show pre-ratio pausing as a function of token segment for all pigeons (amount of time in seconds between token illumination and a response). For Figures 2-3 through 2-6, filled and open circles represent performance under token and tandem components. The large dashed lines represent replications within a token type while the small dashed lines are indicative of replications across chambers (token type). For both tandem and token components across all exchange production schedules, initial-segment rates generally were low and increased as a function of number of tokens earned (Figures 2-3 and 2-4). Pre-ratio pausing was largest for the initial segment and, with a few exceptions, decreased to a small value in the later segments (Figures 2-5 and 2-6). Responding in the tandem components was characterized by low initial-segment rates that gave way to high, constant rates of responding. Responding in the token component was characterized by low initial-segment rates gradually increasing as a function of the number of tokens earned. In summary, these figures show that differences in performance under token and tandem components were comprised of both lower running response rates and longer pausing in the early segments of a token cycle.

PAGE 35

25 832 050100150200250300350400 TOKEN TANDEM 910 RESPONSES PER MINUTE 050100150200250300350400 999SEGMENT 12 050100150200250300 1234 12345678 Figure 2-3. Mean within ratio responses per minute (not including pre-ratio pause) plotted as a function of token production segment, for subjects run primarily in the small token box, from the last 5 sessions of a condition. Points from tandem components represent successive 50 response segments. Open symbols represent data from tandem components, closed from token components, solid lines represent original exposures, large-dashed lines represent replications, and small-dashed lines represent replication across token size.

PAGE 36

26 47 050100150200250300 TOKEN TANDEM 83 RESPONSES PER MINUTE 050100150200250300350400 907SEGMENT 12 050100150200250300 1234 12345678 Figure 2-4. Mean within ratio responses per minute (not including pre-ratio pause) plotted as a function of token production segment, for subjects run primarily in the large token box, from the last 5 sessions of a condition. Points from tandem components represent successive 50 response segments. Open symbols represent data from tandem components, closed from token components, solid lines represent original exposures, dashed lines represent replications, and squares represent replication across token size.

PAGE 37

27 832 0.1110100 TOKEN TANDEM 910 PRE-RATIO PAUSE (SEC) 0.1110100 999SEGMENT 12 0.1110100 1234 12345678 Figure 2-5. Mean pre-ratio pausing plotted as a function of token production segment, for subjects run primarily in the small token box, from the last 5 sessions of a condition. Points from tandem components represent successive 50 response segments. Open symbols represent data from tandem components, closed from token components, solid lines represent original exposures, large-dashed lines represent replications, and small-dashed lines represent replication across token size.

PAGE 38

28 47 0.1110100 TOKEN TANDEM 83 PRE-RATIO PAUSE (SEC) 0.010.1110100 907SEGMENT 12 0.1110100 1234 12345678 Figure 2-6. Mean pre-ratio pausing plotted as a function of token production segment, for subjects run primarily in the large token box, from the last 5 sessions of a condition. Points from tandem components represent successive 50 response segments. Open symbols represent data from tandem components, closed from token components, solid lines represent original exposures, large-dashed lines represent replications, and small-dashed lines represent replication across token size.

PAGE 39

Discussion This experiment investigated behavior maintained under token-reinforcement schedules and equivalent tandem controls. The experimental design allowed for assessment of the effects of tokens under several different exchange-schedule values by parametrically manipulating the exchange schedule across conditions. The primary findings from this experiment, as shown in Figures 2-1 and 2-2, were (a) response rates varied inversely, and pre-ratio pausing directly, with the token production schedule FR value, replicating previous finding concerning exchange schedule manipulations (Bullock and Hackenberg, 2006; Foster et al., 2001; Webbe and Malagodi, 1978), (b) the presence of tokens reduced response rates, and increased pre-ratio pausing, when compared to their absence, and (c) the presence of the tokens engendered a more graded pattern of responding than when they were absent (see Figures 2-3 and 2-4), similar to that seen under extended-chained schedules (Jwaidah, 1973). Lastly, for pigeons exposed to the larger tokens the size of the differences in response rates between the token and tandem components was generally greater, and more consistent, than for pigeons exposed to the smaller tokens. Response rates for pigeons exposed to the larger tokens were higher in most cases under the tandem components than in the accompanying token components under the larger tokens. This result indicates a possible discriminative effect, similar to that reported by Jwaidah (1973) with extended-chained schedules. Figure 2-2 shows that lower overall response rates in the token schedules were primarily a function of long pre-ratio pauses and low response rates in the early links. This result is remarkably similar to that of Jwaidah (1973) and fits within her interpretation that the stimuli in the initial links 29

PAGE 40

30 of the chain (tokens, in the present experiment) served as discriminative stimuli associated with longer delays to primary reinforcement. The interpretation that the tokens primary function was discriminative is further supported by examination of response patterns across segments within an exchange cycle (Figures 2-3 and 2-4). Under simple FR schedules (tandem components in the present experiment) response patterning generally consisted of a pre-ratio pause followed by high, relatively constant rates of responding. However, as shown in Figures 2-3 and 2-4, under the FR 4 and FR 8 exchange schedules (middle and right columns), response patterning within an exchange cycle under the token-reinforcement component was graded with low rates in early segments increasing as more tokens were earned. The response patterning found in the present experiment is once again consistent with the results of Jwaidah (1973) and supports the interpretation of the tokens in the early segments serving a discriminative role. Given the discriminative functions of the tokens it may not be surprising, then, that larger tokens produced larger effects given their greater salience.

PAGE 41

CHAPTER 3 EXPERIMENT 2 Another type of complex sequence schedule, similar to token reinforcement schedules in some respects, involves the presentation of stimuli that are presented briefly after the completion of some simple schedule: second-order schedules of brief stimulus presentation. As with token-reinforcement procedures, the completion of each schedule segment, or unit schedule, produces a discriminable stimulus change (e.g., a flash of light, a tone) and contributes to a higher-order schedule by which primary reinforcement is presented. Some research has shown that brief stimuli can serve a discriminative role, organizing behavior with respect to the temporal proximity of primary reinforcement (Kelleher, 1966). Other research has shown that the effects of brief-stimulus presentation may vary depending on the value of the schedule of brief-stimulus presentation and the primary reinforcement schedule. Kelleher (1966) investigated whether a brief stimulus presented after completion of an FI would be sufficient to maintain pigeons responding under an extended second-order schedule. Pigeons were exposed to an FR 30 (FI 2-min) or an FR 15 (FI 4-min) schedule with a brief stimulus presentation (white key light flash) occurring after completion of each FI. Kelleher notates second-order schedules by listing the schedule of stimulus presentations required for completion first and the schedule of stimulus presentation in parentheses. Thus, FR 30 (FI 2-min) denotes a schedule in which a stimulus is presented after completion of every FI 2-min and that requires 30 FI 2-min completions before primary reinforcement is presented. Performance under each 31

PAGE 42

32 condition in this experiment was compared to performance under a tandem schedule with similar contingencies but the absence of brief-stimulus presentations. Responding in both brief-stimulus conditions was characterized by low rates of responding early in a cycle increasing as a function of proximity to primary reinforcement. Further, within a given FI, response rates had a scalloped pattern with pausing after the presentation of a brief stimulus and with response rates increasing as a function of temporal proximity to the next brief stimulus. Responding under the two brief-stimulus conditions was thus organized with respect to both the FI brief-stimulus schedule and the FR primary-reinforcement schedule. Responding under the tandem-control conditions was markedly different than under the brief-stimulus conditions in that response rates were lower and relatively constant throughout the cycle, with a slight increase as primary reinforcement approached. Kelleher concluded that the brief-stimulus presentations served as conditioned reinforcers, facilitating performance when compared to tandem-control conditions. Lee and Gollub (1971) exposed pigeons to a procedure that arranged for primary reinforcement delivery after 256 responses. A briefly-presented stimulus (0.5 s green light) was presented after a fixed number of responses, varied from 2 to 256 across several conditions. They found an inverted U-shaped function relating response rates to the size of the FR brief-stimulus schedule, with the highest response rates generally occurring under the FR 64 and 128 brief-stimulus presentation conditions. The high rates of responding under the middle brief-stimulus presentation values may be indicative of a conditioned reinforcement function. Evidence for this account is provided by the response patterning with respect to the briefly-presented stimuli, patterning similar to that

PAGE 43

33 seen under simple schedules. Lower response rates seen under the small FR brief-stimulus schedules were thought to indicate a discriminative function, with low rates at the beginning of the cycle due to the pairing of early brief-stimulus presentations with long delays to reinforcement. The present experiment sought to connect findings from research on second-order schedules of brief-stimulus presentation with token reinforcement schedules. The features of token-reinforcement schedules that differ from a brief-stimulus schedule are (a) the duration of stimulus presentation, (b) the correlation between stimulus number and primary-reinforcement magnitude, and (c) the correlation between number of stimuli illuminated and temporal proximity to exchange (Bullock & Hackenberg, 2006; Foster et al., 2001). For example, in token reinforcement schedules, the number tokens earned is inversely proportional to the number of responses remaining, and directly proportional to the number of reinforcers available during the exchange period. To more precisely evaluate the correlation between the number of tokens and both response requirements and temporal proximity to exchange, the present experiment arranged for comparisons between token schedules and several variants of brief-stimulus schedule configurations. As in Experiment 1, a multiple schedule was used to allow for within-session comparisons of response rates and patterning under the different schedule arrangements. Method Subjects Four White Carneau pigeons ( Columba livia ), numbered 47, 83, 832, and 999, served as subjects. All had previously served in Experiment 1: Pigeons 47 and 83 in the large-token box and 832 and 999 in the small-token box.

PAGE 44

34 Apparatus The chamber with the larger tokens was the only one used in the experiment (i.e., the standard three-key pigeon chamber with a row of 12 evenly-spaced stimulus lights from Experiment 1). Procedure The procedure was similar to that of Experiment 1 with the exceptions that the exchange-production schedule was held constant at FR 4, both the left and right keys were used, and in place of the tandem component, a brief-stimulus schedule was used. As in Experiment 1, sessions were comprised of a 2-component multiple-schedule with 2 exposures to each component per session. Component remained in effect for 16, 1.5-s food deliveries, and occurred in a pseudo-random order. Following the completion of a component, a 30 s blackout (intercomponent interval) occurred. Sessions began with the illumination of the white houselight and either the left or right key (either green or yellow depending on component type). Conditions lasted for at least 14 sessions and until response rates were deemed stable via visual inspection. A component began with the illumination of the left or right key, with the position of the key for a component type remaining constant throughout a condition. For components with responses recorded on the left key, tokens where illuminated left to right, while for components in which responses were recorded on the right key, tokens were illuminated right to left. A token schedule was used for one component of the multiple schedule and one of 3 types of brief stimulus schedules for the other. For the brief-stimulus arrangement, a token or tokens were illuminated briefly (0.5 s presentation), accompanied by a tone (0.1 s presentation) following the completion of each FR 50 on the token-production key. Upon the completion of a brief-stimulus

PAGE 45

35 component cycle four tokens illuminated and the exchange period began. The exchange periods were otherwise identical across component types. Table 3-1 lists the order of conditions and number of sessions per condition. Key colors were reversed under replications. Table 3-1. Order of conditions and number of sessions per condition for Experiment 2. Number of sessions per condition is listed in parentheses. Pigeon 47 83 832 999 Token (30) BMS (52) BAS (14) BFS (34) BMS (48) Token (31) BMS (44) BAS (40) BFS (31) BMS (52) Token (18) BMS (38) BAS (34) BFS (70) BMS (24) Token (43) BMS (55) BAS (31) BFS (22) BMS (18) Initially, pigeons were exposed to a condition with token reinforcement schedules in both components, followed by variations of a brief stimulus schedule in one component. One of three types of brief-stimulus components alternated with the token component. In one condition, a brief-moving stimulus (BMS) configuration was used, in which only 1 token flashed at the end of each FR 50 but changed position (left to right or right to left) depending on the number of segments completed. Thus position but not number of tokens was correlated with temporal proximity to exchange. In another condition, a brief-added stimulus (BAS) configuration was used, in which tokens illuminated in the same manner as the token component, but only remained illuminated briefly. Both position of the stimuli and number of stimuli illuminated was correlated with temporal proximity to food delivery. For a third type of condition, a brief-full

PAGE 46

36 stimulus (BFS) configuration was used in which 4 tokens flashed after every 50 responses. In this configuration, no feature of the stimulus itself, other than the number of times it was illuminated, was correlated with food delivery. Results Figures 3-1 and 3-2 show for each pigeon running response rates (response rates factoring out pre-ratio pausing) and pre-ratio pauses, respectively, for each condition in Experiment 2. For each condition the filled bars represent performance under token-reinforcement schedule components while the open bars show performance under the brief-stimulus components. No systematic differences were evident with respect to response rates or pre-ratio pausing between the three brief-stimulus schedule variants examined. Response rates under the brief-stimulus components in two cases were higher, and pre-ratio pausing lower, than those maintained under the token components. For Pigeon 832, differences between the brief-stimulus components and token components were not pronounced, whereas for 999 response rates were generally higher under the token components. Further, for Pigeons 83 and 47, differences in the response rates between the token and brief-stimulus components were due mainly to differences in the early links of a cycle.

PAGE 47

37 47 050100150200250 TOKEN OTHER 999 TOKENBMSBMS revBASBFS 83 832CONDITION TOKENBMSBMS revBASBFSRESPONSES PER MINUTE 050100150200250 Figure 3-1. Mean responses per minute (not including pre-ratio pause) and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the component varied across conditions.

PAGE 48

38 47 0100200300400 TOKEN OTHER 999 TOKENBMSBMS revBASBFS 83 832CONDITION TOKENBMSBMS revBASBFSPRE-RATIO PAUSE (SEC) 050100150 Figure 3-2. Mean pre-ratio pause and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the component varied across conditions. Figures 3-3 and 3-4 show response rates and pre-ratio pausing for token and brief-stimulus components as a function of segment (position in the exchange-production cycle). In general, responding in brief-stimulus components resembled that of the tandem components from Experiment 1: Low initial-link response rates gave way to higher, constant rates in the later links. Performance under the token components also resembled that of token performance in Experiment 1 in that response rates gradually increased as a function of the number of tokens earned. Further, in most cases for both the token and

PAGE 49

39 tandem components, pre-ratio pausing was longest for the initial segments and gave way to small pauses for subsequent segments. 47 RESPONSES PER MINUTE 050100150200250300 83 0100200300400 832 050100150200250300 999 1234 050100150200250300 1234 1234 1234 TOKEN BRIEF SEGMENTBOTH TOKENBASBMSBFS Figure 3-3. Mean responses per minute (not including pre-ratio pause) plotted as a function of token production or brief-stimulus segment from the last 5 sessions of a condition. Open symbols represent data from brief-stimulus components, closed from token components, solid lines represent original exposures, and dashed lines represent replications.

PAGE 50

40 47 PRE-RATIO PAUSE (SEC) 0.1110100 83 0.1110100 832 0.1110100 999 1234 0.1110100 1234 1234 1234 TOKEN BRIEF SEGMENTBOTH TOKENBASBMSBFS Figure 3-4. Mean pre-ratio pausing plotted as a function of token production or brief-stimulus segment from the last 5 sessions of a condition. Open symbols represent data from brief-stimulus components, closed from token components, solid lines represent original exposures, and dashed lines represent replications. Discussion Several brief-stimulus configurations were arranged to incorporate features of token-reinforcement schedules. However, neither a standard-brief stimulus schedule, nor one that varied stimulus position or magnitude as a function of proximity to reinforcement, produced behavior markedly different from what was found in the tandem schedules from the previous experiment. For two subjects, response rates under brief-stimulus components were higher than those under token components. The similarities between the brief-stimulus and tandem components, and the lack of a difference between the variants of the brief-stimulus configurations, may be due to both

PAGE 51

41 the values of the token-production and exchange schedules and the duration of stimulus presentations. Results from Lee and Gollub (1971) indicated that the effects of brief-stimulus presentations vary depending on the value of the stimulus-production schedule. As mentioned earlier, Lee and Gollub (1971) used a procedure in which the number of responses required to produce food remained constant at 256, and across conditions the FR production value of the brief-stimulus was varied. They obtained the highest rates of responding when either 2 or 4 brief stimuli occurred per food presentation. The present procedure had 4 brief-stimulus presentations per primary reinforcement period and it may be that this particular value, similar to Lee and Gollub (1971), produced a high rate of responding. In the absence of parametric variation it is difficult to determine if under other schedule values performance on the present brief-stimulus arrangements would produce behavior similar to that under the token-reinforcement component. This argument may be extended to account for the lack of effects under the different variations of brief-stimulus schedules presently employed. It may be that under different brief-stimulus presentation schedule values differences in performance would be observed. The present findings also are suggestive that the duration of the stimulus presentation is an important variable, perhaps enhancing the discriminative function of tokens by more clearly demarcating each segment. Although the only difference between the brief-added stimulus components and the token components was the duration of stimulus presentation, in some cases the two components had considerably different response rates and patterning. It may be that the continuous display of stimuli makes more salient the temporal correlation between number of tokens and delay to and amount of food available during exchange. Parametric manipulation of stimulus duration in the

PAGE 52

42 context of several token-production and exchange schedules would be needed, however, to more adequately test this possibility.

PAGE 53

CHAPTER 4 EXPERIMENT 3 Similarities between token-reinforcement and extended-chained procedures have been noted several times by previous researchers (Gollub, 1970; Kelleher and Gollub, 1962). Indeed, the long pauses at the beginning and response patterns throughout an extended-chained cycle are similar to those seen in the token components from the first 2 experiments and in other token-reinforcement schedules with FR token-production and exchange schedules (Bullock and Hackenberg, 2006; Foster et al., 2001; Kelleher, 1958; Webbe & Malagodi, 1978). Two methods for more systematically investigating similarities between token-reinforcement and chained schedules would be to (a), make token-reinforcement schedules more similar to chained schedules, and (b), replicate with token-reinforcement schedules previously investigated variations in chained schedules. In particular, the manner of presentation and order of stimuli in chained schedules have been modified to more precisely determine the function of the stimuli demarcating a given link--a procedure that could be readily adapted to token-reinforcement schedules. Kelleher and Fry (1962) examined pigeons responding over a series of conditions involving a traditional-chained schedule, a modified-chained schedule in which the stimuli denoting each link varied from cycle to cycle, and a tandem control. The chained schedule was comprised of three sequential FI schedules. It was found that, following a pre-ratio pause, responding occurred at a high, steady rate under the tandem schedule. Under the traditional-chained schedule, responding was characterized by a long pre-ratio pause and low response rates in the first link, followed by progressively increasing rates 43

PAGE 54

44 in the second and third links. When compared to the first two links of the traditional-chained schedule, response rates under the variable-chained schedule were higher and pre-ratio pauses lower. Unlike performance under the tandem schedule, response rates were positively accelerated in the chained-schedule conditions, a finding indicative of a discriminative function. Although the present experiment used FR rather than FI schedules, Kelleher and Frys (1962) manipulations are readily adaptable to token-reinforcement procedures, and may be similarly revealing of token-stimulus functions. In another study aimed at discovering stimulus function in chained schedules, Byrd (1971) examined pigeons responding when the same stimulus was presented in more than one link of a chain cycle. Pigeons responded on a chained schedule with each link comprised of a 1 min FI and with the number of links varied across conditions (3, 5, 7, and 8). During most conditions the same stimulus was used for the odd numbered links and a distinct stimulus for the even. For instance, under the condition with 7 links, the same stimulus was used for links 1, 3, 5, and 7. The one exception was the 8-link condition, in which the same stimulus was used for the even numbered links. A 7-link control condition was also examined, identical to those previously described with the exception that a distinct stimulus was used for link 7. In keeping with other findings concerning extended-chained schedules, Byrd found that response rates tended to increase from almost zero in link 1 to high rates of responding as a function of temporal proximity to food. However, under conditions with 5 and 7 links, response rates under the later links with the same color stimulus were higher compared to the following distinct stimulus link, despite closer temporal proximity to food of these links. Under the 7-link chain condition with a distinct stimulus for the terminal link, response rates were

PAGE 55

45 lower when compared to the previous 7-link condition. Further, under the 8-link condition response rates under link 1 were still extremely low, but increased under the most temporally distant same color link, link 2. Byrd interpreted these effects as showing that the discriminative properties of stimuli in chained schedules are important determinants of response rate. The response rate increasing effect of having several previous links share the same stimulus as the terminal link may be indicative of a conditioned reinforcing effect. However, they suggest that response-rate increases in chained schedules cannot be unambiguously interpreted as due to conditioned reinforcement. If conditioned reinforcement were the sole factor in determining performance on this procedure, one would expect that response rates in link 1 of the 8 link condition would be higher than those of link 1 from the 7-link condition, due to the production of the stimulus also associated with the terminal link. Both the results of Kelleher and Fry (1962) and Byrd (1971) emphasize the importance of the discriminative properties of stimuli in extended-chain schedules. In a similar vein, the present research manipulated several features of the stimulus-food relations in token reinforcement schedules to assess the stimulus functions of the tokens. Initially the token reinforcement schedule was altered such that it was more procedurally similar to a standard extended-chained schedule. Tokens were presented at the completion of each FR link, but only one reinforcer was available during exchange. If similar mechanisms influence performance on token reinforcement and extended-chain schedules then one would expect to see little difference between performance under a standard-token reinforcement procedure and one that is more like an extended chain. In a second condition, the standard token-reinforcement schedule was compared to one

PAGE 56

46 altered such that the stimulus events preceding an exchange occurred in the reverse order. Reversing the order of stimuli results in perhaps a less distinct stimulus at the beginning of a cycle, 4 tokens present, than under the standard token contingencies, the complete absence of tokens. As in Byrd (1971), the stimuli immediately preceding primary reinforcement may gain more of a conditioned reinforcing effect than those occurring earlier in an extended chain cycle, a finding that may also be wtrue in token-reinforcement schedules. Lastly, a condition was arranged in which the contingency between number of tokens and temporal proximity to an exchange was weakened. In this condition tokens were produced according to a VR schedule while exchanges occurred after 200 responses, irrespective of how many tokens had been produced. Similar to the effects of the variable-order stimulus condition from Kelleher and Fry (1962), one would expect that as the contingency between a number of tokens and proximity to exchange is degraded, performance would come to more closely resemble those from the tandem components described in Experiment 1. Method Subjects and Apparatus The subjects and apparatus were the same as in Experiment 2. Procedure Similar to Experiment 2, a 2-component multiple schedule was used in Experiment 3. Each component type occurred twice per session. In one component, a token-reinforcement schedule was in place while the other component consisted of a variant of a token schedule. In both components, an exchange period occurred after 200 responses; in the token components, tokens were produced after every 50 responses and exchanges after every 4 tokens (FR 4 [50]). Components were ordered pseudo-randomly, with

PAGE 57

47 components remaining in effect for four exchange cycles. A 30-s blackout followed each component. Sessions began with the illumination of the white houselight and the left or right side key (either green or yellow depending on component type). Conditions were in effect for a minimum of 14 sessions and response rates were deemed stable across the last 5 sessions via visual inspection. The token-variant component involved 3 variations on the standard token schedule: reinforcement magnitude, order of token delivery, and schedule of token delivery. In the first, a one-reinforcer token schedule (1 Rein) was in place. This variant is analogous to standard-chained schedules (in which a single reinforcer is available at the end of the terminal link) but with all other features identical to a standard token-reinforcement schedule. That is, identical to the token-reinforcement component, tokens were produced according to an FR 50 and exchange periods occurred after 4 tokens were produced. However, the first exchange response darkened all tokens and produced just one 1.5 s food delivery. Another variant, reverse-order token schedule (reverse), was in place for some conditions. This variant was identical to a standard token-reinforcement schedule with the exception that a token-production cycle began with 4 tokens and every 50 responses extinguished one. Thus the removal of tokens, rather than the presentation, was correlated with temporal proximity to exchange. Following the removal of the last token, 4 tokens were illuminated and an exchange period began. A third variant, broken-contingency token schedule (VR), was also in place for some conditions. Under this variant, responses produced tokens under a VR 50 schedule (up to a maximum of 12). The number of tokens produced, however, was unrelated to

PAGE 58

48 exchange, with exchanges occurring after 200 responses. Under these components it was possible to enter an exchange with fewer or greater than 4 tokens. However, during the exchange, 4 reinforcers were available, with each exchange response darkening a token. If more than 4 tokens had been produced, then additional center key responses were required to darken the remainder before another token-production cycle began. If fewer than 4 tokens had been produced, responses on the center key simply continued to produce food until 4 reinforcers had been obtained. The uncompleted VR value at the end of a cycle was simple used as the first for the next cycle. Table 4-1 lists the order of conditions and number of sessions per condition. Table 4-1. Order of conditions and number of sessions per condition for Experiment 3. Number of sessions per condition is listed in parentheses. Pigeon 47 83 832 999 1 Rein. (47) Reverse (26) VR ( 35 ) 1 Rein. (24) Reverse (17) VR ( 23 ) 1 Rein. (17) Reverse (38) VR ( 30 ) 1 Rein. (28) Reverse (11) VR ( 26 ) Results Figures 4-1 and 4-2 show for each pigeon responses per minute and pre-ratio pauses, respectively, for both component types across conditions. The filled bars represent performance under the token-reinforcement schedule while the open bars show chained-schedule performance. For all subjects except 83, response rates were lower and pre-ratio pauses longer in the one-reinforcer than in the token component. For 3 out of 4 subjects under the reversed order and broken contingency conditions, this relationship was reversed, with response rates lower and pausing longer in the token component. The lone exception in these latter conditions was Pigeon 999, for whom response rates

PAGE 59

49 remained higher in the token component for all conditions in this experiment. Even for this subject, however, response rates were slightly higher under the broken-contingency and reversed-order components than under the one-reinforcer components. 47 RESPONSES PER MINUTE 050100150200250 TOKEN OTHER 83 832 1 rein.reverseVR 050100150200250 999CONDITION 1 rein.reverseVR Figure 4-1. Mean responses per minute (not including pre-ratio pause) and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the other (no-token) component.

PAGE 60

50 47 PRE-RATIO PAUSE (SEC) 0100200300400500 TOKEN OTHER 83 832 1 rein.reverseVR 50100150200250 999CONDITION 1 rein.reverseVR Figure 4-2. Mean pre-ratio pause and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the other (non-token) component. Figures 4-3 and 4-4 show response rates and pre-ratio pausing for each 50-response segment, respectively, for both components across conditions. Filled circles represent responding under the token-reinforcement components and open circles under the components with the chained-schedule variants. For all pigeons under the one-reinforcer contingency components, and for 2 out of 4 subjects under the opposing token components, response patterning was graded with response rates increasing across segments. Interestingly, for Pigeon 999 and to some extent for Pigeon 832, response rates

PAGE 61

51 were fairly constant throughout the token cycle for this condition. Under the reversed-order and broken-contingency components, response rates early in the cycle increased compared to those in the opposing token schedule for all subjects except Pigeon 999. Response patterning across the cycle was similar for both the reversed-order and broken-contingency components in that in most cases response rates were low in the initial segment and remained somewhat constant (Pigeons 832 and 999) or gradually increased (Pigeons 47 and 83). Response rates in the opposing token components, however, were marked by a more accelerated function than the other component in 6 of 8 cases. Similar to the previous experiments, pre-ratio pausing for both components was characterized by long pauses in the initial segment giving way to short, constant pauses thereafter. 47 RESPONSES PER MINUTE 050100150200250300 TOKEN OTHER 83 0100200300400 832 050100150200250300 999SEGMENT 1234 050100150200250300 1234 1234 ONE REINFORCERREVERSED ORDERVR TOKEN PRODUCTION Figure 4-3. Mean responses per minute (not including pre-ratio pause) plotted as a function of token production or 50 response segments from the last 5 sessions of a condition. Open symbols represent data from token variant components while closed symbols represent data from token components.

PAGE 62

52 47 PRE-RATIO PAUSE (SEC) 0.1110100 TOKEN OTHER 83 0.010.1110100 832 0.1110100 999SEGMENT 1234 0.1110100 1234 1234 ONE REINFORCERREVERSED ORDERVR TOKEN PRODUCTION Figure 4-4. Mean pre-ratio pausing plotted as a function of token production or 50 response segments from the last 5 sessions of a condition. Open symbols represent data from token variant components while closed symbols represent data from token components. Discussion The primary findings from this experiment were that, relative to rates in the token-reinforcement schedule, response rates were lower under the chain-like procedure and higher under the broken-contingency and reversed-order procedure. Further, as shown in Figure 4-3, response patterning reflected, to some degree, manner of token presentation.

PAGE 63

53 The one-reinforcer contingency produced lower response rates, compared to the opposing token component, a joint product of longer pre-ratio pauses and lower response rates early in a cycle. Data from the one-reinforcer token schedule indicated that response rates were sensitive to reinforcer magnitude, with rates consistently higher in the token component (4 food deliveries) than in the chained component (1 food delivery). This could be due to the greater reinforcer magnitude in the token component or to the correlation between number of tokens and reinforcer magnitude in that component. That response rates in the initial links of the reverse-order condition remained low relative to later links suggest that low response rates in early links are not simply due to the absence of tokens. These data speak to the importance of the correlation of the tokens with temporal proximity to food. That response rates increased in the broken-contingency component relative to the token component for 3 out of 4 subjects suggests that the correlation between number of tokens and proximity to exchange is important. Although there was a difference in overall response rates between the chain-like and token reinforcement schedules, the qualitative patterns of responding were similar, with response rates increasing across a cycle as tokens accumulated. This finding provides some support for the notion that the discriminative properties of chained and token-reinforcement schedules are similar. The overall increases in response rates under the broken-contingency components are similar to those of the variable-order condition from Kelleher and Fry (1962). Results from both experiments suggest that the presence or absence of a given number of tokens or type of stimulus is arbitrary: what matters is the temporal relation between the tokens and food.

PAGE 64

CHAPTER 5 EXPERIMENT 4 In token-reinforcement schedules the tokens have been shown to have several functions, including conditioned reinforcers (Kelleher and Gollub, 1962) and discriminative stimuli (Bullock and Hackenberg, 2006). Research has shown that pigeons key pecking can be generated irrespective of operant contingencies via stimulus-food relations (Brown & Jenkins, 1968). The typical procedure for generating such autoshaped, or automaintained, behavior is to repeatedly present a keylight, followed by response-independent food delivery (Brown and Jenkins, 1968). Some of the conditions under which such autoshaped keypecking has been generated and maintained are similar to token reinforcement schedules. For example, autoshaped responding can be generated under conditions in which distinct stimuli are presented successively, temporally correlated with food presentation (Ricci, 1973). Ricci (1973) examined pigeons autoshaped key pecking under several stimulus arrangements. In some conditions subjects were exposed to contingencies in which a sequence of 4 colors was presented for 30 s each, with the terminal stimulus followed by food reinforcement. Performance was then compared to that generated under a similar procedure except that just one stimulus was presented for the entire 120 s prior to food delivery. Response distributions under the 4-color conditions were graded, with response probability increasing as a function of the temporal proximity of that stimulus to reinforcement. By contrast, responding under the 1-color conditions was much more uniform throughout the 120 s interval. It was suggested that such autoshaping procedures 54

PAGE 65

55 are similar to chained schedules in that both involve sequential arrangements of stimuli temporally related to food. It is possible that responding on schedules of token reinforcement may be maintained simply by the presentation of tokens, with the number of tokens presented correlated with temporal proximity to food. The present experiment was designed to investigate this possibility. In one component of a multiple schedule tokens were presented response-independently, yoked to their temporal occurrence in the immediately prior token reinforcement component. Under simple schedules, response-independent reinforcement breaks the dependency between responding and food production and results in lower response rates (Lattal, 1972). If the tokens served as conditioned reinforcers then presenting them response independently should result in a substantial decrease or elimination of response rates. On the other hand, if tokens served an eliciting function, in the manner of serial autoshaping, one would expect some maintenance of responding. To examine whether the presence versus absence of the tokens was an important determinant of responding in the yoked component, the tokens were removed from the yoked component in some conditions, while exchanges remained yoked to their temporal occurrence in the preceding token component. These conditions were designed to determine to what degree responding maintained under token-reinforcement schedules is a product of the temporal relations between tokens and food, apart from the contingent production of tokens. To examine the possibility that induction from the token schedule could account for responding in the yoked components, other conditions held constant the key color in both component types. If induction was a determinant of responding in the

PAGE 66

56 yoked component one would expect that by making the two components more similar response rates in the two components would converge. Method Subjects and Apparatus The subjects and apparatus were the same as in Experiments 2 and 3. Procedure Similar to the other experiments reported, a 2-component multiple schedule was used in Experiment 4 with a token-reinforcement component and a yoked component. Each component occurred twice and lasted for 4 exchange cycles. A 30-s blackout followed each component. Sessions began with the illumination of the white houselight and side key associated with the token component (either green or yellow). In the token component tokens were produced every 50 responses and exchanges after every 4 tokens (FR 4 [50]). In the yoked component, tokens and exchange periods were presented response independently, yoked to the times they occurred in the preceding token component. The token components always occurred first and third, the yoked component second and fourth. Two other conditions consisted of (a) holding the token-production key color constant across both components (Yoked Color-Same, or CS), and (b) removing the tokens entirely from the token-production cycle of the yoked component (Yoked No Token, or NT). In this latter condition, 4 tokens illuminated immediately prior to the exchange period while the number of reinforcer deliveries (4) and delays to the exchange period were equal to those in the token component. Conditions were in effect for a minimum of 14 sessions and until response rates were deemed stable across the last 5

PAGE 67

57 sessions via visual inspection. Table 5-1 lists the order of conditions and number of sessions per conditions Table 5-1. Order of conditions and number of sessions per condition for Experiment 4. Number of sessions per condition is listed in parentheses. Pigeon 47 83 832 999 Yoked (34) Yoked NT (14) Yoked (16) Yoked CS (17) Yoked (35) Yoked NT (15) Yoked (20) Yoked CS (30) Yoked (50) Yoked NT (17) Yoked (22) Yoked CS (18) Yoked (25) Yoked NT (14) Yoked (32) Yoked CS (18) Results Figures 5-1 and 5-2 show running response rates (response rates factoring out pre-ratio pausing) and pausing for each pigeon, respectively, across the final 5 sessions in each condition. Filled bars show performance from the token components while open bars show performance from the yoked components. Response rates for all conditions in the standard-token component were higher than in any of the yoked-component variations. Pausing was also generally greater in the yoked than in the token components. Response rates in the standard-yoked components were lower than in to the opposing token components, but never reached zero. Under the no-token (NT) yoked condition, however, responding was almost completely eliminated. In the yoked component with the token-production key color the same as the opposing token component (CS), response rates were either comparable to, or lower than, those in the standard yoked-token conditions for 3 out of 4 pigeons.

PAGE 68

58 47 RESPONSES PER MINUTE 050100150200250 TOKEN YOKED 999 YOKEDYOKED NTYOKEDYOKED CS 050100150200250 83 050100150200250 832CONDITION YOKEDYOKED NTYOKEDYOKED CS 050100150200250 Figure 5-1. Mean responses per minute (not including pre-ratio pause) and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the yoked component.

PAGE 69

59 47 RESPONSE LATENCY (SEC) 050100150200250 TOKEN YOKED 999 YOKEDNO TOKENYOKEDSAME COLOR 050100150200250 83 050100150200250 832CONDITION YOKEDNO TOKENYOKEDSAME COLOR 050100150200250 Figure 5-2. Mean pre-ratio pausing and standard deviations from the last 5 sessions of a condition. Filled bars represent data from the token component while open bars represent data from the yoked component. Figures 5-3 and 5-4 show response rates and pausing for each component type across successive segments in the exchange cycle. Segments consisted of either 50 responses (token component) or the equivalent time periods (yoked component). Response patterning under the token components was graded, increasing as a function of the number of tokens earned and proximity to food. Under the standard token-yoked conditions, responding within a cycle was characterized by extremely low rates for the first one or two segments followed by an increase, resulting in rates close to those of the comparable token component for segments 3 and 4. For all pigeons under the no-token

PAGE 70

60 yoked components response rates were equally low for all 4 segments. Segment pauses, (Figure 5-4) also corresponded to this pattern, with long pauses in the early links of a cycle becoming shorter with increasing numbers of tokens illuminated. (Under the yoked conditions pausing sometimes exceeded the period of time before the first token illuminated. In such cases, the link 1 pause was set to the time period before token presentation. The mean pre-ratio pauses, shown in Figure 5-2, however, were based on the actual time prior to a response, irrespective of token illumination.) 47 RESPONSES PER MINUTE 050100150200250300 TOKEN YOKED 83 0100200300400 832 050100150200250300 999SEGMENT 1234 050100150200250300 1234 1234 YOKEDYOKED COLOR CONSTANTYOKED NO TOKEN Figure 5-3. Mean responses per minute (not including pre-ratio pause) plotted as a function of token production or yoked token-production segment from the last 5 sessions of a condition. For no-token yoked components the segment data are organized around when the token would have occurred. Open symbols represent data from yoked components, closed from token components, solid lines represent original exposures, and dashed lines represent replications.

PAGE 71

61 47 SEGMENT RESPONSE LATENCY (SEC) 0.1110100 TOKEN YOKED 83 0.1110100 832 0.1110100 999SEGMENT 1234 0.1110100 1234 1234 YOKEDYOKED COLOR CONSTANTYOKED NO TOKEN Figure 5-4. Mean pre-ratio pausing plotted as a function of token production or yoked token-production segment from the last 5 sessions of a condition. For no-token yoked components the segment data are organized around when the token would have occurred. Open symbols represent data from yoked components, closed from token components, solid lines represent original exposures, and dashed lines represent replications. Figure 5-5 shows the obtained delays between the last response of a segment and the illumination of a token for that segment, across successive segments within an exchange cycle. For cases in which a response did not occur within a given segment, the duration of the segment was used as the delay. The response-token delays varied both across subjects and across segments within an exchange cycle for individual subjects.

PAGE 72

62 For Pigeon 999, the mean response-token delays were generally quite long, rarely less than 10 s. For the other 3 pigeons, response-token delays were generally high in the early segments, but became shorter in the later links when response rates were high. 47 RESPONSE TOKEN DELAY (SEC) 020406080100 TOKEN OTHER 83 050100150200 832 020406080100 999SEGMENT 1234 020406080100 1234 1234 TOKEN YOKEDTOKEN YOKED REPLICATIONTOKEN YOKEN COLOR SAME Figure 5-5. Mean time between the last response of a segment and the token production plotted as a function of yoked token-production segment from the last 5 sessions of a condition. Discussion Response rates in the yoked components were lower than in the token components, suggesting a role for the dependency between responding and token production (i.e., a reinforcing function). In the yoked components, the presence of tokens maintained

PAGE 73

63 responding at much higher levels than when they were absent. Within a yoked-token component, response rates increased as a function of the temporal proximity to food in much the same way that they did under response-dependent token production. This finding, similar to that reported by Ricci (1973), suggests a discriminative and/or eliciting role for the tokens. When the key color was held constant, response rates were equal to or slightly lower than the regular-yoked components, indicating that induction from the token component does not account entirely for responding in the yoked component. However, because of the previous history of reinforcement on token-reinforcement schedules statements concerning the degree to which the discriminative versus eliciting properties of the stimuli control behavior are limited. Figure 5-5 shows that an adventitious contingency does not explain the present results entirely. Response rates increased in a graded fashion, as a function of the number of tokens earned (see Figure 5-3). There existed a considerable delay between responding and the illumination of the first and second tokens, and relatively short delays between responding and the presentation of the third and fourth tokens. If an adventitious contingency had existed with respect to responding and production of the later tokens, one would expect that responding would be entirely absent prior to the first two token deliveries, rather than the observed graded functions that were found.

PAGE 74

CHAPTER 6 GENERAL DISCUSSION The objective of this series of experiments was to explore systematically the stimulus functions of tokens in token-reinforcement schedules. Performance under schedules of token reinforcement was compared to that under tandem schedules and to that under several token-like schedules, all with equivalent response requirements. Experiment 1 compared token schedules to equivalent tandem schedules and found that response rates under token-reinforcement schedules were lower than under tandem schedules, with response patterning suggesting a discriminative function of the tokens. Experiment 2 compared token schedules to several schedules of briefly-presented token presentation, and found that response rates under token-reinforcement schedules were in some cases lower than under variants of the briefly-presented stimulus schedules. Rates and patterns in the latter were comparable to the tandem components in Experiment 1, suggesting that the continuous display of tokens contributes to their discriminative effects. Experiment 3 compared token schedules to extended chained schedules, and found that response rates under token-reinforcement schedules were lower than under comparable chained schedules when the correlation between token display and temporal proximity to exchange periods was altered. Only when compared to standard-chained schedule, with a single reinforcer at the end of the chain, were response rates higher in token schedules, indicating sensitivity to reinforcement magnitude with stimulus conditions held constant. Experiment 4 compared token schedules to schedules of response-independent token presentation to assess the reinforcing and potential eliciting 64

PAGE 75

65 functions of the tokens. Response rates were reduced under the response-independent schedules, suggesting a reinforcing function, but they were not eliminated, suggesting an eliciting function. Taken together, the results suggest that tokens serve important stimulus functions in token reinforcement schedules, and that the specific function, or functions, depend on the contingencies in which they are embedded. The results of the Experiment 1 correspond to those seen in prior research with token schedules (Bullock & Hackenberg, 2006; Foster et al., 2001), added-stimulus schedules (Zimmerman & Ferster, 1964), and extended-chained schedules (Jwaidah, 1973). Decreases in response rates in the present experiment as a function of increasing the exchange-schedule requirements is consistent with previous token-reinforcement schedule findings using FR exchange schedules (Bullock and Hackenberg, 2006; Foster et al., 2001). The graded pattern of responding found under token-reinforcement schedules in the present research correspond to both previous token-reinforcement research and to the effects off adding incremental stimulus changes reported by Zimmerman and Ferster (1964). Higher rates of responding in components with tandem schedules, when compared to components with token schedules, reported in Experiment 1, correspond to similar manipulations with extended chained schedules (Jwaidah, 1973). Experiment 2 showed that briefly presenting tokens attenuated their discriminative function, irrespective of whether their position and number was correlated with responses/temporal proximity to reinforcement. The lack of a discernable effect between the brief-stimulus conditions, combined with the differences for some pigeons between the brief-stimulus and token components, points to the importance of token display duration. Briefly presenting the tokens may have decreased their salience and

PAGE 76

66 thus disrupted the discriminative properties of the tokens, a result that was perhaps similar to the effects of varying token size in Experiment 1. In other words, the results of Experiment 2 suggest that the functions of tokens in second-order schedules are related to how long they demarcate the completion of each segment. The results of Experiment 3 showed that the discriminative properties of tokens in token-reinforcement schedules are similar in several respects to stimuli in chained schedules. As mentioned earlier, Kelleher and Fry (1962) found that randomizing the order of stimuli delineating the links in an extended chain schedule increased response rates relative to a standard extended-chained schedule. A similar result was found by Kelleher (1958) when chimpanzees pre-ratio pauses decreased as a result of the non-contingent delivery of a large group of tokens prior to the start of a session. In Experiment 3 of the present study, the condition that involved use of a VR schedule of token production that operated independently of the FR 200 exchange-production schedule was analogous to randomizing the stimuli in Kelleher and Fry (1962). The use of a VR token-production schedule in the present research and the procedures used in Kelleher and Fry (1962) and Kelleher (1958) were all similar in that degrading the correlation between tokens and temporal proximity to food resulted in decreased pre-ratio pausing or increased rate of responding. Experiment 4 was designed to assess the conditioned reinforcing and eliciting functions of the tokens by removing the dependency between responding and token presentation. In this experiment responding under a standard token-reinforcement schedule was compared to that under a procedure where tokens were delivered response independently, yoked to their temporal occurrence in the preceding standard token

PAGE 77

67 schedule component. If the tokens served as conditioned reinforcers one would expect that response rates would either substantially decrease or completely cease, as response-independent token delivery would break the contingency between responding and token production. However, if the tokens had an eliciting function, responding should continue, as response-independent token delivery would not alter the token-food correlation. Responding was reduced in yoked conditions, suggesting a reinforcing function, but it was not eliminated, suggesting an eliciting function. The latter result is consistent with those reported by Ricci (1973), described earlier, in which the probability of a response in the presence of a given stimuli increased as a function of temporal proximity to food. A similar relationship was found in Experiment 4, suggesting that tokens may serve to elicit responding in addition to other functions. Similar to the common discriminative properties of stimuli in token reinforcement and chained schedules, research has shown that stimuli in chain schedules also have an eliciting function. In support of this, Dougherty and Lewis (1991) used an omission procedure to investigate further the degree to which extended-chained schedule stimuli have eliciting functions. An omission procedure is one in which the occurrence of a response prevents the delivery of reinforcement. Thus, if responding is maintained by operant contingencies, one would expect an omission procedure to eliminate responding. Conversely, if responding is due to the stimulus-food relations and is independent of operant contingencies, then responding should still occur. Pigeons were exposed to 3 conditions, 2 with an omission contingency and one with a standard chain. In the first, a 2-link chain was in effect, with each link lasting for 60 s and with transition occurring response independently. Responses in the first link terminated the chain and began an

PAGE 78

68 inter-trial interval (ITI). The second condition was similar to the first except that an FI 60 s schedule was in effect for both links. If responses occurred before 60 s had elapsed in the first link then the chain terminated and the ITI began. The third condition was a standard chain with an FI 60 s in both components. That responding occurred in the initial link of the first two conditions (with an omission procedure), in spite of resulting in a lower rate of reinforcement, suggests that the stimuli in these procedures had an eliciting function. Thus, consistent with the results of Ricci (1973) and the present experiment, chain-schedule stimuli also have eliciting functions. Taken together, the results of the experiments presented here suggest that stimuli in token-reinforcement procedures can have a combination of functions. Experiment 1 showed that response rates are lower under token reinforcement schedules when compared to equivalent tandem schedules. This finding, along with a more graded pattern of responding found under the token-reinforcement schedules suggests that tokens have a discriminative function. The results of Experiment 2 suggest that the duration of token presentation is an important determinant of the discriminative functions of tokens. Experiment 3 showed that changing the token production schedule to a VR increased response rates, a finding that again suggests a discriminative function. Results from Experiment 4 showed that response rates continue in the absence of a contingency between responding and token production, a finding that suggests tokens may have a response eliciting function. Further, higher response rates in Experiment 4 under the standard token-reinforcement schedule than under the yoked schedule speaks to the role of a dependency between responses and tokens, and may be indicative of a conditioned reinforcing function. Thus, the present research suggests that the functions of stimuli in

PAGE 79

69 token reinforcement procedure have eliciting, discriminative, and reinforcing properties and that the particular function depends on the contingencies in place. While the present research shows that the tokens have several functions, many dimensions of a token reinforcement procedure have yet to be investigated. Research that varies the length of presentation of a brief-stimulus from very short to almost the entire segment period would allow for a more precise understanding how stimulus function varies as the schedules shift from more brief-stimulus-like to more tokenor chain-like. Manipulations of stimulus durations are particularly suited to experimental preparations that allow for controlled presentations, as opposed to manipulable tokens used in some of the studies discussed earlier. Second, most prior research has conducted comparisons of the various component types under only 1 token production and exchange value. Bullock and Hackenberg (2006) showed that the effects of tokens can vary depending on the context of both the token production and exchange schedules, and the same may also be true of the comparisons of the present research. Although examination of such a wide combination of component types across several token-production and exchange schedules was not practical for the present investigation, these manipulations would yield more detailed knowledge of stimulus function. Third, in the present research for some subjects overall response rates tended to decrease across experiments. Because effects in the present experiment were assessed based on differences within a condition between the two different components this was not a major concern. However, an area for future research would be to investigate how performance varies as a function of amount of exposure to token-reinforcement

PAGE 80

70 schedules. It may be that the determinants of behavior vary as experience with token schedules increases. Lastly, the present procedures involve a dependent arrangement of token production and exchange schedules, with token production contributing to satisfying the exchange-production schedule. This need not be the case, however; the procedure could be modified such that producing an exchange would be independent of token production, given that at least one token had been earned. Under such an arrangement subjects could accumulate a number of tokens before completing a separate exchange ratio. In this case no relationship between number of tokens and proximity to exchange would exist; thus determinants of responding in an accumulation procedure might vary considerably from the token-reinforcement schedules presented presently. Turning to broader issues, token-reinforcement procedures have been used extensively in applied settings with a wide variety of treatment populations (Kazdin & Bootzin, 1972; Kazdin, 1982). That token reinforcement schedules have been so widely used speaks to the importance of understanding of their controlling variables. By using nonhuman subjects in a precisely controlled environment the present research was able to contribute to the literature by showing that tokens have multiple functions depending on the contingencies. Data reported by Field, Nash, Handwerk, and Friman (2004) replicates the effects of exchange-schedule manipulations in a token reinforcement procedure in an applied setting. They found that decreasing the time between opportunities to exchange tokens for primary reinforcers, and thus increasing reinforcement rate, increased the effectiveness of a token economy in managing problem behavior.

PAGE 81

71 The present data and prior token-reinforcement schedule research suggest several potentially fruitful extensions into applied settings. As mentioned earlier, token economies often are utilized as a means of reinforcing pro-social behavior. The present data suggest some of the circumstances that promote a discriminative rather than conditioned reinforcing function. For example, Experiment 3 showed that a correlation with number of tokens and temporal distance to exchange resulted in lower response rates than when this contingency was disrupted. Experiment 2 suggested that the duration of presentation may also determine the discriminative properties of tokens. One extension of these findings to applied research would entail comparing performance under briefly-presented token schedule, a token-schedule with an FR exchange, and a token-schedule with a VR exchange. Comparisons between these three arrangements would allow an assessment of how potential discriminative functions might disrupt responding maintained by tokens and methods of reducing any discriminative functions. Further research along these lines will add to the precision with which token reinforcement systems are implemented, and aid in managing behavior in multiple settings.

PAGE 82

LIST OF REFERENCES Brown, P. L. & Jenkins, H. M. (1968). Auto-shaping of the pigeon's key-peck. Journal of the Experimental Analysis of Behavior, 11, 1-8. Bullock, C. E., & Hackenberg, T. D. (2006). Second-order schedules of token reinforcement in pigeons: Implications for unit price. Journal of the Experimental Analysis of Behavior, 85, 95-106. Byrd, L. D. (1971). Responding in the pigeon under chained schedules of food presentation: The repetition of a stimulus during alternate components. Journal of the Experimental Analysis of Behavior, 16, 31-38. Cowles, J. T. (1937). Food tokens as incentives for learning by chimpanzees. Comparative Psychology Monographs, 14, 1-96. Dougherty, D. M. & Lewis, P. (1991). Elicited responding in chain schedules. Journal of the Experimental Analysis of Behavior, 56, 475-487. Ferster, C. B & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton-Century-Crofts. Field, C. E., Nash, H. M., Handwerk, M. L., & Friman, P. C. (2004). A modification of the token economy for nonresponsive youth in family-style residential care. Behavior Modification, 28(3), 238-456. Foster, T. A. & Hackenberg, T. D. (2004). Unit price and choice in a token-reinforcement context. Journal of Experimental Analysis of Behavior, 81, 5-25. Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token reinforcement with pigeons: Effects of fixedand variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 76, 159-178. Gollub, L. R. (1970). Information on conditioned reinforcement: A review of conditioned reinforcement, edited by Derek P. Hendry. Journal of the Experimental Analysis of Behavior, 14, 361-372. Hackenberg, T. D., & Vaidya, M. (2003). Determinants of pigeons' choices in token-based self-control procedures. Journal of the Experimental Analysis of Behavior, 79, 207-218. 72

PAGE 83

73 Jackson, K., & Hackenberg, T. D. (1996). Token reinforcement, choice, and self-control in pigeons. Journal of the Experimental Analysis of Behavior, 66, 29-49. Jwaideh, A. R. (1973). Responding under chained and tandem fixed-ratio schedules. Journal of the Experimental Analysis of Behavior, 19, 259-267. Kazdin, A. E. (1982). The token economy: A decade later. Journal of Applied Behavior Analysis, 15, 431-445. Kazdin, A. E., & Bootzin, R. R. (1972). The token economy: An evaluative review. Journal of Applied Behavior Analysis, 5, 343-372. Kelleher, R. T. (1956). Intermittent conditioned reinforcement in chimpanzees. Science, 124, 679-680. Kelleher, R. T. (1957). A multiple schedule of conditioned reinforcement with chimpanzees. Journal of Comparative and Physiological Psychology, 49, 571-575. Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281-289. Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475-485. Kelleher, R. T. & Gollub, L. R. (1962). A review of positive conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 5, 543-597. Kelleher, R. T. & Fry, W. (1962). Stimulus functions in chained fixed-interval schedules. Journal of the Experimental Analysis of Behavior, 5, 167-173. Lattal, K. A. (1972). Response-reinforcer independence and conventional extinction after fixed-interval and variable-interval schedules. Journal of the Experimental Analysis of Behavior, 18, 133-140. Lee, J. K. & Gollub, L. R. (1971). Second-order schedules with fixed-ratio components: Variation of component size. Journal of the Experimental Analysis of Behavior. 15, 303-310. Malagodi, E. F. (1967). Fixed ratio schedules of token reinforcement. Psychonomic Science, 8, 469-470. Malagodi, E. F. DeWeese, J. & Johnston, J. M. (1973). Second-order schedules: A comparison of chained, brief-stimulus, and tandem procedures. Journal of the Experimental Analysis of Behavior, 20, 447-460. Malagodi, E. F., Webbe, F. M., & Waddell, T. R. (1975). Second-order schedules of token reinforcement: Effects of varying the schedule of food presentation. Journal of the Experimental Analysis of Behavior, 24, 173-181.

PAGE 84

74 Ricci, J.A. (1973). Key pecking under response-independent food presentation after long simple and compound stimuli. Journal of the Experimental Analysis of Behavior,19, 509-516. Sidman, M. (1960). Tactics of Scientific Research. Boston: Authors Cooperative Inc. Waddell, T. R., Leander, J. D., Webbe, F. M., & Malagodi, E. F. (1972). Schedule interactions in second-order fixed-interval (fixed-ratio) schedules of token reinforcement. Learning and Motivation, 3, 91-100. Webbe, M. F. & Malagodi, E. F. (1978). Second-order schedules of token reinforcement: Comparisons of performance under fixed-ratio and variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 30, 219-224. Williams, B. A. (1994). Conditioned reinforcement: Neglected or outmoded explanatory construct? Psychonomic Bulletin & Review, 1(4), 457-475. Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative Psychology Monographs, 12, 1-72, No. 60. Zimmerman, J., & Ferster, C. B. (1963). Fixed-interval performances with added stimul in monkeys. Journal of the Experimental Analysis of Behavior, 6, 317-322. Zimmerman, J., & Ferster, C. B. (1964). Chained VI performance of pigeons maintained with an added stimulus. Journal of the Experimental Analysis of Behavior, 7, 83-39.

PAGE 85

BIOGRAPHICAL SKETCH I grew up in Oxford, North Carolina, attending public schools. After graduating from J. F. Webb High School in 1994, I enrolled as an undergraduate at the University of North Carolina at Wilmington (UNCW). I graduated from UNCW with a Bachelor of Arts in psychology with honors in the spring of 1999. The following fall I enrolled at the University of Florida in the graduate program in psychology in the behavior analysis area and began working in Dr. Timothy Hackenbergs lab. My research at Florida has focused on choice, self-control, and schedules of token reinforcement. 75


Permanent Link: http://ufdc.ufl.edu/UFE0013785/00001

Material Information

Title: Stimulus Functions in Token-Reinforcement Schedules
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0013785:00001

Permanent Link: http://ufdc.ufl.edu/UFE0013785/00001

Material Information

Title: Stimulus Functions in Token-Reinforcement Schedules
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0013785:00001


This item has the following downloads:


Table of Contents
    Title Page
        Page i
        Page ii
    Acknowledgement
        Page iii
    Table of Contents
        Page iv
        Page v
    List of Tables
        Page vi
    List of Figures
        Page vii
        Page viii
    Abstract
        Page ix
        Page x
    Introduction
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
        Page 8
        Page 9
        Page 10
        Page 11
        Page 12
    Experiment 1
        Page 13
        Page 14
        Page 15
        Page 16
        Page 17
        Page 18
        Page 19
        Page 20
        Page 21
        Page 22
        Page 23
        Page 24
        Page 25
        Page 26
        Page 27
        Page 28
        Page 29
        Page 30
    Experiment 2
        Page 31
        Page 32
        Page 33
        Page 34
        Page 35
        Page 36
        Page 37
        Page 38
        Page 39
        Page 40
        Page 41
        Page 42
    Experiment 3
        Page 43
        Page 44
        Page 45
        Page 46
        Page 47
        Page 48
        Page 49
        Page 50
        Page 51
        Page 52
        Page 53
    Experiment 4
        Page 54
        Page 55
        Page 56
        Page 57
        Page 58
        Page 59
        Page 60
        Page 61
        Page 62
        Page 63
    Discussion
        Page 64
        Page 65
        Page 66
        Page 67
        Page 68
        Page 69
        Page 70
        Page 71
    References
        Page 72
        Page 73
        Page 74
    Biographical sketch
        Page 75
Full Text












STIMULUS FUNCTIONS IN TOKEN-REINFORCEMENT SCHEDULES


By

CHRISTOPHER E. BULLOCK



















A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2006

































Copyright 2006

by

Christopher E. Bullock















ACKNOWLEDGMENTS

I would like to thank my advisor, Dr. Timothy D. Hackenberg, for his guidance and

mentoring during the research and writing portions of this project. I would also like to

thank my family and lab mates for support throughout the course of this project.
















TABLE OF CONTENTS



A C K N O W L E D G M E N T S ................................................................................................. iii

LIST OF TA BLES .............. .............................................. ............ .......... vi

LIST OF FIGURES ......... ....... .................... ............ .... ........... vii

ABSTRACT .............. .................. .......... .............. ix

CHAPTER

1 GENERAL INTRODUCTION ..................................................... ....................

Purpose of the Present R research .................. ... .................................. ...............
Schedules as Tools for Investigating Stimulus Function............... ... ............7

2 EXPERIM ENT 1 .................................... ..... .......... ....... ..... 13

M e th o d ....................................................................................................... 1 7
Subjects ....................................................................................................... 17
Apparatus .................... ......................... 17
Procedure ................ ............................... 19
R results ....................................... ... .................................. 21
Discussion ......................... ...................... 29

3 EXPERIM EN T 2 ................ .............. ................ .......................31

M e th o d ............. ......... .. .............. .. ....................................................... 3 3
Subj ects .................. ............... ............... 33
Apparatus .................... ......................... 34
Procedure ................ ............................... 34
Results ......... ..... ... .......... ............................ ............... 36
Discussion ......................... ...................... 40

4 E X PE R IM E N T 3 ................................................................43

M ethod ..... .................................. ......................... ...... .................... 46
Subjects and Apparatus .............................................. 46
Procedure ........................ ...................... 46










R e su lts .............................................................................................................4 8
D isc u ssio n .............................................................................................................. 5 2

5 EX PERIM EN T 4 ........................................................................... 54

M e th o d .................. ....................................................................................5 6
Subjects and Apparatus ............ ........ ......... .. ........................... 56
Procedure .......................... .................... 56
Results .............. .... ..... .... ............................. ............... 57
D discussion ............... .... .... .....................................................................62

6 GENERAL DISCUSSION .............. .......................... 64

LIST OF REFERENCES ....................................................................... ......... ......... ......... 72

B IO G R A PH IC A L SK E T C H ........................................................................................ 75







































v
















LIST OF TABLES

Table page

2-1. Order of conditions and number of sessions per condition for Experiment 1...........21

3-1. Order of conditions and number of sessions per condition for Experiment 2...........35

4-1. Order of conditions and number of sessions per condition for Experiment 3...........48

5-1. Order of conditions and number of sessions per condition for Experiment 4...........57
















LIST OF FIGURES


Figure page

2-1. Mean responses per minute (not including pre-ratio pause) plotted as a function
of exchange ratio.. .......................... ................ .............................22

2-2. Mean pre-ratio pausing plotted as a function of exchange ratio............................23

2-3. Mean within ratio responses per minute (not including pre-ratio pause) plotted as
a function of token production segment for subjects run primarily in the small
token box .............................................................................25

2-4. Mean within ratio responses per minute (not including pre-ratio pause) plotted as
a function of token production segment for subjects run primarily in the large
token box .............................................................................26

2-5. Mean pre-ratio pausing plotted as a function of token production segment for
subjects run prim arily in the sm all token box .................................. ... ..................27

2-6. Mean pre-ratio pausing plotted as a function of token production segment for
subjects run primarily in the large token box................ ..................28

3-1. Mean responses per minute (not including pre-ratio pause) for each condition ......37

3-2. M ean pre-ratio pause for each condition. ...................................... ...............38

3-3. Mean responses per minute (not including pre-ratio pause) plotted as a function
segm ent ......... ...... ............ ...................................... ........................... 39

3-4. Mean pre-ratio pausing plotted as a function of segment ...........................40

4-1. Mean responses per minute (not including pre-ratio pause) for each condition ......49

4-2. Mean pre-ratio pause for each condition.............. .. .... ................50

4-3. Mean responses per minute (not including pre-ratio pause) plotted as a function
segm ent ......... ...... ............ ...................................... ........................... 5 1

4-4. Mean pre-ratio pausing plotted as a function of segment ...........................52

5-1. Mean responses per minute (not including pre-ratio pause) for each condition ......58









5-2. M ean pre-ratio pausing for each condition.................................... ............... 59

5-3. Mean responses per minute (not including pre-ratio pause) plotted as a function
of seg m ent ........................................................................... 6 0

5-4. Mean pre-ratio pausing and plotted as a function of segment..............................61

5-5. Mean time between the last response of a segment and the token production
plotted as a function of segment................... .................................... ... ............ 62















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

STIMULUS FUNCTIONS IN TOKEN-REINFORCEMENT SCHEDULES

By

Christopher E. Bullock

May 2006

Chair: Timothy D. Hackenberg
Major Department: Psychology

The present study examined pigeons' responding on token-reinforcement

schedules using a two-component multiple schedule with a token-reinforcement schedule

in one component and one of several other schedule types in the other. In Experiment 1

responding under a token-reinforcement schedule was compared to that under an

equivalent tandem schedule. It was found that response rates under the tandem schedule

were higher than under the token and that response patterning in the token-reinforcement

schedule was more graded than under the tandem schedule. In Experiment 2 responding

under a token-reinforcement schedule was compared to that under a series of brief-

stimulus schedule variants. Response rates under brief-stimulus arrangements were

higher than under token arrangements, resembling responding in the tandem components

from Experiment 1. In Experiment 3, responding under a token-reinforcement schedule

was compared to that maintained under several extended-chained schedule variants.

Responding was sensitive to reinforcement magnitude, in that response rates were higher

under token-schedules than under comparable extended-chained schedules with a single









reinforcer. Weakening the correlation between number of tokens and temporal proximity

to reinforcement attenuated the discriminative functions of the tokens. In Experiment 4

responding under a token-reinforcement schedule was compared to that under a

procedure that arranged for response-independent tokens and exchange periods, yoked to

their occurrence in the previous token component. Response rates were reduced but not

eliminated under yoked response-independent token delivery. Only when tokens were

removed entirely was responding eliminated. On the whole, the results from all

experiments suggest that the tokens may serve a variety of stimulus functions-

conditioned reinforcing, discriminative, and eliciting-depending on the contingencies.

Further, the data suggest several points of contact between token-reinforcement,

extended-chain, and other second-order schedules.














CHAPTER 1
GENERAL INTRODUCTION

In a token reinforcement procedure, a token (e.g., a coin, a gold star, a check on a

list) is provided contingent on a particular response. Tokens are then later exchanged for

other reinforcers (e.g., food, access to preferred activities). For example, a child may

earn a gold star for every 5 math problems completed, and at the end of the day can

exchange the stars at a store for candy or toys; a rat may earn a marble for every 20

responses, and when 10 marbles have been produced can exchange the marbles for food.

The tokens have been conceptualized as conditioned (acquired) reinforcers, thought to

gain reinforcing value due to their correlation with primary reinforcers. They may also

serve important antecedent discriminativee and eliciting) functions, signaling temporal

proximity to primary reinforcers. Identifying the conditions under which tokens serve

signaling and/or conditioned reinforcing functions is important in a complete account of

responding under token-reinforcement schedules. Additionally, understanding token-

reinforced behavior is important in that these procedures are often utilized to promote and

maintain prosocial behavior in a variety of academic and clinical settings (Kazdin and

Bootzin, 1972). Thus, understanding the determinants of behavior in token systems is of

theoretical as well as practical importance.

In the first laboratory investigation of token reinforcement, Wolfe (1936) found

that chimpanzees would work for tokens exchangeable for food. He initially established

discrimination between tokens with and without value by arranging exchange

opportunities for one type token (white poker chip) but not for another (brass poker chip).









The chimpanzees were then exposed to a schedule in which a token or food was produced

by a response (lifting a weight). Two variants of this schedule were used, one in which

the weight lifted was constant and the other where the amount of weight increased

following each response. It was found that for both the constant weight and increasing

weight conditions, contingent token delivery maintained behavior in much the same way

as contingent food delivery, suggesting that the tokens were serving as conditioned

reinforcers, acquired through a history of relations with other stimuli.

Cowles (1937) extended the work of Wolfe (1936) by showing that tokens could

maintain behavior under conditions with delayed primary reinforcement in which groups

of tokens were required to produce exchanges for food. Initially, a single token could be

exchanged for food reinforcement. The number of tokens required before an exchange

opportunity became available was gradually increased, until long pauses in responding

occurred. Responding was consistently maintained under such conditions, with 10-30

tokens per exchange, providing the first demonstration of token reinforcement under

intermittent reinforcement schedules. The results showed that tokens could maintain

behavior under conditions in which primary reinforcement was temporally distant.

Following two decades of inactivity, token reinforcement procedures were revived

by Kelleher's work in the 1950s (Kelleher, 1956, 1957, 1958). In one study along these

lines, Kelleher (1958) exposed chimpanzees to a schedule in which tokens (poker chips)

were delivered upon the completion of a fixed-ratio (FR) schedule (FR schedules require

a fixed number of responses for completion). Following the delivery of a fixed number

of poker chips, the animals were given the opportunity to exchange the chips for primary

reinforcement (food). Through the course of the experiment manipulations were made of









the number of responses required to produce a token (token-production schedule) and the

number of tokens required before an exchange opportunity (exchange-production

schedule). The exchange schedule was held constant at an FR 60 except under the

highest token-production schedule value, in which it was reduced to an FR 50. As the

token-production schedule was increased from an FR 30 to an FR 125, response rates

decreased and pausing increased, during the initial portions of a cycle (when no or only a

few tokens had been earned.) Interestingly, when the chimpanzees were given 50 poker

chips at the start of a session long pre-ratio pauses ceased, suggesting a potential

discriminative function of the tokens. That is, a discriminative function was demonstrated

by showing that altering the number of tokens present produced behavior early in a cycle

that was typically seen later in a cycle.

Later studies maintained this emphasis on conditioned reinforcement and temporal

organization of behavior. Malagodi (1967) examined rats' responding on FR 20 token

production schedules, with marbles as token reinforcers. During exchanges, tokens could

be deposited in a receptacle with each deposit producing a food pellet. Fixed numbers of

tokens were required to produce exchange periods, with this number varying from 1 to 8

across groups of sessions. Similar to the token-production effects reported by Kelleher

(1958), as the exchange requirement was increased, response rates decreased and pausing

increased.

Waddell, Leander, Webbe, and Malagodi (1972) examined rats' responding on a

schedule in which tokens (marbles) were produced according to an FR 20 and exchanged

according to a fixed-interval (FI) schedule that varied across conditions. (Fixed-interval

schedules arrange for a consequence to occur following the first response after a set time









interval has expired). Response patterning within the FR 20 token-production schedules

were similar to that seen under simple FR schedules, a pre-ratio pause followed by rapid

rates of responding. Response rates across token-production segments within the

exchange cycle were similar to those seen under simple FI schedules: the rate of FR

token-production sequences increased in proximity to food. Also similar to simple FI

schedules, overall response rates decreased as a function of FI duration.

In a similar vein, Webbe and Malagodi (1978) examined rats' responding on FR

20 token reinforcement schedules with FR or variable ratio (VR) schedule of exchange

production. (Variable ratio schedules deliver a consequence after a number of responses

have been emitted, with the number of responses required varying around some preset

average). Across a series of conditions, the exchange schedule alternated between a VR 6

and FR 6. They found that response rates were higher, and pre-ratio pauses were lower,

under the VR exchange schedule when compared to the FR exchange schedule. These

results are in accord with behavior maintained under simple FR and VR schedules.

However, only one VR and FR exchange value and only one token production value were

used.

Foster, Hackenberg, and Vaidya (2001) extended the work of Webbe and Malagodi

(1978) by examining the influence of exchange-production schedule type and value on

token-reinforced behavior. They also extended prior work by using a different species

(pigeons) and different token reinforcers (lights mounted in a horizontal row above the

response keys in an otherwise standard conditioning chamber). (Such non-manipulable

tokens have a number of advantages with respect to examining stimulus functions of

tokens, as described in more detail below.) In their experiment, the value of the FR and









VR exchange schedule was varied across conditions from 1 to 8 while the token-

production schedule was held constant at FR 50. Response rates decreased and pausing

increased, as a function of ratio size under both schedules. Consistent with the results of

Webbe and Malagodi (1978), response rates under the VR exchange were less affected by

changes in ratio value than those under the FR schedule. This effect, combined with

results reported in the studies discussed above, provide strong evidence that in token-

reinforcement schedules the schedules in place for token production and exchange each

influence behavior in a manner similar to that of analogous simple schedules in isolation.

Bullock and Hackenberg (2006) examined the role of both the token-production

and exchange-production schedules in a token reinforcement procedure with pigeons.

Prior research has typically assessed the effect of manipulating either the token

production or exchange schedule in the context of a fixed value of the other. Bullock and

Hackenberg (2006), however, examined the token production and exchange schedules

with both varied across a range of values. Comparable to FR performance under simple

schedules, response rates decreased as the token-production FR was increased within a

given exchange value. Further, decreases in response rates under the larger FR token-

production schedules were even more pronounced under higher exchange schedules.

Similarly, within a given token-production value, increases in the exchange schedule

produced decreases in response rates, particularly under higher token-production values.

Under the higher exchange-schedule values, responding within an exchange cycle was

graded, with low responses rates early in the cycle increasing as more tokens were

earned. These results indicate that the effects of token production and exchange-schedule

manipulations vary depending on the value of the other schedule. Under lower token-









production schedules, increases in exchange-schedule value had less of an effect than

under higher token-production schedules. However, because the study was not designed

to assess stimulus functions of the tokens directly, more precise statements about the

function of the tokens were limited.

Purpose of the Present Research

Although the studies discussed above demonstrated several important determinants

of responding under token-reinforcement procedures, they failed to isolate the stimulus

functions of the tokens. There are at least three potential functions of stimuli in token

reinforcement schedules: discriminative, reinforcing, and eliciting. Previously neutral

stimuli can gain a discriminative function via their temporal correlation with primary

reinforcement. In other cases stimuli can serve as conditioned reinforcers due to their

correlation with a reduction in delay to or increase in magnitude of primary

reinforcement (Gollub, 1970). A third function is suggested by research on serial

autoshaping in which stimuli correlated with the presentation of food delivered under

some response-independent time-based schedule can elicit responding (Ricci, 1973).

The present research investigates the potential controlling variables discussed

above. The basic methodological strategy was to compare and contrast token schedules

with other reinforcement schedules which have proven useful in revealing stimulus

function. For example, in order to examine if tokens have an effect on behavior one

could compare behavior maintained by a token-reinforcement schedule to a similar

schedule in which they were absent. In the present research, stimulus functions were

examined by using schedules which were similar to token-reinforcement schedules but

had particular features useful for comparison purposes. The following section provides a

brief overview of these schedules and their relation to the present issues.









Schedules as Tools for Investigating Stimulus Function

Token schedules have been conceptualized as second-order schedules of

reinforcement, or "schedules of schedules" with the behavior that produces tokens

considered a unitary response that is itself reinforced according to some other schedule

(Kelleher, 1966). For example, under a second order FR 4 (FR 10) schedule, the first-

order schedule required 10 responses to produce a token and the second-order schedule

required 4 tokens before an opportunity to exchange these tokens for food was presented;

the FR 10 token production schedule can be conceptualized as a unitary response that is

itself reinforced according to an FR 4, hereafter termed an FR 4 (FR 10) token-

reinforcement schedule. Several types of second-order schedules have stimulus

arrangements potentially useful for revealing the function of tokens.

The conventional second-order schedule involves brief-stimulus presentation (e.g.,

a flash of light, a tone) contingent on the completion of a first-order schedule requirement

and primary reinforcement contingent on completion of the second-order schedule

requirement (Gollub, 1970). A token reinforcement schedule differs from this more

conventional arrangement in that completion of the first-order schedule requirement

produces a stimulus that remains present throughout the duration of the second-order

schedule. In addition, the number of stimulus presentations in a token-reinforcement

schedule is directly correlated with the magnitude of reinforcement available upon

completion of the second-order schedule.

Second-order schedules are part of a larger family of sequence schedules that also

includes chained schedules. Chained schedules are schedules that arrange for the

presentation of a stimulus following the completion of some simple schedule (link), with

primary reinforcement delivered after the completion of a number of links. Unlike token-









reinforcement schedules, in chained schedules each successive link may have a different

response requirement and a distinct stimulus is present during each link. Further, similar

to brief-stimulus schedules, in chained schedules there is no correlation between number

of simple schedules completed and magnitude of reinforcement. However, when chained

schedules have the same simple schedule in each link, the simple schedule becomes

analogous to the first-order schedule in token-reinforcement and brief-stimulus schedules.

Comparing and contrasting behavior maintained under token reinforcement schedules to

that maintained under brief-stimulus and chained schedules allows for an examination of

the effects of stimulus duration, stimulus accumulation, and a correlation with number of

stimulus presentations and primary reinforcer magnitude.

According to Kelleher and Gollub (1962), the effects of stimuli in chained or

second-order schedules on responding are difficult to assess. First, the subject may not

be attending to the stimuli presented in the second-order schedule (in this case tokens), in

which case behavior would be a function of the contingency between responding and

reinforcement. Further, responding in the presence of a segment of a token-reinforcement

procedure could be due to the temporal proximity of that segment to primary

reinforcement, the conditioned reinforcing value of the token that responding has

produced in the past, or both. Finally, response rates could be due to some property of

the stimuli themselves, apart from that gained via operant contingencies. Given these

possibilities, statements about the functions of tokens in token reinforcement procedures

require appropriate control procedures.

One method for assessing whether the stimuli in second-order schedules are having

an effect is to employ an equivalent tandem schedule. A tandem schedule, when used as









a control condition, typically arranges for the segment response requirements and

schedules to be identical to those of the second-order schedule of interest with the

exception that there are no discriminative stimuli denoting the completion of a given

segment (Kelleher & Gollub, 1962). For example, Kelleher (1966) examined pigeons'

responding under a second-order schedule in which 30 consecutive FI 2-min schedules

were required for food reinforcement. In this procedure, following the completion of

each FI 2-min schedule, a white key light was briefly illuminated. In order to assess the

effects of the key-light flashes, a tandem procedure was employed in a separate condition

in which the schedule requirement and the total reinforcement were identical, but no

stimuli were presented following the completion of the FI schedules. If the stimuli have

some effect on behavior one would expect differences in performance between the

second-order and tandem procedures. In Experiment 1 of the present investigation, we

compared performance from a token-reinforcement schedule to that of a tandem schedule

equivalent in all respects except for the tokens. Any difference in responding between

the token and tandem schedules would demonstrate that the presence of the tokens have

an effect on behavior.

As mentioned earlier, the stimuli in brief-stimulus schedules differ from those in

token-reinforcement schedules with respect to the duration of presentation, accumulation,

and correlation between stimulus number and position with responses and time before

primary reinforcement. A modified token-reinforcement schedule, modeled after a brief-

stimulus schedule, can thus serve as a basis of comparison to assess the importance of the

procedural differences between token-reinforcement and brief-stimulus schedules in

determining stimulus function. In one condition the briefly-presented token schedule was









identical to that of the token schedule with the exception of presentation duration. In a

second condition the tokens did not accumulate, rather a single token was presented with

its position changing. In a third condition, performance on a token-reinforcement

schedule was compared to that under one that simply flashed all four tokens. Differences

in responding between the 3 briefly-presented token-schedule variants would indicate the

importance of manner of stimulus presentation while differences between the standard

and briefly-presented token schedules would indicate the importance of stimulus

duration.

Another method of assessing the functions of stimuli in second-order schedules

involves altering the order of presentation (Kelleher & Gollub, 1962). Varying the order

by which stimuli are presented in a token-reinforcement schedule may provide

information concerning the importance of a correlation between number of tokens and

temporal proximity to food as a response determinant. For instance an FR 4 (FR 50)

schedule of token reinforcement was used as a standard for comparison in Experiments 2,

3, and 4 of the present study. Under this schedule every 50 responses would illuminate a

token, with the illumination of the fourth token preceding reinforcement. In Experiment

3 of the present investigation, two of the conditions involved an alteration in the manner

of token presentation, the results of which were compared to the standard token schedule.

In one condition the stimulus order was reversed such that a cycle began with four tokens

illuminated that darkened in reverse order as each segment was completed. If the

absence of tokens had an effect on responding in link one, then the reversed order

condition should produce a change in behavior early in the cycle. Another condition in

Experiment three involved tokens illuminating according to a VR 50 schedule but with









the token exchange still occurring after 200 responses and still delivering four food

reinforcers. The VR contingency was designed to weaken the correlation between

number of tokens and proximity to reinforcement. If the contingency was an important

determinant of the effects of the tokens, then the VR token production condition should

produce responding more like that under the tandem conditions.

Altering the contingencies by which stimuli are presented in second-order

schedules allows assessment of the degree to which these stimuli, in addition to their

response-dependent presentation, might affect behavior. If the tokens are presented

response independently, yoked to their temporal occurrence under the regular procedure,

the role of any possible respondent (eliciting) functions of the tokens could be assessed.

For instance, if while under an FR 4 (FR 25) schedule of token reinforcement, a pigeon

earned the first token after 60s, the second after another 45s, the third after another 30s,

and the last after another 15s, then under the yoked procedure, the tokens would be

presented at these temporal intervals, irrespective of responding. The fourth experiment

of this study uses just such an arrangement, with the first and third components of a

session comprised of the standard token-reinforcement schedule, while the second and

fourth components had a schedule in place in which the tokens and exchange periods

were presented independently, yoked to their temporal occurrence in the previous

component. In this case it may be that responses were elicited due to the temporal

relationship between token presentation and food (Ricci, 1973).

Lastly, a method generally employed in the present series of experiment to assess

stimulus function in second-order schedules involved the use of a multiple schedule

(Kelleher and Gollub, 1962). A multiple schedule involves some manner of alternation









between two schedules, each presented independently with a distinct discriminative

stimulus. Throughout the present study a two-component multiple schedule was

employed, allowing for the comparison of a token-reinforcement procedure with some

other schedule within each condition.

Taken together the experiments reported here were designed to examine under what

conditions functions of tokens in token-reinforcement schedules can have conditioned-

reinforcing, discriminative, and eliciting functions. In particular, findings from

experiments involving added-stimulus schedules, extended-chain schedules, token-

reinforcement schedules, and serial autoshaping procedures suggest that the functions of

these stimuli may vary depending on how they are related to the schedule of primary

reinforcement. The present studies were designed not only to examine stimulus function

in token-reinforcement schedules, but to also allow for points of contact between and

give a broader account of the functions of stimuli in second-order and extended-chained

schedules. The results of the present experiments thus lend themselves both to a better

understanding of token-reinforcement schedules and to a broader conceptualization of

how determinants of behavior under token-reinforcement schedules relate to those of

other forms of second-order schedules.














CHAPTER 2
EXPERIMENT 1

Previous research on schedules of token reinforcement has shown that response

rates vary inversely with the value of FR token production and exchange schedules

(Foster et al., 2001; Webbe and Malagodi, 1978). Bullock and Hackenberg (2006)

showed that the relationship between response rates and FR exchange-schedule value

varies depending on the value of the FR token-production schedule. Foster et al., (2001)

found schedule-typical patterns under token production and exchange schedules,

suggestive that the tokens had some function. However, tandem-control conditions would

allow for a more precise characterization of that function (or functions).

An experiment investigating extended-chain schedules by Jwaideh (1973) serves as

an example of utilizing tandem-control conditions to examine stimulus functions and

serves as a potential point of contact between token reinforcement and extended-chain

procedures. Pigeons were exposed to a series of chained schedules, each with an

accompanying equivalent tandem schedules (same response requirement as a chained

schedule but with no stimuli delineating transitions between schedule components, or

links). The number of links in the chain was varied from 1 to 5 with FR schedules in

each link. The total response requirement to complete all links was varied from 12 to 240.

Two additional conditions were conducted, one in which the order of the chain sequence

was reversed and the other in which the terminal stimulus in the chain was also used in

the initial link. Performance under the chained schedule was compared to that of the

equivalent tandem schedule to assess any potential functions of the stimuli. That is,









response requirements were identical in the two schedules; the only difference was

whether each link was (chained) or was not (tandem) correlated with a distinct stimulus.

Overall response rates decreased and pre-ratio pausing increased, both as a direct

function of the number of components and the number of responses to reinforcement.

Further, response rates under tandem conditions were generally higher than equivalent

chained conditions. Reversing the order of the stimuli in the chain resulted in initial

increases in response rates that soon returned to those seen previously under the normal

chained schedule. When the same stimulus was used for the first and last link of the

chain, however, pre-ratio pausing decreased and remained shorter than under the regular

chain procedure. The author suggested that differences in performance under the tandem

and chained schedules demonstrated a function of the stimuli. It was suggested that

stimuli early in the chain came to produce low rates of responding due to their correlation

with long delays to reinforcement, while stimuli in the later links of the chain produced

higher rates of responding due to their correlation with short delays to food.

The experiment by Jwaideh (1973) had the same FR schedule for each link in the

chain and a fixed number of links in the chain per condition. Because the requirement for

each link was constant, each link could be conceptualized as analogous to a token-

production segment. Further, the fixed number of links required to produce

reinforcement is analogous to an FR exchange schedule. The main differences between

the procedure used by Jwaideh (1973) and a token schedule are that in the latter

procedure (a) the number of tokens earned is correlated with the magnitude of

reinforcement (number of food deliveries during an exchange period), and (b) the tokens

accumulate in continuous fashion rather than having a distinct stimulus accompany each

link of a chain schedule.









The accumulation of tokens in token reinforcement procedure resembles stimulus

presentation of another form of second-order schedule, an added-stimulus schedule

(Zimmerman & Ferster, 1963, 1964), which provides a further point of contact with

token-reinforcement schedules. Zimmerman and Ferster (1964) examined responding

under an added-stimulus schedule where pigeons' responding on the left of two keys

resulted in a houselight flash and a voltmeter (a gauge that could be displaced from zero

to maximum of an 80 degree arc) incrementing towards a terminal position, with stimulus

changes reinforced according to a variable-interval (VI) schedule (VI schedules arrange

for a consequence to occur contingent on the first response following some period of time

that varies around a preset average). Once the voltmeter had been fully displaced, the

right key became operative, and each subsequent peck produced food (one per voltmeter

increment). The VI schedule by which the voltmeter incremented (VI 1 min and VI 3

min), the number of increments required to reach the maximum (FR 10 and FR 20), and

the presence/absence of the voltmeter and houselight stimuli were varied systematically

across conditions. Response rates were initially low and accelerated with the number of

voltmeter increments or temporal proximity to primary reinforcement. Increasing the

number of steps for the voltmeter from FR 10 to FR 20 decreased response rates across

both the VI 1 min and VI 3 min increment schedules, an effect similar to that of

increasing the exchange schedule in token-reinforcement schedules (Bullock and

Hackenberg, 2006; Foster et al, 2001). The removal of the voltmeter advance and

houselight flash following the completion of each VI (tandem schedule) resulted in a

more constant rate of responding across the cycle. These results suggest that the added

stimuli served a discriminative function similar to the link stimuli used in extended

chained schedules (Jwaideh, 1973).









The present experiment used a token-reinforcement procedure similar to that of

Bullock and Hackenberg (2006) and Foster et al. (2001). In keeping with the suggestions

of Kelleher and Gollub (1962) that a multiple schedule can serve as part of a control

procedure to investigate the functions of stimuli in second-order schedules, the present

procedure utilized a two-component multiple schedule. One component of the multiple

schedule was comprised of a token-reinforcement schedule and the other component a

tandem schedule with otherwise identical contingencies. The token-production schedule

remained constant at FR 50, a value at which graded patterns of responding across

successive token-production segments have been seen in prior research. This graded

pattern is an important indicator of discriminative functions of added stimuli such as

tokens. The exchange-production FR was varied across conditions in a manner consistent

with prior research in our laboratory (Bullock & Hackenberg, 2006; Foster et al., 2001).

The present experiment is thus a systematic replication (Sidman, 1960) of our prior work,

but with tandem-control conditions to assess the stimulus functions of the tokens.

Half the pigeons in this experiment were exposed to small tokens (light emitting

diodes, or LEDs) while the other half were exposed to larger tokens (jeweled stimulus

lights, 1.5 cm in diameter). Prior published work on token-reinforcement schedules in our

laboratory used the small-token preparation (Foster et al., 2001; Foster & Hackenberg,

2004; Hackenberg & Vaidya, 2003; Jackson & Hackenberg, 1996), but we had

suggestive evidence of more pronounced and systematic effects from the larger tokens.

A comparison of the effect of token size on performance under token-reinforcement

schedules allowed for an assessment of whether the physical properties of the tokens

themselves would produce differential discriminative effects (see Gollub, 1970, for a









discussion of the effects of stimulus properties in second-order schedules of briefly-

presented stimuli).

In sum, the purpose of Experiment 1 was to (a) assess stimulus functions of the

tokens by comparing directly performance under token and equivalent tandem-control

conditions, (b) replicate previous findings of exchange-production FR manipulations, and

(c) evaluate the effects of token size/salience on token-reinforced behavior.

Method

Subjects

Six White Carneau pigeons (Columba livia) (numbered 907, 83, 832, 999, 910, 47)

served as subjects. Pigeon 832 had prior experience with token-reinforcement schedules.

Pigeons were individually housed under a 16.5 hr / 7.5 hr light:dark cycle and had

constant access to water and health grit in home cages. Pigeons were maintained at 80%

+ 20 g of their free-feeding weights with supplemental post-session feeding.

Apparatus

Two standard three-key pigeon chambers with a modified stimulus panel served as

the experimental apparatus. The first chamber (large token chamber) was 35 cm high by

31 cm long by 34.5 cm wide, and had a stimulus panel with three response keys centered

horizontally 10 cm from the ceiling to the key center and 8 cm from the adjacent key(s)

(center to center). Further, a row of 12 evenly spaced stimulus lights with red caps,

approximately 1.5 cm in diameter, was centered 7.5 cm above the response keys (center

to center) and protruded 1.3 cm into the enclosure. The stimulus lights were always

illuminated left to right, in sequential order, and served as tokens in this arrangement.

Food was delivered through an opening centered 10.5 cm under the center key









(approximately 5.5 cm wide and 5 cm tall). This box was also equipped with a Sonalert

that provided an auditory stimulus (0.1 s tone) that accompanied token onset and offset.

The second chamber (small token chamber) was 36 cm high by 50 cm long by 36

cm wide and the intelligence panel had 3 response keys centered vertically 11.5 cm from

the ceiling to the key center and 9 cm from each other (center to center). For this

chamber, a stimulus array of 34 red, evenly spaced, light-emitting diodes (LEDs), 0.4 cm

in diameter, were centered 5 cm above the keys and 1.25 cm apart from each other

(center to center) and protruded 0.3 cm into the enclosure. The LEDs were always

illuminated left to right, in sequential order. An electromechanical stepping switch

(Lehigh Valley Electronics, Model 1427) located on top of the chamber controlled LED

illumination, the operation of which also provided auditory feedback each time a token

was presented or removed. A food hopper opening was centered 11.5 cm below the left

key (approximately 5.5 cm wide and 5 cm tall).

Both chambers had a houselight centered above the token array that provided

diffuse illumination. When operative, side keys were illuminated green or yellow, and

the center key red. Pecks with a force between approximately 0.11-0.14 N (small token

box) and 0.13 N (large token box) were counted. A solenoid-operated hopper could be

raised into the food opening, allowing access to mixed grain. A white light inside the

hopper illuminated during the food presentation. A photo-beam recorded head entry into

the hopper. Continuous white noise and ventilation fans were active during experimental

sessions to mask extraneous sounds. In a separate room a computer equipped with Med-

PC software controlled experimental events and collected data.









Procedure

Preliminary Training. All pigeons were exposed to a series of training conditions

(data not shown) prior to Experiment 1. Naive pigeons were initially adapted to the

experimental chamber with the houselight illuminated and trained to eat food from the

grain hopper. For birds with no history of key pecking, pecks to the center key were

shaped via reinforcing successive approximations. All birds were then exposed to an FR

1 schedule in which pecks on the red illuminated center key produced food access. This

was followed by sessions in which the left side key illuminated; a peck on this key would

darken the side key and illuminate the center key, a peck on the center key would darken

the center key and produce food reinforcement. This training arrangement lasted until

birds were reliably pecking the side and center keys. This was follow by exposure to a

multiple schedule with an FR 100 in effect during both components. Each bird was then

exposed to several days of token-food pairings. These sessions consisted of the

alternating illumination of the left side key within a session (randomly yellow or green).

After the side key was pecked (FR 1) a token was illuminated and a tone was sounded,

after which the side key darkened and the center key illuminated. A peck on the center

key resulted in the darkening of one token and 1.5 s of food (timed from head into

hopper). These sessions lasted for 64 reinforcers and were in effect for 3-4 days.

Standard Procedure. Each session consisted of a 2-component multiple-schedule

with two exposures to each component. Components occurred in a pseudo-random order,

with a component remaining in effect for 16, 1.5-s food deliveries. Components were

separated by 30 s blackouts, or intercomponent intervals. Sessions began with the

illumination of the white houselight and left key (either green or yellow depending on

component type with colors counterbalance across birds). Conditions lasted for at least 14









sessions and until response rates were deemed stable via visual inspection of overall

responses per minute for each component. Data were generally considered stable when

no monotonically increasing or decreasing trends and the highest or lowest points were

not present in the last 5 sessions of a condition.

During the token components, tokens were earned according to an FR 50 token-

production schedule (i.e., 50 responses produced one token) and exchanged according to

an FR exchange-production schedule that varied systematically from FR 2 to 8 across

conditions. Tokens were illuminated left to right. Completing the exchange ratio

requirement produced an exchange period, during which the left key darkened and the

center key illuminated red. A single response on this key darkened the rightmost token

and raised the food hopper for 1.5-s. This exchange period remained in effect until all

tokens earned that cycle had been exchanged for food. The period was followed by an

immediate return to the token-production cycle (left key illuminated) or the inter-

component blackout. After the ratio was completed and the exchange period initiated for

the tandem component, a number of tokens equal to that in the token component was

illuminated. The response requirement and token-exchange stimulus conditions were

otherwise identical for the tandem and token components with the exception that no

stimulus change occurred within the ratio under the tandem schedule (i.e., a fixed-ratio

schedule). Table 2-1 lists the order of conditions and number of sessions per condition.

Key colors were reversed under replications.









Table 2-1. Order of conditions and number of sessions per condition for Experiment 1.
Listed are the token schedule, but each condition also included a tandem
schedule. The superscript a denotes a color reversal, the superscript b denotes
conditions conducted in large token box, while an denotes that a condition
was not completed. Number of sessions per condition is listed in parentheses.
Pigeon
Small Tokens Large Tokens
832 910 999 47 83 907
FR 2[50] (17) FR 2[50](22) FR 2[50](19) FR 2[50] (34) FR 2[50] (35) FR 2[50] (42)
FR 4[50] (22) FR 4[50](70) FR 4[50](56) FR 4[50] (31) FR 4[50] (14) FR 4[50] (21)
FR 8[50] (37) FR 8[50](16)* FR 8[50](61) FR 8[50] (33) FR 8[50] (30) FR 8[50] (26)

FR 4[50]a (28) FR 4[50] a(39) FR 4[50] a(15) FR 4[50] a(19) FR 4[50] (46) FR 4[50] a(49)
FR 4[51 1]']- (53) FR4[50] ab (34)

Results

Figures 2-1 and 2-2 show for each pigeon the means and standard deviations of

the running response rates (response rates factoring out pre-ratio pausing) and pre-ratio

pausing, respectively, as a function of exchange-production ratio across the final 5

sessions of each condition. Graphs in the left and right columns show data for pigeons

typically exposed to the smaller and larger tokens, respectively. Filled points represent

data from token components, open points data from tandem components, with

unconnected points denoting replications. The final conditions for several birds were

replications across chamber type. Thus, for 1 condition 907 was run in the small token

chamber while for 1 condition 999 and 832 were run in the large token chamber. Data

from these conditions are denoted by squares.

Response rates varied inversely, and pre-ratio pausing varied directly, with the

value of the exchange-schedule ratio for both token and tandem components. Further, for

4 out of 6 pigeons response rates in the tandem components were generally higher and

pre-ratio pausing lower than rates in the token components (the exceptions being 999 and








910). Differences in performance under the tandem and token components were greater
for birds exposed to the larger tokens.


SMALL TOKENS


FR 2 [50] FR 4 [50]


FR 8 [50]


FR 2 [50] FR 4 [50]


CONDITION
Figure 2-1. Mean responses per minute (not including pre-ratio pause) and standard
deviations plotted as a function of exchange ratio from the last 5 sessions of a
condition. Left panels show the data from subjects primarily run in the small
token box while right panels show data for subjects primarily run in the large
token box. Open symbols represent data from tandem components, closed
from token components; disconnected symbols represent replications, and
while squares represent data from replications across different chambers and
token sizes.


832


BIG TOKENS


47



^-T-


910



\1


83


999


907 -- Token
-o- Tandem

n^


FR 8 [50]










SMALL TOKENS


FR 2 [50] FR 4 [50]


FR 8 [50]


FR 2 [50] FR 4 [50]


CONDITION


Figure 2-2. Mean pre-ratio pausing and standard deviations plotted as a function of
exchange ratio from the last 5 sessions of a condition. Note that the y-axis is
logarithmic. Left panels show the data from subjects primarily run in the
small token box while right panels show data for subjects primarily run in the
large token box. Open symbols represent data from tandem components,
closed from token components, disconnected symbols represent replications,
while squares represent data from replications across token type.


-*- Token
-o- Tandem


FR 8 [50]


BIG TOKENS









Figures 2-3 and 2-4 show response rates as a function of segment (the ordinal

position within the exchange-production cycle) for pigeons exposed to the small and

large tokens, respectively. Figures 2-5 and 2-6, organized similar to Figures 2-3 and 2-4,

show pre-ratio pausing as a function of token segment for all pigeons (amount of time in

seconds between token illumination and a response). For Figures 2-3 through 2-6, filled

and open circles represent performance under token and tandem components. The large

dashed lines represent replications within a token type while the small dashed lines are

indicative of replications across chambers (token type). For both tandem and token

components across all exchange production schedules, initial-segment rates generally

were low and increased as a function of number of tokens earned (Figures 2-3 and 2-4).

Pre-ratio pausing was largest for the initial segment and, with a few exceptions,

decreased to a small value in the later segments (Figures 2-5 and 2-6). Responding in the

tandem components was characterized by low initial-segment rates that gave way to high,

constant rates of responding. Responding in the token component was characterized by

low initial-segment rates gradually increasing as a function of the number of tokens

earned. In summary, these figures show that differences in performance under token and

tandem components were comprised of both lower running response rates and longer

pausing in the early segments of a token cycle.















400
350 832



03-------------- ------------r -------------------
300
250
S200
150
100






L oo ------------------'*-----------------------------------
50


I--
S400
Z 350 910
300









00
LU 250
0 200
2LU 1503 4 1
Z 100
0
,) 0
LU
nl
300
999
250
200
150_ _- = _

100
5-- TOKEN

0
1 2 1 2 3 4 1 2 3 4 5 6 7 8
SEGMENT


Figure 2-3. Mean within ratio responses per minute (not including pre-ratio pause)

plotted as a function of token production segment, for subjects run primarily
in the small token box, from the last 5 sessions of a condition. Points from

tandem components represent successive 50 response segments. Open
symbols represent data from tandem components, closed from token

components, solid lines represent original exposures, large-dashed lines
represent replications, and small-dashed lines represent replication across
token size.















300
47
250
200
150
100
50
0


S400
S350 83
S300
LU 250
a
200
LU 150
U)
Z 100
D. 50
0
U) 0
LU

300
907
250
200
150 ~~~--D------

100 _
50 TOKEN
S-0-TANDEM
0
1 2 1 2 3 4 1 2 3 4 5 6 7 8

SEGMENT


Figure 2-4. Mean within ratio responses per minute (not including pre-ratio pause)

plotted as a function of token production segment, for subjects run primarily
in the large token box, from the last 5 sessions of a condition. Points from

tandem components represent successive 50 response segments. Open
symbols represent data from tandem components, closed from token
components, solid lines represent original exposures, dashed lines represent
replications, and squares represent replication across token size.
















100

10



1
01

01
100
U)
3
< 10
13-
0
I 1

oQ 01
IL

100


10


832


999


0 1 I I I
1 2 1 2 3 4 1 2 3 4 5 6 7 8

SEGMENT


Figure 2-5. Mean pre-ratio pausing plotted as a function of token production segment,

for subjects run primarily in the small token box, from the last 5 sessions of a

condition. Points from tandem components represent successive 50 response

segments. Open symbols represent data from tandem components, closed
from token components, solid lines represent original exposures, large-dashed

lines represent replications, and small-dashed lines represent replication
across token size.
















100

10

1

01

LU
u,
w 100
LU
u,
V)
D 10

0 1
Q 1

LUI
cr 001
0-


47











83


1 2 1 2 3 4 1 2 3 4 5 6 7 8

SEGMENT


Figure 2-6. Mean pre-ratio pausing plotted as a function of token production segment,
for subjects run primarily in the large token box, from the last 5 sessions of a
condition. Points from tandem components represent successive 50 response
segments. Open symbols represent data from tandem components, closed
from token components, solid lines represent original exposures, large-dashed
lines represent replications, and small-dashed lines represent replication
across token size.


907









Discussion

This experiment investigated behavior maintained under token-reinforcement

schedules and equivalent tandem controls. The experimental design allowed for

assessment of the effects of tokens under several different exchange-schedule values by

parametrically manipulating the exchange schedule across conditions. The primary

findings from this experiment, as shown in Figures 2-1 and 2-2, were (a) response rates

varied inversely, and pre-ratio pausing directly, with the token production schedule FR

value, replicating previous finding concerning exchange schedule manipulations (Bullock

and Hackenberg, 2006; Foster et al., 2001; Webbe and Malagodi, 1978), (b) the presence

of tokens reduced response rates, and increased pre-ratio pausing, when compared to their

absence, and (c) the presence of the tokens engendered a more graded pattern of

responding than when they were absent (see Figures 2-3 and 2-4), similar to that seen

under extended-chained schedules (Jwaidah, 1973). Lastly, for pigeons exposed to the

larger tokens the size of the differences in response rates between the token and tandem

components was generally greater, and more consistent, than for pigeons exposed to the

smaller tokens.

Response rates for pigeons exposed to the larger tokens were higher in most cases

under the tandem components than in the accompanying token components under the

larger tokens. This result indicates a possible discriminative effect, similar to that

reported by Jwaidah (1973) with extended-chained schedules. Figure 2-2 shows that

lower overall response rates in the token schedules were primarily a function of long pre-

ratio pauses and low response rates in the early links. This result is remarkably similar to

that of Jwaidah (1973) and fits within her interpretation that the stimuli in the initial links









of the chain (tokens, in the present experiment) served as discriminative stimuli

associated with longer delays to primary reinforcement.

The interpretation that the tokens primary function was discriminative is further

supported by examination of response patterns across segments within an exchange cycle

(Figures 2-3 and 2-4). Under simple FR schedules (tandem components in the present

experiment) response patterning generally consisted of a pre-ratio pause followed by

high, relatively constant rates of responding. However, as shown in Figures 2-3 and 2-4,

under the FR 4 and FR 8 exchange schedules (middle and right columns), response

patterning within an exchange cycle under the token-reinforcement component was

graded with low rates in early segments increasing as more tokens were earned. The

response patterning found in the present experiment is once again consistent with the

results of Jwaidah (1973) and supports the interpretation of the tokens in the early

segments serving a discriminative role. Given the discriminative functions of the tokens

it may not be surprising, then, that larger tokens produced larger effects given their

greater salience.














CHAPTER 3
EXPERIMENT 2

Another type of complex sequence schedule, similar to token reinforcement

schedules in some respects, involves the presentation of stimuli that are presented briefly

after the completion of some simple schedule: second-order schedules of brief stimulus

presentation. As with token-reinforcement procedures, the completion of each schedule

segment, or unit schedule, produces a discriminable stimulus change (e.g., a flash of

light, a tone) and contributes to a higher-order schedule by which primary reinforcement

is presented. Some research has shown that brief stimuli can serve a discriminative role,

organizing behavior with respect to the temporal proximity of primary reinforcement

(Kelleher, 1966). Other research has shown that the effects of brief-stimulus presentation

may vary depending on the value of the schedule of brief-stimulus presentation and the

primary reinforcement schedule.

Kelleher (1966) investigated whether a brief stimulus presented after completion of

an FI would be sufficient to maintain pigeons' responding under an extended second-

order schedule. Pigeons were exposed to an FR 30 (FI 2-min) or an FR 15 (FI 4-min)

schedule with a brief stimulus presentation (white key light flash) occurring after

completion of each FI. Kelleher notates second-order schedules by listing the schedule of

stimulus presentations required for completion first and the schedule of stimulus

presentation in parentheses. Thus, FR 30 (FI 2-min) denotes a schedule in which a

stimulus is presented after completion of every FI 2-min and that requires 30 FI 2-min

completions before primary reinforcement is presented. Performance under each









condition in this experiment was compared to performance under a tandem schedule with

similar contingencies but the absence of brief-stimulus presentations. Responding in both

brief-stimulus conditions was characterized by low rates of responding early in a cycle

increasing as a function of proximity to primary reinforcement. Further, within a given

FI, response rates had a scalloped pattern with pausing after the presentation of a brief

stimulus and with response rates increasing as a function of temporal proximity to the

next brief stimulus. Responding under the two brief-stimulus conditions was thus

organized with respect to both the FI brief-stimulus schedule and the FR primary-

reinforcement schedule. Responding under the tandem-control conditions was markedly

different than under the brief-stimulus conditions in that response rates were lower and

relatively constant throughout the cycle, with a slight increase as primary reinforcement

approached. Kelleher concluded that the brief-stimulus presentations served as

conditioned reinforcers, facilitating performance when compared to tandem-control

conditions.

Lee and Gollub (1971) exposed pigeons to a procedure that arranged for primary

reinforcement delivery after 256 responses. A briefly-presented stimulus (0.5 s green

light) was presented after a fixed number of responses, varied from 2 to 256 across

several conditions. They found an inverted U-shaped function relating response rates to

the size of the FR brief-stimulus schedule, with the highest response rates generally

occurring under the FR 64 and 128 brief-stimulus presentation conditions. The high rates

of responding under the middle brief-stimulus presentation values may be indicative of a

conditioned reinforcement function. Evidence for this account is provided by the

response patterning with respect to the briefly-presented stimuli, patterning similar to that









seen under simple schedules. Lower response rates seen under the small FR brief-

stimulus schedules were thought to indicate a discriminative function, with low rates at

the beginning of the cycle due to the pairing of early brief-stimulus presentations with

long delays to reinforcement.

The present experiment sought to connect findings from research on second-order

schedules of brief-stimulus presentation with token reinforcement schedules. The

features of token-reinforcement schedules that differ from a brief-stimulus schedule are

(a) the duration of stimulus presentation, (b) the correlation between stimulus number and

primary-reinforcement magnitude, and (c) the correlation between number of stimuli

illuminated and temporal proximity to exchange (Bullock & Hackenberg, 2006; Foster et

al., 2001). For example, in token reinforcement schedules, the number tokens earned is

inversely proportional to the number of responses remaining, and directly proportional to

the number of reinforcers available during the exchange period. To more precisely

evaluate the correlation between the number of tokens and both response requirements

and temporal proximity to exchange, the present experiment arranged for comparisons

between token schedules and several variants of brief-stimulus schedule configurations.

As in Experiment 1, a multiple schedule was used to allow for within-session

comparisons of response rates and patterning under the different schedule arrangements.

Method

Subjects

Four White Carneau pigeons (Columba livia), numbered 47, 83, 832, and 999,

served as subjects. All had previously served in Experiment 1: Pigeons 47 and 83 in the

large-token box and 832 and 999 in the small-token box.









Apparatus

The chamber with the larger tokens was the only one used in the experiment (i.e.,

the standard three-key pigeon chamber with a row of 12 evenly-spaced stimulus lights

from Experiment 1).

Procedure

The procedure was similar to that of Experiment 1 with the exceptions that the

exchange-production schedule was held constant at FR 4, both the left and right keys

were used, and in place of the tandem component, a brief-stimulus schedule was used. As

in Experiment 1, sessions were comprised of a 2-component multiple-schedule with 2

exposures to each component per session. Component remained in effect for 16, 1.5-s

food deliveries, and occurred in a pseudo-random order. Following the completion of a

component, a 30 s blackout (intercomponent interval) occurred. Sessions began with the

illumination of the white houselight and either the left or right key (either green or yellow

depending on component type). Conditions lasted for at least 14 sessions and until

response rates were deemed stable via visual inspection.

A component began with the illumination of the left or right key, with the position

of the key for a component type remaining constant throughout a condition. For

components with responses recorded on the left key, tokens where illuminated left to

right, while for components in which responses were recorded on the right key, tokens

were illuminated right to left. A token schedule was used for one component of the

multiple schedule and one of 3 types of brief stimulus schedules for the other. For the

brief-stimulus arrangement, a token or tokens were illuminated briefly (0.5 s

presentation), accompanied by a tone (0.1 s presentation) following the completion of

each FR 50 on the token-production key. Upon the completion of a brief-stimulus









component cycle four tokens illuminated and the exchange period began. The exchange

periods were otherwise identical across component types. Table 3-1 lists the order of

conditions and number of sessions per condition. Key colors were reversed under

replications.

Table 3-1. Order of conditions and number of sessions per condition for Experiment 2.
Number of sessions per condition is listed in parentheses.

Pigeon

47 83 832 999

Token (30) Token (31) Token (18) Token (43)
BMS (52) BMS (44) BMS (38) BMS (55)
BAS (14) BAS (40) BAS (34) BAS (31)
BFS (34) BFS (31) BFS (70) BFS (22)
BMS (48) BMS (52) BMS (24) BMS (18)



Initially, pigeons were exposed to a condition with token reinforcement schedules

in both components, followed by variations of a brief stimulus schedule in one

component. One of three types of brief-stimulus components alternated with the token

component. In one condition, a brief-moving stimulus (BMS) configuration was used, in

which only 1 token flashed at the end of each FR 50 but changed position (left to right or

right to left) depending on the number of segments completed. Thus position but not

number of tokens was correlated with temporal proximity to exchange. In another

condition, a brief-added stimulus (BAS) configuration was used, in which tokens

illuminated in the same manner as the token component, but only remained illuminated

briefly. Both position of the stimuli and number of stimuli illuminated was correlated

with temporal proximity to food delivery. For a third type of condition, a brief-full









stimulus (BFS) configuration was used in which 4 tokens flashed after every 50

responses. In this configuration, no feature of the stimulus itself, other than the number

of times it was illuminated, was correlated with food delivery.

Results

Figures 3-1 and 3-2 show for each pigeon running response rates (response rates

factoring out pre-ratio pausing) and pre-ratio pauses, respectively, for each condition in

Experiment 2. For each condition the filled bars represent performance under token-

reinforcement schedule components while the open bars show performance under the

brief-stimulus components.

No systematic differences were evident with respect to response rates or pre-ratio

pausing between the three brief-stimulus schedule variants examined. Response rates

under the brief-stimulus components in two cases were higher, and pre-ratio pausing

lower, than those maintained under the token components. For Pigeon 832, differences

between the brief-stimulus components and token components were not pronounced,

whereas for 999 response rates were generally higher under the token components.

Further, for Pigeons 83 and 47, differences in the response rates between the token and

brief-stimulus components were due mainly to differences in the early links of a cycle.












250
47 83
200

150

100


LU 50
50


Z 0


0-

I)


0
0_
0) 250
W 832 999
0 m TOKEN
200 r OTHER

150

100

50

0
TOKEN BMS BMS rev BAS BFS TOKEN BMS BMS rev BAS BFS

CONDITION


Figure 3-1. Mean responses per minute (not including pre-ratio pause) and standard
deviations from the last 5 sessions of a condition. Filled bars represent data
from the token component while open bars represent data from the component
varied across conditions.












400
47 83

300









C)
200


U0) 0




0
o')


O
I-

L 150 -
01 832 999
TOKEN
I OTHER

100-



50




TOKEN BMS BMS rev BAS BFS TOKEN BMS BMS rev BAS BFS

CONDITION


Figure 3-2. Mean pre-ratio pause and standard deviations from the last 5 sessions of a
condition. Filled bars represent data from the token component while open
bars represent data from the component varied across conditions.

Figures 3-3 and 3-4 show response rates and pre-ratio pausing for token and brief-


stimulus components as a function of segment (position in the exchange-production


cycle). In general, responding in brief-stimulus components resembled that of the tandem


components from Experiment 1: Low initial-link response rates gave way to higher,


constant rates in the later links. Performance under the token components also resembled


that of token performance in Experiment 1 in that response rates gradually increased as a


function of the number of tokens earned. Further, in most cases for both the token and













tandem components, pre-ratio pausing was longest for the initial segments and gave way


to small pauses for subsequent segments.


BOTH TOKEN


BAS


BMS


BFS


3 4 1
SEGMENT


4 1 2 3 4


Figure 3-3. Mean responses per minute (not including pre-ratio pause) plotted as a

function of token production or brief-stimulus segment from the last 5

sessions of a condition. Open symbols represent data from brief-stimulus

components, closed from token components, solid lines represent original

exposures, and dashed lines represent replications.


300
250
200
150
100
50
0
400
83
300

200

100

0
300
250
200
150
100
50
0
300
250 9
200
150
100
50
0


4 1


W'







40


BOTH TOKEN BAS BMS BFS

100 47

10


01






g o.
100 83

w 10
W 1




01
0-






100 999 -*- TOKEN

10

1
01
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
SEGMENT


Figure 3-4. Mean pre-ratio pausing plotted as a function of token production or brief-
stimulus segment from the last 5 sessions of a condition. Open symbols
represent data from brief-stimulus components, closed from token
components, solid lines represent original exposures, and dashed lines
represent replications.

Discussion

Several brief-stimulus configurations were arranged to incorporate features of


token-reinforcement schedules. However, neither a standard-brief stimulus schedule, nor


one that varied stimulus position or magnitude as a function of proximity to


reinforcement, produced behavior markedly different from what was found in the tandem


schedules from the previous experiment. For two subjects, response rates under brief-


stimulus components were higher than those under token components.


The similarities between the brief-stimulus and tandem components, and the lack of


a difference between the variants of the brief-stimulus configurations, may be due to both









the values of the token-production and exchange schedules and the duration of stimulus

presentations. Results from Lee and Gollub (1971) indicated that the effects of brief-

stimulus presentations vary depending on the value of the stimulus-production schedule.

As mentioned earlier, Lee and Gollub (1971) used a procedure in which the number of

responses required to produce food remained constant at 256, and across conditions the

FR production value of the brief-stimulus was varied. They obtained the highest rates of

responding when either 2 or 4 brief stimuli occurred per food presentation. The present

procedure had 4 brief-stimulus presentations per primary reinforcement period and it may

be that this particular value, similar to Lee and Gollub (1971), produced a high rate of

responding. In the absence of parametric variation it is difficult to determine if under

other schedule values performance on the present brief-stimulus arrangements would

produce behavior similar to that under the token-reinforcement component. This

argument may be extended to account for the lack of effects under the different variations

of brief-stimulus schedules presently employed. It may be that under different brief-

stimulus presentation schedule values differences in performance would be observed.

The present findings also are suggestive that the duration of the stimulus

presentation is an important variable, perhaps enhancing the discriminative function of

tokens by more clearly demarcating each segment. Although the only difference between

the brief-added stimulus components and the token components was the duration of

stimulus presentation, in some cases the two components had considerably different

response rates and patterning. It may be that the continuous display of stimuli makes

more salient the temporal correlation between number of tokens and delay to and amount

of food available during exchange. Parametric manipulation of stimulus duration in the






42


context of several token-production and exchange schedules would be needed, however,

to more adequately test this possibility.














CHAPTER 4
EXPERIMENT 3

Similarities between token-reinforcement and extended-chained procedures have

been noted several times by previous researchers (Gollub, 1970; Kelleher and Gollub,

1962). Indeed, the long pauses at the beginning and response patterns throughout an

extended-chained cycle are similar to those seen in the token components from the first 2

experiments and in other token-reinforcement schedules with FR token-production and

exchange schedules (Bullock and Hackenberg, 2006; Foster et al., 2001; Kelleher, 1958;

Webbe & Malagodi, 1978). Two methods for more systematically investigating

similarities between token-reinforcement and chained schedules would be to (a), make

token-reinforcement schedules more similar to chained schedules, and (b), replicate with

token-reinforcement schedules previously investigated variations in chained schedules.

In particular, the manner of presentation and order of stimuli in chained schedules have

been modified to more precisely determine the function of the stimuli demarcating a

given link--a procedure that could be readily adapted to token-reinforcement schedules.

Kelleher and Fry (1962) examined pigeons' responding over a series of conditions

involving a traditional-chained schedule, a modified-chained schedule in which the

stimuli denoting each link varied from cycle to cycle, and a tandem control. The chained

schedule was comprised of three sequential FI schedules. It was found that, following a

pre-ratio pause, responding occurred at a high, steady rate under the tandem schedule.

Under the traditional-chained schedule, responding was characterized by a long pre-ratio

pause and low response rates in the first link, followed by progressively increasing rates









in the second and third links. When compared to the first two links of the traditional-

chained schedule, response rates under the variable-chained schedule were higher and

pre-ratio pauses lower. Unlike performance under the tandem schedule, response rates

were positively accelerated in the chained-schedule conditions, a finding indicative of a

discriminative function. Although the present experiment used FR rather than FI

schedules, Kelleher and Fry's (1962) manipulations are readily adaptable to token-

reinforcement procedures, and may be similarly revealing of token-stimulus functions.

In another study aimed at discovering stimulus function in chained schedules, Byrd

(1971) examined pigeons' responding when the same stimulus was presented in more

than one link of a chain cycle. Pigeons responded on a chained schedule with each link

comprised of a 1 min FI and with the number of links varied across conditions (3, 5, 7,

and 8). During most conditions the same stimulus was used for the odd numbered links

and a distinct stimulus for the even. For instance, under the condition with 7 links, the

same stimulus was used for links 1, 3, 5, and 7. The one exception was the 8-link

condition, in which the same stimulus was used for the even numbered links. A 7-link

control condition was also examined, identical to those previously described with the

exception that a distinct stimulus was used for link 7. In keeping with other findings

concerning extended-chained schedules, Byrd found that response rates tended to

increase from almost zero in link 1 to high rates of responding as a function of temporal

proximity to food. However, under conditions with 5 and 7 links, response rates under

the later links with the same color stimulus were higher compared to the following

distinct stimulus link, despite closer temporal proximity to food of these links. Under the

7-link chain condition with a distinct stimulus for the terminal link, response rates were









lower when compared to the previous 7-link condition. Further, under the 8-link

condition response rates under link 1 were still extremely low, but increased under the

most temporally distant same color link, link 2.

Byrd interpreted these effects as showing that the discriminative properties of

stimuli in chained schedules are important determinants of response rate. The response

rate increasing effect of having several previous links share the same stimulus as the

terminal link may be indicative of a conditioned reinforcing effect. However, they

suggest that response-rate increases in chained schedules cannot be unambiguously

interpreted as due to conditioned reinforcement. If conditioned reinforcement were the

sole factor in determining performance on this procedure, one would expect that response

rates in link 1 of the 8 link condition would be higher than those of link 1 from the 7-link

condition, due to the production of the stimulus also associated with the terminal link.

Both the results of Kelleher and Fry (1962) and Byrd (1971) emphasize the

importance of the discriminative properties of stimuli in extended-chain schedules. In a

similar vein, the present research manipulated several features of the stimulus-food

relations in token reinforcement schedules to assess the stimulus functions of the tokens.

Initially the token reinforcement schedule was altered such that it was more procedurally

similar to a standard extended-chained schedule. Tokens were presented at the

completion of each FR link, but only one reinforcer was available during exchange. If

similar mechanisms influence performance on token reinforcement and extended-chain

schedules then one would expect to see little difference between performance under a

standard-token reinforcement procedure and one that is more like an extended chain. In a

second condition, the standard token-reinforcement schedule was compared to one









altered such that the stimulus events preceding an exchange occurred in the reverse order.

Reversing the order of stimuli results in perhaps a less distinct stimulus at the beginning

of a cycle, 4 tokens present, than under the standard token contingencies, the complete

absence of tokens. As in Byrd (1971), the stimuli immediately preceding primary

reinforcement may gain more of a conditioned reinforcing effect than those occurring

earlier in an extended chain cycle, a finding that may also be wtrue in token-

reinforcement schedules. Lastly, a condition was arranged in which the contingency

between number of tokens and temporal proximity to an exchange was weakened. In this

condition tokens were produced according to a VR schedule while exchanges occurred

after 200 responses, irrespective of how many tokens had been produced. Similar to the

effects of the variable-order stimulus condition from Kelleher and Fry (1962), one would

expect that as the contingency between a number of tokens and proximity to exchange is

degraded, performance would come to more closely resemble those from the tandem

components described in Experiment 1.

Method

Subjects and Apparatus

The subjects and apparatus were the same as in Experiment 2.

Procedure

Similar to Experiment 2, a 2-component multiple schedule was used in Experiment

3. Each component type occurred twice per session. In one component, a token-

reinforcement schedule was in place while the other component consisted of a variant of

a token schedule. In both components, an exchange period occurred after 200 responses;

in the token components, tokens were produced after every 50 responses and exchanges

after every 4 tokens (FR 4 [50]). Components were ordered pseudo-randomly, with









components remaining in effect for four exchange cycles. A 30-s blackout followed each

component. Sessions began with the illumination of the white houselight and the left or

right side key (either green or yellow depending on component type). Conditions were in

effect for a minimum of 14 sessions and response rates were deemed stable across the last

5 sessions via visual inspection.

The token-variant component involved 3 variations on the standard token schedule:

reinforcement magnitude, order of token delivery, and schedule of token delivery. In the

first, a one-reinforcer token schedule (1 Rein) was in place. This variant is analogous to

standard-chained schedules (in which a single reinforcer is available at the end of the

terminal link) but with all other features identical to a standard token-reinforcement

schedule. That is, identical to the token-reinforcement component, tokens were produced

according to an FR 50 and exchange periods occurred after 4 tokens were produced.

However, the first exchange response darkened all tokens and produced just one 1.5 s

food delivery.

Another variant, reverse-order token schedule (reverse), was in place for some

conditions. This variant was identical to a standard token-reinforcement schedule with

the exception that a token-production cycle began with 4 tokens and every 50 responses

extinguished one. Thus the removal of tokens, rather than the presentation, was

correlated with temporal proximity to exchange. Following the removal of the last token,

4 tokens were illuminated and an exchange period began.

A third variant, broken-contingency token schedule (VR), was also in place for

some conditions. Under this variant, responses produced tokens under a VR 50 schedule

(up to a maximum of 12). The number of tokens produced, however, was unrelated to









exchange, with exchanges occurring after 200 responses. Under these components it was

possible to enter an exchange with fewer or greater than 4 tokens. However, during the

exchange, 4 reinforcers were available, with each exchange response darkening a token.

If more than 4 tokens had been produced, then additional center key responses were

required to darken the remainder before another token-production cycle began. If fewer

than 4 tokens had been produced, responses on the center key simply continued to

produce food until 4 reinforcers had been obtained. The uncompleted VR value at the

end of a cycle was simple used as the first for the next cycle. Table 4-1 lists the order of

conditions and number of sessions per condition.

Table 4-1. Order of conditions and number of sessions per condition for Experiment 3.
Number of sessions per condition is listed in parentheses.
Pigeon

47 83 832 999

1 Rein. (47) 1 Rein. (24) 1 Rein. (17) 1 Rein. (28)

Reverse (26) Reverse (17) Reverse (38) Reverse (11)

VR (35) VR (23) VR (30) VR (26)

Results

Figures 4-1 and 4-2 show for each pigeon responses per minute and pre-ratio

pauses, respectively, for both component types across conditions. The filled bars

represent performance under the token-reinforcement schedule while the open bars show

chained-schedule performance. For all subjects except 83, response rates were lower and

pre-ratio pauses longer in the one-reinforcer than in the token component. For 3 out of 4

subjects under the reversed order and broken contingency conditions, this relationship

was reversed, with response rates lower and pausing longer in the token component. The

lone exception in these latter conditions was Pigeon 999, for whom response rates







49


remained higher in the token component for all conditions in this experiment. Even for


this subject, however, response rates were slightly higher under the broken-contingency


and reversed-order components than under the one-reinforcer components.


250
47 83
200

150







I-
D




z OTHER
cr 0
0-


z
7 250
U) 832 TOKEN
S200 OTHER
0 2.. T 9 q9T


1 rein. reverse VR 1 rein. reverse VR


CONDITION


Figure 4-1. Mean responses per minute (not including pre-ratio pause) and standard
deviations from the last 5 sessions of a condition. Filled bars represent data
from the token component while open bars represent data from the other (no-
token) component.








50



500
47 83
400


300


200


100


LUJ





O
I--

S 250
uJ
S 832 999 7 TOKEN
20- OTHER
200 -


150


100-


50-



1 rein. reverse VR 1 rein. reverse VR


CONDITION


Figure 4-2. Mean pre-ratio pause and standard deviations from the last 5 sessions of a
condition. Filled bars represent data from the token component while open
bars represent data from the other (non-token) component.


Figures 4-3 and 4-4 show response rates and pre-ratio pausing for each 50-response


segment, respectively, for both components across conditions. Filled circles represent


responding under the token-reinforcement components and open circles under the


components with the chained-schedule variants. For all pigeons under the one-reinforcer


contingency components, and for 2 out of 4 subjects under the opposing token


components, response patterning was graded with response rates increasing across


segments. Interestingly, for Pigeon 999 and to some extent for Pigeon 832, response rates








51



were fairly constant throughout the token cycle for this condition. Under the reversed-


order and broken-contingency components, response rates early in the cycle increased


compared to those in the opposing token schedule for all subjects except Pigeon 999.


Response patterning across the cycle was similar for both the reversed-order and broken-


contingency components in that in most cases response rates were low in the initial


segment and remained somewhat constant (Pigeons 832 and 999) or gradually increased


(Pigeons 47 and 83). Response rates in the opposing token components, however, were


marked by a more accelerated function than the other component in 6 of 8 cases. Similar


to the previous experiments, pre-ratio pausing for both components was characterized by


long pauses in the initial segment giving way to short, constant pauses thereafter.


ONE REINFORCER
300
250 47
200
150
100
0 -- TOKEN
50 -0- OTHER
0
400
83
300
200
100
0
300
250 832
200
150
100
50
300
250 999
200
150
100
50

1 2 3 4


REVERSED ORDER







^/






















1 2 3 4
SEGMENT


VR TOKEN PRODUCTION






























1 2 3 4


Figure 4-3. Mean responses per minute (not including pre-ratio pause) plotted as a
function of token production or 50 response segments from the last 5 sessions
of a condition. Open symbols represent data from token variant components
while closed symbols represent data from token components.










ONE REINFORCER

47



-*-- TOKEN
-0- OTHER


REVERSED ORDER


VR TOKEN PRODUCTION


1 2 3 4 1 2 3
SEGMENT


4 1 2 3 4


Figure 4-4. Mean pre-ratio pausing plotted as a function of token production or 50
response segments from the last 5 sessions of a condition. Open symbols
represent data from token variant components while closed symbols represent
data from token components.

Discussion

The primary findings from this experiment were that, relative to rates in the

token-reinforcement schedule, response rates were lower under the chain-like procedure

and higher under the broken-contingency and reversed-order procedure. Further, as

shown in Figure 4-3, response patterning reflected, to some degree, manner of token

presentation.









The one-reinforcer contingency produced lower response rates, compared to the

opposing token component, a joint product of longer pre-ratio pauses and lower response

rates early in a cycle. Data from the one-reinforcer token schedule indicated that

response rates were sensitive to reinforcer magnitude, with rates consistently higher in

the token component (4 food deliveries) than in the chained component (1 food delivery).

This could be due to the greater reinforcer magnitude in the token component or to the

correlation between number of tokens and reinforcer magnitude in that component. That

response rates in the initial links of the reverse-order condition remained low relative to

later links suggest that low response rates in early links are not simply due to the absence

of tokens. These data speak to the importance of the correlation of the tokens with

temporal proximity to food. That response rates increased in the broken-contingency

component relative to the token component for 3 out of 4 subjects suggests that the

correlation between number of tokens and proximity to exchange is important.

Although there was a difference in overall response rates between the chain-like

and token reinforcement schedules, the qualitative patterns of responding were similar,

with response rates increasing across a cycle as tokens accumulated. This finding

provides some support for the notion that the discriminative properties of chained and

token-reinforcement schedules are similar. The overall increases in response rates under

the broken-contingency components are similar to those of the variable-order condition

from Kelleher and Fry (1962). Results from both experiments suggest that the presence

or absence of a given number of tokens or type of stimulus is arbitrary: what matters is

the temporal relation between the tokens and food.














CHAPTER 5
EXPERIMENT 4

In token-reinforcement schedules the tokens have been shown to have several

functions, including conditioned reinforcers (Kelleher and Gollub, 1962) and

discriminative stimuli (Bullock and Hackenberg, 2006). Research has shown that

pigeons' key pecking can be generated irrespective of operant contingencies via stimulus-

food relations (Brown & Jenkins, 1968). The typical procedure for generating such

autoshaped, or automaintained, behavior is to repeatedly present a keylight, followed by

response-independent food delivery (Brown and Jenkins, 1968). Some of the conditions

under which such autoshaped keypecking has been generated and maintained are similar

to token reinforcement schedules. For example, autoshaped responding can be generated

under conditions in which distinct stimuli are presented successively, temporally

correlated with food presentation (Ricci, 1973).

Ricci (1973) examined pigeons' autoshaped key pecking under several stimulus

arrangements. In some conditions subjects were exposed to contingencies in which a

sequence of 4 colors was presented for 30 s each, with the terminal stimulus followed by

food reinforcement. Performance was then compared to that generated under a similar

procedure except that just one stimulus was presented for the entire 120 s prior to food

delivery. Response distributions under the 4-color conditions were graded, with response

probability increasing as a function of the temporal proximity of that stimulus to

reinforcement. By contrast, responding under the 1-color conditions was much more

uniform throughout the 120 s interval. It was suggested that such autoshaping procedures









are similar to chained schedules in that both involve sequential arrangements of stimuli

temporally related to food.

It is possible that responding on schedules of token reinforcement may be

maintained simply by the presentation of tokens, with the number of tokens presented

correlated with temporal proximity to food. The present experiment was designed to

investigate this possibility. In one component of a multiple schedule tokens were

presented response-independently, yoked to their temporal occurrence in the immediately

prior token reinforcement component. Under simple schedules, response-independent

reinforcement breaks the dependency between responding and food production and

results in lower response rates (Lattal, 1972). If the tokens served as conditioned

reinforcers then presenting them response independently should result in a substantial

decrease or elimination of response rates. On the other hand, if tokens served an eliciting

function, in the manner of serial autoshaping, one would expect some maintenance of

responding. To examine whether the presence versus absence of the tokens was an

important determinant of responding in the yoked component, the tokens were removed

from the yoked component in some conditions, while exchanges remained yoked to their

temporal occurrence in the preceding token component. These conditions were designed

to determine to what degree responding maintained under token-reinforcement schedules

is a product of the temporal relations between tokens and food, apart from the contingent

production of tokens. To examine the possibility that induction from the token schedule

could account for responding in the yoked components, other conditions held constant the

key color in both component types. If induction was a determinant of responding in the









yoked component one would expect that by making the two components more similar

response rates in the two components would converge.

Method

Subjects and Apparatus

The subjects and apparatus were the same as in Experiments 2 and 3.

Procedure

Similar to the other experiments reported, a 2-component multiple schedule was

used in Experiment 4 with a token-reinforcement component and a yoked component.

Each component occurred twice and lasted for 4 exchange cycles. A 30-s blackout

followed each component. Sessions began with the illumination of the white houselight

and side key associated with the token component (either green or yellow). In the token

component tokens were produced every 50 responses and exchanges after every 4 tokens

(FR 4 [50]). In the yoked component, tokens and exchange periods were presented

response independently, yoked to the times they occurred in the preceding token

component. The token components always occurred first and third, the yoked component

second and fourth.

Two other conditions consisted of (a) holding the token-production key color

constant across both components (Yoked Color-Same, or CS), and (b) removing the

tokens entirely from the token-production cycle of the yoked component (Yoked No

Token, or NT). In this latter condition, 4 tokens illuminated immediately prior to the

exchange period while the number of reinforcer deliveries (4) and delays to the exchange

period were equal to those in the token component. Conditions were in effect for a

minimum of 14 sessions and until response rates were deemed stable across the last 5









sessions via visual inspection. Table 5-1 lists the order of conditions and number of

sessions per conditions

Table 5-1. Order of conditions and number of sessions per condition for Experiment 4.
Number of sessions per condition is listed in parentheses.
Pigeon

47 83 832 999

Yoked (34) Yoked (35) Yoked (50) Yoked (25)

Yoked NT (14) Yoked NT (15) Yoked NT (17) Yoked NT (14)

Yoked (16) Yoked (20) Yoked (22) Yoked (32)

Yoked CS (17) Yoked CS (30) Yoked CS (18) Yoked CS (18)

Results

Figures 5-1 and 5-2 show running response rates (response rates factoring out pre-

ratio pausing) and pausing for each pigeon, respectively, across the final 5 sessions in

each condition. Filled bars show performance from the token components while open

bars show performance from the yoked components. Response rates for all conditions in

the standard-token component were higher than in any of the yoked-component

variations. Pausing was also generally greater in the yoked than in the token components.

Response rates in the standard-yoked components were lower than in to the opposing

token components, but never reached zero. Under the no-token (NT) yoked condition,

however, responding was almost completely eliminated. In the yoked component with

the token-production key color the same as the opposing token component (CS), response

rates were either comparable to, or lower than, those in the standard yoked-token

conditions for 3 out of 4 pigeons.























' l


YOKED YOKED NT YOKED YOKED CS


250
999 TOKEN
YOKED
200

150 O

100

50


YOKED YOKED NT YOKED YOKED CS


CONDITION


Figure 5-1. Mean responses per minute (not including pre-ratio pause) and standard
deviations from the last 5 sessions of a condition. Filled bars represent data
from the token component while open bars represent data from the yoked
component.


250
83
200

150

100

50


I3












250 250
47 83
200 200


150 150 -


100 100






Z
z
L-

_1-j
L 250 250
) 832 999 TOKEN
Z m- YOKED
O 200 200

L 150 150 -

100 100 -

50 50

0 0-
YOKED NO TOKEN YOKED SAME COLOR YOKED NO TOKEN YOKED SAME COLOR

CONDITION


Figure 5-2. Mean pre-ratio pausing and standard deviations from the last 5 sessions of a
condition. Filled bars represent data from the token component while open
bars represent data from the yoked component.


Figures 5-3 and 5-4 show response rates and pausing for each component type


across successive segments in the exchange cycle. Segments consisted of either 50


responses (token component) or the equivalent time periods (yoked component).


Response patterning under the token components was graded, increasing as a function of


the number of tokens earned and proximity to food. Under the standard token-yoked


conditions, responding within a cycle was characterized by extremely low rates for the


first one or two segments followed by an increase, resulting in rates close to those of the


comparable token component for segments 3 and 4. For all pigeons under the no-token







60


yoked components response rates were equally low for all 4 segments. Segment pauses,

(Figure 5-4) also corresponded to this pattern, with long pauses in the early links of a

cycle becoming shorter with increasing numbers of tokens illuminated. (Under the yoked

conditions pausing sometimes exceeded the period of time before the first token

illuminated. In such cases, the link 1 pause was set to the time period before token

presentation. The mean pre-ratio pauses, shown in Figure 5-2, however, were based on

the actual time prior to a response, irrespective of token illumination.)

YOKED YOKED COLOR CONSTANT YOKED NO TOKEN


3 4


2 3 4


SEGMENT

Figure 5-3. Mean responses per minute (not including pre-ratio pause) plotted as a
function of token production or yoked token-production segment from the last
5 sessions of a condition. For no-token yoked components the segment data
are organized around when the token would have occurred. Open symbols
represent data from yoked components, closed from token components, solid
lines represent original exposures, and dashed lines represent replications.










YOKED COLOR CONSTANT


1 2 3 4 1 2 3 4 1 2 3 4
SEGMENT

Figure 5-4. Mean pre-ratio pausing plotted as a function of token production or yoked
token-production segment from the last 5 sessions of a condition. For no-
token yoked components the segment data are organized around when the
token would have occurred. Open symbols represent data from yoked
components, closed from token components, solid lines represent original
exposures, and dashed lines represent replications.

Figure 5-5 shows the obtained delays between the last response of a segment and

the illumination of a token for that segment, across successive segments within an

exchange cycle. For cases in which a response did not occur within a given segment, the

duration of the segment was used as the delay. The response-token delays varied both

across subjects and across segments within an exchange cycle for individual subjects.


832


"N


YOKED


YOKED NO TOKEN






62

For Pigeon 999, the mean response-token delays were generally quite long, rarely less

than 10 s. For the other 3 pigeons, response-token delays were generally high in the early

segments, but became shorter in the later links when response rates were high.
TOKEN YOKED TOKEN YOKED REPLICATION TOKEN YOKEN COLOR SAME


1 2 3 4 1 2 3 4 1 2 3 4
SEGMENT

Figure 5-5. Mean time between the last response of a segment and the token production
plotted as a function of yoked token-production segment from the last 5
sessions of a condition.

Discussion

Response rates in the yoked components were lower than in the token components,

suggesting a role for the dependency between responding and token production (i.e., a

reinforcing function). In the yoked components, the presence of tokens maintained


83



1"----. --- ---:


999


- '1 _1









responding at much higher levels than when they were absent. Within a yoked-token

component, response rates increased as a function of the temporal proximity to food in

much the same way that they did under response-dependent token production. This

finding, similar to that reported by Ricci (1973), suggests a discriminative and/or eliciting

role for the tokens. When the key color was held constant, response rates were equal to

or slightly lower than the regular-yoked components, indicating that induction from the

token component does not account entirely for responding in the yoked component.

However, because of the previous history of reinforcement on token-reinforcement

schedules statements concerning the degree to which the discriminative versus eliciting

properties of the stimuli control behavior are limited.

Figure 5-5 shows that an adventitious contingency does not explain the present

results entirely. Response rates increased in a graded fashion, as a function of the

number of tokens earned (see Figure 5-3). There existed a considerable delay between

responding and the illumination of the first and second tokens, and relatively short delays

between responding and the presentation of the third and fourth tokens. If an

adventitious contingency had existed with respect to responding and production of the

later tokens, one would expect that responding would be entirely absent prior to the first

two token deliveries, rather than the observed graded functions that were found.














CHAPTER 6
GENERAL DISCUSSION

The objective of this series of experiments was to explore systematically the

stimulus functions of tokens in token-reinforcement schedules. Performance under

schedules of token reinforcement was compared to that under tandem schedules and to

that under several token-like schedules, all with equivalent response requirements.

Experiment 1 compared token schedules to equivalent tandem schedules and found that

response rates under token-reinforcement schedules were lower than under tandem

schedules, with response patterning suggesting a discriminative function of the tokens.

Experiment 2 compared token schedules to several schedules of briefly-presented token

presentation, and found that response rates under token-reinforcement schedules were in

some cases lower than under variants of the briefly-presented stimulus schedules. Rates

and patterns in the latter were comparable to the tandem components in Experiment 1,

suggesting that the continuous display of tokens contributes to their discriminative

effects. Experiment 3 compared token schedules to extended chained schedules, and

found that response rates under token-reinforcement schedules were lower than under

comparable chained schedules when the correlation between token display and temporal

proximity to exchange periods was altered. Only when compared to standard-chained

schedule, with a single reinforcer at the end of the chain, were response rates higher in

token schedules, indicating sensitivity to reinforcement magnitude with stimulus

conditions held constant. Experiment 4 compared token schedules to schedules of

response-independent token presentation to assess the reinforcing and potential eliciting









functions of the tokens. Response rates were reduced under the response-independent

schedules, suggesting a reinforcing function, but they were not eliminated, suggesting an

eliciting function. Taken together, the results suggest that tokens serve important

stimulus functions in token reinforcement schedules, and that the specific function, or

functions, depend on the contingencies in which they are embedded.

The results of the Experiment 1 correspond to those seen in prior research with

token schedules (Bullock & Hackenberg, 2006; Foster et al., 2001), added-stimulus

schedules (Zimmerman & Ferster, 1964), and extended-chained schedules (Jwaidah,

1973). Decreases in response rates in the present experiment as a function of increasing

the exchange-schedule requirements is consistent with previous token-reinforcement

schedule findings using FR exchange schedules (Bullock and Hackenberg, 2006; Foster

et al., 2001). The graded pattern of responding found under token-reinforcement

schedules in the present research correspond to both previous token-reinforcement

research and to the effects off adding incremental stimulus changes reported by

Zimmerman and Ferster (1964). Higher rates of responding in components with tandem

schedules, when compared to components with token schedules, reported in Experiment

1, correspond to similar manipulations with extended chained schedules (Jwaidah, 1973).

Experiment 2 showed that briefly presenting tokens attenuated their

discriminative function, irrespective of whether their position and number was correlated

with responses/temporal proximity to reinforcement. The lack of a discernable effect

between the brief-stimulus conditions, combined with the differences for some pigeons

between the brief-stimulus and token components, points to the importance of token

display duration. Briefly presenting the tokens may have decreased their salience and









thus disrupted the discriminative properties of the tokens, a result that was perhaps

similar to the effects of varying token size in Experiment 1. In other words, the results of

Experiment 2 suggest that the functions of tokens in second-order schedules are related to

how long they demarcate the completion of each segment.

The results of Experiment 3 showed that the discriminative properties of tokens in

token-reinforcement schedules are similar in several respects to stimuli in chained

schedules. As mentioned earlier, Kelleher and Fry (1962) found that randomizing the

order of stimuli delineating the links in an extended chain schedule increased response

rates relative to a standard extended-chained schedule. A similar result was found by

Kelleher (1958) when chimpanzee's pre-ratio pauses decreased as a result of the non-

contingent delivery of a large group of tokens prior to the start of a session. In

Experiment 3 of the present study, the condition that involved use of a VR schedule of

token production that operated independently of the FR 200 exchange-production

schedule was analogous to randomizing the stimuli in Kelleher and Fry (1962). The use

of a VR token-production schedule in the present research and the procedures used in

Kelleher and Fry (1962) and Kelleher (1958) were all similar in that degrading the

correlation between tokens and temporal proximity to food resulted in decreased pre-ratio

pausing or increased rate of responding.

Experiment 4 was designed to assess the conditioned reinforcing and eliciting

functions of the tokens by removing the dependency between responding and token

presentation. In this experiment responding under a standard token-reinforcement

schedule was compared to that under a procedure where tokens were delivered response

independently, yoked to their temporal occurrence in the preceding standard token









schedule component. If the tokens served as conditioned reinforcers one would expect

that response rates would either substantially decrease or completely cease, as response-

independent token delivery would break the contingency between responding and token

production. However, if the tokens had an eliciting function, responding should continue,

as response-independent token delivery would not alter the token-food correlation.

Responding was reduced in yoked conditions, suggesting a reinforcing function, but it

was not eliminated, suggesting an eliciting function. The latter result is consistent with

those reported by Ricci (1973), described earlier, in which the probability of a response in

the presence of a given stimuli increased as a function of temporal proximity to food. A

similar relationship was found in Experiment 4, suggesting that tokens may serve to elicit

responding in addition to other functions. Similar to the common discriminative

properties of stimuli in token reinforcement and chained schedules, research has shown

that stimuli in chain schedules also have an eliciting function.

In support of this, Dougherty and Lewis (1991) used an omission procedure to

investigate further the degree to which extended-chained schedule stimuli have eliciting

functions. An omission procedure is one in which the occurrence of a response prevents

the delivery of reinforcement. Thus, if responding is maintained by operant

contingencies, one would expect an omission procedure to eliminate responding.

Conversely, if responding is due to the stimulus-food relations and is independent of

operant contingencies, then responding should still occur. Pigeons were exposed to 3

conditions, 2 with an omission contingency and one with a standard chain. In the first, a

2-link chain was in effect, with each link lasting for 60 s and with transition occurring

response independently. Responses in the first link terminated the chain and began an









inter-trial interval (ITI). The second condition was similar to the first except that an FI

60 s schedule was in effect for both links. If responses occurred before 60 s had elapsed

in the first link then the chain terminated and the ITI began. The third condition was a

standard chain with an FI 60 s in both components. That responding occurred in the

initial link of the first two conditions (with an omission procedure), in spite of resulting in

a lower rate of reinforcement, suggests that the stimuli in these procedures had an

eliciting function. Thus, consistent with the results of Ricci (1973) and the present

experiment, chain-schedule stimuli also have eliciting functions.

Taken together, the results of the experiments presented here suggest that stimuli in

token-reinforcement procedures can have a combination of functions. Experiment 1

showed that response rates are lower under token reinforcement schedules when

compared to equivalent tandem schedules. This finding, along with a more graded

pattern of responding found under the token-reinforcement schedules suggests that tokens

have a discriminative function. The results of Experiment 2 suggest that the duration of

token presentation is an important determinant of the discriminative functions of tokens.

Experiment 3 showed that changing the token production schedule to a VR increased

response rates, a finding that again suggests a discriminative function. Results from

Experiment 4 showed that response rates continue in the absence of a contingency

between responding and token production, a finding that suggests tokens may have a

response eliciting function. Further, higher response rates in Experiment 4 under the

standard token-reinforcement schedule than under the yoked schedule speaks to the role

of a dependency between responses and tokens, and may be indicative of a conditioned

reinforcing function. Thus, the present research suggests that the functions of stimuli in









token reinforcement procedure have eliciting, discriminative, and reinforcing properties

and that the particular function depends on the contingencies in place.

While the present research shows that the tokens have several functions, many

dimensions of a token reinforcement procedure have yet to be investigated. Research that

varies the length of presentation of a brief-stimulus from very short to almost the entire

segment period would allow for a more precise understanding how stimulus function

varies as the schedules shift from more brief-stimulus-like to more token- or chain-like.

Manipulations of stimulus durations are particularly suited to experimental preparations

that allow for controlled presentations, as opposed to manipulable tokens used in some of

the studies discussed earlier.

Second, most prior research has conducted comparisons of the various component

types under only 1 token production and exchange value. Bullock and Hackenberg

(2006) showed that the effects of tokens can vary depending on the context of both the

token production and exchange schedules, and the same may also be true of the

comparisons of the present research. Although examination of such a wide combination

of component types across several token-production and exchange schedules was not

practical for the present investigation, these manipulations would yield more detailed

knowledge of stimulus function.

Third, in the present research for some subjects overall response rates tended to

decrease across experiments. Because effects in the present experiment were assessed

based on differences within a condition between the two different components this was

not a major concern. However, an area for future research would be to investigate how

performance varies as a function of amount of exposure to token-reinforcement









schedules. It may be that the determinants of behavior vary as experience with token

schedules increases.

Lastly, the present procedures involve a dependent arrangement of token

production and exchange schedules, with token production contributing to satisfying the

exchange-production schedule. This need not be the case, however; the procedure could

be modified such that producing an exchange would be independent of token production,

given that at least one token had been earned. Under such an arrangement subjects could

accumulate a number of tokens before completing a separate exchange ratio. In this case

no relationship between number of tokens and proximity to exchange would exist; thus

determinants of responding in an accumulation procedure might vary considerably from

the token-reinforcement schedules presented presently.

Turning to broader issues, token-reinforcement procedures have been used

extensively in applied settings with a wide variety of treatment populations (Kazdin &

Bootzin, 1972; Kazdin, 1982). That token reinforcement schedules have been so widely

used speaks to the importance of understanding of their controlling variables. By using

nonhuman subjects in a precisely controlled environment the present research was able to

contribute to the literature by showing that tokens have multiple functions depending on

the contingencies. Data reported by Field, Nash, Handwerk, and Friman (2004)

replicates the effects of exchange-schedule manipulations in a token reinforcement

procedure in an applied setting. They found that decreasing the time between

opportunities to exchange tokens for primary reinforcers, and thus increasing

reinforcement rate, increased the effectiveness of a token economy in managing problem

behavior.









The present data and prior token-reinforcement schedule research suggest several

potentially fruitful extensions into applied settings. As mentioned earlier, token

economies often are utilized as a means of reinforcing pro-social behavior. The present

data suggest some of the circumstances that promote a discriminative rather than

conditioned reinforcing function. For example, Experiment 3 showed that a correlation

with number of tokens and temporal distance to exchange resulted in lower response rates

than when this contingency was disrupted. Experiment 2 suggested that the duration of

presentation may also determine the discriminative properties of tokens. One extension

of these findings to applied research would entail comparing performance under briefly-

presented token schedule, a token-schedule with an FR exchange, and a token-schedule

with a VR exchange. Comparisons between these three arrangements would allow an

assessment of how potential discriminative functions might disrupt responding

maintained by tokens and methods of reducing any discriminative functions.

Further research along these lines will add to the precision with which token

reinforcement systems are implemented, and aid in managing behavior in multiple

settings.















LIST OF REFERENCES


Brown, P. L. & Jenkins, H. M. (1968). Auto-shaping of the pigeon's key-peck. Journal
of the Experimental Analysis ofBehavior, 11, 1-8.

Bullock, C. E., & Hackenberg, T. D. (2006). Second-order schedules of token
reinforcement in pigeons: Implications for unit price. Journal of the Experimental
Analysis ofBehavior, 85, 95-106.

Byrd, L. D. (1971). Responding in the pigeon under chained schedules of food
presentation: The repetition of a stimulus during alternate components. Journal of
the Experimental Analysis of Behavior, 16, 31-38.

Cowles, J. T. (1937). Food tokens as incentives for learning by chimpanzees.
Comparative Psychology Monographs, 14, 1-96.

Dougherty, D. M. & Lewis, P. (1991). Elicited responding in chain schedules. Journal
of the Experimental Analysis ofBehavior, 56, 475-487.

Ferster, C. B & Skinner, B. F. (1957). Schedules of reinforcement. New York:
Appleton-Century-Crofts.

Field, C. E., Nash, H. M., Handwerk, M. L., & Friman, P. C. (2004). A modification of
the token economy for nonresponsive youth in family-style residential care.
Behavior Modification, 28(3), 238-456.

Foster, T. A. & Hackenberg, T. D. (2004). Unit price and choice in a token-reinforcement
context. Journal ofExperimental Analysis ofBehavior, 81, 5-25.

Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token
reinforcement with pigeons: Effects of fixed- and variable-ratio exchange
schedules. Journal of the Experimental Analysis ofBehavior, 76, 159-178.

Gollub, L. R. (1970). Information on conditioned reinforcement: A review of conditioned
reinforcement, edited by Derek P. Hendry. Journal of the Experimental Analysis of
Behavior, 14, 361-372.

Hackenberg, T. D., & Vaidya, M. (2003). Determinants of pigeons' choices in token-
based self-control procedures. Journal of the Experimental Analysis ofBehavior,
79, 207-218.






73


Jackson, K., & Hackenberg, T. D. (1996). Token reinforcement, choice, and self-control
in pigeons. Journal of the Experimental Analysis of Behavior, 66, 29-49.

Jwaideh, A. R. (1973). Responding under chained and tandem fixed-ratio schedules.
Journal of the Experimental Analysis ofBehavior, 19, 259-267.

Kazdin, A. E. (1982). The token economy: A decade later. Journal ofApplied Behavior
Analysis, 15, 431-445.

Kazdin, A. E., & Bootzin, R. R. (1972). The token economy: An evaluative review.
Journal ofApplied Behavior Analysis, 5, 343-372.

Kelleher, R. T. (1956). Intermittent conditioned reinforcement in chimpanzees. Science,
124, 679-680.

Kelleher, R. T. (1957). A multiple schedule of conditioned reinforcement with
chimpanzees. Journal of Comparative and Physiological Psychology, 49, 571-575.

Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with
chimpanzees. Journal of the Experimental Analysis ofBehavior, 1, 281-289.

Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal
of the Experimental Analysis ofBehavior, 9, 475-485.

Kelleher, R. T. & Gollub, L. R. (1962). A review of positive conditioned reinforcement.
Journal of the Experimental Analysis ofBehavior, 5, 543-597.

Kelleher, R. T. & Fry, W. (1962). Stimulus functions in chained fixed-interval
schedules. Journal of the Experimental Analysis ofBehavior, 5, 167-173.

Lattal, K. A. (1972). Response-reinforcer independence and conventional extinction after
fixed-interval and variable-interval schedules. Journal of the Experimental Analysis
ofBehavior, 18, 133-140.

Lee, J. K. & Gollub, L. R. (1971). Second-order schedules with fixed-ratio components:
Variation of component size. Journal of the Experimental Analysis ofBehavior. 15,
303-310.

Malagodi, E. F. (1967). Fixed ratio schedules of token reinforcement. Psychonomic
Science, 8, 469-470.

Malagodi, E. F. DeWeese, J. & Johnston, J. M. (1973). Second-order schedules: A
comparison of chained, brief-stimulus, and tandem procedures. Journal of the
Experimental Analysis ofBehavior, 20, 447-460.

Malagodi, E. F., Webbe, F. M., & Waddell, T. R. (1975). Second-order schedules of
token reinforcement: Effects of varying the schedule of food presentation. Journal
of the Experimental Analysis ofBehavior, 24, 173-181.









Ricci, J.A. (1973). Key pecking under response-independent food presentation after long
simple and compound stimuli. Journal of the Experimental Analysis of
Behavior, 19, 509-516.

Sidman, M. (1960). Tactics of Scientific Research. Boston: Authors Cooperative Inc.

Waddell, T. R., Leander, J. D., Webbe, F. M., & Malagodi, E. F. (1972). Schedule
interactions in second-order fixed-interval (fixed-ratio) schedules of token
reinforcement. Learning and Motivation, 3, 91-100.

Webbe, M. F. & Malagodi, E. F. (1978). Second-order schedules of token
reinforcement: Comparisons of performance under fixed-ratio and variable-ratio
exchange schedules. Journal of the Experimental Analysis ofBehavior, 30, 219-
224.

Williams, B. A. (1994). Conditioned reinforcement: Neglected or outmoded explanatory
construct? Psychonomic Bulletin & Review, 1(4), 457-475.

Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative
Psychology Monographs, 12, 1-72, No. 60.

Zimmerman, J., & Ferster, C. B. (1963). Fixed-interval performances with added stimul
in monkeys. Journal of the Experimental Analysis ofBehavior, 6, 317-322.

Zimmerman, J., & Ferster, C. B. (1964). Chained VI performance of pigeons maintained
with an added stimulus. Journal of the Experimental Analysis ofBehavior, 7, 83-
39.















BIOGRAPHICAL SKETCH

I grew up in Oxford, North Carolina, attending public schools. After graduating

from J. F. Webb High School in 1994, I enrolled as an undergraduate at the University of

North Carolina at Wilmington (UNCW). I graduated from UNCW with a Bachelor of

Arts in psychology with honors in the spring of 1999. The following fall I enrolled at the

University of Florida in the graduate program in psychology in the behavior analysis area

and began working in Dr. Timothy Hackenberg's lab. My research at Florida has focused

on choice, self-control, and schedules of token reinforcement.