<%BANNER%>

Second-Order Schedules of Token Reinforcement: Combined Effects of Token-Production and Exchange-Schedule Manipulations


PAGE 1

SECOND-ORDER SCHEDULES OF TOKEN REINFORCEMENT: COMBINED EFFECTS OF TOKEN-PRODUCTI ON AND EXCHANGE-SCHEDULE MANIPULATIONS By CHRISTOPHER BULLOCK A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE UNIVERSITY OF FLORIDA 2003

PAGE 2

Copyright 2003 by Christopher Bullock

PAGE 3

ACKNOWLEDGMENTS I thank my parents for their support and encouragement throughout my graduate studies. I would also like to thank my mentor, Timothy Hackenberg, for his encouragement and guidance in the research and writing of this research project. iii

PAGE 4

TABLE OF CONTENTS Page ACKNOWLEDGMENTS.................................................................................................iii LIST OF FIGURES.............................................................................................................v ABSTRACT.......................................................................................................................vi CHAPTER 1 INTRODUCTION........................................................................................................1 2 METHOD.....................................................................................................................5 Subjects.........................................................................................................................5 Apparatus......................................................................................................................5 Procedure......................................................................................................................6 3 RESULTS...................................................................................................................11 4 DISCUSSION.............................................................................................................18 LIST OF REFERENCES...................................................................................................26 BIOGRAPHICAL SKETCH.............................................................................................28 iv

PAGE 5

LIST OF FIGURES Figure page 2-1. Sample of an FR 2 (FR 50) token-reinforcement schedule.......................................10 3-1. Mean responses per minute and standard deviations for each pigeon under constrained consumption conditions plotted as a function of small, medium, and large token-production ratios, and as a function of exchange ratio..........................13 3-2. Mean responses per minute plotted as a function of token-production segment for each pigeon under constrained consumption conditions..........................................14 3-3. Mean latency plotted as a function of token-production segment for each pigeon under constrained consumption conditions..............................................................15 3-4. Mean total responses for each pigeon under unconstrained consumption conditions plotted as a function of unit price.............................................................................16 3-5. Mean consumption (total seconds access to food) for each pigeon under unconstrained consumption conditions plotted as a function of unit price..............17 4-1. Mean consumption rate as a function of unit price for each pigeon under constrained consumption conditions plotted on log-log coordinates..........................................24 4-2. Mean reciprocal responses per minute and modified unit price for each pigeon plotted as a function of unit price and exchange schedule.......................................25 v

PAGE 6

Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Master of Science SECOND-ORDER SCHEDULES OF TOKEN REINFORCEMENT: COMBINED EFFECTS OF TOKEN-PRODUCTION AND EXCHANGE-SCHEDULE MANIPULATIONS By Christopher Bullock May 2003 Chair: Timothy D. Hackenberg Major Department: Psychology Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule and were exchanged for 2.5 s access to food according to a fixed-ratio (token-exchange) schedule. The token-production and token-exchange schedules were manipulated systematically across conditions. Response rates varied inversely with the token-production schedule for a given token-exchange schedule value. Response rates also varied inversely with the token-exchange schedule for a given token-production value, particularly at the higher token-production ratios. Further, under higher token-production and exchange-schedule values, response rates tended to increase in token-production segments closer to exchange periods and food. Several probe conditions were also studied in a closed economy that permitted unconstrained food consumption. Under these conditions, response rates were less sensitive to token-production schedule manipulations than under standard (constrained consumption) vi

PAGE 7

conditions in which sessions were limited to 48 food presentations. Results were analyzed using a modification of the behavioral economic concept of unit price (a cost-benefit ratio comprising responses per unit of food delivery). vii

PAGE 8

CHAPTER 1 INTRODUCTION A second-order schedule of reinforcement is one in which a pattern of behavior reinforced according to one schedule is treated as a unitary response reinforced according to a second schedule (Kelleher, 1966). A token-reinforcement schedule is a second-order schedule in which responses produce tokens according to one schedule (the token-production schedule) and opportunities to exchange those tokens for primary reinforcement according to a second schedule (the exchange schedule) (Kelleher, 1958; Malagodi, 1967). Previous research has shown that response rates and patterns maintained under token-reinforcement schedules vary systematically as a function of both the token-production schedules (Kelleher, 1958) and the exchange schedules (Foster, Hackenberg & Vaidya, 2001; Waddell, Leander, Webbe, & Malagodi, 1972; Webbe & Malagodi, 1978). Most research on token-reinforcement schedules has involved fixed-ratio (FR) components (schedules that require a specified number of responses for reinforcement). Similar to FR performance reinforced with food, response rates under FR token-production and exchange schedules vary inversely with the FR value (Foster, et al., 2001; Kelleher, 1958; Malagodi, 1967; Webbe & Malagodi, 1978;). Kelleher (1958), for example, found that chimpanzees' rates of lever pressing decreased as the FR token-production requirement increased from 30 to 100 with the exchange ratio held constant at FR 60. Foster et al. (2001) found that pigeons response rates decreased as the FR exchange schedule increased from 1 to 8 with the token-production ratio held constant at 1

PAGE 9

2 FR 50. In both studies, response-rate decrements were mainly a function of increased pausing early in an FR cycle. With FR token-production and exchange schedules, a potentially relevant variable emerges: the ratio of responses per unit of reinforcer, closely related to the economic concept of unit price (DeGrandpre, Bickel, Hughes, Layng, & Badger, 1993; Hursh, 1978). When the exchange ratio is manipulated with token-production ratio held constant, changes in reinforcer magnitude are proportional to changes in exchange ratio. As a result, the unit price remains constant across variations in the exchange schedule. For example, in the Foster et al. (2001) study, with a token-production ratio of 50 and an exchange ratio of 2, FR 2 [FR 50], to use the standard nomenclature,100 responses produced 2 tokens (each exchangeable for 2 s access to food), or 25 responses per second access to food. So too under the FR 8 [FR 50] condition, 400 responses produced 8 tokens (16 s access to food), or 25 responses per second access to food. According to a literal version of the unit price concept, response output should be roughly constant under such conditions of equivalent unit price, without regard to the specific values of its cost and benefit components(Madden, Bickel, & Jacobs, 2000). Contrary to these predictions, however, response output varied substantially across conditions of equal unit price. Responding then was governed not by responses per unit of food delivery (unit price) but by responses per exchange period (exchange ratio). Another way of examining the viability of the unit price concept with token-reinforcement procedures is through manipulations of the token-production ratio. Unlike exchange-ratio manipulations, altering the token-production ratio while holding constant the exchange ratio changes the unit price. For example, in the Kelleher (1958) study,

PAGE 10

3 with an FR 30 token-production schedule and an FR 60 exchange schedule, 1800 responses produced 60 tokens, a unit price of 30. When the token-production schedule was increased to 100, while holding constant the exchange schedule ratio at 60, 6000 responses produced 60 tokens, a unit price of 100. In accordance with unit price predictions, rates of lever pressing decreased as the FR requirement and unit price of the token-production schedule increased. Thus, unlike exchange-ratio manipulations, in which unit price is invariant with respect to changes in ratio requirements; with token-production manipulations, unit price changes with changes in ratio requirements. The purpose of the present experiment was to clarify how both the token-production schedule and exchange schedule contribute to maintaining behavior on token-reinforcement schedules while assessing the applicability of unit price in accounting for this behavior. Although response rates and patterns in token-reinforcement procedures have been shown to vary with the values of both FR token-production and exchange schedules, previous studies have typically manipulated only one of these variables while holding the other constant. The effects of manipulating one of the schedules in a token-reinforcement procedure may depend on the others value. Using a preparation similar to that of Foster et al. (2001), with stimulus lights serving as token reinforcers, FR token-production and exchange schedules were parametrically manipulated on a within-subject basis across a wider range of schedule values than examined in prior research, allowing for a more detailed analysis of their effects. The present experiment included three token-production values (small, medium, and large) and three exchange values (2, 4, and 8); and was designed such that most of the possible combinations of these two schedules were experienced by the subjects.

PAGE 11

4 Unit price experiments are typically conducted in a closed economy, where all food is obtained via interaction with the experimental conditions (Hursh, 1980). A closed economy is often contrasted with an open economy, where supplemental feeding occurs outside of experimental sessions. In the Foster et al. (2001) study, birds received all or nearly all of their daily food intake within sessions (functionally a closed economy). However, because supplementary feeding was sometimes necessary, it can be argued that this was not a genuine closed economy, rendering a unit-price analysis inappropriate. In the present study, therefore, three probe conditions were conducted under a closed economy where all food was obtained via interaction with experimental contingencies. Parametric evaluation of token-production and exchange ratios in closed and open economic contexts allows for a stronger investigation of the unit price concept and for an extension of previous token-reinforcement studies. Unit price considerations aside, the study had as its main objective the extension of previous work in the realm of token-reinforcement procedures. The results should further clarify the contributions of token-production and exchange schedules to performance on token-reinforcement procedures.

PAGE 12

CHAPTER 2 METHOD Subjects Four male White Carneau pigeons ( Columba livia ) (numbered 702, 732, 774, 1855) served as subjects. Each pigeon had prior experience with token-reinforcement schedules. Subjects were housed individually under a 16.5 hour / 7.5 hour light dark cycle and had continuous access to water and health grit in home cages. Pigeons were maintained at approximately 20g of their 80% free feeding weights for most conditions of the experiment. In a few conditions (see below), the weight restrictions were removed, such that weights were free to vary above the 80% level. Apparatus Experimental sessions were conducted in an enclosure 36 cm high by 50 cm long by 36 cm wide. An intelligence panel on a wall of the enclosure was equipped with three response keys centered vertically 11.5 cm from the ceiling to the key center and 9 cm from each other (center to center). Thirty-four evenly spaced red light emitting diodes (LEDs) were centered 5 cm above the keys and 1.25 cm apart from each other (center to center) and protruded 0.3 cm into the enclosure. The LEDs (hereafter referred to as tokens) were always illuminated left to right, in sequential order. The presentation and removal of tokens was controlled by an electromechanical stepping switch (Lehigh Valley Electronics, model 1427, Lehigh Valley, PA), the operation of which also provided auditory feedback each time a token was presented or removed. Centered above the token array was a yellow house light that provided the enclosure with diffuse 5

PAGE 13

6 illumination. When active, side keys were illuminated green, and the center key was illuminated red. Pecks exceeding approximately 11 to 14 g of force were counted. A hopper aperture was centered 11.5 cm below the left key. A solenoid-operated food hopper could be raised into this opening, allowing access to mixed grain. Food presentation was accompanied by illumination of a yellow light located inside the opening. A photo-beam mounted across the aperture recorded head entry into the hopper. Continuous white noise and ventilation fans were active during experimental sessions to mask extraneous sounds. Experimental contingencies were programmed, and data collected, using a computer equipped with Med-pc software, located in a separate room. Procedure Because the pigeons had previous experience with token-reinforcement schedules, no training was necessary. A session began with the illumination of a green side key (key position counterbalanced across subjects). Responses on this key (designated the token-production key), were reinforced according to a FR schedule, that is, a fixed number of responses illuminated one token. A separate exchange schedule was arranged with respect to the number of tokens needed to produce an exchange period. During this exchange period, signaled by illumination of the red center key, a single response darkened the rightmost token and produced 2.5 s access to food (timed from head entry into hopper). The exchange period continued until all tokens earned during that cycle were exchanged, followed immediately by the illumination of the green side key and the beginning of the next cycle. Sessions continued until 48 tokens had been exchanged for food. (Because Pigeon 774 was consistently overweight, sessions from the last 5 conditions ended after only 32 tokens had been exchanged). Figure 1 is a procedural schematic of an FR 50 token production FR 2 exchange token-reinforcement schedule.

PAGE 14

7 All pigeons were exposed to a series of conditions in which FR values of both the token-production and exchange schedules were varied systematically across conditions. Conditions included FR exchange-schedule values of 2, 4, and 8 and FR token-production values of 25, 50, and 100. A variable ratio (VR) schedule 12.5 (on which reinforcement depended with equal probability on either 12 or 13 responses) was added as the lowest token-production schedule for Pigeon 1855 because response rates could not be maintained reliably at the highest ratios. Table 1 shows the sequence of conditions and the number of sessions conducted under each. (When reporting the results of this experiment, conditions are referred to by the values of the token-production and exchange schedules. The token exchange-schedule value is listed first, followed by the value of the token-production schedule listed in brackets. For example, a condition with an FR 50 token production FR 2 token exchange would be referred to by the abbreviation FR 2 [FR 50] or simply as 2 [50].) The experiment consisted of two parts. In Part 1 (constrained consumption conditions) pigeons were maintained at approximately 80% of their free-feeding weights. This was accomplished by holding constant the number of food deliveries each session at 48, and providing supplementary post-session feedings as needed. In Part 2 (unconstrained consumption conditions) sessions lasted until 10 min elapsed without a response. Because pigeons in this phase were usually above 80% weights, generally no supplementary feeding was needed. For some conditions in this phase for Pigeons 702 and 1855, not enough food was earned to maintain their respective 80% free feeding weights. Supplementary feedings were therefore provided. In three instances, when responding had weakened to the point that sessions were not being completed consistently, or 80% weight not maintained in unconstrained consumption

PAGE 15

8 conditions, conditions were changed arbitrarily in the absence of stability. Data from these conditions were not included in the analysis. Further, several conditions with no tokens were also run, but were not included in the present analysis. In Part 1 conditions, the token-production ratio and the exchange ratio were systematically varied across conditions. In Part 2 conditions, the token-exchange schedule was held constant at FR 2, while the token-production schedule was varied across conditions. During this phase a limited hold of 10 s was placed on hopper cycles, such that the amount of time the hopper was raised was restricted. The expiration of the 10 s limited hold was treated as a hopper entry, starting the 2.5 s food timer. Each condition was in place for a minimum of 20 sessions and until performance was deemed stable via visual inspection of daily response rates. For several conditions, due to a programming error, the initial-link latencies to respond were sometimes lost for given exchange cycles. In those instances where greater than 5% of the total initial-link latencies were lost from the last 5 sessions of a condition, the condition was replicated. For two of the subjects, Pigeons 774 and 1855, five replications were required (Table 2-1).

PAGE 16

9 Table 2-1. Order of conditions and number of sessions per condition listed as: exchange schedule [production schedule] (number of sessions) Pigeon 702 732 774 1855 Part 1 4[25] (48) 4[50] (23) 4[100] (36) 2[100] (21) 4[50] (24) 8[25] (28) 2[50] (27) 2[25] (25) 2[100] (25) 4[25] (29) 4[50] (56) 4[100] (23) 2[100] (53) 8[25] (25) 2[50] (22) 2[25] (22) 4[25] (57) 4[50] (38) 4[100] (37) 2[100] (33) 8[25] (41) 2[50] (19) 2[25] (20) 4[25] (64) 4[50] (33) 4[12.5] (28) 2[50] (84) 2[25] (20) 2[12.5] (20) 8[12.5] (38) Part 2 2[100] (70) a 2[25] (24) 2[50] (24) 2[100] (25)a 2[100] (30) 2[50] (28) 2[25] (28) 2[100] (40) 2[50] (19) 2[25] (36) 2[50] (23) a 2[25] (28) 2[12.5] (27) 2[50] (24) Part 1 2[50] (26) 4[100] (20) a 2[50] (33) 2[25] (21) 2[50] (24) 2[100] (24) 2[25] (22) b 4[25] (21) b 4[50] (21) b 4[100] (26) b 8[25] (20)b 2[50] (20) 4[50] (54) 2[50] (23) 2[25] (25) 2[12.5] (27) 4[25] (20) 4[12.5] (21) 8[12.5] (27) a Condition ended arbitrarily b 32 reinforcers per session

PAGE 17

10 After all tokens are exchanged the c y cle re p eats. A peck on the center key darkens one token and produces 2.5-s access to food. FR1 FR50 FR50 an exchange 2 tokens produce an 50 p ecks p roduce 1 token Figure 2-1. Sample of an FR 2 (FR 50) token-reinforcement schedule

PAGE 18

CHAPTER 3 RESULTS All data analyses are based on the last five sessions from each condition. Figure 3-1 shows responses per min from Part 1 conditions, plotted as a function of token-production ratio value (small, medium, and large), and as a function of exchange-ratio value within a given token-production ratio. Response rates tended to decrease as the token-production schedule increased. Under higher token-production ratios response rates also tended to decrease as the exchange schedule increased. For two pigeons, 732 and 774, replications of conditions were consistent with original exposures. For the other two pigeons, however, some subsequent exposures to conditions failed to replicate the initial findings. Replications occurred after an intervening history on unconstrained consumption conditions. In most of these instances, response rates increased when compared to initial exposures. Figure 3-2 shows responses per min across successive token-production segments for Part 1 conditions under all combinations of token-production and exchange schedules. A token segment is defined as the portion of an exchange cycle that occurs during the production of a given token. Graphs in the left, center, and right columns show response rates under FR 2, 4, and 8 exchange schedules, respectively. For all subjects, response rates were bi-valued, with lower initial-link rates giving way to higher, relatively constant rates in subsequent links. Under the FR 2 and FR 4 exchange ratios, the initial-link response rates were generally lowest in conditions with higher token-production ratios (FR 50 and FR 100). Under the FR 4 exchange ratio, rates under the highest token11

PAGE 19

12 production schedules tended to increase more gradually than under the smaller token-production ratios. Figure 3-3 shows for each token-production schedule, from Part 1 conditions, the pre-ratio pause across successive token-production segments under each exchange schedule. The data in this figure are organized as in Figure 3-2, with the graphs in each column showing data for a given exchange schedule and graphs in each row specific to an individual pigeon. For all pigeons pre-ratio pausing was longest in the initial link of the exchange ratio, and relatively short and undifferentiated thereafter. Within a given exchange ratio, initial-link pausing was directly related to the requirements of the token-production schedules. Increases in the exchange-schedule value from 2 to 4 also tended to produce increased initial-link pauses, but generally only when comparing conditions under the highest token-production value. For lower token-production FR values, increases in the exchange schedule either had no effect on initial-link pausing or produced small increases. Figures 3-4 and 3-5 show total response output and total consumption, respectively, for Part 2 conditions plotted as a function of unit price on log coordinates. For all subjects total response output tended to increase as unit price increased. Total consumption, however, decreased very little, if at all, with increases in unit price.

PAGE 20

13 702 2[25]4[25]8[25]2[50]2[50]4[50]4[50]2[100]2[100]4[100] RESPONSES PER MINUTE 04080120160200240 732 2[25]2[25]4[25]8[25]2[50]2[50]4[50]2[100]4[100] 04080120160200240 smallsmallsmallsmallmediummediummediummediumlargelargelargelarge1855CONDITION 2[12.5]2[12.5]4[12.5]4[12.5]8[12.5]8[12.5]2[25]2[25]4[25]4[25]2[50]2[50]2[50]4[50]4[50] 04080120160200240 774 2[25]2[25]4[25]4[25]8[25]8[25]2[50]2[50]4[50]4[50]2[100]2[100]4[100]4[100] 04080120160200240 Initial exposure replications Figure 3-1. Mean responses per minute and standard deviations for each pigeon under constrained consumption conditions plotted as a function of small, medium, and large token-production ratios, and as a function of exchange ratio.

PAGE 21

14 FR 2 FR 8 FR 4 300 702 200 small medium large 100 0 300 732 200 100 RESPONSES PER MINUTE 0 300 774 200 100 0 400 1855 300 200 100 0 1 2 2 4 1 2 3 4 6 8 SEGMENT Figure 3-2. Mean responses per minute plotted as a function of token-production segment for each pigeon under constrained consumption conditions. Open symbols represent initial exposures to a condition, filled symbols represent replications.

PAGE 22

15 FR 8 FR 2 FR 4 1000 702 100 Small Medium Large 10 1 0.1 1000 732 100 10 PRE-RATIO PAUSE (SEC) 1 0.1 1000 774 100 10 1 0.1 1000 1855 100 10 1 0.1 1 2 1 2 3 4 2 4 6 8 SEGMENT Figure 3-3. Mean latency plotted as a function of token-production segment for each pigeon under constrained consumption conditions. Open symbols represent initial exposures to a condition, filled symbols replications.

PAGE 23

16 10000 10000 702 732 1000 1000 TOTAL RESPONSES 10 40 10 40 10000 10000 774 1855 1000 1000 10 40 20 5 UNITPRICE Figure 3-4. Mean total responses for each pigeon under unconstrained consumption conditions plotted as a function of unit price.

PAGE 24

17 702 100 10 20 5 1855 100 10 40 10 732 40 10 774 40 10 100 TOTAL SECONDS FOOD ACCESS 10 100 10 UNIT PRICE Figure 3-5. Mean consumption (total seconds access to food) for each pigeon under unconstrained consumption conditions plotted as a function of unit price.

PAGE 25

CHAPTER 4 DISCUSSION The results of the present experiment are consistent with those previously reported on extended chain and second-order schedules with ratio components. Similar to Kelleher (1958), increases in the token-production ratio at a given exchange ratio decreased response rates. Similar to Foster et al. (2001), increases in the exchange schedule produced lower overall response rates, but generally only under the larger token-production ratios. In the context of the lowest token-production value, response rates varied less, if at all, with the exchange ratio. Further, for both token-production and exchange-schedule manipulations, decreases in response rates were primarily a result of longer pre-ratio pausing and weak behavior early in the ratio (see Figures 3-2 and 3-3), a finding also consistent with previous research (Foster et al., 2001; Kelleher, 1958; Webbe et al., 1978). Also similar to these previous findings, response rates were low, and pauses high, in early links of all exchange cycles. The present results also correspond to those reported by with extended chain and token schedules in regard to the gradually increasing rates seen under combinations of higher token-production and exchange-schedule values (Foster et al., 2001; and Jwaidah, 1973). The present results then both replicate and extend previous investigations with token-reinforcement procedures, manipulating the token-production and exchange schedules across a wider range of values than previously examined. The results also have implications for the unit price concept. Researchers have typically examined unit price with a closed economy, defined as one in which the total 18

PAGE 26

19 consumption of a reinforcer is limited by a subjects interaction with the contingencies. By contrast, an open economy is one in which total consumption is controlled by the experimenter. The majority of conditions in the present experiment involved an open economy in that the total consumption for a session was kept constant at 48 reinforcers. However, for 3 of 4 subjects, daily consumption within a session was sufficient to preclude post-session feedings, a feature generally consistent with closed economies. In that the majority of subjects consumption of food occurred solely via contact with the experimental contingencies, the economic conditions might be considered a functional closed economy, suggesting the potential applicability of a unit price analysis. The main dependent measures in unit price experiments are total consumption and response output (Hursh et. al, 1988; Hursh, 1980, 1984; Madden, Bickel, and Jacobs, 2000). Madden et. al (2000) noted two predictions of unit price. First, when the unit price of a reinforcer increases, one can expect decreases consumption of that reinforcer. Second, as unit price of a reinforcer is increased, overall responding increases to some peak, with further price increases decreasing responding. Stated differently, the functions relating total consumption and responding to unit price are negatively accelerated and bitonic, respectively. Data from Part 2 conditions were generally in accord with these predictions. Total response output varied directly with unit price in all cases. As Figures 3-4 and 3-5 show, total responding generally increased, while consumption remained constant or decreased, as unit price increased. The number of unconstrained consumption conditions included in this analysis is insufficient for an examination of the shape of the full function for these two measures.

PAGE 27

20 Because both consumption and responding were restricted in constrained consumption conditions, analysis of total consumption and responding as a function of unit price is not feasible. However, as Sumpter, Temple, and Foster (1999) point out, for contingencies where absolute consumption is restricted, consumption rate is still free to vary and may also be sensitive to unit price manipulations. In Sumpter et al. (1999), consumption rate was examined in sessions that ended after either 30 reinforcers had been delivered or 40 minutes had passed. Consumption rate proved an orderly measure and was found to vary as a function of unit price, similar to total consumption in other contexts. Similarly, response rates are also free to vary in conditions with restricted response output. In light of the successful use of unit price as an account of consumption rate in Sumpter et al. (1999), consumption rates were used in the analysis of constrained consumption conditions. Additionally, because total responding was restricted in the constrained consumption conditions, response rate was investigated as another measure potentially sensitive to unit price manipulations. Figure 4-1 shows consumption rate (s access to food per min) from Part 1 conditions plotted on log coordinates as a function unit price. Consumption rate decreased as unit price increased, a finding consistent with unit price predictions. As in Sumpter et al. (1999) consumption rate in this case therefore serves as a suitable proxy for total consumption with regard to unit price predictions. Under all exchange ratios, consumption rate generally decreased with increases in unit price. However, under higher exchange ratios, sharper decreases in consumption rate tended to occur with increases in the token-production schedule. For Part 1 conditions with higher exchange values, the function relating consumption rate to unit price has a steeply decreasing slope.

PAGE 28

21 However, for conditions with the same unit price, consumption rates sometimes varied inversely with token exchange-schedule value. This mirrors the variation in response rates mentioned earlier for these same conditions, and is not in strict accord with unit price predictions. As mentioned earlier, a literal version of the unit price concept predicts that two schedules of reinforcement with the same response-reinforcer ratios should engender equivalent response output, regardless of the particular response requirements or reinforcer amounts that comprise the ratio. In the present study, decreases in responses rates that occurred as a function of increasing the token-production schedule is consistent with the unit price formulation. Further, the lack of consistent exchange-schedule effects (where unit price is held constant) at lower token-production values is also in accord with the unit price equation. The rate decreases which occurred as a function of increases in the exchange schedule at higher token-production schedule values (where the cost benefit ratio remains the same), however, are not predicted by unit price. When attempting to account for departures from unit price predictions in his data set, Madden et al. (2000) found that a modification of the unit price concept made better ordinal predictions than a traditional unit price account. The modified unit price equation is given by P = (FR + H) / V (4-1) where P is modified unit price, FR is total number of responses prior to reinforcement, H is handling costs (in the present study, the number of responses to exchange each token, equal to the exchange-schedule value), and V is the reinforcer value (Mazur, 1987). The equation for reinforcer value is written

PAGE 29

22 V = A / 1 + kD (4-2) where A is the reinforcer amount, D is the reinforcer delay, and k is a scaling constant (set to 1 s-1 for the present analysis). A modified unit price analysis was conducted on response rates generated in the present experiment, with s of food access used for reinforcer amount and the average time from the illumination of the token-production key to exchange used for reinforcer delay. Figure 4-2 shows modified unit price and the reciprocal of responses per minute for each condition as a function of unit price. As with the traditional unit price formulation, response rates are expected to vary inversely with modified unit price. The reciprocal of responses per minute was thus used to allow ease of inspection, as the reciprocal measures would be expected to vary directly with modified unit price. Figure 4-2 is organized similar to Figure 2-1 in that conditions are plotted with respect to increasing unit price (Figure 2-1 is organized with respect to token-production value) and within unit price, as a function of increasing exchange-schedule values. The gray bars situated directly over each condition label represent the modified unit price for that condition. The black bar directly to the left of a given gray bar represents the reciprocal of responses per minute for that condition. The modified unit price equation allows for better ordinal predictions than a standard unit price account with respect to exchange-schedule manipulations. That is, within a given unit price, this formulation correctly predicts the direction of the variations in responding in the majority of cases. A standard unit price account, based on nominal programmed values, is silent with respect to such variations. Additional research is

PAGE 30

23 needed to determine the full extent to which unit price, and modified unit price, are useful metrics for token-reinforcement schedules. In summary, the present research replicated the results of token-production and exchange-schedule manipulations of previous token-reinforcement studies (Foster et. al., 2001; Kelleher, 1958) in showing that response rates vary systematically as a function of FR token-production and exchange schedules. Unit price does a reasonably good job in accounting for the effects of token-production manipulations in both constrained and Part 2 conditions. The modified unit price formulation, however, provides better ordinal predictions of the effects of exchange-schedule manipulations than a traditional unit price formulation.

PAGE 31

24 732 702 10 10 1 1 SECONDS FOOD ACCESS PER MINUTE 0 0 10 40 10 40 774 1855 10 10 1 1 FR 2 exchange FR 4 exchange FR 8 exchange 0 0 10 40 20 5 UNIT PRICE Figure 4-1. Mean consumption rate as a function of unit price for each pigeon under constrained consumption conditions plotted on log-log coordinates. Open symbols represent initial exposures to a condition, filled symbols replications.

PAGE 32

25 702 22ce48222ce44224 102103104105 1855 222ce4488222ce442222ce44 102103104105106 732 222ce48222ce422ce4 774EXCHANGE SCHEDULE 222ce4488222ce44222ce44 MODIFIED UNIT PRICE 102103104105 10204010102020201040405702 732 102103104105 0.000.020.040.06 774 0.000.020.040.060.080.10 1855 0.000.020.040.06 RECIPROCALRESPONSES PER MINUTE MODIFIED UNIT PRICE RECIPROCAL RESPONSES PER MINUTE .343 Figure 4-2. Mean reciprocal responses per minute and modified unit price for each pigeon plotted as a function of unit price and exchange schedule. Conditions denoted with the letters ce were run under a closed economy with unconstrained consumption.

PAGE 33

LIST OF REFERENCES DeGrandpre, R. J., Bickel, W. K., Hughes, J. R., Layng, M. P., & Badger, G. (1993) Unit price as a useful metric in analyzing effects of reinforcer magnitude. Journal of the Experimental Analysis of Behavior, 60, 641-666. Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token reinforcement with pigeons: Effects of fixedand variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 76, 159-178. Gollub, L. R. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Stadden (Eds.), Handbook of operant behavior (pp. 288-312) Englewood Cliffs, NJ: Prentice Hall. Hursh, S. R. (1978). The economics of daily consumption controlling foodand water-reinforced responding. Journal of the Experimental Analysis of Behavior, 29, 475-491. Hursh, S. R. (1980). Economic concepts for the analysis of behavior. Journal of the Experimental Analysis of Behavior, 34, 219-238. Hursh, S. R. (1984). Behavioral economics. Journal of the Experimental Analysis of Behavior, 42, 435-452. Hursh, S. R., Raslear, T. F., Shurtleff, D., Bauman, R., & Simmons, L. (1988). A cost-benefit analysis of demand for food. Journal of the Experimental Analysis of Behavior, 50, 419-440. Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281-289. Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475-485. Madden, G. J., Bickel, W. K. & Jacobs, E. A. (2000). Three predictions of the economic concept of unit price in a choice context. Journal of the Experimental Analysis of Behavior, 73, 45-64. Malagodi, E. F. (1967). Fixed-ratio schedules of token reinforcement. Psychonomic Science, 8, 469-470. 26

PAGE 34

27 Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 55-73). Hillsdale, NJ: Erlbaum. Sumpter, C. E., Temple, W., & Foster, T. M. (1999). The effects of differing response type and price manipulations on demand measures. Journal of the Experimental Analysis of Behavior, 71, 329-354. Waddell, T. R., Leander, J. D., Webbe, F. M., & Malagodi, E. F. (1972). Schedule interactions in second-order fixed-interval (fixed-ratio) schedules of token reinforcement. Learning and Motivation, 3, 91-100. Webbe, F. M., & Malagodi, E. F. (1978). Second-order schedules of token reinforcement: Comparisons of performance under fixed-ratio and variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 30, 219-224.

PAGE 35

BIOGRAPHICAL SKETCH Christopher Bullock graduated from J. F. Webb High school in the spring of 1994. He then enrolled at the University of North Carolina at Wilmington (UNCW) in the Fall of 1994. He graduated from UNCW in the spring of 1999 with a Bachelor of Arts in psychology with honors. The next fall he enrolled in the Behavior Analysis graduate studies program in the Department of Psychology at the University of Florida. He is presently continuing his education and conducting research in Behavior Analysis. 28


Permanent Link: http://ufdc.ufl.edu/UFE0000654/00001

Material Information

Title: Second-Order Schedules of Token Reinforcement: Combined Effects of Token-Production and Exchange-Schedule Manipulations
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0000654:00001

Permanent Link: http://ufdc.ufl.edu/UFE0000654/00001

Material Information

Title: Second-Order Schedules of Token Reinforcement: Combined Effects of Token-Production and Exchange-Schedule Manipulations
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0000654:00001


This item has the following downloads:


Full Text











SECOND-ORDER SCHEDULES OF TOKEN REINFORCEMENT: COMBINED
EFFECTS OF TOKEN-PRODUCTION AND EXCHANGE-SCHEDULE
MANIPULATIONS














By

CHRISTOPHER BULLOCK


A THESIS PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE

UNIVERSITY OF FLORIDA


2003

































Copyright 2003

by

Christopher Bullock















ACKNOWLEDGMENTS

I thank my parents for their support and encouragement throughout my graduate

studies. I would also like to thank my mentor, Timothy Hackenberg, for his

encouragement and guidance in the research and writing of this research project.
















TABLE OF CONTENTS
Page

A C K N O W L E D G M E N T S ......... ................................................................................... iii

LIST OF FIGURES .............. ... ................................ ......... ..v

ABSTRACT ........ .............. ............. ..... .......... .......... vi

CHAPTER

1 IN TR O D U C TIO N ......................................................................... .... .. ........

2 M E T H O D .............................................................................. 5

Su objects ...................................... .................................... .................... 5
A p p a ratu s ................................................................................. 5
P rocedu re .......................................................................... . 6


3 R E S U L T S ........................................................................................................1 1

4 D ISC U SSIO N ..................................................... 18

L IST O F R E FE R E N C E S .............................................................................. 26

B IO G R A PH IC A L SK E T C H ........................................................................................ 28
















LIST OF FIGURES


Figure page

2-1. Sample of an FR 2 (FR 50) token-reinforcement schedule............................... ..10

3-1. Mean responses per minute and standard deviations for each pigeon under
constrained consumption conditions plotted as a function of small, medium, and
large token-production ratios, and as a function of exchange ratio........................ 13

3-2. Mean responses per minute plotted as a function of token-production segment for
each pigeon under constrained consumption conditions ....................................14

3-3. Mean latency plotted as a function of token-production segment for each pigeon
under constrained consumption conditions. .................................. .................15

3-4. Mean total responses for each pigeon under unconstrained consumption conditions
plotted as a function of unit price. ............................. .... ..................... 16

3-5. Mean consumption (total seconds access to food) for each pigeon under
unconstrained consumption conditions plotted as a function of unit price.............. 17

4-1. Mean consumption rate as a function of unit price for each pigeon under constrained
consumption conditions plotted on log-log coordinates. .......................................24

4-2. Mean reciprocal responses per minute and modified unit price for each pigeon
plotted as a function of unit price and exchange schedule .............. ...............25
















Abstract of Thesis Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Master of Science

SECOND-ORDER SCHEDULES OF TOKEN REINFORCEMENT: COMBINED
EFFECTS OF TOKEN-PRODUCTION AND EXCHANGE-SCHEDULE
MANIPULATIONS

By

Christopher Bullock

May 2003

Chair: Timothy D. Hackenberg
Major Department: Psychology

Four pigeons were exposed to second-order schedules of token reinforcement,

with stimulus lights serving as token reinforcers. Tokens were earned according to a

fixed-ratio (token-production) schedule and were exchanged for 2.5 s access to food

according to a fixed-ratio (token-exchange) schedule. The token-production and token-

exchange schedules were manipulated systematically across conditions. Response rates

varied inversely with the token-production schedule for a given token-exchange schedule

value. Response rates also varied inversely with the token-exchange schedule for a given

token-production value, particularly at the higher token-production ratios. Further, under

higher token-production and exchange-schedule values, response rates tended to increase

in token-production segments closer to exchange periods and food. Several probe

conditions were also studied in a closed economy that permitted unconstrained food

consumption. Under these conditions, response rates were less sensitive to token-

production schedule manipulations than under standard (constrained consumption)









conditions in which sessions were limited to 48 food presentations. Results were

analyzed using a modification of the behavioral economic concept of unit price (a cost-

benefit ratio comprising responses per unit of food delivery).














CHAPTER 1
INTRODUCTION

A second-order schedule of reinforcement is one in which a pattern of behavior

reinforced according to one schedule is treated as a unitary response reinforced according

to a second schedule (Kelleher, 1966). A token-reinforcement schedule is a second-order

schedule in which responses produce tokens according to one schedule (the token-

production schedule) and opportunities to exchange those tokens for primary

reinforcement according to a second schedule (the exchange schedule) (Kelleher, 1958;

Malagodi, 1967).

Previous research has shown that response rates and patterns maintained under

token-reinforcement schedules vary systematically as a function of both the token-

production schedules (Kelleher, 1958) and the exchange schedules (Foster, Hackenberg

& Vaidya, 2001; Waddell, Leander, Webbe, & Malagodi, 1972; Webbe & Malagodi,

1978). Most research on token-reinforcement schedules has involved fixed-ratio (FR)

components (schedules that require a specified number of responses for reinforcement).

Similar to FR performance reinforced with food, response rates under FR token-

production and exchange schedules vary inversely with the FR value (Foster, et al., 2001;

Kelleher, 1958; Malagodi, 1967; Webbe & Malagodi, 1978;). Kelleher (1958), for

example, found that chimpanzees' rates of lever pressing decreased as the FR token-

production requirement increased from 30 to 100 with the exchange ratio held constant at

FR 60. Foster et al. (2001) found that pigeons' response rates decreased as the FR

exchange schedule increased from 1 to 8 with the token-production ratio held constant at









FR 50. In both studies, response-rate decrements were mainly a function of increased

pausing early in an FR cycle.

With FR token-production and exchange schedules, a potentially relevant variable

emerges: the ratio of responses per unit of reinforcer, closely related to the economic

concept of unit price (DeGrandpre, Bickel, Hughes, Layng, & Badger, 1993; Hursh,

1978). When the exchange ratio is manipulated with token-production ratio held

constant, changes in reinforcer magnitude are proportional to changes in exchange ratio.

As a result, the unit price remains constant across variations in the exchange schedule.

For example, in the Foster et al. (2001) study, with a token-production ratio of 50 and an

exchange ratio of 2, FR 2 [FR 50], to use the standard nomenclature,100 responses

produced 2 tokens (each exchangeable for 2 s access to food), or 25 responses per second

access to food. So too under the FR 8 [FR 50] condition, 400 responses produced 8

tokens (16 s access to food), or 25 responses per second access to food. According to a

literal version of the unit price concept, response output should be roughly constant under

such conditions of equivalent unit price, without regard to the specific values of its cost

and benefit components(Madden, Bickel, & Jacobs, 2000). Contrary to these predictions,

however, response output varied substantially across conditions of equal unit price.

Responding then was governed not by responses per unit of food delivery (unit price) but

by responses per exchange period (exchange ratio).

Another way of examining the viability of the unit price concept with token-

reinforcement procedures is through manipulations of the token-production ratio. Unlike

exchange-ratio manipulations, altering the token-production ratio while holding constant

the exchange ratio changes the unit price. For example, in the Kelleher (1958) study,









with an FR 30 token-production schedule and an FR 60 exchange schedule, 1800

responses produced 60 tokens, a unit price of 30. When the token-production schedule

was increased to 100, while holding constant the exchange schedule ratio at 60, 6000

responses produced 60 tokens, a unit price of 100. In accordance with unit price

predictions, rates of lever pressing decreased as the FR requirement and unit price of the

token-production schedule increased. Thus, unlike exchange-ratio manipulations, in

which unit price is invariant with respect to changes in ratio requirements; with token-

production manipulations, unit price changes with changes in ratio requirements.

The purpose of the present experiment was to clarify how both the token-

production schedule and exchange schedule contribute to maintaining behavior on token-

reinforcement schedules while assessing the applicability of unit price in accounting for

this behavior. Although response rates and patterns in token-reinforcement procedures

have been shown to vary with the values of both FR token-production and exchange

schedules, previous studies have typically manipulated only one of these variables while

holding the other constant. The effects of manipulating one of the schedules in a token-

reinforcement procedure may depend on the others value. Using a preparation similar to

that of Foster et al. (2001), with stimulus lights serving as token reinforcers, FR token-

production and exchange schedules were parametrically manipulated on a within-subject

basis across a wider range of schedule values than examined in prior research, allowing

for a more detailed analysis of their effects. The present experiment included three

token-production values (small, medium, and large) and three exchange values (2, 4, and

8); and was designed such that most of the possible combinations of these two schedules

were experienced by the subjects.









Unit price experiments are typically conducted in a closed economy, where all food

is obtained via interaction with the experimental conditions (Hursh, 1980). A closed

economy is often contrasted with an open economy, where supplemental feeding occurs

outside of experimental sessions. In the Foster et al. (2001) study, birds received all or

nearly all of their daily food intake within sessions (functionally a closed economy).

However, because supplementary feeding was sometimes necessary, it can be argued that

this was not a genuine closed economy, rendering a unit-price analysis inappropriate. In

the present study, therefore, three probe conditions were conducted under a closed

economy where all food was obtained via interaction with experimental contingencies.

Parametric evaluation of token-production and exchange ratios in closed and open

economic contexts allows for a stronger investigation of the unit price concept and for an

extension of previous token-reinforcement studies. Unit price considerations aside, the

study had as its main objective the extension of previous work in the realm of token-

reinforcement procedures. The results should further clarify the contributions of token-

production and exchange schedules to performance on token-reinforcement procedures.














CHAPTER 2
METHOD

Subjects

Four male White Carneau pigeons (Columba livia) (numbered 702, 732, 774,

1855) served as subjects. Each pigeon had prior experience with token-reinforcement

schedules. Subjects were housed individually under a 16.5 hour / 7.5 hour light dark

cycle and had continuous access to water and health grit in home cages. Pigeons were

maintained at approximately 20g of their 80% free feeding weights for most conditions of

the experiment. In a few conditions (see below), the weight restrictions were removed,

such that weights were free to vary above the 80% level.

Apparatus

Experimental sessions were conducted in an enclosure 36 cm high by 50 cm long

by 36 cm wide. An intelligence panel on a wall of the enclosure was equipped with

three response keys centered vertically 11.5 cm from the ceiling to the key center and 9

cm from each other (center to center). Thirty-four evenly spaced red light emitting

diodes (LEDs) were centered 5 cm above the keys and 1.25 cm apart from each other

(center to center) and protruded 0.3 cm into the enclosure. The LEDs (hereafter referred

to as tokens) were always illuminated left to right, in sequential order. The presentation

and removal of tokens was controlled by an electromechanical stepping switch (Lehigh

Valley Electronics, model 1427, Lehigh Valley, PA), the operation of which also

provided auditory feedback each time a token was presented or removed. Centered above

the token array was a yellow house light that provided the enclosure with diffuse









illumination. When active, side keys were illuminated green, and the center key was

illuminated red. Pecks exceeding approximately 11 to 14 g of force were counted. A

hopper aperture was centered 11.5 cm below the left key. A solenoid-operated food

hopper could be raised into this opening, allowing access to mixed grain. Food

presentation was accompanied by illumination of a yellow light located inside the

opening. A photo-beam mounted across the aperture recorded head entry into the hopper.

Continuous white noise and ventilation fans were active during experimental sessions to

mask extraneous sounds. Experimental contingencies were programmed, and data

collected, using a computer equipped with Med-pc software, located in a separate room.

Procedure

Because the pigeons had previous experience with token-reinforcement schedules,

no training was necessary. A session began with the illumination of a green side key

(key position counterbalanced across subjects). Responses on this key (designated the

token-production key), were reinforced according to a FR schedule, that is, a fixed

number of responses illuminated one token. A separate exchange schedule was arranged

with respect to the number of tokens needed to produce an exchange period. During this

exchange period, signaled by illumination of the red center key, a single response

darkened the rightmost token and produced 2.5 s access to food (timed from head entry

into hopper). The exchange period continued until all tokens earned during that cycle

were exchanged, followed immediately by the illumination of the green side key and the

beginning of the next cycle. Sessions continued until 48 tokens had been exchanged for

food. (Because Pigeon 774 was consistently overweight, sessions from the last 5

conditions ended after only 32 tokens had been exchanged). Figure 1 is a procedural

schematic of an FR 50 token production FR 2 exchange token-reinforcement schedule.









All pigeons were exposed to a series of conditions in which FR values of both the

token-production and exchange schedules were varied systematically across conditions.

Conditions included FR exchange-schedule values of 2, 4, and 8 and FR token-

production values of 25, 50, and 100. A variable ratio (VR) schedule 12.5 (on which

reinforcement depended with equal probability on either 12 or 13 responses) was added

as the lowest token-production schedule for Pigeon 1855 because response rates could

not be maintained reliably at the highest ratios. Table 1 shows the sequence of conditions

and the number of sessions conducted under each. (When reporting the results of this

experiment, conditions are referred to by the values of the token-production and

exchange schedules. The token exchange-schedule value is listed first, followed by the

value of the token-production schedule listed in brackets. For example, a condition with

an FR 50 token production FR 2 token exchange would be referred to by the abbreviation

FR 2 [FR 50] or simply as 2 [50].) The experiment consisted of two parts. In Part 1

(constrained consumption conditions) pigeons were maintained at approximately 80% of

their free-feeding weights. This was accomplished by holding constant the number of

food deliveries each session at 48, and providing supplementary post-session feedings as

needed. In Part 2 (unconstrained consumption conditions) sessions lasted until 10 min

elapsed without a response. Because pigeons in this phase were usually above 80%

weights, generally no supplementary feeding was needed. For some conditions in this

phase for Pigeons 702 and 1855, not enough food was earned to maintain their respective

80% free feeding weights. Supplementary feedings were therefore provided. In three

instances, when responding had weakened to the point that sessions were not being

completed consistently, or 80% weight not maintained in unconstrained consumption









conditions, conditions were changed arbitrarily in the absence of stability. Data from

these conditions were not included in the analysis. Further, several conditions with no

tokens were also run, but were not included in the present analysis.

In Part 1 conditions, the token-production ratio and the exchange ratio were

systematically varied across conditions. In Part 2 conditions, the token-exchange

schedule was held constant at FR 2, while the token-production schedule was varied

across conditions. During this phase a limited hold of 10 s was placed on hopper cycles,

such that the amount of time the hopper was raised was restricted. The expiration of the

10 s limited hold was treated as a hopper entry, starting the 2.5 s food timer.

Each condition was in place for a minimum of 20 sessions and until performance

was deemed stable via visual inspection of daily response rates. For several conditions,

due to a programming error, the initial-link latencies to respond were sometimes lost for

given exchange cycles. In those instances where greater than 5% of the total initial-link

latencies were lost from the last 5 sessions of a condition, the condition was replicated.

For two of the subjects, Pigeons 774 and 1855, five replications were required (Table 2-

1).









Table 2-1. Order of conditions and number of sessions per condition listed as:
exchange schedule [production schedule] (number of sessions)
Pigeon
702 732 774 1855
Part 1


4[25]
4[50]
4[100]
2[100]
4[50]
8[25]
2[50]
2[25]
2[100]
Part 2
2[100]
2[25]
2[50]
2[100]
Part 1
2[50]
4[100]


(48)
(23)
(36)
(21)
(24)
(28)
(27)
(25)
(25)

(70)a
(24)
(24)
(25)a

(26)
(20)a


4[25]
4[50]
4[100]
2[100]
8[25]
2[50]
2[25]



2[100]
2[50]
2[25]


2[50]
2[25]


(29)
(56)
(23)
(53)
(25)
(22)
(22)


4[25]
4[50]
4[100]
2[100]
8[25]
2[50]
2[25]


(30)
(28)
(28)



(33)
(21)


2[100]
2[50]
2[25]


2[50]
2[100]
2[25]
4[25]
4[50]
4[100]
8[25]


(57)
(38)
(37)
(33)
(41)
(19)
(20)



(40)
(19)
(36)


(24)
(24)
(22)b
(21)b
(21)b
(26)b
(20)b


a Condition ended arbitrarily
b 32 reinforcers per session


4[25]
4[50]
4[12.5]
2[50]
2[25]
2[12.5]
8[12.5]



2[50]
2[25]
2[12.5]
2[50]

2[50]
4[50]
2[50]
2[25]
2[12.5]
4[25]
4[12.5]
8r12.51


(64)
(33)
(28)
(84)
(20)
(20)
(38)



(23)a
(28)
(27)
(24)

(20)
(54)
(23)
(25)
(27)
(20)
(21)
(27)












FR50



50 pecks produce
1 token



FR50




2 tokens produce an
an exchange


FR1


A peck on the center key darkens one token
and produces 2.5-s access to food.




After all tokens are exchanged
the cycle repeats.


Figure 2-1. Sample of an FR 2 (FR 50) token-reinforcement schedule














CHAPTER 3
RESULTS

All data analyses are based on the last five sessions from each condition. Figure 3-

1 shows responses per min from Part 1 conditions, plotted as a function of token-

production ratio value (small, medium, and large), and as a function of exchange-ratio

value within a given token-production ratio. Response rates tended to decrease as the

token-production schedule increased. Under higher token-production ratios response rates

also tended to decrease as the exchange schedule increased. For two pigeons, 732 and

774, replications of conditions were consistent with original exposures. For the other two

pigeons, however, some subsequent exposures to conditions failed to replicate the initial

findings. Replications occurred after an intervening history on unconstrained

consumption conditions. In most of these instances, response rates increased when

compared to initial exposures.

Figure 3-2 shows responses per min across successive token-production segments

for Part 1 conditions under all combinations of token-production and exchange schedules.

A token segment is defined as the portion of an exchange cycle that occurs during the

production of a given token. Graphs in the left, center, and right columns show response

rates under FR 2, 4, and 8 exchange schedules, respectively. For all subjects, response

rates were bi-valued, with lower initial-link rates giving way to higher, relatively constant

rates in subsequent links. Under the FR 2 and FR 4 exchange ratios, the initial-link

response rates were generally lowest in conditions with higher token-production ratios

(FR 50 and FR 100). Under the FR 4 exchange ratio, rates under the highest token-









production schedules tended to increase more gradually than under the smaller token-

production ratios.

Figure 3-3 shows for each token-production schedule, from Part 1 conditions, the

pre-ratio pause across successive token-production segments under each exchange

schedule. The data in this figure are organized as in Figure 3-2, with the graphs in each

column showing data for a given exchange schedule and graphs in each row specific to

an individual pigeon. For all pigeons pre-ratio pausing was longest in the initial link of

the exchange ratio, and relatively short and undifferentiated thereafter. Within a given

exchange ratio, initial-link pausing was directly related to the requirements of the token-

production schedules. Increases in the exchange-schedule value from 2 to 4 also tended

to produce increased initial-link pauses, but generally only when comparing conditions

under the highest token-production value. For lower token-production FR values,

increases in the exchange schedule either had no effect on initial-link pausing or

produced small increases.

Figures 3-4 and 3-5 show total response output and total consumption,

respectively, for Part 2 conditions plotted as a function of unit price on log coordinates.

For all subjects total response output tended to increase as unit price increased. Total

consumption, however, decreased very little, if at all, with increases in unit price.







13






240 -
200 702 small medium large
160 -
120 -
80-
40

2[25] 4[25] 8[25] 2[50] 2[50] 4[50] 4[50] 2[100] 2[100] 4[100]

240 732 small medium rge
200 732 large
W 160-
S 120-
z 80
2 40
LW 0
0- 2[25] 2[25] 4[25] 8[25] 2[50] 2[50] 4[50] 2[100] 4[100]
) 240 -
LU 772 small medium large
(0 200 774
O 160 Initial exposure
0a replications
S 120
WL 80
40
0-
2[25] 2[25] 4[25] 4[25] 8[25] 8[25] 2[50] 2[50] 4[50] 4[50] 2[100] 2[100] 4[100] 4[100]

240 small medium large
2001- 1855
160
120
80

0
2[125] 2[125] 4[125] 4[125] 8[125] 8[125] 2[25] 2[25] 4[25] 4[25] 2[50] 2[50] 2[50] 4[50] 4[50]

CONDITION


Figure 3-1. Mean responses per minute and standard deviations for each pigeon under
constrained consumption conditions plotted as a function of small, medium,
and large token-production ratios, and as a function of exchange ratio.












FR 4


1 2 1 2 3 4
SEGMENT


S --- small
-A- medium
--- large


2 4 6 8


Figure 3-2. Mean responses per minute plotted as a function of token-production
segment for each pigeon under constrained consumption conditions. Open
symbols represent initial exposures to a condition, filled symbols represent
replications.


200

100

0

300
732

200

100

0

300
774

200

100

0












FR2
1000
702
100

10

1

0.1

1000
732
100

10



0.1

1000
774
100

10

1

0.1

1000
1855
100

10



0.1
1 2


FR4


1 2 3 4
SEGMENT


FR8


-0- Small
-A- Medium
--- Large


2 4 6 8


Figure 3-3. Mean latency plotted as a function of token-production segment for each
pigeon under constrained consumption conditions. Open symbols represent
initial exposures to a condition, filled symbols replications.








16



10000- 10000
702 732











CO 1000 1000
LU 10 40 10 40
CO
z
0
09
CO
LU
10000- 10000
,<
S774 1855











1000 1000
10 40 5 20



UNIT PRICE


Figure 3-4. Mean total responses for each pigeon under unconstrained consumption
conditions plotted as a function of unit price.
H-




















conditions plotted as a function of unit price.

























10








100 -


10 10 4
10 40


732


410
40 10


* 1855


UNIT PRICE
Figure 3-5. Mean consumption (total seconds access to food) for each pigeon under
unconstrained consumption conditions plotted as a function of unit price.














CHAPTER 4
DISCUSSION

The results of the present experiment are consistent with those previously reported

on extended chain and second-order schedules with ratio components. Similar to

Kelleher (1958), increases in the token-production ratio at a given exchange ratio

decreased response rates. Similar to Foster et al. (2001), increases in the exchange

schedule produced lower overall response rates, but generally only under the larger

token-production ratios. In the context of the lowest token-production value, response

rates varied less, if at all, with the exchange ratio. Further, for both token-production and

exchange-schedule manipulations, decreases in response rates were primarily a result of

longer pre-ratio pausing and weak behavior early in the ratio (see Figures 3-2 and 3-3), a

finding also consistent with previous research (Foster et al., 2001; Kelleher, 1958; Webbe

et al., 1978). Also similar to these previous findings, response rates were low, and pauses

high, in early links of all exchange cycles. The present results also correspond to those

reported by with extended chain and token schedules in regard to the gradually increasing

rates seen under combinations of higher token-production and exchange-schedule values

(Foster et al., 2001; and Jwaidah, 1973). The present results then both replicate and

extend previous investigations with token-reinforcement procedures, manipulating the

token-production and exchange schedules across a wider range of values than previously

examined.

The results also have implications for the unit price concept. Researchers have

typically examined unit price with a closed economy, defined as one in which the total









consumption of a reinforcer is limited by a subject's interaction with the contingencies.

By contrast, an open economy is one in which total consumption is controlled by the

experimenter. The majority of conditions in the present experiment involved an open

economy in that the total consumption for a session was kept constant at 48 reinforcers.

However, for 3 of 4 subjects, daily consumption within a session was sufficient to

preclude post-session feedings, a feature generally consistent with closed economies. In

that the majority of subjects' consumption of food occurred solely via contact with the

experimental contingencies, the economic conditions might be considered a functional

closed economy, suggesting the potential applicability of a unit price analysis.

The main dependent measures in unit price experiments are total consumption and

response output (Hursh et. al, 1988; Hursh, 1980, 1984; Madden, Bickel, and Jacobs,

2000). Madden et. al (2000) noted two predictions of unit price. First, when the unit

price of a reinforcer increases, one can expect decreases consumption of that reinforcer.

Second, as unit price of a reinforcer is increased, overall responding increases to some

peak, with further price increases decreasing responding. Stated differently, the functions

relating total consumption and responding to unit price are negatively accelerated and

bitonic, respectively.

Data from Part 2 conditions were generally in accord with these predictions.

Total response output varied directly with unit price in all cases. As Figures 3-4 and 3-5

show, total responding generally increased, while consumption remained constant or

decreased, as unit price increased. The number of unconstrained consumption conditions

included in this analysis is insufficient for an examination of the shape of the full

function for these two measures.









Because both consumption and responding were restricted in constrained

consumption conditions, analysis of total consumption and responding as a function of

unit price is not feasible. However, as Sumpter, Temple, and Foster (1999) point out, for

contingencies where absolute consumption is restricted, consumption rate is still free to

vary and may also be sensitive to unit price manipulations. In Sumpter et al. (1999),

consumption rate was examined in sessions that ended after either 30 reinforcers had

been delivered or 40 minutes had passed. Consumption rate proved an orderly measure

and was found to vary as a function of unit price, similar to total consumption in other

contexts. Similarly, response rates are also free to vary in conditions with restricted

response output. In light of the successful use of unit price as an account of consumption

rate in Sumpter et al. (1999), consumption rates were used in the analysis of constrained

consumption conditions. Additionally, because total responding was restricted in the

constrained consumption conditions, response rate was investigated as another measure

potentially sensitive to unit price manipulations.

Figure 4-1 shows consumption rate (s access to food per min) from Part 1

conditions plotted on log coordinates as a function unit price. Consumption rate

decreased as unit price increased, a finding consistent with unit price predictions. As in

Sumpter et al. (1999) consumption rate in this case therefore serves as a suitable proxy

for total consumption with regard to unit price predictions. Under all exchange ratios,

consumption rate generally decreased with increases in unit price. However, under

higher exchange ratios, sharper decreases in consumption rate tended to occur with

increases in the token-production schedule. For Part 1 conditions with higher exchange

values, the function relating consumption rate to unit price has a steeply decreasing slope.









However, for conditions with the same unit price, consumption rates sometimes varied

inversely with token exchange-schedule value. This mirrors the variation in response

rates mentioned earlier for these same conditions, and is not in strict accord with unit

price predictions.

As mentioned earlier, a literal version of the unit price concept predicts that two

schedules of reinforcement with the same response-reinforcer ratios should engender

equivalent response output, regardless of the particular response requirements or

reinforcer amounts that comprise the ratio. In the present study, decreases in responses

rates that occurred as a function of increasing the token-production schedule is consistent

with the unit price formulation. Further, the lack of consistent exchange-schedule effects

(where unit price is held constant) at lower token-production values is also in accord with

the unit price equation. The rate decreases which occurred as a function of increases in

the exchange schedule at higher token-production schedule values (where the cost benefit

ratio remains the same), however, are not predicted by unit price.

When attempting to account for departures from unit price predictions in his data

set, Madden et al. (2000) found that a modification of the unit price concept made better

ordinal predictions than a traditional unit price account. The modified unit price equation

is given by

P = (FR + H)/ V (4-1)

where P is modified unit price, FR is total number of responses prior to reinforcement, H

is handling costs (in the present study, the number of responses to exchange each token,

equal to the exchange-schedule value), and V is the reinforcer value (Mazur, 1987). The

equation for reinforcer value is written









V = A/ 1 +kD (4-2)

where A is the reinforcer amount, D is the reinforcer delay, and k is a scaling constant

(set to 1s-1 for the present analysis). A modified unit price analysis was conducted on

response rates generated in the present experiment, with s of food access used for

reinforcer amount and the average time from the illumination of the token-production key

to exchange used for reinforcer delay.

Figure 4-2 shows modified unit price and the reciprocal of responses per minute for

each condition as a function of unit price. As with the traditional unit price formulation,

response rates are expected to vary inversely with modified unit price. The reciprocal of

responses per minute was thus used to allow ease of inspection, as the reciprocal

measures would be expected to vary directly with modified unit price. Figure 4-2 is

organized similar to Figure 2-1 in that conditions are plotted with respect to increasing

unit price (Figure 2-1 is organized with respect to token-production value) and within

unit price, as a function of increasing exchange-schedule values. The gray bars situated

directly over each condition label represent the modified unit price for that condition.

The black bar directly to the left of a given gray bar represents the reciprocal of responses

per minute for that condition.

The modified unit price equation allows for better ordinal predictions than a

standard unit price account with respect to exchange-schedule manipulations. That is,

within a given unit price, this formulation correctly predicts the direction of the variations

in responding in the majority of cases. A standard unit price account, based on nominal

programmed values, is silent with respect to such variations. Additional research is









needed to determine the full extent to which unit price, and modified unit price, are useful

metrics for token-reinforcement schedules.

In summary, the present research replicated the results of token-production and

exchange-schedule manipulations of previous token-reinforcement studies (Foster et. al.,

2001; Kelleher, 1958) in showing that response rates vary systematically as a function of

FR token-production and exchange schedules. Unit price does a reasonably good job in

accounting for the effects of token-production manipulations in both constrained and Part

2 conditions. The modified unit price formulation, however, provides better ordinal

predictions of the effects of exchange-schedule manipulations than a traditional unit price

formulation.

































10 40








774







0



A


10 40








1855



A





O FRexchange

0 FR 2 exchange
A FR 8 exchange
E FR 8 exchange
A


UNIT PRICE




Figure 4-1. Mean consumption rate as a function of unit price for each pigeon under
constrained consumption conditions plotted on log-log coordinates. Open
symbols represent initial exposures to a condition, filled symbols replications.


702
o
[O


732

o o


0


A





























2 2ce 4 8 2 2 2ce 4 4 2 2 4



774

I ITI


10 20 40













I lIllI ll I


-008


- 006


-004


002

102
000


2 22ce4 4 8 8 2 22ce4 4 2 22ce4 4


105.

10-


104-




103.




102






0 10106 -


2 2 2ce 4 8 2 2 2ce 4 2 2ce 4



1855


5


10













jjIl 1V


343
20


/




II


2 22ce4 4 8 8 2 22ce4 4 2 2 22ce4 4


EXCHANGE SCHEDULE
I MODIFIED UNIT PRICE
I RECIPROCAL RESPONSES PER MINUTE


Figure 4-2. Mean reciprocal responses per minute and modified unit price for each
pigeon plotted as a function of unit price and exchange schedule. Conditions
denoted with the letters ce were run under a closed economy with
unconstrained consumption.


IinIllIHI I II


m
000 U-
0a
ooo
C-0
m 0

CP
z
c
m-I
M


006


-004


-002


'' "* *' *


"


,l,


,n ,nl, ,H ll ,


- 002
















LIST OF REFERENCES


DeGrandpre, R. J., Bickel, W. K., Hughes, J. R., Layng, M. P., & Badger, G. (1993).
Unit price as a useful metric in analyzing effects of reinforcer magnitude. Journal
of the Experimental Analysis of Behavior, 60, 641-666.

Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of
token reinforcement with pigeons: Effects of fixed- and variable-ratio exchange
schedules. Journal of the Experimental Analysis of Behavior, 76, 159-178.

Gollub, L. R. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig &
J. E. R. Stadden (Eds.), Handbook of operant behavior (pp. 288-312) Englewood
Cliffs, NJ: Prentice Hall.

Hursh, S. R. (1978). The economics of daily consumption controlling food- and water-
reinforced responding. Journal of the Experimental Analysis of Behavior, 29, 475-
491.

Hursh, S. R. (1980). Economic concepts for the analysis of behavior. Journal of the
Experimental Analysis of Behavior, 34, 219-238.

Hursh, S. R. (1984). Behavioral economics. Journal of the Experimental Analysis of
Behavior, 42, 435-452.

Hursh, S. R., Raslear, T. F., Shurtleff, D., Bauman, R., & Simmons, L. (1988). A cost-
benefit analysis of demand for food. Journal of the Experimental Analysis of
Behavior, 50, 419-440.

Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with
chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281-289.

Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of
the Experimental Analysis of Behavior, 9, 475-485.

Madden, G. J., Bickel, W. K. & Jacobs, E. A. (2000). Three predictions of the economic
concept of unit price in a choice context. Journal of the Experimental Analysis of
Behavior, 73, 45-64.

Malagodi, E. F. (1967). Fixed-ratio schedules of token reinforcement. Psychonomic
Science, 8, 469-470.









Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M.
L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis
of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement
value (pp. 55-73). Hillsdale, NJ: Erlbaum.

Sumpter, C. E., Temple, W., & Foster, T. M. (1999). The effects of differing response
type and price manipulations on demand measures. Journal of the Experimental
Analysis of Behavior, 71, 329-354.

Waddell, T. R., Leander, J. D., Webbe, F. M., & Malagodi, E. F. (1972). Schedule
interactions in second-order fixed-interval (fixed-ratio) schedules of token
reinforcement. Learning and Motivation, 3, 91-100.

Webbe, F. M., & Malagodi, E. F. (1978). Second-order schedules of token
reinforcement: Comparisons of performance under fixed-ratio and variable-ratio
exchange schedules. Journal of the Experimental Analysis of Behavior, 30, 219-
224.















BIOGRAPHICAL SKETCH

Christopher Bullock graduated from J. F. Webb High school in the spring of 1994.

He then enrolled at the University of North Carolina at Wilmington (UNCW) in the Fall

of 1994. He graduated from UNCW in the spring of 1999 with a Bachelor of Arts in

psychology with honors. The next fall he enrolled in the Behavior Analysis graduate

studies program in the Department of Psychology at the University of Florida. He is

presently continuing his education and conducting research in Behavior Analysis.