<%BANNER%>

Some Effects of Economic Context on Pigeons' Choices in Situations of Diminishing Returns

Permanent Link: http://ufdc.ufl.edu/UFE0024394/00001

Material Information

Title: Some Effects of Economic Context on Pigeons' Choices in Situations of Diminishing Returns
Physical Description: 1 online resource (86 p.)
Language: english
Creator: Yankelevitz, Rachelle
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: choice, diminishing, foraging, optimal, pigeons, reinforcement, returns, token
Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Pigeons choices were studied in a laboratory-based situation of diminishing returns, in which the rate of gain steadily decreased with time spent in the situation. The study was designed as a laboratory analogue of a depleting patch of resources like that faced by an animal foraging in the wild, and to test the predictions of three optimization models: a global-maximization model, a local-maximization model, and an intermediate-term maximization model based on the sum of reciprocals of upcoming delays (the summed-immediacies model). Pigeons chose between fixed-interval (FI) schedules and progressive-interval (PI) schedules of token reinforcement (illumination of a stimulus light). Immediate-exchange conditions were similar to procedures typically used with pigeons: each token was exchanged for brief access to food immediately after it was earned. In delayed-exchange conditions, 11 tokens were earned before an exchange phase occurred. Delayed-exchange conditions were analogous to procedures more commonly used with human subjects, in which a fixed number of tokens are earned before exchanging them for other reinforcers. Three experiments were conducted, each of which systematically manipulated a variable to more fully distinguish between the predictions of the models: FI duration (Experiment 1), PI incrementation size (Experiment 2), and inter-trial interval (Experiment 3). Across experiments, choices in immediate-exchange conditions were in better accord with the predictions of the summed-immediacies model, conforming to previous results with pigeons, whereas choices in delayed-exchange conditions were in better accord with the global-maximization model, conforming to previous results with humans. In delayed-exchange conditions, trials farthest from upcoming exchange periods resulted in lower response rates, higher latencies, and a higher probability of choosing the FI schedule compared to trials near exchange, reducing overall reinforcement rate relative to immediate-exchange conditions. Differences in choices between exchange types suggest that decision rules are influenced by the economic context in which they are used, although these differences are modulated by local schedule effects.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Rachelle Yankelevitz.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Hackenberg, Timothy D.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024394:00001

Permanent Link: http://ufdc.ufl.edu/UFE0024394/00001

Material Information

Title: Some Effects of Economic Context on Pigeons' Choices in Situations of Diminishing Returns
Physical Description: 1 online resource (86 p.)
Language: english
Creator: Yankelevitz, Rachelle
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2009

Subjects

Subjects / Keywords: choice, diminishing, foraging, optimal, pigeons, reinforcement, returns, token
Psychology -- Dissertations, Academic -- UF
Genre: Psychology thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: Pigeons choices were studied in a laboratory-based situation of diminishing returns, in which the rate of gain steadily decreased with time spent in the situation. The study was designed as a laboratory analogue of a depleting patch of resources like that faced by an animal foraging in the wild, and to test the predictions of three optimization models: a global-maximization model, a local-maximization model, and an intermediate-term maximization model based on the sum of reciprocals of upcoming delays (the summed-immediacies model). Pigeons chose between fixed-interval (FI) schedules and progressive-interval (PI) schedules of token reinforcement (illumination of a stimulus light). Immediate-exchange conditions were similar to procedures typically used with pigeons: each token was exchanged for brief access to food immediately after it was earned. In delayed-exchange conditions, 11 tokens were earned before an exchange phase occurred. Delayed-exchange conditions were analogous to procedures more commonly used with human subjects, in which a fixed number of tokens are earned before exchanging them for other reinforcers. Three experiments were conducted, each of which systematically manipulated a variable to more fully distinguish between the predictions of the models: FI duration (Experiment 1), PI incrementation size (Experiment 2), and inter-trial interval (Experiment 3). Across experiments, choices in immediate-exchange conditions were in better accord with the predictions of the summed-immediacies model, conforming to previous results with pigeons, whereas choices in delayed-exchange conditions were in better accord with the global-maximization model, conforming to previous results with humans. In delayed-exchange conditions, trials farthest from upcoming exchange periods resulted in lower response rates, higher latencies, and a higher probability of choosing the FI schedule compared to trials near exchange, reducing overall reinforcement rate relative to immediate-exchange conditions. Differences in choices between exchange types suggest that decision rules are influenced by the economic context in which they are used, although these differences are modulated by local schedule effects.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Rachelle Yankelevitz.
Thesis: Thesis (Ph.D.)--University of Florida, 2009.
Local: Adviser: Hackenberg, Timothy D.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2009
System ID: UFE0024394:00001


This item has the following downloads:


Full Text

PAGE 1

1 SOME EFFECTS OF ECONOMIC CONTEXT ON PIGEONS CHOICES IN SITUATIONS OF DIMINISHING RETURNS By RACHELLE L. YANKELEVITZ A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF TH E REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2009

PAGE 2

2 2009 Rachelle L. Yankelevitz

PAGE 3

3 To my mom and dad

PAGE 4

4 ACKNOWLEDGMENTS My advisor Tim Hackenbergs contributions to this pap er and to my education have been profound. Beyond editing the content of this work, his guidance, patience, and mentorship over the past six years have shaped my academic, professional, and everyday behavior extensively. I thank Mena Morales, my trusted friend and source of great reassurance. I thank Leonardo Andrade, Chris Bullock, Anthony DeFulio, Carla Lagorio, and Bethany Raiff for running experimental sessions and offering input. My committee members, Dr. Marc Branch, Dr. Jane Brockmann, Dr. David Smith, and Dr. Tim Vollmer, broadened my focus and for that I am thankful. Finally, I thank Spider, Monkey, Noodle, and the others, who remind me that as long as you have the basics in life, it is easy to be happy.

PAGE 5

5 TABLE OF CONTENTS ACKNOWLEDGMENTS .................................................................................................................... 4 page LIST OF TABLES ................................................................................................................................ 7 LIST OF FIGURES .............................................................................................................................. 8 ABSTRACT ........................................................................................................................................ 10 CHAPTER 1 GENERAL INTRODUCTION .................................................................................................. 12 Introduction ................................................................................................................................. 12 The Marginal Value Theorem ............................................................................................. 12 Diminishing Returns in the Laboratory: Progressive Schedule Choice Procedures ...... 16 Current G oals ....................................................................................................................... 20 2 EXPERIMENT 1 ......................................................................................................................... 25 Introduction ................................................................................................................................. 25 Method ......................................................................................................................................... 25 Subjects ................................................................................................................................ 25 Apparatus ............................................................................................................................. 26 Preliminary Training ............................................................................................................ 27 Experimental Procedure ...................................................................................................... 28 Results .......................................................................................................................................... 30 Discussion .................................................................................................................................... 34 3 EXPERIMENT 2 ......................................................................................................................... 48 Introduction ................................................................................................................................. 48 Method ......................................................................................................................................... 48 Subjects and Apparatus ....................................................................................................... 48 Preliminary Training ............................................................................................................ 49 Experimental Procedure ...................................................................................................... 49 Results .......................................................................................................................................... 49 Discussion .................................................................................................................................... 51 4 EXPERIMENT 3 ......................................................................................................................... 63 Introduction ................................................................................................................................. 63 Method ......................................................................................................................................... 63 Experimental Procedure ...................................................................................................... 63 Results .......................................................................................................................................... 64

PAGE 6

6 Discussion .................................................................................................................................... 66 5 GENERAL DISCUSSION ......................................................................................................... 76 Future Directions ......................................................................................................................... 79 Laboratory Analogues of Foraging ............................................................................................ 80 LIST OF REFERENCES ................................................................................................................... 83 BIOGRAPHICAL SKET CH ............................................................................................................. 86

PAGE 7

7 LIST OF TABLES Table page 2 1 Order of conditions and the number of sessions conducted in each in Experiment 1 ....... 39 3 1 Order of conditions and the number of sessions conducted in each in Experiment 2 ....... 51 4 1 Order of conditions and the number of sessions conducted in each in Experiment 3 ....... 67

PAGE 8

8 LIST OF FIGURES Figure page 1 1 Optimal patch time as a function of travel time ................................................................... 23 1 2 Optimal patch time as a function of patch density ............................................................... 24 2 1 Predicted switch points as a function of FI value ................................................................. 40 2 2 Predictions of each model and mean switch points as a function of FI value .................... 41 2 3 Deviation of mean switch point from prediction as a function of FI value ........................ 42 2 4 Percentage of FI choices occurring at each PI value as a function of FI value .................. 43 2 5 Mean responses per minute as a function of choice position .............................................. 44 2 6 Mean choice latency as a function of choice position ......................................................... 45 2 7 Probability of switching as a function of choice position.................................................... 46 2 8 Mean reinforcers per minute as a function of FI value ........................................................ 47 3 1 Predicted switch points as a function of PI value ................................................................. 55 3 2 Predictions of each model and mean switch points as a function of PI value .................... 56 3 3 Deviation of mean switch point from prediction as a func tion of PI value ........................ 57 3 4 Percentage of FI choices occurring at each PI value as a function of PI value .................. 58 3 5 Mean responses per minute as a function of choice position .............................................. 59 3 6 Mean choice latency as a function of choice position ......................................................... 60 3 7 Probability of switch ing as a function of choice position.................................................... 61 3 8 Mean reinforcers per minute as a function of PI value ........................................................ 62 4 2 Predictions of each model and mean switch points as a function of ITI value .................. 69 4 3 Deviation of mean switch point from prediction as a function of ITI value ...................... 70 4 4 Percentage of FI choices occurring at each PI value as a function of ITI value ................ 71 4 5 Mean responses per minute as a function of choice position .............................................. 72 4 6 Mean choice latency as a function of choice position ......................................................... 73

PAGE 9

9 4 7 Probability of switching as a function of choice position.................................................... 74 4 8 Mean reinforcers per minute as a function of ITI value ...................................................... 75

PAGE 10

10 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy SOME EFFECTS OF ECONOMIC CONTEXT ON PIGEONS CHOICES IN SITUATIONS OF DIMINISHING RETURNS By Rachelle L. Yankelevitz May 2009 Chair: Timothy D. Hackenberg Major: Psychology Pigeons choices were s tudied in a laboratory -based situation of diminishing returns, in which the rate of gain steadily decreased with time spent in the situation. The study was designed as a laboratory analogue of a depleting patch of resources like that faced by an animal fo raging in the wild, and to test the predictions of three optimization models: a global maximization model, a local -maximization model, and an intermediate term maximization model based on the sum of reciprocals of upcoming delays (the summed-immediacies mo del). Pigeons chose between fixedinterval (FI) schedules and progressive -interval (PI) schedules of token reinforcement (illumination of a stimulus light). Immediate -exchange conditions were similar to procedures typically used with pigeons: each token was exchanged for brief access to food immediately after it was earned. In delayed -exchange conditions, 11 tokens were earned before an exchange phase occurred. Delayed -exchange conditions were analogous to procedures more commonly used with human subjec ts, in which a fixed number of tokens are earned before exchanging them for other reinforcers. Three experiments were conducted, each of which systematically manipulated a variable to more fully distinguish between the predictions of the models: FI durati on (Experiment 1), PI increment ation size (Experiment 2), and inter trial interval (Experiment 3). Across experiments, choices in immediate -exchange conditions were in

PAGE 11

11 better accord with the predictions of the summed -immediacies model, conforming to previous results with pigeons, whereas choices in d elayed -exchange conditions were in better accord with the global -maximization model, conforming to previous results with humans. In delayed exchange conditions, trials farthest from upcoming exchange periods resul ted in lower response rates, higher latencies, and a higher probability of choosing the FI schedule compared to trials near exchange, reducing overall reinforcement rate relative to immediate-exchange conditions. Differences in choices between exchange ty pes suggest that decision rules are influenced by the economic context in which they are used, although these differences are modulated by local schedule effects.

PAGE 12

12 CHAPTER 1 GENERAL INTRODUCTION Introduction A central assumption of cost -benefit models is t hat of maximization of benefits over time. Whether the costs and benefits are defined in relation to prices and commodities, net energy gain, or net reinforcement, behavior is assumed to reflect a balance of costs and benefits subject to constraints. Amo ng the most fundamental questions in such maximization models concern the local behavioral mechanisms, or decision rules, governing the cost benefit tradeoffs. The present study aims to better understand these local decision mechanisms in a common decisi on-making context: a situation of diminishing returns. A situation of diminishing returns is characterized by a gradually decreasing rate of gain for continued work on a particular task. This feature is termed resource depression (Charnov, Orians, & Hyat t, 1976). There is usually an option to leave the current task and begin another, but this option entails a period of little or no gain. A simple illustration is a hummingbird foraging for nectar among the flowers in a small field. The bird first harvest s nectar from the flowers that provide the most energy per unit time. Having depleted t he best flowers, it may begin visiting the lower yield flowers, and so on. At some point, this diminishing rate of return will drop below that of the environment as a whole. When should the bird switch from the depleting pat c h to another patch? How do such switches relate to the overall maximization of gains? The Marginal Value Theorem The field of behavioral ecology has conceptualized a variety of situations faced by animals as problems of optimization. One such problem is the question of when a forager should switch patches. Optimization models assume some currency is being maximized given certain constraints, and various optimization models make testable predict ions about which currency and

PAGE 13

13 constraints are relevant. Charnov (1976) systematized the patch-switching question by formulating the marginal value theorem. The model begins with some simplifying assumptions. First, as an animal spends more time consumin g the commodity in a patch, the rate of return of that commodity in that patch decreases. This is what defines the situation as one involving diminishing returns. Second, it assumes that foraging episodes are divided into two mutually exclusive phases: between patch (travel) time and within -patch (exploitation) time. That is, an animal cannot energetically profit while traveling between patches, and it cannot switch patches while exploiting the current one. Completion of these two phases (travel and exp loitation) makes up a single foraging episode. Given these two assumptions, the marginal value theorem makes quantitative predictions about when an animal should switch patches to maximize overall net gain. Figure 1 1 depicts these relationships in grap hical form. The figure shows the predicted lengths of foraging episodes in two patches having the same rate of gain but different travel times. Curve A represents cumulative gain as a function of time spent exploiting either patch. This is a negatively accelerating positive function because of the assumption that the rate of gain decreases as time spent in the patch increases, and this shape defines the situation as one of diminishing returns. The vertical line divides the foraging episode into travel t ime and patch time. The gain curve is flat to the left of the vertical line, indicating that there is no energetic gain during travel time. A line drawn from the origin and tangent to the gain curve represents the highest possible rate of return. For ex ample, the point of intersection between Line L and the gain curve corresponds to OPT,L, which is the optimal amount of time to spend in a patch associated with travel time L. The other patch has an identical gain curve but a shorter travel time, S. As t he beginning of the travel period moves closer in time (rightwards along the xaxis)

PAGE 14

14 to the beginning of the patch time, the point of intersection of the tangent line moves leftwards along the gain curve, as in the transition between lines L and S. The ma ximum gain in the patch with travel time S is the point OPT,S. In summary, all else held equal, longer travel times should produce longer within-patch residence times, and shorter travel times should produce shorter within -patch residence times. The margi nal value theorem also makes predictions about the impact of changes in rate of gain on patch times. Figure 1 2 presents a scenario in which the optimal patch times are predicted for patches R and P having the same travel times but differing prey densitie s. Patch R is a relatively rich patch, and patch P is a relatively poor patch. As in Figure 1 1, tangent lines drawn from the start of the travel time intersect each curve at the optimal patch times for each patch. The richer patch R is associated with a shorter patch time, and the poorer patch P is associated with a longer patch time. The prediction that an animal would leave a richer patch sooner results from the assumption that the next patch is of the same quality as the current patch, so the animal would be moving from one rich patch to another or one poor patch to another. As an optimization model, the marginal value theorem is silent with respect to the local decision rules that produce optimization in the long run. Rather, it assumes optimizat ion subject to constraints and proper specification of currency. In optimal foraging models, the currency is typically energy gain, a proxy for long term fitness maximization. Constraints include characteristics of the environment (e.g., rate of resource depletion) or the organism (e.g., perceptual or memory restrictions) that limit the range of behavior that is possible. One possible decision rule is global maximization, which assumes long term maximization of gain across the entire foraging episode ( d efined as the travel time plus the patch time). This pattern of maximization requires foregoing immediate outcomes in favor of longer -term ones,

PAGE 15

15 specifically at the point of switching between patches. An animal maximizing overall net rate of return (amou nt of food per unit time) on the basis of the entire foraging episode must occasionally choose to enter a period of zero net gain to subsequently access a replenished patch. The opportunity cost of this longterm gain is that relatively easily accessible prey items are left in the patch, unconsumed. A second type of decision rule is local maximization, which assumes that animals maximize not over the entire foraging episode, but only with respect to the immediately upcoming choice between the short term p ayoff for staying in the patch and the short term payoff for leaving the patch. An animal whose choices are described by a temporally local maximization rule will switch patches only when the marginal rate of return for pursuing one more item in the curre nt patch reaches the point of equality with the marginal rate of return for switching patches and pursuing one prey item in the new patch Adopting this local maximization rule entails staying in the patch for much longer than the global -maximization rule predicts. The local -maximization rule, just like the global -maximization rule, predicts that animals will vary their patch times based on changes in travel time. If travel time requirements are relatively high, then switching patches is associated with a relatively high cost. In this case, the marginal rate of return must diminish considerably in the current patch in order for switching patches to be worthwhile. If, instead, travel costs are low, then the current rate of gain may only slightly diminis h before the point at which the payoffs from staying and switching are equal. Large travel costs predict longer patch times, and small travel costs predict shorter patch times. The function relating switch points to travel costs is qualitatively the same (but quantitatively different) according to the local and global -maximization rules.

PAGE 16

16 In the area of relating within -patch rate of gain to patch times, the local -maximization model varies qualitatively from the global -maximization rule The local model predicts that regardless of whether the rate of gain decreases steeply or gradually, the animal will switch patches at the point of equality. Therefore, patch times will not be affected by changes in within patch rate of gain. The local and global -maxi mization models represent end points of a continuum. Both assume the foraging animal will maximize rate of return. The global model assumes maximization over an extended sequence of outcomes: the consumption of several prey items in the patch and the even tual travel to the next patch. The local model assumes maximization over only the next outcome, weighing the cost of pursuing one more prey item in the current patch against the cost of switching patches and then pursuing one prey item in the new patch. Thus, the two predictions are of the same class, both belonging to a family of optimization models. They differ in their assumptions about the time frame over which maximization occurs, with the global model including a long time frame and the local model including a short time frame. Diminishing Returns in the Laboratory: Progressive-Schedule Choice Procedures The two models each make quantitative predictions about switch points in situations of diminishing returns. These predictions can be difficult to test, however, in the naturalistic settings in which they were derived. Laboratory models can be useful for comparing predictive models which require background variables to be held constant while key variables are manipulated (Fantino, 1985; Fantino & Abarca, 1985; Hackenberg, 1998; Lea, 1981). The diminishing returns paradigm has been translated into the laboratory and studied with a variety of species including chimpanzees (Hodos & Trumble, 1967), monkeys (Hineline & Sodetz, 1987), pigeons (Hackenb erg & Hineline, 1992; Mazur & Vaughan, 1987; Neuman, Ahearn, & Hineline, 1997; Neuman, Ahearn, & Hineline, 2000; Wanchisen, Tatham, & Hineline, 1988), and humans

PAGE 17

17 (Hackenberg & Axtell, 1993; Hackenberg & Joker, 1994; Jacobs & Hackenberg, 1996; Wanchisen, Tatham, & Hineline, 1992). The general procedure consists of a concurrent choice between a progressive schedule and a fixed schedule. The progressive schedule starts out at a low value, and each time it is chosen, its requirement increases by the paramete r value. For example, consider time -based schedules choice s between a fixed and a progressivelyincreasing interval schedule (technically, a progressive interval, or PI, schedule) T he first time a progressive interval 8 s (PI 8 s) schedule is chosen, a 0 s delay is in place ( the first response towards the requirement results in a single food delivery). The next choice of the PI requires an 8 -s delay before a response results in food delivery; on the next PI choice a 16-s delay is in place, and so on, the delay incrementing by 8 s with each successive choice of the progressive schedule. The progressive schedule is procedurally analogous to the patch time as described above: the time spent in the patch consuming items. Successive choices of the progressi ve schedule decrease the rate of gain associated with that schedule, just as consumption of successive prey items within a patch decreases the marginal rate of return. A second, simultaneously available fixed interval (FI) schedule opposes the PI schedule, and remains the same throughout the session. In the standard procedure, completing the FI schedule resets the PI schedule to its lowest value. Choosing the FI schedule and then encountering the PI schedule at its lowest value is procedurally analogous t o an animal entering a travel period and then accessing a replenished patch. This setup allows the subject to choose between persisting in a patch (choosing the progressive schedule) versus switching patches (choosing the fixed schedule). The main depe ndent measure is the value of the progressive schedule when the fixed is chosen, termed the

PAGE 18

18 switch point. The local -maximization model predicts that on each trial, the animal will choose the shorter schedule; that is, repeatedly choose the progressive sch edule until its parameter value is equal to that of the fixed schedule before choosing the fixed schedule. The global maximization model predicts the animal will adopt a pattern of switching that yields the highest overall rate of reinforcement, which ent ails switching well before the point predicted by the local -maximization strategy. Pigeons switch points in several variations of this procedure have fallen between those predicted by short term and long term maximization (Mazur & Vaughan, 1987; Wanchise n, Tatham, & Hineline, 1988; Hackenberg & Hineline, 1992; Neuman, Ahearn, & Hineline, 1997; Neuman, Ahearn, & Hineline, 2000). To account for such intermediate -term maximization, Shull and Spear (1987), d rawing on an earlier suggestion by McDiarmid and Ri lling (1965), proposed that the value of a series of reinforcers can be calculated by the summed reinforcer immediacies comprising the sequence, n i iD V11 (1) where V is the value of a series of choices, Di is the delay between a choice response and the delivery of consequence i and n is the number of reinforcers in the series. It is conventional on studies such as these to use n = 4 because over the range of schedule parameters used by previous researchers, varying n from about 3 to it s maximum (aggregating across all possible choices) produces little quantitative variation in predictions (Wanchisen, Tatham, & Hineline, 1988). This value will be used in the present study. Setting n =4 maintains contact with previous literature, but more importantly, it allows this model to fall at intermediate ranges on a continuum anchored by global maximization on one end and local maximization on the other. The quantitative predictions of the summed -immediacies model fall in between the prediction s of

PAGE 19

19 the global and local models. All three models are optimization models in that they assume maximization over some time frame, but the summed immediacies model differs in that it includes an extra constraint: delayed reinforcers may be less valuable th an immediate reinforcers. The model predicts how many times an animal will choose the PI before switching to the FI by comparing the values of persisting versus switching at each choice point. Value is calculated based on the delay to each reinforcer, an d these delays are weighted in proportion to their order of occurrence. That is, immediately upcoming delays are weighted more heavily than temporally remote delays Qualitatively, the summed -immediacies model predicts that increases in the fixed schedul e (analogous to travel cost) and increases in the progressive schedule (analogous to decreases in within-patch rate of gain) will both result in increased switch points. Summarizing the results across experiments, pigeons responding on the diminishing ret urns procedure have produced switch points in accord with the summed -immediacies model (Hackenberg & Hineline, 1992; Neuman, Ahearn, & Hineline, 1997; Neuman, Ahearn, & Hineline, 2000; Wanchisen, Tatham, & Hineline, 1988). Human performance in laboratorybased situations of diminishing returns has varied from that of pigeons. Hackenberg and Axtell (1993) presented human subjects with choices between progressive and fixed schedules of points later exchangeable for money. Switch points conformed reasonably well to both the summed immediacies model and global -maximization model; however, in conditions when the two models predictions diverged the most, switches conformed most to global maximiz ation Similar results were reported by Jacobs and Hackenberg (1996) in a more extensive parametric study of humans choices between FI and PI schedules of points exchangeable for money: switching generally conformed to the predictions of the global -maximization model

PAGE 20

20 The difference in human and pigeon performances ins pires a search for the relevant differences in the procedures used with each species. In most laboratory research with human subjects, including that reviewed above, choices produce conditioned token reinforcers points later exchangeable for money rather than unconditioned reinforcers food used in analogous research with pigeons. Unlike the latter, which are consumed as they are earned, token reinforcers accumulate for later exchange and consumption. This may render the tokens less effective in the short run, as their value depends on their relation to more remote sources of reinforcement available during exchange periods. Thus, perhaps a shift in the economic context, from one in which reinforcers are consumed as they are earned to one in which they acc umulate for later exchange, gives rise to different decision rules. If so, then modifying the economic context may be expected to produce different choice patterns. Current Goals The primary goal of the current experiments was to examine the role of the economic context on pigeons choice in situations of diminishing returns. Choices produced token reinforcers in the form of stimulus lamps mounted in a horizontal array above the response keys in an otherwise standard experimental chamber for pigeons. The tokens served as a form of currency akin to point/money reinforcers in analogous lab experiments with human subjects. This type of token reinforcement procedure has been used in several experiments from our laboratory, including second -order schedules (Bullock & Hackenberg, 2006; Foster, Hackenberg, & Vaidya, 2001), choice and self -control (Foster & Hackenberg, 2004; Hackenberg & Vaidya, 2003; Jackson & Hackenberg, 1996), punishment ( Pietras & Hackenberg, 2005; Raiff, Bullock, & Hackenberg, 2008), and reinforcer accumulation ( Yankelevitz, Bullock, & Hackenberg, 2008). In the present methods, pigeons choices produced token reinforcers exchangeable for food during scheduled exchange periods. In some conditions ( i mmediate e xchange ) the exchange

PAGE 21

21 periods immediately followed token delivery, permitting immediate consumption of each reinforcer as it was earned. One might expect choice patterns under these conditions to resemble those found in prior research with pigeon subjects, which were well -described by the summed immediacies model. In other conditions ( d elayed e xchange ), tokens accumulated across choice trials, with exchange periods scheduled after 11 tokens had been earned. These procedures were seen as more analogous to those used in laboratory res earch with human subjects, in which the delay to tokens on each trial varied but the number of choices required before entering the temporally -remote exchange period was the same regardless of which choice had been made. Also like the cumulative display o f points, the continuous display of tokens in the present study was correlated with temporal proximity to and number of reinforcers available during exchange. Together, these procedural factors make the delayed -exchange conditions more akin to the economic contingencies in research with human subjects. If this economic context is a primary factor in differences between human and pigeon performance in situations of diminishing returns, then one might expect to see more human -like performance under delayed -exchange conditions and more pigeonlike performance under immediate -exchange conditions. More specifically, one might expect the switch points to be better described by the global maximization rule under delayedexchange conditions and by the summed immediacies rule under immediate -exchange conditions. A second, more specific, goal was to assess the descriptive adequacy of the 3 models Because the models often make similar point predictions, the schedule parameters will be altered across experiment s in ways that produce changes not only in the point predictions but in the form of the functions predicted by the models. While comparing the three models to obtained data, they should be viewed as a family of models, each making related assumptions about the local

PAGE 22

22 decision variables controlling choice. Each is a type of optimization model in that it assumes maximization over a certain time frame, but they differ in the time frame under consideration and, in the case of the summedimmediacies model, the method of calculating the value of the currency maximized. Comparing the models to pigeons switch points provides a way of experimentally analyzing the local decision rules which combine to produce optimization.

PAGE 23

23 Figure 1 1 Optimal patch time as a function of travel time. The gain curve (Curve A) is a negatively accelerating positive function of patch time. If a patch has travel time L, then the optimal duration of foraging within it (OPT,L) is the point of intersection between the gain curve for the patch and a tangent line drawn from the start of the travel time (Line L). If the patch has a shorter travel time, S, then Line S is drawn, and the optimal patch time decreases to OPT,S. Adapted from Stephens & Krebs (19 8 6).

PAGE 24

24 Figure 1 2 Optim al patch time as a function of patch density. Prey within patch R are densely distributed, and prey in patch P are sparsely distributed. Patches R and P have the same travel time, T. The optimal duration of foraging within patches R and P (OPT,R and OPT, P, respectively) are the points of intersection between each gain curve and its tangent line drawn from the start of the travel time. The denser patch has a shorter predicted patch time than the sparser patch. Adapted from Hanson (1987) and Krebs (19 78)

PAGE 25

25 CHAPTER 2 EXPERIMENT 1 Introduction The first goal of Experiment 1 was to investigate whether switch points in delayed exchange conditions reflected more sensitivity to longterm outcomes than did switch points in immediate -exchange conditions. In delayed exchange conditions, because the first 10 choices of each set of 11 are temporally distal from food delivery, the choice reinforcer delays may have less impact than in immediate -exchange conditions. If this is the case, then subjects will switch earlier in delayed than in immediate exchange. The second goal was to investigate the impact of the FI duration on switch points. Previous research has shown that increasing FI duration increases switch points in pigeons (Hackenberg & Hineline, 1992) and humans (Hackenberg & Axtell, 1993; Jacobs & Hackenberg, 1996). Figure 2 1 shows the predicted switch points of the global -maximization model, the summed immediacies model, and the local -maximization model (separate data paths) as a function of FI schedule value All three models predict that switch points will increase with increases in the FI schedule, but the slopes of the predicted functions differ. The global model predicts the least sensitivity to FI value, and the local model predicts the most. The pred ictions in Figure 2 1 apply to both the immediate and delayed -exchange conditions when calculated with respect to token deliveries. Method Subjects Four male White Carneau pigeons ( Columba livia ), numbered 14, 18, 7 and 456, served as subjects. All pi geons were experimentally nave Pigeons were housed individually under a 16.5hr/7.5 -hr light/dark cycle (lights on from 7:00 a.m. until 11:30 p.m.) in a temperature and

PAGE 26

26 humidity -controlled room and had continuous access to water and grit in their home c ages. Pigeons were maintained at 80 % + 20g of their free -feeding weights by supplementary post session feeding of mixed grain Apparatus Two standard experimental chamber s w ere used. Subjects 14 and 7 completed sessions in a chamber with inside dimensions of 490 mm long, by 360 mm wide, by 380 m m high. Subjects 18 and 456 completed sessions in a chamber with inside dimensions of 310 mm long, by 350 mm wide, by 340 mm high. Both chambers had similar response panel dimensions and differed mostly in inter ior size; for ease of exposition, dimensions of the response panel used for subjects 18 and 456 are in parentheses following the dimensions of the chamber used for subjects 14 and 7. Three response keys 2 5 mm (25 mm) in diameter were arrayed horizontall y across the intelligence panel. The keys could be illuminated yellow, red, or green, and required a force of 0.25 N (0.25 N) to be operated. The centers of each key were 108 m m (96 mm) below the ceiling of the chamber and 89 m m (82 mm) apart. Above the three response keys was a row of twelve red lights, which served as tokens. Each token light had a diameter of 13 m m (15 mm) and protruded 13 m m into the chamber. The distance between the centers of each token was 29 m m (25 mm) and the centers of the t oken lights were 5 0 m m (45 mm) above the centers of the response keys. The order of illumination of the tokens was always from left to right. Food delivery consisted of timed access to mixed grain through a centrally located rectangular aperture (5 0 m m t all by 58 m m wide (54 mm tall by 57 mm wide) ), located 12 0 m m (125 mm) below the middle key. A photocell mounted in the food aperture ensured that each food delivery was of equal duration within subject timed from head entry into the hopper. Food access durations were held constant throughout the experiment but varied across subjects because of differences number of trials completed in pre -experimental conditions: 1 s for Subject 14, 1.5 s for Subject

PAGE 27

27 7, and 2.5 s for Subjects 18 and 456. A Sonalert wa s mounted behind the stimulus panel and emitted a 0.1 -s tone when tokens were illuminated or extinguished. A white houselight centered above the token array was on throughout the session, and white noise masked external sounds continuously. Contingencies were programmed and data collected using a computer and MED PC software located in an adjacent room. Preliminary Training Because no subjects had previous experimental histories, key pecking was shaped by reinforcing with food successive approximations to pecking. All pigeons then received several training sessions in which the final performance was established via principles of backward chaining. In the first set of training sessions, each cycle began with illumination of n stimulus lights (tokens) al ong with the red center (exchange) key. Across sessions, n was increased from 3 to 12. One response on the exchange key turned off the key and the rightmost token and produced a tone and 2 -s access to food. The red key was then reilluminated, and the next token was exchanged for food by pecking the exchange key. When all tokens were exchanged for food, n tokens were reilluminated and the cycle began anew, until several groups of n tokens had been exchanged for food. Pigeons 14, 7, and 18 each required a maximum of 5 sessions to complete this training phase, and Pigeon 456 required 12 sessions. In the second training program, trials began with illumination of the center blue key, one peck on which produced the center red key and a tone and illuminated t he leftmost token. One peck on the center red key exchanged the leftmost token for food, after which the center blue key was reilluminated and the cycle started again. This training program was in place for one session and ended after 20 reinforcers were delivered. In the third training program, the first 5 trials began with presentation of the green key (randomly located on the right or left side across trials), a single peck on which produced the center blue key, which then functioned as in the previou s training

PAGE 28

28 program. The next 5 trials within a session were identical except that the yellow key instead of the green key was presented. The third training program was in place for a maximum of two sessions, and each session ended when 40 reinforcers wer e delivered. The final training phase was identical to the previous phase in that 5 trials began with illumination of the green key, and 5 trials began with illumination of the yellow key. The difference was that a fixed -interval (FI) schedule was now ar ranged on the yellow key, and the interval was increased from 5 to 60 s across sessions, and the green key presented a progressive interval (PI) schedule, in which successive completions of the schedule requirement increased the requirement across trials. The size of this increment (the value of the PI schedule) was increased from 1 to 8 seconds across sessions. Each session ended when 40 reinforcers were delivered. All pigeons completed the final training phase within 9 sessions. Experimental Procedure In the terminal procedure, ea ch trial began with illumination of the two side keys, one yellow and the other green The positions of the yellow and green keys were randomly assigned across the two side keys at the start of each trial. A single peck to e ither key initiated its schedule requirement and extinguished the unchosen alternative. The yellow key was correlated with an FI x s schedule, in which the first peck after x seconds satisfied the requirement. The green key was correlated with a PI 8 s s chedule, in which the interval increased by 8 s across successive trials in which it was chosen. For example, the first time the PI schedule was chosen, 0 s passed before the first peck satisfied the requirement (the first response after the schedule was chosen produced food). The second time the PI was chosen, the interval was 8 seconds, and so on. Thus, repeated choices of the PI schedule increased the PI value until the session ended or until the FI schedule was selected. Choosing the FI schedule res et the PI schedule back to its lowest value of 0 s. To enhance discriminabilty of the reset contingency, the PI key flashed on

PAGE 29

29 for 0.5 s and off for 0.5 s when it was at its lowest value. A single peck on either choice key extinguished the unchosen alter native and initiated the programmed response requirement on that key. Satisfying either interval requirement extinguished the chosen key and illuminated the blue center key, one peck on which produced one token. Requiring one peck to the center blue key served to center the pigeons head between choice alternatives and to increase discriminability of token delivery. Tokens could be exchanged for food during scheduled exchange periods, signaled by a red center key. The number of tokens required prior to a n exchange period constituted one of the main independent variables in the experiment and had two values: 1 and 11. In immediate exchange conditions, an exchange period was made available following each token delivery, whereas in the delayed-exchange cond ition, an exchange period was made available after 11 tokens were earned. In both conditions, exchange was signaled by darkening of both side keys and illuminating the center red (exchange) key, one peck on which turned off the most recently earned token and produced a tone and 2 -s access to food. In the delayed-exchange condition, the exchange key was repeatedly presented until all 11 tokens were exchanged for food. When the last token was exchanged, the cycle restarted with presentation of the two choi ce keys. The only deviation from this trial structure was that each session of all types began with forced exposure to the FI schedule in effect: on the first trial of each session, only the FI key was presented. The second trial and all subsequent trial s were choices between the FI and PI schedules. Sessions of either type ended following 44 food deliveries or 75 minutes, whichever came first. The other independent variable was the size of the FI schedule, which varied systematically across conditions, from 64 s to 48 s to 32 s. There were a total of 6 experimental conditions per subject (excluding replications), defined by a unique combination of FI size and

PAGE 30

30 exchange -delay. The immediate -exchange and delayed-exchange conditions alternated at each FI v alue, yielding a 3 x 2 within-subject factorial design. The order of immediate and delayed exchange per FI value was counterbalanced across subjects, with two (Pigeons 14 and 456) receiving immediate exchange first at each FI value, and the other two (Pig eons 18 and 7) receiving delayed exchange first at each FI value. One pair of conditions, typically the FI 64 s condition pair, was replicated per subject. The primary dependent measure was the value of the PI schedule when the pigeon chose the FI sched ule. This value is termed the switch point defined as the value of the PI when the FI is chosen. Table 1 shows the order of conditions and the number of sessions conducted in each. Conditions lasted for a minimum of 10 sessions, and remained in effect until median switch points we re deemed stable 5 consecutive sessions that did not include the highest or lowest points of the condition and were free of monotonic trends. Responding grew too weak in some delayed -exchange conditions to meet stability requir ements, in that fewer than half the reinforcers in a session were obtained, or there were no choices of the FI schedule. In these cases, conditions were terminated prematurely, and these data were excluded from the analysis. Some of these incomplete condi tions were later replicated. Results All analyses are based on the last 5 sessions of each condition. Figure 2 2 shows each models predicted switch points as in Figure 2 1 (dashed lines) and switch points for each subject calculated as the mean of each sessions median switch point (symbols). Error bars represent one standard deviation from the mean. Each panel shows switch points from both delayedand immediate -exchange conditions. Switch points increased systematically with FI size under immediate -exchange conditions for all subjects, and were consistently higher than delayedexchange switch points. For two subjects (7 and 18), switch points also increased with FI size

PAGE 31

31 under delayed-exchange conditions, but the functions were flatter than those obt ained under immediate -exchange conditions. For the other two subjects (14 and 456), delayed-exchange switch points did not increase systematically with FI size. Subjects 14 and 456 were also the two subjects who experienced the immediate -exchange conditi on before the delayed -exchange condition within each FI value. Results of replicated conditions were similar to original conditions except for Pigeons 18 and 7 under FI 64 delayed exchange, a condition which resulted in low and variable switching. Immedi ate -exchange switch points were generally near or slightly higher than the predictions of the summed immediacies model. Delayed -exchange switch points were closest to the predictions of the global -maximization model, but deviations from these predictions were in the negative direction in 9 of 15 cases. Figure 2 3 quantitatively compares switch points for each subject to the predictions of three models. Each of the three data paths represents the absolute difference between obtained switch points and one set of predictions. In delayed-exchange conditions (left column), obtained switch points deviate least from the predictions of the global -maximization model. The summed immediacies model using four reinforcers was the second -best predictor of obtained s witch points, and pigeons always switched well before the equality point (the switch points predicted by the local -maximization model). In comparing deviations across FI values, note that the predictions of the three models diverge as FI value increases. Thus, small changes in switch points between FI 48 s and FI 64 s conditions resulted in increasing deviations from predictions across these conditions. In immediate -exchange conditions, no single model was always the best predictor. For two out of four subjects (subjects 18 and 7), switches deviated least from the summed immediacies model, especially at higher FI values where the predictions diverge. For

PAGE 32

32 the other two subjects (subjects 14 and 456, who experienced immediate exchange before delayed at ea ch FI value), no single model was the best predictor of switch points. Figure 2 4 shows the percentage of switches occurring at each PI value for each subject. Columns display individual FI values. Dark bars represent data from delayed exchange, and lig ht bars data from immediate exchange. Filled and open points above bars represent median switch points in delayed and immediate exchange, calculated as the median of all switch points in the last 5 sessions of each condition. Horizontal error bars repres ent inter -quartile ranges. Efficiency functions (solid curves), scaled to the righthand y axis, are theoretical functions showing the session -wide rate of reinforcement that would result from a consistent pattern of switching at each PI value. These func tions are theoretical in that they assume an invariant switching pattern across trials and that they do not include response latencies. These assumptions cannot be met by any organism, but efficiency functions are useful points of reference for comparison to obtained data. The efficiency functions in Figure 24 are scaled uniformly across graphs with respect to the highest point across all efficiency functions but are arbitrarily scaled with respect to percentages of switches. In other words, efficiency functions predict PI values at which percentage of switches peak if choices are sensitive to long term outcomes, and comparisons across efficiency functions reveal the relative magnitude of these peaks. Within graphs, however, the vertical placement of ea ch efficiency function does not predict the height of percentage -of -switches distributions. Comparisons of delayedand immediate exchange distributions with efficiency functions show that distribution peaks were often more well -defined at lower FI value s where efficiency function peaks were also sharper. As the efficiency function flattened with increases in the FI value, inter -quartile ranges increased, indicating that choices

PAGE 33

33 were more widely dispersed. Across FI values, median switch points under de layed -exchange were generally lower than (shifted left of) those under immediate -exchange. Figure 2 5 shows mean responses per minute as a function of choice position. The response rate numerator is the total responses during the interval, excluding the choice response which initiated the interval. The response rate denominator is the time from the start of the interval to the time of the token -producing peck. These graphs combine all trials from the last 5 sessions of all conditions (FI 32 s, 48 s, 64 s, and the replication) within each type (delayed or immediate exchange) for each subject. In delayed -exchange graphs (left column), trials are grouped into choice position bins based on proximity to the upcoming exchange period. For example, the bar at choice position 11 in each delayed-exchange graph represents data from trials that immediately preceded an exchange period (trials 11, 22, 33, and 44). Choice position 2 includes trials 2, 13, 24, and 35, and choice position 1 includes trials 12, 23, and 24. Trial 1 was excluded from these analyses because it was a forced exposure to the FI schedule. Immediateexchange choice position bins include the same trial numbers as do delayed-exchange bins. Error bars represent standard deviations from the mean s. For three out of four subjects, response rates were lower in delayed exchange than in immediate exchange. For these same three subjects, delayed -exchange response rates were a slight increasing function of choice position. Immediate exchange response rates did not vary systematically as a function of choice position. Figure 2 6 shows mean choice latencies as a function of choice position. Latencies are the time from onset of both choice keys to the single schedule initiating peck. Details of the fig ure are as in Figure 2 5 except that y axes are on log scales. In delayed-exchange conditions, latencies were an inverse function of choice position. In immediate -exchange conditions,

PAGE 34

34 latencies were unrelated to choice position but were uniformly near th e mean latency across delayed -exchange bins. Figure 2 7 shows probability of choosing the FI as a function of choice position. Details are as in Figure 2 5. In delayed-exchange conditions, for all subjects except 456, choosing the FI was more likely at l ower choice positions than at higher choice positions. In immediate exchange conditions, choosing the FI was unrelated to choice position. Figure 2 8 shows mean reinforcers per minute as a function of FI value for both delayed and immediate -exchange conditions. Error bars represent one standard deviation. Rates of reinforcement were consistently higher in immediate exchange than in delayed exchange. In immediate -exchange conditions, reinforcement rates decreased with increases in FI value. There was n o consistent relation between delayed exchange reinforcement rates and FI value. Discussion Patterns of switching differed between delayed and immediate-exchange conditions, indicating sensitivity to the time frame over which food is delivered. Immedia te -exchange switch points were higher and more sensitive to changes in FI duration than were delayed exchange switch points. The immediate -exchange conditions in this experiment were procedurally similar to the arrangements used by previous researchers. T he results of the current experiment therefore replicate previous results with pigeons (Hackenberg & Hineline, 1992; Wanchisen, Tatham, & Hineline, 1988) and humans (Hackenberg & Axtell, 1993; Jaco b s & Hackenberg, 1996), showing that switch points are dire ctly related the size of the fixed schedule Deviations from predicted switch points put a sharper quantitative point on the difference between the time frame s over which choices were made in the two exchange conditions. In delayed exchange, pigeons choi ces best conformed to the predictions of the global maximization model, both in terms of the quantitative level of switch points and the relatively

PAGE 35

35 flat shape of the function relating delayed-exchange switch points to FI value The higher slopes of the im mediate -exchange switch point functions in Figure 2 2 relative to the del ayed -exchange functions suggest that the summed immediacies and local -maximization models best characterized the effect of FI on switching. The deviation analysis (Figure 2 3) show s that the summed immediacies model was the better predictor of switches. Because immediate exchange was like conditions used in previous experiments with pigeon subjects, this is in accord with previous results which found that the summed immediacies mod el best predicted switch points (Hackenberg & Hineline, 1992; Mazur & Vaughan, 1987; Neuman, Ahearn, & Hineline, 1997; Neuman, Ahearn, & Hineline, 2000; Wanchisen, Tatham, & Hineline, 1988). Comparing distributions of switches to efficiency functions in Fi gure 2 4 shows how choices corresponded to theoretical maximization of rate of reinforcement. The peaks of the efficiency functions correspond to the switch point predicted by the global -maximization model. The shape of the function shows how switching a t PI values other than the optimal switch point decreases the theoretical maximum rate of reinforcement. Moving left along the function away from the peak, the curve sharply decreases; this means that switching before the optimal point is associated with a relatively sharp decrease in the maximum rate of reinforcement. Moving right along the function away from the peak, the curve gradually decreases, indicating that switching beyond the optimal point decreases rate of reinforcement at a more moderate rate Thus, efficiency functions predict the shape of the variability in switching across trials. In Figure 24, delayed -exchange distributions were often shifted left of the efficiency functions peak, whereas immediate -exchange distributions were often shi fted right of the peak. Although delayedexchange switches conformed better to global maximiz ation than to the other models, the fact

PAGE 36

36 that delayed -exchange switches were below the optimal point suggests that the rate of reinforcement in delayed exchange m ay have been lower than in immediate. Data on schedule effects augment the picture of the differences in performance produced by delayed and immediate exchange. First, delayed-exchange response rates were generally lower than immediate exchange response rates. These delayed -exchange conditions were procedurally similar to second -order schedules of token reinforcement (see review by Hackenberg, 2009). In these studies, tokens are produced according to one schedule of reinforcement (the token -production s chedule) with exchange periods arranged according to a second schedule of reinforcement (the exchange -production schedule). In general, response rates are directly related to the temporal proximity to exchange, with response rates increasing across succes sive token-production segments leading up to exchange periods and food (Bullock & Hackenberg, 2006; Foster et al., 2001) Similarly, in a token accumulation context, Yankelevitz et al. (2008) found lower response rates in conditions in which pigeons accum ulated several tokens before exchanging them relative to conditions in which pigeons immediately exchanged tokens for food. Latencies to choice showed patterns similar to those produced in previous research. Webbe and Malagodi (1978) required, in some conditions, six completions of a fixedratio token -production schedule before exchange was available. Rats pre ratio pauses took on a bi valued pattern, with long pauses occurring before beginning the first fixed-ratio schedule and fairly uniformly shor t pauses occurring before each subsequent fixed ratio schedule. Other researchers have found similar patterns (Bullock & Hackenberg, 2006; Foster et al., 2001; Jwaideh, 1973; Kelleher, 1957). In the current study, a similar general relation was found

PAGE 37

37 bet ween choice latencies and proximity to upcoming exchange, though the function was somewhat more graded. Overall, response rates and latencies in delayed exchange were sensitive to proximity to the upcoming exchange period. That token productions early i n each cycle of delayed -exchange conditions were treated differently than later token productions is further illustrated by the fact that proximity to upcoming exchange influenced the probability of choosing the FI. For three of four pigeons, probabilitie s of choosing the FI in the trials farthest from the next exchange period were at least twice as large as the probabilities in trials closest to the next exchange period. This suggests that the individual delays to token delivery influenced choice more as exchange approached. These findings are consistent with prior results showing dependence of token -based choices on temporal proximity to exchange periods (Jackson & Hackenberg, 1996). These schedule effects also illuminate why immediate-exchange rates of reinforcement exceeded delayed -exchange rates of reinforcement. Based on the deviations of switches from the predictive models, delayed -exchange switches conformed best to the predictions of global maximization One might then infer that delayed-exchang e conditions would result in a higher overall rate of reinforcement. Figure 2 8 shows, however, that the opposite was true. Lower switch points did not translate into higher session -wide rates of reinforcement, mainly due to the schedule effects describe d above. Another reason for immediate -exchange reinforcement rates to exceed delayed -exchange reinforcement rates is the particular way in which delayed -exchange switches deviated from the global -maximization model. In Figure 2 4, efficiency functions indicate that switching at any value besides the point of the peak on the function results in long-term costs. However, the functions are asymmetrical in that they rise to their peaks at a higher rate than they fall after the

PAGE 38

38 peaks. This implies that sw itching one PI value before the peak is more costly than switching one PI value after the peak. Although switches deviated least from global maximiz ation compared to other models, in 12 out of 15 exposures to delayed-exchange across subjects (including re plications), mean switch points were below the points predicted by strict adherence to global maximiz ation The direction of the deviation contributed to low overall rates of reinforcement in delayed exchange.

PAGE 39

39 Table 2 1. Order of conditions and the number of sessions conducted in each in Experiment 1 Pigeon Exchange condition FI PI Number of sessions 14 Immediate 64 8 28 14 Delayed* 64 8 22 14 Immediate 48 8 13 14 Delayed 48 8 29 14 Immediate 32 8 32 14 Delayed 32 8 19 14 Immediate^ 64 8 23 14 De layed^ 64 8 20 18 Delayed 64 8 14 18 Immediate 64 8 13 18 Delayed 48 8 25 18 Immediate 48 8 24 18 Delayed 32 8 22 18 Immediate 32 8 13 18 Delayed^ 64 8 31 18 Immediate^ 64 8 11 7 Delayed 64 8 28 7 Immediate 64 8 38 7 Delayed 48 8 14 7 Immediate 48 8 17 7 Delayed 32 8 23 7 Immediate 32 8 11 7 Delayed^ 64 8 11 7 Immediate^ 64 8 13 456 Immediate 64 8 18 456 Delayed 64 8 16 456 Immediate 48 8 16 456 Delayed 48 8 12 456 Immediate 32 8 17 456 Delayed 32 8 17 456 Immediate^ 48 8 33 456 Delayed^ 48 8 30 Indicates no more than 11 reinforcers were earned per session in the last 5 sessions of this condition, or there were no switches in this condition. These conditions are not included in the results. ^Indicates this condition is a replication of a previous condition.

PAGE 40

40 Figure 2 1 Predicted switch points as a function of FI value. Data paths represent predictions of each model.

PAGE 41

41 Figure 2 2 Predictions of each model (dashed lines) and mean switch points (symbols) as a function of FI value. Unconnected points represent replicated -condition switch points. Data are from the last 5 sessions of each condition. Error bars indicate one standard deviation from means of last 5 sessions of each condition

PAGE 42

42 Figure 2 3 Deviation of mean switch point from prediction as a function of FI value. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. The left column presents delayed -exchange conditions, and the right column presents immediate -e xchange conditions. Symbols correspond to predictive models. All data are from the last 5 sessions of the first exposure to each condition, except for Pigeon 14s delayed-exchange FI 64, PI 8 condition in which data are presented from the replication bec ause the first exposure produced unstable responding.

PAGE 43

43 Figure 2 4 Percentage of FI choices occurring at each PI value as a function of FI value. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. Each column represents a single FI value, indicated at the top of the column. Black bars represent delayed -exchange conditions, and white bars represent immediate -exchange conditions. Points above bars represent medians of distributions having matching colors, and bars extending horizontally from points represent inter quartile ranges. Note y axis scaled individually for Pigeon 14s FI 64 conditions. All data are from the last 5 sessions of the first exposure to each condition, except for Pigeon 14s delayed-excha nge FI 64, PI 8 condition in which data are presented from the replication because the first exposure produced unstable responding.

PAGE 44

44 Figure 2 5 Mean responses per minute as a function of choice position. Delayed-exchange conditions are in the left col umn, and immediate -exchange conditions are in the right column. In delayed exchange, choices in position 1 are farthest from the upcoming exchange phase, and choices in position 11 are closest to the upcoming exchange phase. Immediate -exchange bins conta in the same trial numbers as delayed -exchange bins do. Note y axes scaled individually for subject 14. Each row presents graphs from a single subject. Error bars indicate one standard deviation from the mean

PAGE 45

45 Figure 2 6 Mean choice latency as a fun ction of choice position. Details are as in Figure 2 5, except that y axes are on log scales.

PAGE 46

46 Figure 2 7 Probability of switching as a function of choice position. Details are as in Figure 2 5.

PAGE 47

47 Figure 2 8 Mean reinforcers per minute as a funct ion of FI value. Closed symbols represent delayed -exchange conditions, and open symbols represent immediate exchange conditions. Unconnected points represent replicated conditions. Data are from the last 5 sessions of each condition. Error bars indicat e one standard deviation from the mean of the last 5 sessions of each condition.

PAGE 48

48 CHAPTER 3 EXPERIMENT 2 Introduction Experiment 1 showed that pigeons switch points were lower in delayed-exchange conditions than in immediate -exchange conditions and t hat response patterning varied both across exchange condition types and across choice positions within delayed-exchange conditions. Experiment 2 aimed to further compare delayed and immediate exchange in the context of manipulations of the PI value. Wher eas all three predictive models indicated that increasing the FI value in Experiment 1 should increase switch points, the models differ on the outcome of increasing the PI value. Figure 3 1 shows the predictions of each of the three models (separate data paths) as a function of PI value. The global -maximization model and summed immediacies models both predict that increasing the PI value will increase switch points. On the other hand, if choices are made with respect to the payoff associated with each in dividual alternative on a choiceby-choice basis (as predicted by the local -maximization model), then switch points should be insensitive to PI size, as the equality point does not vary with the PI size. The role of immediate and delayed exchange manipu lations is as in Experiment 1: increasing the time frame over which outcomes are realized should decrease the influence of any single delay, bringing choices into greater conformity with global maximiz ation in delayed exchange. Method Subjects and Apparat us Three subjects (Pigeons 14, 18, and 456) from Experiment 1 continued to Experiment 2. Pigeon 7 died unexpectedly soon after the start of Experiment 2 and was replaced by Pigeon 235, a male White Carneau pigeon.

PAGE 49

49 Preliminary Training Pigeon 235 had no prior experience with key pecking, so key pecking was shaped by reinforcing with food successive approximations to pecking He then completed the three phases of preliminary backward -chaining described above in 7 sessions. His training was supplemented w ith delayed-exchange training conditions. These conditions were identical to the final training phase in Experiment 1, but exchange occurred only after a defined number of tokens were earned. Across three delayed -exchange training phases, the number of t okens required before exchange was increased across phases from 6 to 9 to 11 tokens, while the schedule parameters of the FI and PI schedules were increased within phases from 30 and 7, respectively, to 70 and 8, respectively. Pigeon 235 completed sessions in the same chamber as Pigeon 14, and his programmed food access duration was 2 s. Experimental Procedure The experimental procedure was identical to that of Experiment 1 except that the FI value was held constant at 64 s across conditions and the PI v alue was varied from 4 s to 8 s to 16 s across conditions. The immediate -exchange and delayed -exchange conditions alternated at each PI step size per subject, counterbalanced across subjects. Pigeon 235 continued in the sequence established for Pigeon 7, with delayed -exchange conditions preceding immediate -exchange conditions per PI step size. Table 2 shows the order of conditions and the number of sessions conducted in each. Session -termination and stability criteria were as in Experiment 1. Results Al l figures follow the format of those in Experiment 1 and are based on the last 5 sessions of each condition. Figure 3 2 shows the switch points predicted by each model, as in Figure 3 1 (dashed lines), with switch points for each pigeon calculated as the mean of each sessions median switch point (symbols). Error bars represent one standard deviation from the

PAGE 50

50 mean. Switch points were consistently higher in immediate exchange than in delayed exchange. For 3 out of 4 pigeons, the functions relating switch points to PI step size were steeper under delayed -exchange conditions than under immediate -exchange conditions, resulting in a convergence at the highest step size of 16 s. Immediate -exchange switch points generally increased across the range of PI value s. Results of replications of immediate -exchange conditions were similar to original conditions, but results of delayed-exchange replications varied from originals. The largest differences between original and replicated conditions were for Pigeons 18 and 235, which were also the only subjects for whom the delayed -exchange PI 4 replication followed immediate -exchange PI 16. Immediate -exchange PI 16 produced high switch points, and elevated switch points in delayed -exchange PI 4 replication conditions com pared to originals, may be due to carry -over from recent experience. Comparing predicted to actual switch points shows that immediate -exchange switch points were near the predictions of the summed immediacies model. Delayed -exchange switch points were ge nerally closer to the predictions of the global -maximization model but were lower than its predictions in 10 of 17 cases. Figure 3 3 shows deviations of switch points from the predictions of the 3 models. In delayed exchange, switches always conformed mos t to the predictions of global maximiz ation In immediate exchange, switches were best predicted by the summed immediacies model in 10 of 12 cases across subjects. Figure 3 4 shows the percentage of switches occurring at each PI value for each subject. Details are as in Figure 2 4. The distribution of switches in delayed exchange is shifted leftwards compared to the distribution of switches in immediate exchange and delayed-exchange

PAGE 51

51 functions peak to the left of the peak of the theoretical functions w hile immediate -exchange functions peak to the right Figure 3 5 shows mean responses per minute as a function of choice position. Details are as in Figure 2 5. In delayed-exchange conditions, response rates in three out of four pigeons increased modest ly as a function of choice position. Immediate exchange response rates were higher than delayed -exchange rates and did not vary systematically as a function of choice position. Figure 3 6 shows mean choice latencies as a function of choice position for each pigeon. Details are as in Figure 2 6. In delayed exchange conditions, latencies were an inverse function of choice position for all pigeons. In immediate -exchange conditions, latencies were unrelated to choice position but were consistently near the mean latency across delayed -exchange bins. Figure 3 7 shows probability of switching as a function of choice position. In delayedexchange conditions, the probability of choosing the FI was an orderly inverse function of choice position for all pigeons. In immediate -exchange conditions, choosing the FI was unrelated to choice position. Figure 3 8 shows mean reinforcers per minute as a function of PI value for both delayed and immediate -exchange conditions. Error bars represent one standard deviation. Rates of reinforcement were consistently higher in immediate exchange than in delayed exchange. In immediate -exchange conditions, reinforcement rates decreased with increases in PI value. There was no consistent relation between delayed exchange reinfor cement rates and PI value. Discussion Results of Experiment 2 were similar to results of Experiment 1 in that switch points depended on the time frame over which food was delivered. Immediate -exchange switch points were higher than delayed -exchange switc h points. Immediate -exchange switch points increased

PAGE 52

52 with FI value across the range of PI values and conformed quantitatively and qualitatively to the summed im mediacies model. Calculated deviations of obtained switch points from each model showed that i n conditions most like those typically used with pigeons (immediate exchange), switch points were best predicted by the summed -immediacies model, resembling previous results with pigeons (Hackenberg & Hineline, 1992; Mazur & Vaughan, 1987; Wanchisen, Tatha m, & Hineline, 1988). Although the effect of PI value on choices has been examined previously with human subjects (Jacobs & Hackenberg, 1996), this variable has not been manipulated previously in experiments with pigeons. These data therefore extend to a new procedural variation the finding that pigeons choices on the diminishing returns procedure are well -characterized by the summed immediacies model. Delayed -exchange switch points were best predicted by the global -maximization model. This result is c onsistent with previous results with human subjects and token reinforcement procedures (Hackenberg & Axtell, 1993; Jacobs and Hackenberg, 1996). As in Experiment 1, these results suggest that the delayed -exchange condition shares some key features with pr ocedures typically used with human subjects. Deviations of distributions of switches from efficiency functions suggest that switch patterns in delayed -exchange were less efficient in terms of long term outcomes than were immediate -exchange switches. Pat terns of response rates, latencies, and probabilities of choosing the FI were similar to those in Experiment 1, although delayedexchange response rates were less sensitive to proximity to upcoming exchange in Experiment 2 than in Experiment 1. These patt erns contribute to the view of token schedules from Experiment 1. Rates of reinforcement were lower in delayed exchange than in immediate exchange because of the

PAGE 53

53 combined influence of schedule effects and negative deviations of switch points from the glob al maximization model.

PAGE 54

54 Table 3 1. Order of conditions and the number of sessions conducted in each in Experiment 2 Pigeon Exchange condition FI PI Number of sessions 14 Immediate 64 4 13 14 Delayed 64 4 25 14 Immediate 64 8 28 14 Delayed 64 8 27 14 Immediate 64 16 28 14 Delayed 64 16 25 14 Immediate^ 64 4 21 14 Delayed^ 64 4 20 18 Delayed 64 4 12 18 Immediate 64 4 17 18 Delayed** 64 4 3 18 Delayed 64 8 37 18 Immediate 64 8 34 18 Delayed 64 16 53 18 Immediate 64 16 31 18 Delayed^ 64 4 61 1 8 Immediate^ 64 4 41 235 Delayed 64 4 33 235 Immediate 64 4 35 235 Delayed 64 8 40 235 Immediate 64 8 38 235 Delayed 64 16 30 235 Immediate 64 16 12 235 Delayed^ 64 4 21 235 Immediate^ 64 4 29 235 Delayed^ 64 4 32 456 Immediate 64 4 51 456 Delay ed 64 4 52 456 Immediate 64 8 20 456 Delayed 64 8 67 456 Immediate 64 16 13 465 Delayed 64 16 49 456 Immediate^ 64 4 16 456 Delayed^ 64 4 57 ** Indicates condition was run by accident and therefore not carried out to stability criteria and not inclu ded in results. ^Indicates this condition is a replication of a previous condition.

PAGE 55

55 Figure 3 1 Predicted switch points as a function of PI value. Data paths represent predictions of each model.

PAGE 56

56 Figure 3 2 Predictions of each model (dashed line s) and mean switch points (symbols) as a function of PI value. Unconnected points represent replicated-condition switch points. Data are from the last 5 sessions of each condition. Error bars indicate one standard deviation from the mean of the last 5 s essions of each condition.

PAGE 57

57 Figure 3 3 Deviation of mean switch point from prediction as a function of PI value. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. The left column presents delayed -exchange c onditions, and the right column presents immediate -exchange conditions. Symbols correspond to predictive models. All data are from the last 5 sessions of the first exposure to each condition.

PAGE 58

58 Figure 3 4 Percentage of FI choices occurring at each PI value as a function of PI value. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. Conditions having the same PI value are grouped by column, and the PI value is indicated at the top of the column. Black bars represent delayed -exchange conditions, and white bars represent immediate -exchange conditions. Points above bars represent medians of distributions having matching colors, and bars extending horizontally from points represent inter -quartile ranges. All d ata are from the last 5 sessions of the first exposure to each condition. Note x axes scaled individually by column.

PAGE 59

59 Figure 3 5 Mean responses per minute as a function of choice position. Delayed-exchange conditions are in the left column, and imm ediate -exchange conditions are in the right column. In delayed exchange, choices in position 1 are farthest from the upcoming exchange phase, and choices in position 11 are closest to the upcoming exchange phase. Immediate -exchange bins contain the same trial numbers as delayed -exchange bins do. Each row presents graphs from a single subject. Error bars indicate one standard deviation from the mean

PAGE 60

60 Figure 3 6 Mean choice latency as a function of choice position. Details are as in Figure 3 5, exce pt that y axes are scaled logarithmically.

PAGE 61

61 Figure 3 7 Probability of switching as a function of choice position. Details are as in Figure 3 5.

PAGE 62

62 Figure 3 8 Mean reinforcers per minute as a function of PI value. Closed symbols represent delayed -ex change conditions, and open symbols represent immediate exchange conditions. Unconnected points represent replicated conditions. Data are from the last 5 sessions of each condition. Error bars indicate one standard deviation from the mean of the last 5 sessions of each condition

PAGE 63

63 CHAPTER 4 EXPERIMENT 3 Introduction The goal of Experiment 3 was to further distinguish between the predictive models. Mazur and Vaughan (1987) pointed out that the quantitative predictions of global and local maximization c an closely coincide at certain parameter values, and the predictions may be difficult to distinguish from one another. Their solution was to insert an inter trial interval (ITI) after each reinforcer (with reinforcers delivered after each choice). Figure 4 1 shows the predictions of each model as a function of ITI value. The presence of the ITI affects the predictive models differently. Adding an ITI to each choice uniformly changes the delays in an entire experimental session. Similarly, adding an ITI does not change the delays to the upcoming reinforcer, because the ITI is, by definition, beyond the end of the current trial. Therefore, the predictions of the local and globa l -maximization models are unchanged by changes in the value of the ITI across conditions. In contrast, the summed -immediacies model calculates value from a single choice point, and delays later in the sequence of delays are assumed to have less influence on choices than delays at the beginning of the sequence. Adding an ITI furth er diminishes the influence of the later delays in the sequence, and the benefits of resetting the PI to its lowest value are not weighed as heavily. For this reason, higher ITI values result in higher predicted switch points according to the summedimmed iacies model. If switch points increase with ITI value, then the summed -immediacies model is supported. Method Experimental Procedure All four subjects from Experiment 2 served in Experiment 3. The procedure was like that of Experiments 1 and 2 except that the FI value was held constant at 64 s and the PI value was

PAGE 64

64 constant at 8 s. The only other variation was the addition of an inter -trial interval (ITI) between trials in which all lights in the chamber were extinguished. The ITI occurred before each presentation of the choice keys and was varied from 0 s to 24 s to 48 s across conditions. These values were chosen because the predictions derived from them enabled comparisons between the three descriptive models. As before, the immediate -exchange and delayed -exchange conditions alternated at each ITI duration, with counterbalancing across subjects. Table 3 shows the order of conditions and the number of sessions conducted in each. Session termination and stability criteria were as in Experiment s 1 a nd 2 Results All figures follow the format of those in Experiment 1 and are based on the last 5 sessions of each condition. Figure 42 shows the predicted switch points of each model. As i n Figure 4 1, theoretical functions (dashed lines) provide refere nce points against which to compare mean switch points for each pigeon (symbols). Error bars represent one standard deviation from the mean. Switch points were consistently higher under immediate than delayed exchange. For three of four pigeons, switch points increased as a function of ITI value in immediate exchange. Switch points either did not change or were a slightly decreasing function under delayed exchange conditions. Results of replications were similar to original conditions. Immediate excha nge switch points were near the predictions of the summedimmediacies model. Delayed exchange switch points were near the predictions of the global -maximization model, although in 11 of 13 cases were lower than predicted. Figure 4 3 compares deviations in obtained switch points from predicted switch points under each of three models. In delayed exchange, global maximiz ation best predicted switch points In immediate exchange, switches were best predicted by the summed immediacies model in 6 of 12 origina l exposures to conditions. As the ITI value increased, switch points conformed

PAGE 65

65 increasingly well to the local -maximization model. However, when the ITI was 48 s, switching according to the local -maximization model and switching according to the summed im mediacies model differed by only one PI step. Because it is difficult at higher ITI values to distinguish quantitatively between the summed immediacies model and the local -maximization model, it is worth noting that the shape of the switch point function as shown in Figure 4 2 better conforms to the positively sloped predictions of the summedimmediacies model than the flat line of the loca l -maximization model. Figure 4 4 shows the percentage of switches occurring at each PI value for each pigeon. Delayed -exchange distributions were shifted to the left of immediate -exchange distributions for all pigeons. Delayed -exchange distributions often peaked to the left of, and immediate -exchange distributions peaked to the right of, the peak of the theoretical func tions. At the highest ITI value, when the efficiency function was the flattest, inter -quartile ranges of each exchange condition nearly ceased to overlap, indicating that the distributions of switch points differed the most between delayed and immediate exchange at the highest ITI. Figure 4 5 shows mean responses per minute as a function of choice position. Details are as in Figure 2 5. In delayed-exchange conditions, response rates increased slightly as a function of choice position for 2 out of 4 pigeons. In immediate exchange, response rates were higher than in delayed exchange conditions, and there was no relation between response rates and choice position. Figure 4 6 shows mean choice latencies as a function of choice position. Details are as in F igure 2 6. L atencies were an inverse function of choice position for all pigeons in delayedexchange conditions, but were not systematically related to choice position in immediate exchange conditions.

PAGE 66

66 Figure 4 7 shows probability of switching as a functi on of choice position. In delayed exchange conditions, the probability of choosing the FI was an orderly inverse function of choice position for all pigeons. In immediate -exchange conditions, choosing the FI was unrelated to choice position. Figure 4 8 s hows mean reinforcers per minute as a function of ITI value for both delayed and immediate -exchange conditions. Error bars represent one standard deviation. Rates of reinforcement were consistently higher in immediate exchange than in delayed exchange. In both exchange conditions, reinforcement rates decreased with increases in IT I value. Discussion Delayed -exchange switch points conformed closely to the predictions of the global maximization model quantitatively and qualitatively. Immediate -exchange switch points conformed to the summed immediacies model when the ITI was low, and they conformed quantitatively to the local-maximization model at the highest ITI value The overall linearly increasing shape of the function relating switch points to ITI value, however, suggests that the summed immediacies model provides a better description of the data overall. Response rates, latencies, and probabilities of choosing the FI were similar to those in Experiment 2. Delayed exchange switch points occurred a t lower PI values than were predicted by global maximiz ation and in combination with local schedule effects, this resulted in a low rate of reinforcement compared to in immediate exchange.

PAGE 67

67 Table 4 1. Order of conditions and the number of sessions cond ucted in each in Experiment 3 Pigeon Exchange condition FI PI ITI Number of sessions 14 Immediate 64 8 0 13 14 Delayed 64 8 0 12 14 Immediate 64 8 24 85 14 Delayed 64 8 24 55 14 Immediate 64 8 48 40 14 Delayed* 64 8 48 95 14 Immediate^ 64 8 24 76 14 Delayed^ 64 8 24 67 18 Delayed 64 8 0 28 18 Immediate 64 8 0 37 18 Delayed* 64 8 24 89 18 Immediate 64 8 24 19 18 Delayed* 64 8 24 20 18 Immediate 64 8 48 69 18 Delayed 64 8 48 24 18 Immediate^ 64 8 24 28 18 Delayed^ 64 8 24 28 235 Delayed 64 8 0 23 235 Immediate 64 8 0 18 235 Delayed 64 8 24 23 235 Immediate 64 8 24 35 235 Immediate 64 8 48 28 235 Delayed 64 8 48 17 235 Immediate^ 64 8 24 26 235 Delayed^ 64 8 24 74 456 Immediate 64 8 0 24 456 Delayed 64 8 0 74 456 Immediate 64 8 24 78 456 Delayed 64 8 24 52 456 Immediate 64 8 48 65 456 Delayed* 64 8 48 27 456 Immediate^ 64 8 24 54 456 Delayed^* 64 8 24 59 Indicates no more than 11 reinforcers were earned per session in the last 5 sessions of this condition, or there were no sw itches in this condition. These conditions are not included in the results. ^Indicates this condition is a replication of a previous condition.

PAGE 68

68 Figure 4 1 Predicted switch points as a function of PI value. Data paths represent predictions of each mod el.

PAGE 69

69 Figure 4 2 Predictions of each model (dashed lines) and mean switch points (symbols) as a function of ITI value. Unconnected points represent replicated-condition switch points. Data are from the last 5 sessions of each condition. Error bars indicate one standard deviation from the mean of last 5 sessions of each condition

PAGE 70

70 Figure 4 3 Deviation of mean switch point from prediction as a function of ITI value. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. The left column presents delayed -exchange conditions, and the right column presents immediate -exchange conditions. Symbols correspond to predictive models. All data are from the last 5 sessions of the first exposure to each condition except for Pigeon 18s delayed-exchange ITI 24 s condition in which data are presented from the replication because the first exposure produced unstable responding.

PAGE 71

71 Figure 4 4 Percentage of FI choices occurring at each PI value as a function of ITI valu e. Each row presents data from a single subject, and subject numbers are in the leftmost graphs. Black bars represent delayed -exchange conditions, and white bars represent immediate -exchange conditions. Points above bars represent medians of distributions having matching colors, and bars extending horizontally from points represent inter quartile ranges. All data are from the last 5 sessions of the first exposure to each condition except for Pigeon 18s delayed -exchange ITI 24 s condition in which data a re presented from the replication because the first exposure produced unstable responding Note x axes scaled individually by column.

PAGE 72

72 Figure 4 5 Mean responses per minute as a function of choice position. Delayed-exchange conditions are in the left column, and immediate -exchange conditions are in the right column. In delayed exchange, choices in position 1 are farthest from the upcoming exchange phase, and choices in position 11 are closest to the upcoming exchange phase. Immediate -exchange bins contain the same trial numbers as delayed -exchange bins do. Each row presents graphs from a single subject. Error bars indicate one standard deviation from the mean.

PAGE 73

73 Figure 4 6 Mean choice latency as a function of choice position. Details are as in F igure 4 5, except that y axes are scaled logarithmically.

PAGE 74

74 Figure 4 7 Probability of switching as a function of choice position. Details are as in Figure 4 5.

PAGE 75

75 Figure 4 8 Mean reinforcers per minute as a function of ITI value. Closed symbols repr esent delayed -exchange conditions, and open symbols represent immediate exchange conditions. Unconnected points represent replicated conditions. Data are from the last 5 sessions of each condition. Error bars indicate one standard deviation from the mea n of the last 5 sessions of each condition.

PAGE 76

76 CHAPTER 5 GENERAL DISCUSSION The primary goal of these experiments was to investigate how economic context impacted choices between FI and PI schedules. Across all three experiments, immediate -exchange swit ch points were higher than delayed-exchange switch points. In Experiment 1, immediate -exchange switches were more sensitive to changes in the FI schedule than were delayed -exchange switches, indicating that the summed -immediacies model and local -maximizat ion model were better characterization s of immediate -exchange switches than was the global -maximization model. Calculated deviations from the summedimmediacies model were smallest overall. In Experiment 2, calculated deviations indicated again that the summed immediacies model best predicted immediate -exchange switch points. In some conditions of Experiment 3, the quantitative predictions were similar, but the shape of the function relating switch points to ITI value indicated that the summed -immediacie s model was a better fit than the local -maximization model. Overall, based on conformity to the point predictions and the slo pes of predictions, the summed immediacies model best predicted immediate-exchange switches, replicating previous research with pi geons and similar exchange structures (Hackenberg & Hineline, 1992; Neuman, Ahearn, & Hineline, 1997; Neuman, Ahearn, & Hineline, 2000; Wanchisen, Tatham, & Hineline, 1988). Delayed -exchange switches deviated least from the predictions of the global -maximi zation model compared to the other models. This resembled previous results showing that humans responding on diminishing -returns laboratory procedures switched at points close to those that maximized overall rate of reinforcement (Hackenberg & Axtell, 1993; Jacobs and Hackenberg, 1996).

PAGE 77

77 Although the global -maximization model generates predictions based on the assumption that animals maximize rates of reinforcement over an entire series of PI choices followed by an FI, conformity to these predictions in the present study did not yield maximal reinforcement rates. Pigeons in the current experiments obtained lower rates of reinforcement in delayedthan immediate -exchange conditions. Examining response rates, latencies, and probabilities of switching within delayed -exchange cycles revealed patterns that systematically shifted reinforcement rates away from their theoretical maxima. Aside from schedule effects, constraints mediate the relation between currency (such as the maximization of rate of gain) and deci sion variables (the definition of the problem being analyzed), and they fall into two categories: intrinsic and extrinsic (Stephens & Krebs, 1986). Intrinsic constraints include basic phylogenetic factors such as the time required to perceive stimuli, max imum rate of behavior, and time horizons, to name a few. Extrinsic constraints are features of the environment that limit potential maximization of the currency within the decision variables. A bear cannot catch more salmon than are in the stream. In th e current experiments, extrinsic constraints included the negatively accelerating rate of gain associated with persistence on the PI schedule and the cost associated with resetting the PI to its lowest value. Decision variables were switch points, differe nt levels of which were associated with different types of behavioral mechanisms maximizing the currency of rate of gain in different ways. Other experiments have studied the impact on choice of the time frame over which reinforcers are delivered in token settings. As reviewed by Hackenberg (2009), Jackson and Hackenberg (1996, Experiment 4) presented pigeons with choices between 1 token delivered immediately or 3 tokens delivered after a delay of 6 s. In equal delay conditions, exchange periods occurre d 9.5 s after choices were made, regardless of which option was chosen. In

PAGE 78

78 unequal delay conditions, exchange occurred 0.1 s after choices of the smaller reinforcer and 10 s after choices of the larger reinforcer. Overall rates of token and food delivery were held constant across conditions. The equal -delay condition resembled procedures used with humans in which within -session choice options present tokens at locally varying rates, but overall delays to exchange are held constant. Specifically, in proc edures with human subjects, tokens are exchanged for money at the end of experimental participation and the delay to this exchange period does not depend on specific choice patterns. In these procedures, human subjects usually choose the larger, more dela yed reinforcer over the smaller, sooner reinforcer. Jackson and Hackenbergs pigeons, like human subjects, strongly preferred the larger reinforcer in equal delay conditions. Preference reversed in unequal delay conditions conditions more like those used with pigeon subjects whose responses are followed by food. The delayed -exchange conditions in the current experiment were designed along logical lines similar to those behind Jackson and Hackenbergs equal delay conditions. If delay to the exchange pe riod is an important determinant of behavior in token production components, then delayed -exchange conditions should produce switch points different from those in immediate exchange conditions. Specifically, increasing the exchange -production requirements should shift behavior in token -production components towards control by longer term outcomes. The rates and patterns of behavior in the current study suggest that the delayed-exchange conditions be characterized as extended chained schedules. One proposed role of tokens in token reinforcement schedules is that of conditioned reinforce rs --initially neutral stimuli that acquire the ability to reinforce by pairing with food during exchange periods. Tokens did not function just like food in this procedure, or results from immediate and delayed -exchange conditions would be indistinguishable. Instead, response rates and latencies were systematically

PAGE 79

79 related to time from the exchange period and food. This pattern resembles that seen with other extended -chained scheduled and suggests a discriminative role of tokens. On the whole, the results support the view that delayed -exchange cycles can be conceptualized as extended chains. Future Directions Future studies may investigate the role of schedule parameters no t manipulated in the current study. One variable which is likely to have impacted the current results is the number of tokens required prior to exchange in delayed -exchange conditions. The current delayedexchange conditions required 11 tokens before exc hange a requirement that is somewhat higher than that used in other token reinforcement procedures: FR 6 (Webbe & Malagodi, 1978), FR 8 (Bullock & Hackenberg, 2006; Foster et al. 2001), FR 10 (Jackson & Hackenberg, 1996). Perhaps requiring a lower number o f tokens under delayed exchange would better maintain choice patterns under the present procedure some conditions of which had to be excluded due to weakly maintained behavior. More stable responding may also be maintained by requiring a variable number of tokens to be earned rather than a fixed number as in the current study. Foster et al. (2001) showed that pre -ratio pausing in early token -production segments was lower when the number of tokens required per exchange was an average of 8 compared to when it was fixed at 8. Numerous procedural variations are found in considering alternative patch depletion frameworks. Pigeons foraging amongst scattered grains select grain nonrandomly, so when all preferred items are consumed, the patch may instantaneousl y deplete, creating a more bivalued or stepwise rate of depletion (Baum, 1987). This pattern of depletion violates the assumptions of the marginal value theorem, but studying it may illuminate behavioral mechanisms at work in certain environments. Stephe ns and Krebs (1986) describe several hypothetical gain functions,

PAGE 80

80 including one in which the rate of depletion is initially low but increases approaching asymptote (a sigmoid function). Laboratory Analogues of Foraging Theoretical models of maximization are useful in generating quantitative predictions about behavior, but any quantitative model requires a set of assumptions to be precisely fulfilled in order for the predictions to be valuable. Laboratory models of naturalistic settings allow for this wit h the added benefit of allowing relevant variables to be precisely manipulated (Fantino, 1985; Fantino & Abarca, 1985). External validity is sacrificed in the sense that the laboratory setting may not capture every fine detail of the naturalistic setting in which evolutionary pressures operate. For example, risk of predation has likely influenced evolved foraging patterns, but this factor is usually removed in the laboratory. Therefore, conclusions from laboratory analogues cannot be assumed to translate directly into a field setting. Questions about animal behavior can only be extensively explored by translational studies bridging this methodological and theoretical gap between laboratory and field. The current experiments fit into a class of experiments aiming to make foraging models more realistic. Shettleworth (1988) defined this class as containing studies which bring to bear predictions of optimal foraging theory and predictions based in psychology on a single situation. Such experiments can shed light on the behavioral processes at work while potentially refining the constraints of the foraging model. It is important to note that the implications of the current operant foraging analogy are tempered by the assumptions made in its formulation, incl uding that all patches in the environment are of a single type, that prey are encountered stochastically, and that subjects enter each patch fully informed about its schedule parameters rather than gaining information while exploiting the patch. This fina l assumption is addressed by only changing experimental parameters when switch points reached stability criteria. The analogy to a naturalistic foraging

PAGE 81

81 situation would be extended if variations on these features and others were accounted for in the labor atory procedure, but should this be the goal of future studies, the procedure can be modified to fit those needs. The goal of these experiments was to utilize a narrowly-specified version of a situation of diminishing returns as a point of comparison for d ecision -making under differing economic contexts. Certain features of this procedure proved useful in addressing this goal. First, the procedure differs from some other self control tasks in that choices on individual trials are not directly related to l ongor short term maximization, but instead, sequences of choices comprise the behavioral units of analysis. Second, the procedural analogy between naturalistic models of patch choice and laboratory models of diminishing returns such as the current exper iment suggest the marginal value theorem as a useful set of assumptions to which more -psychological approaches may be compared. The implications of these results can be applied to problems beyond the realm of foraging, or the procurement of food. Many decisions faced by human and nonhuman animals involve a tradeoff between a higher local rate of return and a higher global rate of return, and these tradeoffs are embedded within a context of stimuli paired with these outcomes. The fields of behavioral ec onomics, behavioral ecology, and behavior analysis have the common goal of the systematic analysis of behavior. They use different terms for inputs and outputs to the system, including cost and return, constraints and currency, and response requirements a nd reinforcement, and these different terms reflect differences in how inputs and outputs are conceptualized depending on each fields preferred level of analysis. For example, behavioral ecology most often points to causes of behavior which are relevant in an evolutionary time frame, whereas behavior analysis often explains behavior on an individual level and as a result of

PAGE 82

82 mechanisms operating within an animals lifetime. Because of these differences, these fields are complimentary pieces of a complete explanation of behavior.

PAGE 83

83 LIST OF REFERENCES Baum, W. M. (1987). Random and systematic foraging, experimental studies of depletion, and schedules of reinforcement. In A. C. Kamil, J. R. Krebs, & H. R. Pulliam (Eds.), Foraging behavior (pp. 587 607). New Yor k: Plenum. Bullock, C. E., & Hackenberg, T. D. (2006). Second-order schedules of token reinforcement with pigeons: Implications for unit price. Journal of the Experimental Analysis of Behavior, 85, 95 106. Charnov, E. L. (1976). Optimal foraging: The mar ginal value theorem. Theoretical Population Biology, 9, 129136. Charnov, E. L., Orians, G. H., & Hyatt, K. (1976). The ecological implications of resource depletion. American Naturalist, 110, 247259. Fantino, E. (1985). Behavior analysis and behaviora l ecology: A synergistic coupling. The Behavior Analyst, 8, 151157. Fantino, E., & Abarca, N. (1985). Choice, optimal foraging, and the delayreduction hypothesis. Behavioral and Brain Sciences, 8, 315330. Foster, T. A., & Hackenberg, T. D. (2004). Choice and unit price in a token reinforcement context. Journal of the Experimental Analysis of Behavior, 81, 5 25. Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token reinforcement with pigeons: Effects of fixedand vari able ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 76, 159178. Hackenberg, T. D. (2009). Token reinforcement: A review and analysis. Journal of the Experimental Analysis of Behavior, 91, 257286. Hackenberg, T. D. (1998). Laboratory methods in human behavioral ecology. In K. A. Lattal & M. Perone (Eds.), Handbook of research methods in human operant behavior (pp. 541577). New York: Plenum. Hackenberg, T. D., & Axtell, S. A. M. (1993). Human's choices in situations of t ime based diminishing returns. Journal of the Experimental Analysis of Behavior, 59, 445470. Hackenberg, T. D., & Hineline, P. N. (1992). Choice in situations of time -based diminishing returns: Immediate versus delayed consequences of action. Journal of the Experimental Analysis of Behavior, 57, 67 80. Hackenberg, T. D., & Joker, V. R. (1994). Instructional versus schedule control of humans' choices in situations of diminishing returns. Journal of the Experimental Analysis of Behavior, 62, 367383.

PAGE 84

84 Hack enberg, T. D., & Vaidya, M. (2003). Determinants of pigeons' choices in token -based self contro l procedures. Journal of the Experimental Analysis of Behavior, 79, 207218. Hanson, J. (1987). Tests of optimal foraging using an operant analogue. In A. C. Kamil, J. R. Krebs, & H. R. Pulliam (Eds.), Foraging behavior (pp. 335 362). New York: Plenum. Hineline, P. N., & Sodetz, F. J. (1987). Appetitive and aversive schedule preferences: Schedule transitions as intervening events. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Q uantitative analysis of behavior (Vol. 5): The effect of de lay and of intervening events on reinforcement value (pp. 141 157). Hillsdale, NJ: Erlbaum. Hodos, W., & Trumbule, G. H. (1967). Strategies of schedule preference in chimpanzees. Journal of the Experimental Analysis of Behavior, 10, 50 3 4 14. Jackson, K., & Hackenberg, T. D. (1996). Token reinforcement, choice, and self -control in pigeons. Journal of the Experimental Analysis of Behavior, 66, 29 49. Jacobs, E. A., & Hackenberg, T. D. (1996). Humans' choices in situations of time based diminishing returns: Effects of fixed interval duration and progressive interval step size. Journal of the Experimental Analysis of Behavior, 65, 5 19. Jwaideh, A. R. (1973). Responding under chained and tandem fixed ratio schedules. Journal of the Experimental Analysis of Behavior, 19, 259 267. Kelleher, R. T. (1958). Fixedratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281 289. Krebs, J. R. (1978). Optimal foraging: Decision rules for predators. In J. R. Krebs & N. B. Davies (Eds.), Behavioural Ecology: An evolutionary approach (pp. 23 53). Oxford: Blackwell. Lea, S. E. G. (1981). Correlation and contiguity in foraging behavior. In P. Harzem & M. Seiler (Eds.), A dvances in analysis of behavior (Vol. 2 ): Predictability, correlation, and contiguity (pp. 355 406). New York: Wiley. Mazur, J. E., & Vaughan, W., Jr. (1987). Molar optimization versus delayed reinforcement as explanations of choice between fixed ratio and progressive ratio schedules. Journal of the Experimental Analysis of Behavior, 48 251261. McDiarmid, C. G., & Rilling, M. E. (1965). Reinforcement delay and reinforcement rate as determinants of schedule preference. Psychonomic Science, 2 195196. Neuman, P., Ahearn, W. H., & Hineline, P. N (1997). Pigeons' choices between fixedratio and geometrically escalating schedules. Journal of the Experimental Analysis of Behavior, 68, 357374.

PAGE 85

85 Neuman, P., Ahearn, W. H., & Hineline, P. N. (2000). Pigeons' choices between fixedratio and linear or geometric escalating schedules. Journal of the Experimental Analysis of Behavior, 73, 93 102. Pietras, C. J., & Hackenberg, T. D. (2005). Response -cost punishment via token loss with pigeons. Behavioural Processes, 69, 343356. Raiff, B. R., Bullock, C. E., & Hackenberg, T. D. (2008). Response -cost punishment with pigeons: Further evidence of response suppression via token loss. Learning & Behavior, 36, 29 41. Shettleworth, S. J. (1988). Foraging as operant behavior and operant behavior as foraging: What have we learned? Psychology of Learning and Motivation, 22, 1 49. Shull, R. L., & Spear, D. J. (1987). Detention time after reinforcement: Effects due to delay of reinforcement? In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitativ e analyses of behavior (Vol. 5): The effect of delay and of intervening events on reinforcement value (pp. 187 204). Hillsdale, NJ: Erlbaum. Stephens, D. W., & Krebs, J. R. (1986). Foraging theory. Princeton, NJ: Princeton University Press. Wanchisen, B. A., Tatham, T. A., & Hineline, P. N. (1988). Pigeons' choices in situations of diminishing returns: Fixed versus progressive ratio schedules. Journal of the Experimental Analysis of Behavior, 50 375394. Wanchisen, B. A., Tatham, T. A., & Hineline, P. N. (1992). Human choice in "counterintuitive" situations: Fixed -versus progressive -ratio schedules. Journal of the Experimental Analysis of Behavior, 58, 67 85. Webbe, F. M., & Malagodi, E. F. (1978). Second-order schedules of token reinforce ment: Compari sons of performance under fixedratio and variable ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 30, 219224. Yankelevitz, R. L., Bullock, C. E., & Hackenberg, T. D. (2008). Reinforcer accumulation in a token reinforcement co ntext. Journal of the Experimental Analysis of Behavior, 90, 283 299.

PAGE 86

86 BIOGRAPHICAL SKETCH Rachelle Yankelevitz graduated from Winter Park High School in May 1999. She completed her undergraduate education at the University of Florida in 2003, earning bo th a B.S. in p sychology (summa cum laude) and a B.S. in b usiness a dministration (cum laude) with a m ajor in m arketing. She entered the University of Florida graduate program in behavior analysis a nd then earned her M.S. in p sychology in May 2006. She pas sed her qualifying exam in May 2008.