<%BANNER%>

Response-Independent Conditioned Reinforcement in an Observing Procedure


PAGE 1

1 RESPONSE-INDEPENDENT CONDITIONED REINFORCEMENT IN AN OBSERVING PROCEDURE By ANTHONY L. DEFULIO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2007

PAGE 2

2 2007 Anthony L. DeFulio

PAGE 3

3 To my best friend Kirsty, and to Andria, who knows me best.

PAGE 4

4 ACKNOWLEDGMENTS I thank my grandparents, Amelia and Ralph, without whom none of my accomplishments would be possible. I thank my advisor, Tim Ha ckenberg, for his guidance and patience. I thank Katie Saulsgiver, a great friend and wonderful person without whom several of the analyses in the present manuscript would not be possible. I thank Rachel Cassidy and Stacy Strothers, whose countless hours of data analysis were esse ntial in completing the experiments described here. I thank my surrogate siblings, Chris Bullo ck, Bethany Raiff, Rachelle Yankelevitz, Carla Lagorio and Leo Andrade, who conducted many of the experimental sessions and provided me with valuable discussion and feedback. I thank Raul Avilla, who provided suggestions that were critical in the formulation of the experimental procedures. I thank my committee members, Marc Branch, Jesse Dallery, Alan Spect or, and David Daegling, each of wh om was a critical ingredient in my intellectual development. Lastly, I thank Obie for being a constant source of positive reinforcers in a world of lean schedules.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS...............................................................................................................4 LIST OF TABLES................................................................................................................. ..........7 LIST OF FIGURES................................................................................................................ .........8 ABSTRACT....................................................................................................................... ............10 CHAPTER 1 GENERAL INTRODUCTION..............................................................................................12 Assessing Conditioned Reinforcement...................................................................................13 Response-Reinforcer Dependency..........................................................................................16 Observing and Attention........................................................................................................ .21 Uncertainty-Reduction Theory........................................................................................22 Conditioned Reinforcement.............................................................................................24 The Present Study.............................................................................................................. .....26 2 EXPERIMENT 1................................................................................................................... .28 Method......................................................................................................................... ...........29 Subjects....................................................................................................................... .....29 Apparatus...................................................................................................................... ...29 Preliminary Training.......................................................................................................29 Experimental Procedure..................................................................................................31 Results........................................................................................................................ .............33 Discussion..................................................................................................................... ..........36 3 EXPERIMENT 2................................................................................................................... .48 Method......................................................................................................................... ...........48 Subjects and Apparatus...................................................................................................48 Procedure...................................................................................................................... ...49 Results........................................................................................................................ .............51 Discussion..................................................................................................................... ..........54 4 DISCUSSION..................................................................................................................... ....70 Density of S+ and S-........................................................................................................... ....70 Response-Reinforcer Dependency..........................................................................................73 Multiple Functions of Stimuli.................................................................................................76 Observing and Attending........................................................................................................79 Future Directions.............................................................................................................. ......82

PAGE 6

6 LIST OF REFERENCES............................................................................................................. ..85 BIOGRAPHICAL SKETCH.........................................................................................................90

PAGE 7

7 LIST OF TABLES Table page 2-1 Order of conditions and number of sessions per condition for Experiment 1...................39 3-1 Order of conditions and number of sessions per condition for Experiment 2...................56

PAGE 8

8 LIST OF FIGURES Figure page 2-1 Rate of responses on the observing key as a function of rate of response-independent stimulus presentations........................................................................................................40 2-2 Rate of responses on the food key as a function of rate of response-independent stimulus presentations........................................................................................................41 2-3 Discrimination index of food-key respondi ng in the presence of schedule-correlated stimuli as a function of rate of respon se-independent stimulus presentations...................42 2-4 Discrimination index of food-key respondi ng in the presence of the mixed stimulus as a function of rate of responseindependent stimulus presentations...............................43 2-5 Discrimination index of observing-key responding in the presence of the mixed stimulus as a function of rate of resp onse-independent stimulus presentations................44 2-6 Relative rate of respondi ng on the food key as a func tion of rate of responseindependent stimulus presentations...................................................................................45 2-7 Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation for three subjec ts (457, 250, 1774) in Experiment 1......................46 2-8 Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation for three subj ects (96, 289, 1770) in Experiment 1........................47 3-1 Observing responses per minute as a f unction of the duration of the schedulecorrelated stimulus............................................................................................................ .57 3-2 Rate of responses on the observing ke y as a function of dependency ratio.......................58 3-3 Normalized group mean rate of respon ses on the observing key as a function of dependency ratio............................................................................................................... .59 3-4 Rate of responses on the food key as a function of dependency ratio...............................60 3-5 Discrimination index of food-key respondi ng in the presence of schedule-correlated stimuli as a function of dependency ratio..........................................................................61 3-6 Discrimination index of food-key respondi ng in the presence of the mixed stimulus as a function of dependency ratio......................................................................................62 3-7 Discrimination index of observing-key responding in the presence of the mixed stimulus as a function dependency ratio............................................................................63 3-8 Relative rate of respondi ng on the food key as a func tion of rate of responseindependent stimulus presentations...................................................................................64

PAGE 9

9 3-9 Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation for three subjec ts (457, 250, 1774) in Experiment 2......................65 3-10 Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation for three subj ects (96, 289, 1770) in Experiment 2........................66 3-11 Cumulative record showing terminal performance for Pigeon 250 from the last session of the 0/4 dependency ratio c ondition in Phase 1 of Experiment 2.......................67 3-12 Cumulative record showing reacquisition of baseline performance in the first session of a replication of the 4/0 dependency rati o condition in Phase 1 of Experiment 2..........68 3-13 Cumulative record showing stable baselin e performance in the first session of a replication of the 4/0 dependency ratio condition in Phase 1 of Experiment 2.................69

PAGE 10

10 Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy RESPONSE-INDEPENDENT CONDITIONED REINFORCEMENT IN AN OBSERVING PROCEDURE By Anthony L. DeFulio May 2007 Chair: Timothy D. Hackenberg Major: Psychology Pigeons pecks on a response key produced f ood according to a variable-interval (VI) schedule, in which responses produced food ever y 40 s, on average. These VI periods, or components, alternated in irregular fashion, w ith extinction (EXT) comp onents, in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in e ffect. In Experiment 1, these stimuli could be produced every 20 s, on average, by responses on the observing key, and also occurred independently of responding. The rate of such response-independent stimulus presentation did not produce systematic cha nges in observing behavi or. Across phases in Experiment 1, the proportion of session time spen t in the EXT component was adjusted from 0.5 to 0.75 (Group A), and 0.25 (Group B). Increases in the proportion of session time spent in EXT produced increases in the rate of observing. In Experiment 2, density of stimulus presentation was held constant while the dependency between observing and presentation of the stimuli was decreased across conditions. Results indicate that rate of obser ving decreased as the dependency between responses and stimulus pr esentations was decreased. Furt hermore, results indicate that discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Overall, these resu lts show that conditione d reinforcers function

PAGE 11

11 similarly to unconditioned reinforcers with resp ect to response-consequence dependencies, and that stimulus control is greatly enhanced unde r conditions in which th e relevant stimuli are produced by an organisms behavior.

PAGE 12

12 CHAPTER 1 GENERAL INTRODUCTION Pavlov (1927) demonstrated that contiguous pairings of a neutra l stimulus and an unconditioned stimulus such as food produced a tr ansfer of stimulus function, such that the previously neutral stimulus came to elicit res ponses similar to those elicited by the unconditioned stimulus. Pavlovs work emphasized the role of antecedent stimuli in the control of behavior, but the pairing procedure was soon adapted fo r the study of operant conditioning. Thorndike (1898) had already shown that consequences pl ayed a critical role in the acquisition and maintenance of behavior. Thus, it was a simple ne xt step to demonstrate that Pavlovs pairing procedure would produce similar resu lts when the stimuli were used as consequences rather than as antecedents to behavior. Williams (1929) showed that exposing rats to a white discrimination box before delivering food as a consequence for maze running was suffici ent to transfer the reinforcing function of food to the discrimination box. Similarly, severa l researchers showed that poker chips paired with food could be effective in the maintena nce of chimpanzees weight-lifting behavior (Cowles, 1937; Wolfe, 1936). With conditioned reinforcers it was also po ssible to build long sequential chains of behavior in rats, pigeons, and chimpanzees with only minimal support by unconditioned reinforcers (Kelleher, 1966). Thus, conditioned reinforcement might play an important role in the explanation of similar behavior in humans: People rarely engage in behavior which produc es or avoids biologica lly significant stimuli directly. Many people are virtually never deprived of food and wa ter. If one is to account for human behavior in terms of reinforcemen t principles one must look elsewhere for the reinforcers the applicability of the princi ples of behavior c ontrol may depend upon our understanding of conditioned reinfo rcement. (Schwartz, 1978, p. 281).

PAGE 13

13 Assessing Conditioned Reinforcement Although conditioned reinforcement is a valuable explanatory device, empirical evidence suggests that cautious extrapolati on of the principle is in order. Bugelski (1938) conducted an experiment in which rats lever presses produced a click followed by the delivery of food. He then compared performance under conditions in which both the click and the food were unavailable to conditions in which the click continued, but no food was delivered. Although responding was more persistent when the click was presented, the effects were transient and weak. Similar results have been obtai ned by many researchers (Miller, 1951). Not only is continued pairing a requiremen t for continued efficacy of a conditioned reinforcer, but when such stimuli are paired wi th a single kind of unc onditioned reinforcer, the effectiveness of the conditioned reinforcer is dependent on the same establishing operations as the unconditioned reinforcer. For example, if a rat is not water-deprived, then stimuli paired with the delivery of water will not serve as effective reinforcers (Estes, 1949). However, there is no evidence to suggest that repeated presentati on of a conditioned reinfo rcer produces satiation of the kind produced when an animal is given extended access to food. Immediacy is also an important determinant of the reinforcing value of both unconditioned and conditioned reinforcers. It has been demons trated that an unsignale d delay to the delivery food lowers response rates whether responses duri ng the delay increase th e delay (Skinner, 1938) or not (Dews, 1960). Similarly, Wolfe (1936) show ed that tokens paired with food maintained chimpanzees behavior when the tokens were delivered immediately, even when exchanges for food were delayed. When the token delivery was delayed, however, responding deteriorated considerably. Another important piece of evidence that conditioned reinforcers are similar to unconditioned reinforcers has been generated by the new-response procedure. This procedure

PAGE 14

14 begins with repeated pairings of a neutral stimulus and a unconditioned reinforcer. The formerly neutral stimulus is then used to train a new response. Kellehe r (1961) showed th at patterns of behavior generated by conditioned reinforcers in the new response procedure were similar to those generated by unconditioned reinforcers. Using food preceded by a brief click as the consequence, he first trained pigeons to peck a key according to a fixed-interval (FI) 5-min schedule of reinforcement, in which food is pr oduced by the first peck after 5 min has passed from either the start of the session or the prev ious food delivery. This arrangement typically generates a pattern of responding that is positively accelerated acr oss the 5-min interval. Once the characteristic pattern deve loped, Kelleher disconnected the food hopper and used the click alone as the consequence for key pecking. The schedule of reinforcement was then changed to a differential reinforcement of other behavior (DRO ) schedule. Under this contingency, clicks were produced whenever the pigeon did not peck th e response key for a period of 10 s. Kelleher found that pigeons pecked the response key according to the same low-rate pattern typically generated by food reinforcement under simila r conditions. During the same session, the schedule of reinforcement was changed to a fixed ra tio (FR) 1. According to this schedule, every peck to the response key produced a click. Th roughout the remainder of the session the number of pecks required to produce a click was gradually increased. This procedure produced high rates of behavior, interrupted by brief pauses following the delivery of a click, a pattern of behavior identical to that produced by FR schedules when food is used as the reinforcer. Thus, Kelleher demonstrated that sc heduling of conditioned reinfo rcers generated novel response patterns similar to th ose generated by unconditioned reinforcers. Additional evidence that conditioned reinforcer s produce similar patterns of behavior to those produced by unconditioned reinforcement comes from experiments employing

PAGE 15

15 second-order and chained schedules. As in all chained schedule proc edures, a sequence of components, or links, each consisting of a simple schedule of reinforcement, are arranged such that the consequence for completing the terminal link is a unconditioned reinforcer. The consequence for the completion of early links in the chain is the production of the stimuli associated with the next link, and thus greater temporal proximity to unconditioned reinforcement. Second-order schedules are compound schedules in which, the behavior specified by a schedule contingenc y is treated as a unitary res ponse that is itself reinforced according to some schedule of unconditioned reinforcement (Kelleher, 1966). An understanding of this kind of schedule fo llows easily from a consideration of the nomenclature used to describe them. For exam ple, in a FR 4 (VI 30 s) second-order chained schedule, each component requir es a single response after a va riable amount of time has passed, averaging 30 s. Unconditioned reinforcement is produced after every 4th component schedule. The consequence for completion of the other com ponent schedules is an exteroceptive stimulus change (e.g., a key color change) as sociated with greater temporal proximity to food. The effects of these chained schedules are often compared to those of tandem schedules. Tandem schedules are identical to chained schedules with the exception that no exteroceptive stimulus changes occur. Comparisons of this sort are therefore us eful for determining the function of the stimuli in second-order schedules. Gollub (1958) conducted this kind of comparison, and made several important observations. First, responding in the chained schedule was stronger in the terminal link and weaker in the initial link relative to responding in the tand em schedule. Second, the overall pattern of responding generated by the chained sc hedule resembled the pa ttern generated by the component schedule when unconditioned reinforcer was delivered after the completion of each

PAGE 16

16 schedule. This was the case even when chai ns were composed of 5 components. Thus, exteroceptive stimulus changes ac ross early links in th ese chains were producing effects similar to those produced by food as a consequence alth ough they had never been directly paired with food, or even paired with another stimulus that had been directly paired with food. In summary, stimuli that are repeatedly pa ired with unconditioned reinforcers have many similarities to unconditioned reinforcers. Pa tterns of behavior ge nerated by schedules of conditioned reinforcement are similar in char acter to those generated by schedules of unconditioned reinforcement, and behavior is well maintained by the delivery of conditioned reinforcers, as demonstrated by both secondorder and chained schedules. Moreover, the operations that modulate the effectiveness of uncondi tioned reinforcers, such as deprivation with respect to the unconditioned rein forcer and immediacy of delive ry, produce similar changes in the effectiveness of conditioned reinforcers. Response-Reinforcer Dependency Another operation useful in comparing unconditi oned and conditioned reinforcement is the delivery of response-independent reinforcement. There has been substa ntial interest in the effects of delivering response-independent uncond itioned reinforcers, but very few studies have examined the effects of delivering res ponse-independent conditioned reinforcers. The first study of response-independent deli very of reinforcers was conducted by Skinner (1948). Skinner was interested in the patter ns of behavior that developed when food was delivered at regular intervals in dependent of behavior. He noted that pigeons exposed to such fixed-time (FT) schedules eventually developed idiosyncratic stereotypic response patterns that coincided with the delivery of food. Skinner te rmed such patterns superstitious behavior, and argued that such patterns were a function of accide ntal reinforcement of behavior at the moment of food delivery. Although later more detailed analyses called Skinners interpretation into

PAGE 17

17 question (cf. Timberlake & Lucas, 1985), the ex periment provided important evidence that reinforcement has powerful strengthening effects on behavior whether or not it is dependent on responding. Zeiler (1968) investigated the effects of re sponse-independent rein forcers on pigeons key pecking. Pigeons were first trained to peck a response key by the method of successive approximations using response-dependent reinforcer s. In later conditions pigeons were exposed to various response-independent schedules of f ood delivery. In addition to the FT schedules used in previous studies, Zeile r investigated the effects of va riable-time (VT) schedules of response-independent reinforcement, in which food is delivered aperiodica lly and independent of behavior. Zeiler found that FT and VT schedules of reinforcement maintained similar patterns of behavior as FI and VI schedules of reinforcement, respectively. Rate of pecking was lower, however, under FT and VT schedules than their response-dependent counterparts. Of particular interest to the present line of re search are studies that explore the combinative effects of response-dependent and response-in dependent schedules of reinforcement. Investigations of this sort involve FT or VT schedules s uperimposed on underlying responsedependent schedules of reinforcement. Such ar rangements are typically re ferred to as conjoint schedules (Catania, Deegan, & Cook, 1966). Th e effects of such schedules depend on the particular composite schedules used to construct the conjoint schedule. Herrnstein and Morse (1957) reported increases in response rate when signaled FT food deliveries were superimposed on a DRO schedule of reinforcement. Lattal and Bryan (1976) found that response rates either increased or were unchanged when a VT 150 s schedule was superimposed on responses maintained by a FI schedule. When FT schedules were later superimposed on the FI schedule, unsystematic results were obtained. Zeiler (1977) also

PAGE 18

18 obtained unsystematic results when FT schedules were superimposed on behavior maintained by VI schedules. Despite the variation in these resu lts, the most consistent effect of superimposing response-independent schedules on response-dependent schedules of reinforcement is a decrease in response rates (Burgess & Wearden, 1986). Edwards, Peek, and Wolfe (1970) found that reinforcers delivered according to a FT schedule lowered rates of behavi or maintained on a FR schedule of reinforcement, and Zeiler (1979) obtained similar results. The arrangemen ts that have produced the most consistent reduction in rate of behavior re lative to response-dependent base line performance, however, are conjoint VI VT schedules. Rachlin and Baum (1972) found that response rates decreased proportionally with the proportion of reinforcers delivered by the response-dependent schedule. Other researchers have reported similar result s (e.g., Imam & Lattal, 1988; Lattal, 1974). Superimposing a response-independent sc hedule on a response-dependent schedule changes both the temporal relations between responding and food (dependency), and the discriminative control that can arise either as a function of ch anges in the dependency, or as a function of differences in interfood intervals (Lat tal & Abreu-Rodrigues, 1997). The effects of conjoint schedule arrangements of this kind may therefore be a function of the interaction of these variables. Imam and Lattal (1988) showed that decreases in response-reinforcer contiguity accounted for observed decreases in rate of beha vior under conjoint VI VT conditions. These results are similar to those obtained when a de lay to reinforcement is programmed on a simple VI schedule (Sizemore & Lattal, 1977). Whethe r disparities in the results obtained when arrangements other than conjoi nt VI VT can be explained by changes in response-reinforcer contiguity alone, or require a more complex analys is is still unclear. The consistency of the

PAGE 19

19 results generated by conjoint VI VT arrangem ents, however, recommends them as a useful starting point in the examination of response-inde pendent delivery of conditioned reinforcement. Marr and Zeiler (1974) investigated the e ffects of response-dependent and responseindependent brief-stimulus presentations on food-ma intained key pecking of two pigeons. An FI 15-min schedule of food delivery was arranged throughout the experiment. In all conditions, food was delivered 0.5 s after the response that satisfied the schedule requirement. In experimental conditions, the brief stimulus (a 0.5 -s key-color change) was produced according to FI 3 min, VI 3 min, FT 3 min and VT 3 min schedu les, respectively. Each of these arrangements was examined under conditions in which the brief stimulus was paired with the food schedule, and under conditions in which the brief stimulus wa s never paired with food. Early conditions in which the brief stimulus was unpaired with food ha d little effect on behavior. As the experiment progressed, however, the paired and unpaired conditi ons produced similar effects. With respect to the paired conditions, both FI and FT schedules of brief stimulus presentation produced positively accelerated patterns of responding simila r to those typically produced by FI schedules of food reinforcement. In contrast, schedules of VI and VT brief stimul us presentation produced less post-stimulus pausing. This difference is si milar to the difference in response patterning obtained with VI schedules of food reinforcement compared to those obtained with FI schedules. While the FI and VI schedules of brief-stimulus presentation enhanced rates of behavior early in the FI-food interval, FT and VT brief stimulus pr esentations suppressed th is responding. For one pigeon, global rates were substantially reduced in paired FT and VT conditions relative to response-dependent conditions. The other pigeo n, however, showed littl e difference in overall rates across these conditions. The authors concluded that resp onse-independent delivery of

PAGE 20

20 conditioned reinforcers (paired brie f stimuli) has similar effects on the patterning of behavior as the delivery of unconditioned reinfor cers under similar circumstances. One might also ask whether response-independ ent delivery of conditioned reinforcement had similar effects on behavior maintained by conditioned reinforcement. To answer this question, Lieving, Reilly, and Lattal (2006) used an observing-response procedure in which key pecks produced food and treadle presses produced food-correlated stimuli. In Experiment 1, treadle presses during VI food co mponents produced 5-s presentati ons on the food key of stimuli previously correlated with the availability of f ood. In the experimental condition, treadle presses were rendered ineffective, and V I-correlated stimuli were presented according to a VT schedule during VI food components. The VT schedule was yoked to each pigeons own performance during baseline, such that the de nsity of VI-correlated stimulus presentations was held constant across conditions. All pigeons showed a substan tial decrease in rate of treadle pressing during this condition. When the base line conditions were reinstated (response-independent stimulus presentations), responding recove red to levels similar to thos e observed during the original baseline. In Experiment 2, the effects of adding a delay to response-dependent VI-correlated stimulus presentations were examined. Pressing th e treadle during the delay reset the delay to its maximum value, which was manipulated across conditions. The authors found that response rates decreased as an inverse function of delay. Experiment 3 investigated the effects of signaling the delay, and employed a second response key instead of a treadle as the observing response. During signaled delay conditions, th e observing key was darkened during the delay. All subjects pecked the observing key at slight ly higher rates during th e signaled-delay condition compared to the unsignaled condition.

PAGE 21

21 The results of each of these experiments ar e similar to those obtained when similar manipulations are conducted on behavior mainta ined by unconditioned reinforcement. Thus, the limited available literature s uggests that the delivery of re sponse-independent conditioned reinforcers has similar effects as the response-in dependent delivery of unconditioned reinforcers. The present study was designed to investigate the effects of response-independent delivery of schedule-correlated stimuli in the context of an observing pr ocedure. The present study was also concerned with the discriminability of the background reinforcement contextvariables thought to modulate the reinforcing efficacy of the si gnals. This pertains to the broader issue of stimulus control, and attention in particular. Interestingly, it was this issue of attention, rather than conditioned reinforcement, that inspired the observing procedure. Observing and Attention The observing procedure was developed by Wyc koff (1952, 1969) to investigate attention. The original procedure involved pigeons as subj ects and an experimental chamber equipped with two manipulanda. Pecks on a lighted response ke y produced food on 50% of the trials; pressing a treadle just above the floor changed the color of the food key such that a stimulus that was uncorrelated with the trial outcome was replaced with a stimulus th at was correlated either with the presence (S+) or absence (S-) of food, depending on the programmed outcome for that trial. This procedure was deemed relevant to questio ns concerning attenti on because the observing response (treadle press) served only to produ ce correlated stimuli, but had no effect on the availability of food. The proced ure thus allowed for precise measurement of responses deemed analogous to orienting responses such as head or whole body movements toward a stimulus. For several decades following Wyckoffs orig inal report, the observing procedure was a popular choice for researchers inte rested in investigating atten tion in non-human animals. Research questions during this pe riod were largely focused on determining what was maintaining

PAGE 22

22 the observing response. While most researchers agreed that reinforcement was integral to an explanation of observing behavior there was considerab le disagreement about its precise role. Two general kinds of theories emerged in an attempt to explain the maintenance of observing behavior, one of which appealed to the proce ss of uncertainty reducti on, and the other to the process of conditioned reinforcement (Fantino, 1977). Uncertainty-Reduction Theory According to uncertainty-reduction theory, presentation of the Sshould reinforce observing to the extent that such a presentation reduces the uncertainty of a situation (Hendry, 1969). This account, which arose from informa tion theory in applied mathematics and was supported by cognitvely-oriented theorists, predicts that obser ving behavior should decrease as the observing context becomes more predictabl e. For example, in observing procedures, components in which food is available typically alternate with compone nts in which food is unavailable. In a given experi mental session, contingencies are least predictable when 50% of the session is spent in each component, and b ecome more predictable with changes in the proportion of session time in the presence of the tw o stimuli. Because the presentation of S+ and Sreduce uncertainty to an equal extent, the f unction relating observing be havior to proportion of time spent in one of the components should be a symmetrical inverted-U, with a peak at 50%. Wilton and Clements (1971) tested this hypothe sis in a trial-based observing procedure. Pigeons produced outcome-correlated color changes on a lighted response key according to a fixed-interval (FI) 15-s schedule. Completing th e FI requirement produced a trial outcome (light and food, or light only) after a 35s delay, provided that no key peck s occurred in the 5-s prior to the outcome. In one condition the probability of a food outcome was 0.2, and in a subsequent condition this probability was changed to 0.8. B ecause these values deviate from 0.5 in equal

PAGE 23

23 measure, uncertainty-reduction theory predicts that these values s hould generate the same rate of response. However, the authors found that pigeon s reliably pecked at higher rates during the FI period in the 0.2 probability of food condition. Kendall (1973) used a similar procedure, but allo wed trials to progress in the absence of an observing response. When the probability of f ood at the end of a tria l was parametrically manipulated across conditions ( 0.25, 0.5, and 0.75), the rate of obser ving was inversely related to the probability that a trial terminated in food: observing increased as the probability of food decreased. Similarly, McMillan (1974) tested uncertainty -reduction theory, but used a wider range of probability values. Pigeons produced food by p ecking one response key according to a mixed schedule with variable-interval (VI) 70-s a nd extinction (EXT) components (MIX VI EXT). Mixed schedules are compound schedul es of reinforcement in which no exteroceptive stimuli are correlated with the respective component schedules. A single peck on a second key (the observing key) changed the MIX schedule to a multiple (MULT) schedule, in which schedulecorrelated stimuli were available for 20 s. A pr obability gate that de termined which of the component schedules was in effect was sampled ev ery 40 s. The probability of a VI component was manipulated across seven condi tions, including probabilities of 0.0 and 1.0. Contrary to the predictions of uncertainty-reduc tion theory, McMillan found an asymmetrical inverted-U function, with the highest rates of observing at moderately low probability values for 5 of 6 pigeons. Dinsmoor (1983) notes that such re sults are quite common and that the symmetrical inverted-U function relating probability of observing to probability of a positive outcome predicted by uncertainty-reduc tion has never been obtained.

PAGE 24

24 Conditioned Reinforcement The failure of uncertainty-reduc tion theory has led many resear chers to favor explanations of observing based instead on conditioned rein forcement. Although different versions of conditioned reinforcement theory have been used to account for the data generated by experiments employing the observing procedure, all are based on some type of stimulusreinforcer pairing. That is, observing is ma intained by stimuli correlated with unconditioned reinforcement (for reviews, see Dinsmoor, 1983; Fantino, 1977). One apparent problem for explanations based on contiguous stimulus parings is that they seem to require an additional assumpti on that the reinforc ing value of the S+ is greater than the punishing value of the S-. As Dinsmoor (1983) and others have pointed out, however, animals will, to the extent that it is po ssible, maximize their exposure to S+ and minimize their exposure to S-, for example, by turning away from the re sponse key. This account, known as selective observing, preserves a view based on con tiguous stimulus-stimulus pairings. The pairing view is further supported by data showing that presentations of Sdo not maintain observing behavior (Dinsmoor, Browne, & Lawrence, 1972). It has also been shown that observing does not persist when component durations are very brief (Branch, 1973). According to Dinsmoor, this fits well with a selective observing account because, When selectivity of observing is largely precludedby re striction of the length of time for which the S+ is available, artificial obser ving responses become difficult or impossible to maintain. A more quantitatively sophist icated version of the pa iring view of conditioned reinforcement is Fantinos delay-reduction theory. In brief, the theory states that the value of a conditioned reinforcer is a function of the relative reduction in delay to unconditioned reinforcement signaled by its presentation (F antino, 1977). Delay-reduction theory therefore

PAGE 25

25 predicts that stimuli which signa l an increase in the delay to unconditioned reinforcement should not be effective conditioned reinfo rcers, and that behavior whic h produces them should therefore not be maintained. Initially developed to describe the effects of conditioned reinforcement on choice in concurrent-chains procedures, delay-reduction theory has subs equently been extended to observing procedures. For example, Case and Fan tino (1981) exposed pigeons to MIX FI FI FI schedules in which pecks on one response ke y produced food according to one of three alternating FI schedules. P ecks on a second key converted the MIX to a MULT schedule, replacing the uncorrelated stimulus with stimuli that were correlated with the FI schedule in effect on the food key, and in some conditions pr oduced a different (uncorrelated) stimulus. Observing was well maintained by all stimuli that signaled a reduction in the delay to food relative to the mixed stimulus. Contrary to the predictions of delay-reduction theory, observing behavior was weakly maintained by stimuli that signaled an increase in th e delay to food. The results of a second experiment, however, showed that the rate of obser ving was higher when such behavior produced a second, uncorrelated, s timulus than when it produced a stimulus correlated with an increase in delay to food. Th e authors concluded that sensory reinforcement was responsible for maintaining a low level of observing behavior, a nd that delay-reduction theory therefore accounted well for the obtained data. However, uncertainty-reduction theory could not account for the present data, because according to this theory, the reduction in uncertainty provided by stimuli that signal an increas e in the delay to food should be reinforcing. The authors noted, however, that other theori es of conditioned reinforcement could account equally well for their data. Again, delay-reduc tion theory has not been shown to outperform accounts of observing behavior base d on contiguous pairing of stim uli. Delay-reduction theory

PAGE 26

26 is only favored to the extent that it offers precise quantitative pr edictions that can be demonstrated to apply across a range of conditioned reinforcement procedures, including the observing and concurrent-chains procedures. The Present Study The purpose of the present investigation was to further explore the effects of responseindependent conditioned reinforcement on observing behavior. Unlike the Lieving et al. (2006) study, the present study also manipul ated the proportion of session ti me spent in either VI or EXT components of the compound schedule opera ting on the food key. This manipulation was conducted to examine the possibility that the effects of response-independent delivery of schedule-correlated stimuli are modulated by the proportion of S+ versus Sexposures. For example, a greater proportion of Sexposures may produce greater proportional decreases in rate of observing, or greater proporti onal decreases in measures of discriminative control by the schedule-correlated stimuli. Furthermore, the effects of changing the proportion of session time spent in each component of the compound schedule are well replicated and thus serve as solid ground in what is otherwise a relatively novel procedure. Another procedural feature adopted in the present study co ncerned preliminary training with the component schedules. Mixedand multiple-schedule training was begun simultaneously, and training on the observing key did not begin until control by the mixed and multiple stimuli was clearly evident. This tr aining technique was suggested by Branch (1973), because the common technique of introducing the mixed stimulus at the same time as the observing response, and after multiple schedule training, may disrupt food key performance in the presence of the mixed stimulus. Another difference between the present study and Lieving et al. is that observing responses were reinforced according to VI schedules, rather than a FR 1 schedule of

PAGE 27

27 reinforcement. Variable-interval schedules ar e commonly chosen as baselines because they minimize post-reinforcement pausing and engender steady intermediate rates of responding. This is especially important in the present st udy because unlike the Lieving et al. study, which examined response-dependent and response-independent schedules of conditioned reinforcement in isolation, the present study at tempted to demonstrate a more graded function relating response rate to various combinations of conjoint VI VT schedules of conditioned reinforcement. Perhaps the most unique procedural feature of the present study c oncerned the locus of the correlated stimuli. In the majority of prior research on the topic, the mixed and multipleschedule stimuli are presented on a response key. In the present st udy, these stimuli were presented using the houselight, with different flash rates corresponding to the different schedules. This procedural feature is releva nt to the issue of se lective observing. Be cause the flashing of the houselight is visible throughout the experime ntal space, these stimuli cannot be escaped, thereby minimizing the possibility of selective observing.

PAGE 28

28 CHAPTER 2 EXPERIMENT 1 Rachlin and Baum (1972, Experiment III-B) inve stigated the effects of conjoint VI VT schedules of primary reinforcement on pigeons key pecking. These authors found that increasing the proportion of primary reinforcer s per session delivered by the VT (responseindependent) schedule produced decreases in the ra te of key pecking. Th e purpose of the present experiment was to determine whether a similar relation is obtained when response-independent deliveries of conditioned reinforcers are superi mposed on behavior maintained by conditioned reinforcement. Pecks on a response key produced food according to a mixed VI 40-s EXT schedule of reinforcement. Pecks on a second (observing) key produced brief access to stimuli correlated with the component currently in effect on the food key. During baseline conditions, a VI 20-s schedule of stimulus presentations was arranged on the observing key. Across experimental conditions, VT schedules of in creasing density were superimposed on the underlying VI contingency ope rating on the observing key. Another variable of interest was the proporti on of total session time spent in each of the component schedules operating on the food key. After the manipulation described above was conducted under conditions in which 50% of total session time was spent in the EXT component of the mixed schedule on the food key, pigeons we re exposed to a second series of conditions identical to the first with the exception that eith er 25 or 75% of the total session time was spent in the EXT component. This manipulation wa s accomplished by alternatively increasing or decreasing the values of the distribution used to determine the duration of the EXT component.

PAGE 29

29 Method Subjects One female (250) and five male (96, 289, 457, 1770, 1774) adult White Carneau pigeons ( Columba livia ) served as subjects. All subjects were nave with respect to all experimental procedures and maintained at approximately 83% of their fr ee-feeding body weights. The subjects were housed in individua l cages in a colony room where they had free access to fortified water and grit. The colony was kept on a ligh t/dark cycle with a light duration of 16.5 hours beginning at 7AM. Apparatus Sessions were conducted in a st andard three-key operant cham ber. The chamber was lightand sound-attenuating and located in a dark room. An exhaust fan in the chamber and a white noise speaker outside the chamber served to fu rther mask outside noise. The experimental chamber measured 30.5 cm x 35 cm x 35 cm. Keys were arranged 23.5 cm from the floor of the chamber and were spaced 5.7 cm apart. Only the left and center ke ys were operable. A houselight provided general illumination in th e chamber. A solenoid-operated food hopper was used to deliver 3 s access to mixed grain. Respon se keys required a forc e of 0.12 N to register a response. The left key was illuminated red, and the center key white for all subjects across all training and experimental conditions All experimental events were controlled and data collected by a standard PC using MED-PC IV softwa re and located in an adjacent room. Preliminary Training Subjects were run in experiment al sessions seven days per week at approximately the same time every day. Each session began with a 5-mi n blackout period, during wh ich all lights in the experimental chamber were off and responses produced no scheduled consequences. The pigeons were first trained to eat from the food hopper and then trained with food to peck the

PAGE 30

30 white center key via the method of successi ve approximations. Once key-pecking was established, all pigeons were exposed to 21 sess ions in which pecking produced food according to a variable-interval (VI) 40-s schedule of rein forcement. Before the introduction of the left (observing) key, all pigeons were exposed to training sessions in which the schedule of food reinforcement on the center key alternated betw een VI 40 s and extinction (EXT) components. The VI components terminated with food pr esentation. Duration of EXT components was determined by drawing from a 20-element equal probability (Flesher-Hoffman) distribution of values that were identical to bu t independent of the values of the VI 40-s distribution. The VI and EXT components alternated pseudo-randomly su ch that the components could be presented in either order, but with the restriction that an exposure to each component was completed before a random determination of the next component type was made. During these sessions and througho ut the rest of the experime nt, the houselight delineated the S+, S-, and mixed conditions. Specifically, the housel ight flashed at a hi gher rate (on and off every 0.1 s) to signal the VI component (S+) and at a lower rate (eve ry 0.66 s) to signal the extinction component (S-). A steady-on houselight was used to signal the mixed schedule. All stimuli used in the terminal procedure were in troduced together in the training sessions. Alternation between mixed and mu ltiple (schedule-correlated) s timulus types occurred every 10min in the 60-min training sessions. Training se ssions are therefore describable in standard reinforcement-schedule nomencla ture as MULT (MULT VI 40 s EXT / MIX VI 40 s EXT). Training continued until a clear discrimination developed betw een the 3 stimulus conditions, as determined by visual inspection of cumulative records. Following the initial training sessions, the left (observing) key was introduced. In the absence of observing responses, the mixed stim ulus was in place continuously. When the

PAGE 31

31 observing requirement was satisfied the mixed stim ulus was replaced for 3 s with the schedulecorrelated stimulus that corresponded to the sche dule (VI or EXT) currently in effect on the food key. The stimulus changed from Sto S+ imme diately if components changed from EXT to VI during the 3-s stimulus presentation. Because VI components terminated with food delivery, the schedule-correlated stimulus did not change from S+ to Sduring a single presentation. A change-over delay (COD) that prevented the de livery of food for 1.5 s after an observing key peck was in place throughout training and all experimental conditions. Initially the observing-key sc hedule requirement was FR 1. That is, every peck on the observing key changed the houselight from steady-on to flashing correlated with the contingency of reinforcement in place on the food key. To f acilitate acquisition of observing, initial exposure to response-produced schedule-correl ated stimuli was also controlled in such a way that the first 5 exposures to schedule-correla ted stimuli in a session were S+. If the observing key was pecked while an EXT component was in effect then th e schedule-correlated stim ulus was not produced unless the pigeon had produced the S+ at least five times during that session. After several sessions this restriction was lif ted, and the observing schedule was leaned to a VI 10 s across several sessions. Due to low observing rates, Pigeon 96 was exposed to an additional condition in which schedule-correlated stimuli were produc ed only when observing responses were closely (5 s or less) spaced. When observing rates were moderate and stable, the observing schedule was changed to a VI 20-s schedule and Experiment 1 began. Experimental Procedure The schedule of food reinfor cement arranged for pecks on the center key alternated between VI 40 s and EXT, as described above. Sessions ended after 40 food presentations or 75 minutes, whichever came first. The schedule of reinforcement on the observing key was VI 20 s throughout the experiment.

PAGE 32

32 A variable-time (VT) schedule of respons e-independent presentation of schedulecorrelated stimuli was superimposed on the VI 20-s observing sche dule across blocks of sessions, or conditions, within a phase. Experimental phases we re comprised of blocks of conditions, and began with a baseline condition in which no response-independent presentations of schedule-correlated stimuli occurred, followed by a series of conditions in which response-independent presentations of schedul e-correlated stimuli were presented with increasing density (VT 40 s, VT 20 s, and VT 10 s, respectively). No COD was arranged between these presentations and other experime ntal events. Each phase concluded with a replication of the baseline condition. Thus, a pha se consisted of 5 conditions. Conditions were terminated after a minimum of 14 sessions had been conducted and a judg ment of stability had been made. Stability was determined by vi sual inspection of se ssion rates of observing responses for individual subjects. Stability criteria required that no trend appear in the last five sessions of a condition and that neither the highest or lowest session rates of observing responses were contained in the last five sessions of a condition. Across phases the proportion of session time spent in EXT was systematically manipulated. This was accomplished by changing the values in the distribution comprising the durations of EXT components. During Phase 1, pi geons spent approximatel y 50% of the session in EXT. After completion of Phase 1 pigeons were split into two groups. Pigeons in Group A (250, 457, 1774) were exposed to Phase 2, in whic h approximately 75% of total session time was spent in EXT. Pigeons in Group B (96, 289, 1770) were exposed to Phase 3, in which approximately 25% of total session time was sp ent in EXT. The original plan for this experiment was for all pigeons to experience all phases. However, the within-phase changes in the rate of response-independent stimulus pres entation did not produce orderly changes in the

PAGE 33

33 rate of observing behavior. Therefore, Experime nt 2 began after 2 phases of Experiment 1 had been completed. The order of conditions and number of sessions for each condition in Experiment 1 appear in Table 2-1. Results Figure 2-1 shows mean rate of observing respons es as a function of the average number of response-independent stimulus presentations per mi n for the last five sessions of each condition in Experiment 1. Each graph shows data for an individual pigeon. Subject number is shown in the upper-left corner of each graph. Data for Gr oup A pigeons are presented in the left column, and data for Group B pigeons are presented in th e right column. Each data path represents results from a single phase. In general, response rates varied as a function of the proportion of total session time spent in EXT. Specifi cally, for 25 of 29 within-subject across-phase comparisons, rate of observing behavior increa sed as proportion of tota l session time spent in EXT increased. Data were less orderly with respect to within-phase manipulations. Across conditions within Phase 1 (50% EXT), 3 pigeons (250, 289, 457) showed no effect of average number of response-independent pr esentations of the schedule-corre lated stimuli. Data for two of the remaining pigeons (1770, 1774) were char acterized by a U-shaped function, whereas data for a third (Pigeon 96) were ch aracterized by a linearly decreas ing function. Across conditions within Phase 2 (75% EXT), the data were mi xed: no effect for Pigeon 457, an inverted-U function for Pigeon 1774, and a linearly decreasi ng function for Pigeon 250. Across conditions within Phase 3 (25% EXT), the data were simi larly mixed: an inverted-U function for Pigeon 289, and a decreasing function for Pigeon 1770. Ob serving behavior was not well maintained across conditions in Phase 3 for Pigeon 96. Figure 2-2 shows mean rate of responses on the food key as a function of the average number of response-independent st imulus presentations per min for th e last five sessions of each

PAGE 34

34 condition in Experiment 1. Data are displayed similarly to Figure 2-1. For 23 of 29 withinsubject across-phase comparisons rate of pecking on the food key varied as an inverse proportion of total session time spent in EXT. Comparisons across condi tions within Phase 1 indicate small idiosyncratic ch anges in rate of pecking on the food key as a function of the density of response-independent stimulus pres entations. For 3 pigeons (250, 289, 457), rates were highest during baseline cond itions, and then decreased but leveled off across conditions in which response-independent stimuli were presen ted. For Pigeon 96, rates were lowest during baseline conditions, but increased sharply when response-independent stimuli were introduced. Data for Pigeon 1770 indicate an inverted-U shaped function, while data for Pigeon 1774 indicate no effect. Within Phases 2 and 3, the data indicate no effect for all pigeons across all conditions. The lone exception to this was a de crease in rate of peck ing for Pigeon 1770 during the condition in which response-independent stimu li were presented at a rate of 6 per min. To assess the extent to which the schedule-co rrelated stimuli contro lled responses to the food key, Figure 2-3 shows discri mination index (DI) as a functi on of the average number of response-independent stimulus presentations per mi n for the last five sessions of each condition in Experiment 1. Data are displayed similarly to Figure 2-1. The DI valu es shown in Figure 2-3 were calculated using Equation 2-1, where R1 is the key-pecking rate in the presence of S+, and R2 the rate of key pecking in the presence of S-. 2 1 1 R R R DI (2-1) Figure 2-3 indicates strong contro l by the schedule-correlated stimuli across all subjects and conditions, with mean DI above 0.9 in 35 of 59 cases, and above 0.7 in 58 of 59 cases. DI remained relatively stable across all conditions fo r 5 of 6 pigeons. Pigeon 96 showed a slight

PAGE 35

35 decrease in DI as the rate of response-independent deliveries of schedule-correlated stimuli was increased. A second DI analysis was conducted on food-key responses in the presence of the mixed stimulus to determine the extent to which othe r events, such as food delivery, were serving a discriminative function for responses to the food key. Equation 1 wa s used for this analysis, with the rate of response during VI serving as R1, and the rate of response during EXT as R2. Figure 2-4 shows mixed stimulus DI, and is arranged similarly to Figure 2-1. These data indicate that all pigeons were more likely to peck th e food key during VI than during EXT across all conditions and phases. A slight in creasing trend is apparent across conditions in 7 of 12 phases. No effect is apparent across conditions in four phases. Phase 1 (50% EXT) data for Pigeon 96 show a high DI during baseline c onditions, but a more moderate DI across conditions in which stimuli were occasionally delivered independent of responding. A comparison of Figure 2-4 and Figure 2-3 shows that DI was higher in the presen ce of schedule-correlated stimuli than in their absence in all conditions for all subjects. Figure 2-5 shows a third DI analysis and is ar ranged similarly to the preceding figures. Responses on the observing key were analyzed in identical fashion to the analysis of food key responses in Figure 2-4. Data show DI valu es below 0.4 in 55 of 59 conditions across all subjects. Thus, all pigeons were more likely to peck the observing ke y during EXT than during VI in the presence of the mixed stimulus. Pige on 96 was relatively more likely to peck the observing key during VI than the other subjects across Phase 1 conditions. Across conditions a slight decreasing trend is a pparent in 9 of 12 phases. Figure 2-6 shows relative rate of response on the food key (Pfood) as a function of the average number of response-independent stimulus presentations per min. Data were analyzed

PAGE 36

36 using Equation 2-2, where Rfood is the rate of pecking the food key in the presence of the mixed stimulus, Robs is the rate of pecking the observing ke y in the presence of the mixed stimulus. obs food food food R R R P (2-2) As in the preceding figures, each data path repres ents results from a single phase. Data indicate that as proportion of total session time spent in EXT increased, the relative rate of responding on the food key decreased. Relative rates were roughly similar across conditions within a phase for all subjects. The only exception was for Phase 1 da ta for Pigeon 96. Relative rate of responding on the food key was approximately 0.6 during ba seline conditions, and increased sharply to approximately 0.9 across conditions in which re sponse-independent stimuli were presented. Figures 2-7 and 2-8 show the contiguity between observing key p ecks and subsequent response-independent presentations of the schedule -correlated stimuli. Each dot shows a single delay from an observing key peck to the next response-independent pres entation of a stimulus, and triangles indicate median values. If no observing key pecks occurred between two responseindependent stimulus presentations, then the latte r presentation was excluded from the analysis. For six of six pigeons, data show that as the ra te of response-independent stimulus presentations was increased, greater contiguity between observing key pecks and response-independent stimulus presentations was obtained, both in term s of the median delay and the spread in the distributions of obtained delays. Discussion Taken together, the results indicate that be havior was strongly controlled by the schedulecorrelated stimuli, and was systematically relate d to the proportion of total session time spent in EXT. Increasing the density of response-independent stimulus pres entations in the context of a

PAGE 37

37 constant VI 20-s on the observing key, however, pr oduced only relatively weak effects across all dependent measures. Although this latter effect appe ars to be at odds with comp arable manipulations involving unconditioned reinforcement, it was not due to lack of experimental control. First, the orderly changes in observing behavior across phases, across which proportion of session time spend in EXT components was manipulated, are similar to those obtained in previous experiments on observing behavior (e.g., Kendall, 1973; McMi llan, 1974; Wilton & Clements, 1971). Second, the discrimination index for food-key responding in the presence of the multiple stimuli remained high across all conditions of Experi ment 1. These results indicate that S+ was serving both discriminative and conditioned reinforcing func tions, and that the reinforcing efficacy of the S+ did not change apprec iably across conditions. As noted previously, changes in the response-re inforcer dependency have been implicated as a critical controlling variab le in the study of response-inde pendent reinforcement (Lattal & Abreu-Rodrigues, 1997). Across conditions in Ex periment 1, the background VI schedule of reinforcement for observing key pecks remained cons tant. Thus, it is possible that more extreme disruption of the dependency between observing behavior and stimulus presentations could produce more systematic changes in observing be havior. The relativel y subtle changes in dependency were accompanied by an increase in the probability of contiguous pairings of observing responses and response-in dependent delivery of the sche dule-correlated stimuli due to the increasing density of response-independent stimulus presentations across conditions (see Figures 2-7 and 2-8). This account is consistent with data showing th at the pattern of key pecking produced by a VI schedul e of food presentation is relati vely unchanged when the VI schedule is replaced with a VT schedule (Marr & Zeiler, 1974; Zeiler, 1968). This possibility

PAGE 38

38 was explored in Experiment 2 by holding th e density of schedule-correlated stimulus presentations constant across sessions, and manipulating the dependency via concomitant changes in both the VI and VT schedules operating on the observing key.

PAGE 39

39 Table 2-1. Order of conditi ons and number of sessions pe r condition for Experiment 1. Group A Group B Condition/ Phase 457 250 1774 96 289 1770 Phase 1 (50% EXT) VI 20" 55 56 53 32 35 22 VI 20" + VT 40" 25 23 29 19 36 22 VI 20" + VT 20" 20 18 17 16 40 18 VI 20" + VT 10" 15 14 15 14 15 18 VI 20" 29 17 52 14 30 28 Phase 2 (75% EXT) VI 20" 37 20 74 VI 20" + VT 40" 14 16 15 VI 20" + VT 20" 52 21 24 VI 20" + VT 10" 25 30 27 VI 20" 49 26 13 Phase 3 (25% EXT) VI 20" 60 39 25 61 VI 20" + VT 40" 41 17 16 VI 20" + VT 20" 34 38 16 VI 20" + VT 10" 20 63 22 VI 20" 20

PAGE 40

40 Figure 2-1. Rate of responses on the observing key as a function of rate of response-independent stimulus presentations. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indicate standard deviations.

PAGE 41

41 Figure 2-2. Rate of responses on the food key as a function of rate of response-independent stimulus presentations. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indicate standard deviations.

PAGE 42

42 Figure 2-3. Discrimination inde x of food-key responding in the presence of schedule-correlated stimuli as a function of rate of response-i ndependent stimulus presentations. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error ba rs indicate standard deviations.

PAGE 43

43 Figure 2-4. Discrimination inde x of food-key responding in the pr esence of the mixed stimulus as a function of rate of response-indepe ndent stimulus presentations. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error ba rs indicate standard deviations.

PAGE 44

44 Figure 2-5. Discrimination i ndex of observing-key responding in the presence of the mixed stimulus as a function of rate of resp onse-independent stimulus presentations. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Erro r bars indicate standa rd deviations.

PAGE 45

45 Figure 2-6. Relative rate of responding on the food key as a function of rate of responseindependent stimulus presentations. Sym bols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indicate standard deviations.

PAGE 46

46 Figure 2-7. Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation as a function of th e rate of response-independent stimulus presentations for three subjects (457, 250, 1774) in Experiment 1. Points show individual obtained delays from the last five sessions of each condition. Open triangles indicate median values.

PAGE 47

47 Figure 2-8. Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation as a function of th e rate of response-independent stimulus presentations for three subjects (96, 289, 1770) in Experiment 1. Points show individual obtained delays from the last five sessions of each condition. Open triangles indicate median values.

PAGE 48

48 CHAPTER 3 EXPERIMENT 2 To manipulate the dependency between obser ving responses and the presentation of schedule-correlated stimuli, a procedure was em ployed that mimicked a previous study designed to investigate the relationship between dependency and unconditioned reinforcement. Lattal (1974) arranged a procedure in which the rate of reinforcement was held constant while the dependency between pigeons key pecks and food de livery was manipulated in a graded fashion across conditions. This was accomplished by firs t establishing baseline performance according to a VI schedule of reinforcement, and followi ng this with a condition in which a similarly valued VT schedule arranged food deliveries independent of responding. Following these conditions, a series of intermediate dependency conditions were conducted in which every third or every tenth reinforcer deliver y was contingent on a response. In addition, one pigeon was exposed to a condition in which every third rein forcer was delivered independent of responding, but the remaining reinforcers were contingent on a response. T hus, conditions were conducted in which 100, 66, 33, 10, and 0% of the reinforcer s were response-dependent. Results indicated that the rate of key pecking decreased as the proportion of response-dependent reinforcers decreased. In the present experiment a sim ilar manipulation was conducted, but the technique used to accomplish this was slightly different, involving conjoint VI VT schedules in various combinations, as described below. Method Subjects and Apparatus Subjects and apparatus were identical to those used in Experiment 1.

PAGE 49

49 Procedure All procedural details were identical to thos e of Experiment 1, unless otherwise noted. Experiment 2 began immediately after Experime nt 1 without a break. Prior to Phase 1 a condition was conducted in which the duration of schedule-correlated stimulus presentation was 10 s. All other procedural detail s were identical to those of the baseline condition of Phase 1 of Experiment 2. This condition was conducted to determine if increasing the stimulus duration would produce higher rates of pecking on the obs erving key. After the completion of this condition the duration of schedule-correlated stimul us presentation was reduced to 3-s. Because most pigeons pecked on the observing key more frequently when the stim ulus duration was 3-s, this stimulus duration was used throughout Experiment 2. Across conditions within a phase contingencies were changed such that the programmed density of schedule-correlated stimuli was held constant, while the level of dependency between observing responses and schedulecorrelated stimulus presenta tions was manipulated. During baseline conditions, schedule-correlated stimuli were produced according to a VI 15-s schedule, for an average of 4 response-dependent stim ulus presentations per min, and 0 responseindependent stimulus presentations (4, 0). Acro ss conditions within Phase 1, the average number of response-dependent stimulus presentations per min was decreased by one, and the average number of response-independent st imulus presentations was increa sed by one, until all stimulus presentations occurred independe nt of responding. Thus, the sc hedule of stimulus production operating on the observing key during the second c ondition of Phase 1 was a conjoint VI 20 s VT 60 s (3, 1). This condition was followed by conditi ons in which the schedule was conjoint VI 30 s VT 30 s (2, 2), conjoint VI 60 s VT 20 s (1, 3), and then VT 15 s (0, 4). The phase was concluded with a replication of the VI 15 s baseline (4, 0).

PAGE 50

50 Manipulation of the other major independent variable (proportion of session time in EXT) followed the pattern established in Experiment 1, except that all subjects received all levels of the independent variable: 50% (Phase 1) 75% (Phase 2), and 25% (Phase 3). This was accomplished by changing the mean of the recta ngular distribution tied to EXT component duration to 180 s (Phase 2) or 20 s (Phase 3). Also unlike Experiment 1, component durations in Experiment 2 were determined by two identical independent 11-element rectangular distributions averaging 60 s. In this way, the delivery of food on the VI schedule would not signal the absence of food (EXT). The order of conditions and number of sessions for each condition in Experiment 2 appear in Table 3-1. Conditions were terminated af ter a minimum of 14 sessions had been conducted and a judgment of stability had been made. St ability was determined by visual inspection of session rates of observing responses for individual subjects. St ability criteria required that no trend appear in the last five sessions of a conditio n and that neither the highest or lowest session rates of observing responses were contained in the last five sessions of a condition. When Phase 3 began rates of observing behavi or dropped substantially. Thus, guidelines were established to determine the sequence of conditions in Phase 3 dependent on individual pigeons performance. If the mean rate of obser ving key pecks across the la st five sessions of the Phase 3 baseline condition was less than one per minute, then no conditions involving response-independent stimulus pr esentation were conducted. In stead, an A-B-A reversal was conducted in which the proportion of total session time spent in EXT was changed from 25% to 50%, and then back to 25%. Only Pigeon 250 followed this sequence of conditions. If the mean rate of observing key pecks across the last five sessions of the Phase 3 baseline was greater than one per minute, but less than 1/3 of the rate observed in the Phase 2 baseline replication

PAGE 51

51 condition, then an A-B-A reversal was conducte d in which the dependency ratio was changed from 4/0 to 0/4, and then back to 4/0. Only Pigeon 457 followed this sequence of conditions. The sequence of Phase 3 conditions for Pigeons 1774, 96, and 289 was identical to the sequence of conditions conducted in Phase 2. Pigeon 1770 was not exposed to Phase 3 conditions because this subject required substantially more time to complete the first 2 phases. Results Figure 3-1 is a comparison of th e rates of observing key pecks in the presence of the mixed stimulus across conditions in which the duration of schedule-correlated stimulus presentations was manipulated from 10 s (black bars) to 3 s (gre y bars). Each set of bars show data for an individual subject, as indicated on the X-axis. For 4 of 6 pigeons observing rates were higher when the stimulus duration was 3 s than when it was 10 s. Rates were similar under both stimulus durations for Pigeon 457 and were higher under the 10-s dur ation for Pigeon 1774. Figure 3-2 shows for each pigeon the mean rates of observing responses for the last five sessions of each condition as a function of the dependency ratio, defined as the average number of schedule-correlated stimulus presentations available per min from VI and VT schedules, respectively. Within-phase functions are char acterized by an inverse relationship between the dependency of observing key pecks on the pres entation of schedule-corre lated stimuli and the rate of pecking the observing key. A decreasi ng trend in rate of observing key pecks across conditions is apparent in all 16 phases across a ll subjects. Comparisons between conditions of maximal and minimal dependency (4/0 and 0/4 dependency ratios, respectively) show lower rates in the latter for 16 of 16 comparisons. Co mparisons between Phases 1 and 2 indicate that observing rates did not change substantially wh en the proportion of session time spent in EXT was increased from 50% to 75%, although rates were slightly higher under Phase 2 conditions

PAGE 52

52 (75% EXT) in 14 of 21 cases. Phase 3 conditions (25% EXT) generated lower rates of observing than comparable conditions in Phases 1 and 2. Figure 3-3 shows normalized group mean observing key pecks for all conditions of Experiment 2. This analysis was conducted to assess the extent to which changes in the proportion of session time spent in EXT interacted with changes in the dependency ratio. The extensive overlap of the three functions on this figure indicate that these variables produced additive, rather than interactive eff ects on the rate of observing key pecks. Figure 3-4 shows mean response rate on the f ood key as a function of dependency ratio. None of the pigeons showed systematic change s in response rate across conditions within a phase. Across-phase comparisons indicate food-key response rates were inversely related to the proportion of session time spent in EXT for four pigeons (457, 250, 289, 1770). Pigeons 96 and 1774 showed the highest rates of responding to th e food key when the proportion of session time spent in EXT was 50%. These results are similar to those obtained in Experiment 1. Figure 3-5 shows the discrimination index fo r food-key responding in the presence of the schedule-correlated stimuli. Discrimination i ndex was calculated with Equation 1, using the same measures as in Figure 2-3. As in Experi ment 1, this measure indicates strong control of food-key responding by schedule-correlated stimuli fo r all pigeons across all conditions. A clear decreasing trend is apparent in 16 of 16 cases across conditions within a phase. Between-phase comparisons indicate that proportion of session time spent in EXT did not affect discriminability of the schedule-correlated stimuli, as indicated by the overlapping curves for 5 of 6 subjects. Figure 3-6 shows the discrimination index fo r food-key responding in the presence of the mixed stimulus, calculated as in Figure 2-4. For all pigeons the disc rimination-index values were approximately 0.6 acro ss all conditions and phases.

PAGE 53

53 Figure 3-7 shows the discrimination index for observing responses in the presence of the mixed stimulus. The discrimination-index values were roughly stable across conditions, with values generally ranging from 0.3 to 0.5. This is consistent with the stability observed in the discrimination index for food-key responding shown in Figure 3-6, and indicates that, in the presence of the mixed stimulus, pigeons were mo st likely to peck the observing key during EXT, and most likely to peck the food key during VI. Figure 3-8 shows the relative rate of res ponding on the food key in the presence of the mixed stimulus, calculated according to Equatio n 2. Relative rates were above 0.6 for all pigeons under all conditions, and above 0.8 in 54 of 73 cases. Comparisons across conditions within a phase indicate an increa se in the relative rate of f ood key responding as dependency ratio decreased. Across-phase comparisons show th at the highest relative rates were obtained in Phase 3 (25% EXT) in 5 of 5 cases. Comparis on of relative rates obtained in Phases 1 and 2 show no difference in 5 of 6 cases. Pigeon 457 s howed slightly higher relative rates during Phase 1 (50% EXT) than Phase 2 (75% EXT). The contiguities between observing key peck s and subsequent response-independent presentations of the schedule-co rrelated stimuli are shown in Figures 3-9 and 3-10. These figures are arranged similarly to Figures 2-7 and 2-8. The data show that temporal contiguity between these events decreased as the dependency ratio was decreased for all pigeons, as evidenced by increases in the median delay and th e spread in the distributions of obtained delays. Figures 3-11, 3-12, and 3-13 are representativ e cumulative records from two consecutive sessions for Pigeon 250. Figure 3-11 shows the fina l few minutes of the last session of the 0/4 dependency ratio condition in Phase 1 (50% EXT). All schedule-correlated stimulus presentations were response independent in this condition. The observi ng key record for this

PAGE 54

54 figure is generally flat, and is punctuated by two brief periods of activity during EXT components. Figure 3-12 shows the beginni ng of the following session, in which the dependency ratio was changed to 4/0. No peck s on the observing key occurred for the first three minutes of this session. The first peck on th e observing key produced S-, after which rate of observing key pecks briefly increased. Figure 3-13 shows the typical pattern of responding during baseline conditions. Note that this r ecord shows performance after approximately 3.5 minutes had passed from the end of Figure 3-12. Thus, reacquisition of terminal response patterns was rapid. Note the bursts of respondi ng to the observing key during EXT components, as well as the higher rates of respons e on the food key during VI components. Discussion The results of the present experiment show th at observing responses were controlled by the dependency between observing responses and the pres entation of schedule-correlated stimuli. Specifically, decreases in the dependency ratio decreased rates of re sponding on the observing key. This finding extends the findings of Latta l (1974) to behavior maintained by conditioned reinforcement and is consistent with the findings of Liev ing et al. (2006). It is interesting to note th at the discrimination index fo r food-key responding in the presence of the multiple stimuli also decreased as the dependency between observing key pecks and schedule-correlated stimulus pr esentations decreased. This wa s a product of incr eases in rate of responding on the food key during Spresentations (data not shown) As shown in Figure 3-7, responses on the observing key were more likely during EXT components throughout the experiment. Thus, decreases in the rate of observing reduced competition between the observing and food keys, especially during EXT components, as response-independent Spresentations became more likely. This pattern of behavior al so indicates that the pi geons did not move away from the intelligence panel during Spresen tations, as suggested by a selective observing

PAGE 55

55 account of observing behavior. Personal observa tions of the experimental sessions further substantiate that the pigeons in this experiment typically remained close to the intelligence panel throughout the sessions. It seem s likely that delivering the sche dule-correlated stimuli via the houselight prevented the development of selective observing. Lastly, increasing the duration of schedule-correlated stimul us presentations generally reduced rate of responding on the observing key in the present experiment. This finding is limited because no reversal conditions were conducted. Dinsmoor, Mulvaney, & Jwaideh (1981) showed that pigeons prefer long-duration S+ presentations over short duration S+ presentations. However, measures of preference do not necessarily correspond to measures of rate of response. For example, it is possible that one schedule of reinforcement would produce higher rates of response than another schedule of reinforcement wh en exposure to both schedules is forced, but that the schedule that engendered th e lower rates of behavior would be chosen more frequently in a preference assessment. Further experimentation is required to determine the precise relations that controlled this outcome.

PAGE 56

56 Table 3-1. Order of conditi ons and number of sessions pe r condition for Experiment 2. Condition/ Phase 457 250 1774 96 289 1770 Phase 1 (50% EXT) VI 15"a 20 20 20 15 29 20 VI 15" 42 36 16 72 24 49 VI 20" + VT 60" 14 22 20 28 18 21 VI 30" + VT 30" 14 19 58 21 35 57 VI 60" + VT 20" 14 23 19 28 19 34 VT 15" 17 16 41 15 22 18 VI 15" 25 24 24 25 32 57 Phase 2 (75% EXT) VI 15" 25 14 15 22 14 16 VI 30" + VT 30" 38 17 14 15 14 36 VT 15" 18 37 16 17 17 21 VI 15" 29 40 18 15 27 Phase 3 (25% EXT) VI 15" 41 28 32 25 24 VI 30" + VT 30" 22 19 27 VT 15" 16 13 16 16 VI 15"b 14 VI 15" 16 25 Note: a Schedule-correlated stimulus durati on was 10 s during this condition, b 50% of session time spent in EXT during this condition.

PAGE 57

57 Figure 3-1. Observing responses per minute as a function of the durat ion of the schedulecorrelated stimulus. Error bars indicate standard deviations.

PAGE 58

58 Figure 3-2. Rate of responses on the observing key as a functi on of dependency ratio. Symbols indicate proportion of total session time spent in EXT. Error bars indicate standard deviations. Unconnected point s are data from replicated conditions. The open circle indicates a replication of the 50% EXT baseline condition during Phase 3.

PAGE 59

59 Figure 3-3. Normalized group m ean rate of responses on the observing key as a function of dependency ratio. Symbols indicate proportion of total se ssion time spent in EXT.

PAGE 60

60 Figure 3-4. Rate of responses on the food key as a function of dependency ratio. Symbols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indi cate standard deviation. The open circle indicates a replication of the 50% EXT baseline condition during Phase 3.

PAGE 61

61 Figure 3-5. Discrimination inde x of food-key responding in the presence of schedule-correlated stimuli as a function of dependency rati o. Symbols indicate proportion of total session time spent in EXT. Unconnected point s are data from replicated conditions. Error bars indicate standard de viations. The open circle in dicates a replication of the 50% EXT baseline condition during Phase 3.

PAGE 62

62 Figure 3-6. Discrimination inde x of food-key responding in the pr esence of the mixed stimulus as a function of dependency ratio. Sym bols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indicate standard deviations. The open circ le indicates a replication of the 50% EXT baseline condition during Phase 3.

PAGE 63

63 Figure 3-7. Discrimination i ndex of observing-key responding in the presence of the mixed stimulus as a function dependency ratio. Symbols indicate proportion of total session time spent in EXT. Unconnected points ar e data from replicat ed conditions. Error bars indicate standard deviations. The ope n circle indicates a re plication of the 50% EXT baseline condition during Phase 3.

PAGE 64

64 Figure 3-8. Relative rate of responding on the food key as a function of rate of responseindependent stimulus presentations. Sym bols indicate proportion of total session time spent in EXT. Unconnected points are data from replicated conditions. Error bars indicate standard deviations. The open circ le indicates a replication of the 50% EXT baseline condition during Phase 3.

PAGE 65

65 Figure 3-9. Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation as a function of the dependency ratio for three subjects (457, 250, 1774) in Experiment 2. Points show individual obtained delays from the last five sessions of each condition. Open triangles indicate median values.

PAGE 66

66 Figure 3-10. Delay from an observing key peck to a response-independent schedule-correlated stimulus presentation as a function of the dependency ratio for three subjects (96, 289, 1770) in Experiment 2. Points show individual obtained delays from the last five sessions of each condition. Open triangles indicate median values.

PAGE 67

67 Figure 3-11. Cumulative record showing terminal performance for Pigeon 250 from the last session of the 0/4 dependency ratio condition in Phase 1 of Experiment 2. The upper line shows food key pecks and the lower line shows observing key pecks. Tick marks at the zero point of the Y-axis show food deliveries. The upper event record shows the schedule in operation on the food key, w ith an upward displacement of the record indicating EXT. The lower event record in dicates when schedule-correlated stimuli were presented, with a downward displacem ent of the record indicating the presence of schedule-correlated stimuli. Note th e break on the Y-axis and corresponding change in scale.

PAGE 68

68 Figure 3-12. Cumulative record showing reacqui sition of baseline performance in the first session of a replication of the 4/0 de pendency ratio condition in Phase 1 of Experiment 2. Data are arranged as in Fi gure 3-11. Note the break on the Y-axis and corresponding change in scale.

PAGE 69

69 Figure 3-13. Cumulative record showing stable baseline performance in the first session of a replication of the 4/0 dependency ratio condi tion in Phase 1 of Experiment 2. Data are arranged as in Figure 3-11. Note th e break on the Y-axis and corresponding change in scale

PAGE 70

70 CHAPTER 4 DISCUSSION In Experiments 1 and 2, across-phase change s in the proportion of session time spent in EXT components of a mixed schedule of food delivery produced orderly changes in rate of observing key pecks. In both experiments, pige ons observed at lowest rates in the 25% EXT Phases. In Experiment 1, the 75% EXT Phase pr oduced the highest rates of observing, while in Experiment 2 similar observing rates were pr oduced by the 50% EXT and 75% EXT Phases. In the presence of the mixed stimulus, obser ving responses were most likely during EXT components for all pigeons throughout both experiments. In Experiment 1, the rate of responseindependent schedule-c orrelated stimulus presentations was increased acros s sessions in the context of a constant response-dependent schedule of conditioned reinforcement. This ma nipulation produced unsystematic changes in the rate of observing key pecks, and did not produ ce appreciable changes in the discriminative control by the schedule-correlated stimuli. In Experiment 2, density of schedule-correlated stimulus presentations was held constant acro ss conditions and the comp osite schedules of the conjoint VI VT sche dule operating on the observing key were varied to manipulate the dependency between responses and schedule-correla ted stimulus presentations across conditions. Results showed orderly decrease s in the rate of observing ke y pecks and the discriminative control by the schedule-correlate d stimuli as the dependency between pecks and stimulus presentations was decreased. The implications of these results for theories of conditioned reinforcement, observing, and attention will be discussed in this chapter. Density of S+ and SThe effects of the across-phase manipulation in both experiments, in which the proportion of session time spent in the respective component schedules operating on the food key was

PAGE 71

71 varied, provides a point of comp arison to prior research (e.g., McMillan, 1974; Kendall, 1973). Comparisons across the baseline conditions of ea ch phase in Experiment 1 show results very similar to those reported by Kendall. In genera l, when a larger proportion of session time was spent in EXT components, rates of observing behavior were higher. Results from baseline conditions in Experiment 2 show a slightly different relationship, however. Wilton and Clements (1971) described a theoretical function re lating the reinforcing value of S+ presentations as a function of the probability of their presentation. According to this function, the 50% and 75% EXT Phases shoul d produce identical rates of response on the observing key, but the 25% EXT Phase should pr oduce considerably lower rates of observing behavior. The data obtained in Experiment 2 are broadly consistent with this prediction; rate of observing behavior was similar across the 50% and 75% EXT Phases, but dropped considerably during the 25% EXT Phase. An additional comparison relevant to the present study was conducted by McMillan (1974). Two pigeons spent more time in the presence of schedule-co rrelated stimuli under conditions in which the probability of a positiv e outcome was 0.2 (roughly similar to the 75% EXT Phase in the present experiment) than when this probability was 0.5 (comparable to the 50% conditions). Another 2 pigeons showed si milar performance under th ese conditions and the remaining 2 pigeons spent more time in the pres ence of the schedule-correlated stimuli during the 0.5 probability condition. The reason for the difference between the resu lts of the across-phase manipulations in Experiments 1 and 2 is easily explained. Duri ng baseline conditions in these experiments, no response-independent stimuli were presented. Thus, the only difference between these conditions was the method used for switching be tween components operating on the food key.

PAGE 72

72 In Experiment 1, VI components terminated with food delivery, wh ile in Experiment 2 component alternations were unrelated to othe r experimental events. The method used in Experiment 1 enhances component discriminabili ty in the presence of the mixed stimulus. A comparison of the baseline discriminability indi ces for food-key responding in the presence of the mixed stimulus indicates that discriminab ility was roughly similar during baseline 50% EXT conditions across the two experiments, but that the respective components were more discriminable during 75% EXT baseline in Expe riment 1 than in the same condition in Experiment 2. Why this would produce the diffe rence in baseline observing rates across the two experiments requires a consideration of the typica l pattern of behavior displayed by the pigeons in both experiments. The discrimination indices of observing key respond ing in the presence of the mixed stimulus indicate that observing p ecks were more likely during EXT components under all conditions in both experiments (see Figures 2-5 and 3-7). Kelleher, Riddle, and Cook (1962) provided the following account of the pattern of behavior they observed under base line conditions similar to those used in the present study: When observing responses produced the negative stimulus, the bird paused on both keys. When the negative stimulus went off, the bi rd responded on the observing key until the negative or positive stimulus appeared. If the positive stimulus appeared, the bird responded at a high rate on the food producing key. When the positive stimulus went off, the bird continued to res pond on the food-producing key in the mixed stimulus (p. 6). Similar patterns of behavior we re evident in the present stud y, as shown in Figure 3-13. As in the present experiments, Kelleher et al. reported that most of the observing key pecks occurred during EXT components. Thus, it could be expected that extended bouts of observing key pecks would follow food deliveries during th e Experiment 1 75% EXT baseline condition, when such deliveries signaled periods of food un availability three times as long as under 50%

PAGE 73

73 EXT. In contrast, food deliveries in Experime nt 2 could be followed by another food delivery only seconds later in both 75% and 50% EXT conditions. The present data are also consistent with delay-reduction theory. Consider the most consistent finding from the across-phase manipulati ons in the present experimentsthat rate of observing behavior was lower in the 25% EXT Phases than in the 50% EXT Phases. The interreinforcement interval in the presence of the mixed stimulus was 60 s in the former case and 80 s in the latter case. In both cases presentation of the S+ signaled a reduc tion in the average delay to food delivery (40 s). This indicates a far gr eater reduction in the delay to food in the 50% EXT condition. Delay-reduction theory therefor e predicts that this condition would produce higher response rates. This account matches th e obtained result in the present experiments. These results are also consistent, however, with modern pairing theories, because these theories predict that the strength of a conditioned res ponse is in part a function of the conditional probability of an unconditioned stimulus in the pres ence of a conditioned stimulus relative to the conditional probability of an unconditioned stimulus in the absence of a conditioned stimulus (Rescorla, 1967). Thus, the results of the across-phase manipulation support conditionedreinforcement based theories of observing behavior in general, but do not differentially support a particular version of such theories. The re sults do not support any version of uncertaintyreduction theories, however, because such theories predict that the 25% and 75% EXT phases should produce similar rates of observing beha vior because these situations are equally uncertain. Response-Reinforcer Dependency Rachlin and Baum (1972, Experiment II I-B) compromised the dependency between pecking and access to food by superimposing various VT schedules of food presentation on a VI 3-min schedule that was held constant across co nditions. In comparison, Lattal (1974) held the

PAGE 74

74 density of food presentations constant acros s conditions and varied the proportion of food deliveries that were dependent upon a response. Similar results were obtained in both experiments; as the number of response-independen t reinforcer deliveries increased the rate of food key pecking decreased. The present experime nts suggest that whether similar functions are obtained for behavior maintained by conditione d reinforcement depends on which of these methods are used to compromise the depende ncy between responding an d the presentation of conditioned reinforcers. Sp ecifically, increasing the dens ity of response-independent presentations of schedule-correl ated stimuli in the context of a constant underlying responsedependent schedule did not produce systematic effects on observing behavior in Experiment 1. However, holding the number of sc hedule-correlated stimulus pres entations per min constant and reducing the proportion of these presentations delivered accordi ng to the response-dependent schedule of stimulus delivery systematically re duced rate of response to the observing key in Experiment 2. Thus, Experiment 2 extends the findings of Lattal to situations in which the response-independent delivery of conditioned reinforcers is superimposed on behavior maintained by conditioned reinforcement. It is not immediately obvious why the manipula tions conducted in Experiment 1 failed to replicate the effect reported by Rachlin and Baum (1972). One possibility concerns the overall density of reinforcement. Rachlin and Baum conducted a control condition in which a conjoint VI 45 s VT 180 s schedule of reinforcement ope rated on the food key. Although this schedule delivered the same proportion of response-indepe ndent reinforcement as another condition with the same subjects (conjoint VI 180 s VT 720 s), ra te of key pecking was considerably higher in the condition with the higher overa ll density of reinforcement for 3 of 4 pigeons. Thus, it seems reasonable to conclude that increasing the dens ity of reinforcement compresses the temporal

PAGE 75

75 relations between response-indepe ndent events and behavior ma intained by response-dependent presentation of those same events. This interp retation is corroborated by data from the present experiments showing that the different methods used to disrupt the dependency between observing key pecks and stimulus presentations in Experiments 1 and 2 changed the contiguity between these events in different ways. Specifically, increasing th e rate of response-independent stimulus presentations in th e context of a continuous b ackground schedule of responsedependent stimulus presentations increased th e contiguity between response and putative conditioned reinforcer in Experiment 1. Thus within-phase manipulations in Experiment 1 produced conflicting changes with respect to dependency and con tiguity of events. In contrast, decreases in the dependency ratio across condi tions in Experiment 2 produced concomitant decreases in the contiguity between responses and response-independent stimulus changes. Given these outcomes, it seems likely that disc riminative control by the response-independent schedule was disrupted by the contiguity be tween responding and conditioned reinforcement during Experiment 1. It is also possible that conditioned reinfor cers are fundamentally different from primary reinforcers with respect to the effects of change s in the response-reinforcer dependency. This seems very unlikely, given the extent to which the voluminous literature on conditioned reinforcement supports the position that c onditioned reinforcers f unction similarly to unconditioned reinforcers in most respects, as re viewed in the introduction. The results of both Marr and Zeiler (1974) and Lieving, et al. (2006), the only previ ous studies involving responseindependent conditioned reinforcement, al so argue against th is possibility. Perhaps the strongest evidence fo r rejecting the possibility th at conditioned reinforcers are somehow fundamentally different from primary reinforcers with respect to response-independent

PAGE 76

76 reinforcement is Experiment 2 of the present investigation. As discussed by Burgess and Wearden (1986), the manipulation of response-rein forcer dependency conducted in Experiment 2 confounds the effects of reducing the rate of response-dependen t reinforcement and increasing the rate of response-independent reinforcement. However, the endpoint conditions, in which only response-dependent or res ponse-independent pres entations of schedul e-correlated stimuli were available, clearly illustra te the importance of dependency in conditioned reinforcement. Further experimentation designed to isolate the relative contributions of the confounded operations in Experiment 2 is warranted, but th e graded function relating dependency to rate of observing behavior suggests dependency, as form ulated in Experiment 2, as an important controlling variable. Multiple Functions of Stimuli There is one other alternativ e explanation for the discrepa ncy between the results of Experiment 1 and the results of Rachlin and Baum (1973). Some theorists have argued the conditioned reinforcing function of any stimulus depends on its establishment as a discriminative stimulusthe so-called discrimination hypothesi s (Keller & Schoenfeld, 1950). Keller and Schoenfeld cited two experiments in support of the discrimination hypothesis. The first of these showed that when a light was repeatedly paired with food after a rat ha d already collected the food from the food cup, the light was ineffective as a reinforcer in later sessions in which it was applied as a consequence for a new response (Schoenfeld, Antonitis, & Bersh, 1950). The second experiment began by establishing a light as a discriminative stimulus by reinforcing rats lever-presses in its presence but not in its abse nce. Then, under extinction conditions in which food was never delivered, the light was presented either as an ante cedent for some rats and as a consequence for others. No differences in perfor mance during extinction were obtained. In later extinction sessions, the temporal relation between lever presses a nd the light was reversed; rats

PAGE 77

77 formerly exposed to the light as an antecedent were now exposed to the light as a consequence for lever pressing, and vice-versa. No changes in performance were prod uced by changes in the temporal relation between responding and stimul us presentation (Dinsmoor, 1950). Based on the results of these experiments, Ke ller and Schoenfeld concluded that a discriminative function is a necessary precondition for a conditioned reinforc ing function, and that discriminative and conditioned reinforcing functi ons are interchangeable. Other authors, notably Kelleher (1966), have argued against the discrimination hypothesis of conditioned reinforcement. Kelleher noted that the failure to obtain a reinforcing function in the experiment by Schoenfeld et al. was similar to failures to produce an eliciting function using similar arrangements in Pavlovian conditioning pr ocedures. More importantly, Kelleher cited an experiment by Stein (1958), which showed that rats developed a preference for a lever that produced a tone after being exposed to a Pavlovi an conditioning procedure in which the tone was paired with intracranial brain stimulation. In this arrangement the tone never served a discriminative function, yet the results indicate th at it was an effective conditioned reinforcer. In a similar vein, Kelleher (1961) used a pa iring procedure to es tablish a click as a reinforcer, and then used the click alone to ge nerate patterns of beha vior similar to those generated by unconditioned reinforcer (as describe d earlier in the General Introduction). As in the experiment by Stein, the putative conditione d reinforcer never served a discriminative function, yet produced patterns of behavior consiste nt with a reinforcing function. As noted in several extensive reviews on conditioned reinforcem ent, the notion that stimuli can have multiple functions is not new, and does not jeopardize an account of the results of experiments involving stimulus-response chains, second-order schedule s, or observing procedures in terms of conditioned reinforcement (Gollub, 1970; Kelleher, 1966; Kelleher & Gollub, 1962).

PAGE 78

78 Other researchers have taken a more extr eme position than that proposed by Keller and Schoenfeld (1950). While Keller and Schoenfeld argued that a conditione d reinforcing function depends on a discriminative function, Baum (1973), and Staddon (1983), for example, have advocated for the abandonment of conditioned reinforcement as an explanation for performance in chained schedules. An experiment by Schuste r (1969, Experiment 3) is sometimes cited to support the claim that conditioned reinforcement is an unnecessary concept. In this experiment pigeons were exposed to a concurrent-chains procedure. Concurrent -chain schedules are schedules in which satisfying th e response requirement on one of two concurrently available response keys (the initial link) produces another schedule of reinfo rcement (the terminal link) on the chosen response key, and darkens the other key. Food is delivered when the response requirement is completed in the terminal link. In Schusters experi ment, identical VI 60-s schedules operated during the initial links. Th is procedure differed s lightly from typical concurrent-chains procedures, how ever, in that each peck produ ced a 1-s blackout on both keys, during which responses were ineffective. Comp letion of the response requirement on either key produced the terminal-link stimulus on that key a nd darkened the other key. Each terminal link provided brief access to food according to a VI 30-s schedule of reinforcement, but one key also provided presentations according to an FR 11 schedule of stimu li paired with food delivery. Pigeons in this experiment reliably preferred the response key that did not produce additional stimuli, although for 2 of 5 subjects the preference s were weak. Additionally, all pigeons pecked at a higher rate in the terminal link that included additional stimulus presentations than in the terminal link that did not. Schuster reasoned that if the stimuli were conditioned reinforcers, then the pigeons would prefer the response key that increased the number of stimulus presentations, and thus concluded that the stimuli in chained sche dules do not serve a reinforcing

PAGE 79

79 function. He attributed the higher rates in the less preferred terminal link to the discriminative function of the stimuli paired with access to food. Gollub (1970) argued that instead of invalidating theories of conditioned reinforcement, Schusters results simply corroborate earlier work by Fantino (1968), showing that higher rates of behavior in one terminal-link produces pr eference for the alternative link. Baum (1973) suggested that Gollubs interpretation was incorrec t, because Fantinos results are restricted to situations in which higher response rates are required, and so on. Schusters results remain controversial, and experiments employing a variety of procedures outside the scope of the present discussion have been conducted to illustrate that the concept of conditioned reinforcement outperforms availa ble alternatives (e.g., Dunn & Williams, 1991; Dunn, Williams, & Royalty, 1987; Royalty, Williams, & Fantino, 1987). These experiments, in concert with the cogent review s of the literature by Gollub (1977, 1970) and Kelleher (1966) provide strong evidence for the viability of th e concept of conditioned reinforcement. In contrast, there is relatively litt le evidence that supports extreme versions of the discrimination hypothesis, as advanced by Baum, Staddon, and, mo re recently, Davison and Baum (2006). That conditioned reinforcers also func tion as discriminative stimuli in many, and perhaps most cases is uncontroversial, however. Observing and Attending Returning to the issue of observing, co rrelated stimuli undoubtedly serve multiple functions, discriminative as well as reinforcing. Indeed, the observing pr ocedure, as developed by Wyckoff (1952, 1969), was design ed to investigate aspects of stimulus control concerned with attention. The observing res ponse is an experimental analogue of responses such as turning toward a stimulus, which presumably increases c ontact between receptor cells and the stimulus. This has led some researchers to distinguish between natural and artificial observing

PAGE 80

80 responses, with the stimulus-producing responses in observing procedures in the latter category (e.g. Dinsmoor, 1983). Although this natural/a rtificial distinction may be intuitively reasonable, there are many exceptions, in which naturally occurring conditions involve artificial stimulus-producing responses. Imagine a situation in which a child is playing a video game while her father is giving her important instructi ons concerning things she must do later in the day. If she continues to press buttons on the game controller and st are at the screen, the father is likely to command, Pay attention to me! This command is essentially equivalent to, Pause the game and look at me, which specif ies a response quite like an observing response in Wyckoffs procedure. The concept of attention is more comm only invoked under circumstances that are analogous to those pertaining to stimulus control more broadly. For example, consider the phrase, He wouldnt have been hit by the bus if he had been paying at tention. Note that in this example attention is used as an explanation for th e lack of behavior change in the presence of a relevant stimulus. While such comments ar e convenient in everyda y discourse, they are unsatisfactory as explanations in a scientific a ccount because they appeal to a concept that was inferred from the behavior it is said to explain. As noted by Terrace (1966): The use of attention as an explanatory principl e in these instances is begging the question, and seems to be nothing more than a mask for our ignorance concerning the establishment of stimulus control (p. 289). The distinction between attention as an e xplanation for behavior and attention as a description of behavior has led researchers to address the concep t of attention by determining the conditions under which a stimulus comes to cont rol behavior. Heinemann, Chase, and Mandell (1968), for example, showed that whether a pigeon s behavior is altered by changes in a stimulus depends on how stimulus changes relate to continge ncies of reinforcement. In this experiment,

PAGE 81

81 responses on one of two concurrently availabl e response keys produced food. When the keys were red, pecks to the left key produced access to food in the presence of a 300-Hz tone, and pecks to the right key produced acce ss to food in the presence of a 1000-Hz tone. When the keys were green, only pecks to the right key produ ced access to food. Pecks on incorrect keys produced an inter-trial interval of variable duration, during which the response keys were turned off. The inter-trial interval followed food deliv eries as well. After highly accurate performance was obtained, the probability of reinforcement for correct responses was leaned to 0.5. In the final condition, tones ranging in frequency fr om 450 Hz to 3200 Hz (but not 1000 Hz) were presented across the 50% of trials in which f ood delivery was not programmed. The relationship between the frequency of the tone on the unreinfor ced trials and the proba bility of a right-key peck was examined. When the response keys we re red the pigeons behavior was controlled by the frequency of the tone. Frequencies nearer to 1000 Hz produced a higher probability of right key pecks. When the response keys were green however, pigeons pecked the right key almost exclusively, regardless of the fr equency of the tone. In colloq uial language, the pigeons learned to attend to the tone when the keys were red, a nd ignore the tone when the keys were green. This result shows that contingencies of reinforcement are critical determinants of stimulus control, and therefore attention. For example, our hypo thetical bus accident victims behavior may have been under the control of st imuli signaling the availability of other kinds of reinforcement, such as the smell of food from a nearby rest aurant. Thus, the observing procedure addresses attention in terms of orienting responses, and the discrimination literature addresses attention in terms of stimulus control. In Experiment 2, a function relating decr eases in the dependency between observing responses and the production of sc hedule-correlated stimuli to d ecreases in rate of observing

PAGE 82

82 behavior was shown. The discrimination index used to measure control of food-key responding by the schedule-correlated stimuli shows concomitant decreases in stimulus control as a function of decreases in the dependency. Because th e schedule-correlated stimuli were presented diffusely throughout the experimental space, this re sult cannot be a function of the pigeons lack of contact with the stimulus. Instead, the decreas e in stimulus control is a direct function of changes in the dependency, as evidenced by th e graded function relating these variables. Although this finding is rather intuitive (as furthe r demonstrated by the use of the same word to describe these relations in everyday language), I know of no other st udy employing an observing procedure that provides direct evidence that these two kinds of attention are related, although theorists interested in these procedures have presumed as much (e.g., Dinsmoor, 1995). That this function has not been demonstrated previously is probably a function of the decades of controversy regarding why observi ng responses were maintained in Wyckoffs procedure. Although Wyckoff developed the observing procedure to address attentional processes, in which antecedent-behavior rela tions are critical, most studies employing the observing procedure emphasize behavior-consequen ce relations. The present study shows that the observing procedure has not ye t been fully exploited as a met hod for the study of attention. Future Directions Future experiments could employ variants of the present procedure to pit the level of dependency between observing responses and schedul e-correlated stimulus presentations (or the response requirement on the observing key) agains t the strength of the correlation between the stimuli and the schedule operating on the food ke y. Such a manipulation would facilitate a comparison of the effects of effort and di scriminative control on observing behavior. Studies combining the observing proce dure and discriminated timeout-avoidance procedures would also be interesting. Fo r example, responses on one key might postpone

PAGE 83

83 periods of timeout from respons e-independent reinforcement, while responses on a second key might produce stimuli correlated with temporal proxi mity to timeout. Another possibility is an experiment in which response-independent presenta tions of schedule-correl ated stimuli would be prevented by pecks on an avoidance key. The proportion of session time spent in EXT components might then be manipulated. This pro cedure could be used to determine whether the rate of avoidance behavior is proportional to the number of Spresentations. Arrangements such as this would be useful for the study of attent ion in the context of aversive control. Any of these arrangements, including the present procedure, could be employed as terminal links in choice proce dures designed to assess preferen ce in relation to, for example, response-dependent versus responseindependent presentations of sche dule correlated stimuli. It would also be informative to offer choices be tween response-independent presentations of Sonly, S+ only, or both schedule-corr elated stimuli. Clearly, there is much work to be done if we are to improve upon conventional accounts of at tention (most of which have not progressed much beyond James, 1890), and the possibilities o ffered by the observing procedure are almost endless. Another direction for future stud ies it to extend quantitative m odels of choice to situations involving response-independent conditioned reinfor cement. For example, an experiment could be arranged in which the schedule requirement of the VI component of a conjoint VI VT schedule operating on the observing key could be varied across conditions within an experimental phase. The VT com ponent could then be manipulated across phases, with the same series of changes to the VI sc hedule conducted in each phase of the experiment. A parametric manipulation of this kind would produce data amenable to modern quantitative analysis,

PAGE 84

84 allowing for a determination of the extent to wh ich currently available models can account for the effects of response-independent delivery of conditioned reinforcement. In closing, it may be worthwhile to consider some possible applica tions of the present results. The relation between the dependency of schedule-correlated stimulus presentations on observing behavior and the contro l of food-key pecking shown in the present procedure has clear relevance to instructional design. The presen t results suggest, for example, that requiring a response to produce instructions would enhan ce the discriminative control exerted by an instruction. For example, if a company was inte rested in providing empl oyees with training in behavioral safety in the workplace, an inte ractive computer-based program in which the employee controlled the delivery and removal of the instructions thr oughout the lesson should produce considerably better results that ev en the best-made instructional video.

PAGE 85

85 LIST OF REFERENCES Baum, W. M. (1973). The corre lation-based law of effect. The Journal of the Experimental Analysis of behavior, 20, 137-153. Branch, M. N. (1973). Observing responses in pigeons: Effects of sche dule component duration and schedule value. The Journal of the Experiment al Analysis of Behavior, 20, 417-428. Bugelski, R. (1938). Extinction with and without sub-goal reinforcement. Journal of Comparative Psychology, 26, 121-134. Burgess, I. S., & Wearden, J. H. (1986). Superimposition of response-independent reinforcement. The Journal of the Experimental Analysis of Behavior, 45, 75-82. Case, D. A., & Fantino, E. (1981). The delayreduction hypothesis of conditioned reinforcement and punishment: Observing behavior. The Journal of the Expe rimental Analysis of Behavior, 35, 93-108. Catania, A. C., Deegan, J. F., & Cook, L. (1966). Concurrent fixed-ratio and avoidance responding in the squirrel monkey. Journal of the Experimental Analysis of Behavior, 9, 227-231. Chomsky, N. (1959). A review of Skinners Verbal Behavior. Language, 35, 26-58. Cowles, J. T. (1937). Food-tokens as incentives for learning by chimpanzees. Comparative Psychology Monographs, 14, 1-96. Davison, M., & Baum, W. M. (2006). Do conditional reinforcers count? Journal of the Experimental Analysis of Behavior, 86, 269-283. Dews, P. B. (1960). Free-opera nt behavior under condi tions of delayed reinforcement: I. CRFtype schedules. Journal of the Experimental Analysis of Behavior, 3, 221-234. Dinsmoor, J. A. (1950). A quantitative comp arison of the discriminative and reinforcing functions of a stimulus. Journal of Experime ntal Psychology, 40, 458-472. Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693-728. Dinsmoor, J. A. (1995). Stimulus Control: I. The Behavior Analyst, 18, 51-68. Dinsmoor, J. A., Browne, M. P., & Lawrence, C. E. (1972). A test of the negative discriminative stimulus as a reinforcer of observing. Journal of the Experimental Analysis of Behavior, 18, 79-85.

PAGE 86

86 Dunn, R., Williams, B., & Royalty P. (1987). Devaluation of s timuli contingent on choice: Evidence for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 48, 117-131. Estes, W. K. (1949). A study of motivating condi tions necessary for secondary reinforcement. Journal of Experime ntal Psychology, 39, 306-310. Fantino, E. (1968). Effects of requi red rates of responding upon choice. Journal of the Experimental Analysis of Behavior, 11, 15-22. Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, 723-730. Gollub, L. R. (1958). The chaining of fixe d-interval schedules. Unpublished doctoral dissertation, Harvard University. Gollub, L. R. (1970). Information on conditioned reinforcement: A review of Conditioned Reinforcement, edited by Derek P. Hendry. Journal of the Experi mental Analysis of Behavior, 14, 361-372. Heineman, E. G., Chase, S., & Mandell, C. (1968). Discriminative control of attention. Science, 160, 553. Hendry, D. P. (1969). Reinforcing value of inform ation: Fixed-ratio schedules. In D. P. Hendry (Ed.), Conditioned reinforcement (pp. 300-341). Homewood, IL: Dorsey Press. Herrnstein, R. J. (1961). Relative and absolute st rength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267-272. Herrnstein, R. J., & Morse, W. H. (1957). Some effects of response-independent reinforcement on maintained operant behavior. Journal of Comparative and Physiological Psychology, 50, 461-467. Imam, A. A., & Lattal, K. A. (1988). Eff ects of alternative re inforcement sources: A reevaluation. Journal of the Experimental Analysis of Behavior, 50, 261-271. James, W. (1890). The principles of psychology. New York, NY: Henry Holt and Company. Kelleher, R. T. (1961). Schedules of rein forcement during experimental extinction. Journal of the Experimental Analysis of Behavior, 4, 1-5. Kelleher, R. T, (1966). Chaini ng and conditioned reinforcement. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp. 160-212). Englewood Cliffs, NJ: Prentice-Hall.

PAGE 87

87 Kelleher, R. T., & Gollub L. R. (1962). A review of positive conditioned reinforcement. Journal of the Experimental A nalysis of Behavior, 5(4, Suppl.), 543-597. Kelleher, R. T., Riddle, W. C., & Cook, L. (1962). Observing responses in pigeons. Journal of the Experimental Analysis of Behavior, 5, 3-13. Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology: A systematic text in the science of behavior. East Norwalk, CT: Appleton-Century-Crofts, Inc. Kendall, S. B. (1973). Effects of two proce dures for varying information transmission on observing responses. Journal of the Experimental Analysis of Behavior, 20, 73-83. Lattal, K. A. (1974). Combinations of res ponse-reinforcer dependence and independence. Journal of the Experimental Analysis of Behavior, 22, 357-362. Lattal, K. A., & Abreu-Rodrigues, J. (1997). Response-independent events in the behavior stream. Journal of the Experimental Analysis of Behavior, 68, 375-398. Lattal, K. A., & Bryan, A. J. (1972). Effects of concurrent response-independent reinforcement on fixed-interval schedule performance. Journal of the Experimental Analysis of Behavior, 26, 495-504. Lieving, G. A., Reilly, M. P., & Lattal, K. A. (2006). Disruption of responding maintained by conditioned reinforcement: Alterations in re sponse-conditioned-reinforcer relations. Journal of the Experimental Analysis of Behavior, 86, 197-209. Marr, M. J., & Zeiler, M. D. (1974). Sc hedules of response-i ndependent conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 21, 433-444. Mazur, J. E. (1997). Learning and behavior (4th ed.). Upper Saddle River, NJ: Prentice-Hall. McMillan, J. C. (1974). Average uncertainty as a determinant of observing behavior. Journal of the Experimental Analysis of Behavior, 22, 401-408. Miller, N. E. (1951). Learnable drives a nd rewards. In S. S. Stevens (Ed.), Handbook of experimental psychology (435-472). Oxford, England: Wiley. Pavlov, I. (1927). Conditioned reflexes. New York, NY: Oxford University Press. Pierce, W. D., & Cheney, C. D. (2004). Behavior analysis and learning (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Prokasy, W. F. Jr. (1956). The acquisition of observing responses in the absence of differential external reinforcement. Journal of Comparative and Physiological Psychology, 49, 131134.

PAGE 88

88 Rachlin, H., & Baum, W. M. (1972). Alterna tive reinforcement: Does the source matter? Journal of the Experimental Analysis of Behavior, 18, 231-241. Rescorla, R. A. (1967). Pavlovian conditio ning and its proper control procedures. Psychological Review, 74, 71-80. Royalty, P., Williams, B. A., & Fantino, E. (1987) Effects of delayed conditioned reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 47, 41-56. Schoenfeld, W. N., Antonitis, J. J., & Bersh, P. J. (1950). A preliminary study of the training conditions necessary for conditioned reinforcement. Journal of Experimental Psychology, 40, 40-45. Schuster, R. H. (1969). A functional analysis of conditioned reinforcement. In D. P. Hendry (Ed.), Conditioned reinforcement (pp. 192-234). Homewood, IL: Dorsey Press. Schwartz, B. (1978). Psychology of learning and behavior. Oxford, England: W. W. Norton & Co. Sizemore, O. J., & Lattal, K. A. (1977). De pendency, temporal contiguity, and responseindependent reinforcement. Journal of the Experimental Analysis of Behavior, 27, 119125. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Oxford, England: Appleton-Century. Skinner, B. F. (1948). Superstition in the pigeon. Journal of Experime ntal Psychology, 38, 168-172. Staddon, J. E. R. (1983). Animal learning and behavior. Cambridge, England: Cambridge University Press. Stein, L. (1958). Secondary reinforcement established with subcortical stimulation. Science, 127, 466-467. Terrace, H. S. (1966). Stimulus Control. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp. 271-344). Englewood Cliffs, NJ: Prentice-Hall. Thorndike, E. L. (1898). Animal intelligence : An experimental study of the associative processes in animals. Psychological Monographs, 2, 1-109. Timberlake, W., & Lucas, G. A. (1985). Th e basis of superstitous behavior: Chance contingency, stimulus substitution, or appetitive behavior? Journal of the Experimental Analysis of Behavior, 44, 279-299.

PAGE 89

89 Williams, K. A. (1929). The reward value of a conditioned stimulus. University of California Publications in Psychology, 4, 31-55. Williams, B. A., & Dunn, R. (1991). Pref erence for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 55, 37-46. Wilton, R. N., & Clements, R. O. (1971). The ro le of information in the emission of observing responses: A test of two hypotheses. Journal of the Experimental Analysis of Behavior, 16, 161-166. Wolfe, J. B. (1936). Effectiveness of token rewards in chimpanzees. Comparative Psychology Monographs, 12, 1-72. Wycoff, L. B., Jr. (1952). The role of obser ving responses in disc rimination learning. Psychological Review, 59, 68-78. Wycoff, L. B., Jr. (1969). The ro le of observing responses in disc rimination learning. In D. P. Hendry (Ed.), Conditioned reinforcement (pp. 237-260). Homewood, IL: Dorsey Press. Zeiler, M. D. (1968). Fixed and variable sche dules of response-independent reinforcement. Journal of the Experimental Analysis of Behavior, 11, 405-414. Zeiler, M. D. (1977). Elimination of reinfo rced behavior: Intermittent schedules of notresponding. Journal of the Experimental Analysis of Behavior, 27, 23-32.

PAGE 90

90 BIOGRAPHICAL SKETCH Born in New Jersey, and raised in Philadelphi a, PA by my grandparents, I first arrived in Florida in 1987 at the age of 13. In my first few years in Florida I attended Largo High School and learned to play the guitar. After earning a Florida Undergradua tes Scholarship I decided to attend the University of Florida, and arrived in Gainesville in 1992. I put myself through school with a combination of grants, lo ans, and part-time jobs. When part-time jobs became scarce, I donated plasma to have money for food. One ni ght I was robbed at gunpoint while working in a Subway restaurant on NW 13th Street, just north of University Avenue. After another robbery about a month later, I decided to quit that job and work at Toys R Us. Although there was no shortage of guns in Toys R Us, they were all made of plastic and shot foam bullets. It was about that time that I decided to become a psychology ma jor, more as a default option rather than out of any kind of real passion. After sitting through classes listening to endless serious presentations of ridiculous theories, volunteeri ng as a counselor for the local suicide hotline where the principle technique was to paraphrase what the caller said and repeat it back to them, and as an assistant in a rat lab where no one ev er talked about why what they were doing was relevant, I was seriously entertaining the idea of changing majors. It was then that I met Dr. Hank Pennypacker, and things began to change qu ickly. I began to understand the practicality and power of the study of behavior in its own right. The silly theo ries were all gone, and replaced with a simple, elegant set of descrip tions about the relationship between behavior and environment. The techniques were easy to appl y in everyday life, and I thought I might be onto something. I enrolled in a course called Beh aviorism in Contemporary Society taught by Ed Malagodi, whos insightful guidance led me to an understanding that what I had stumbled upon was not a simple collection of techniques for changing behavior, but a thoroughgoing worldview. It had become easy to be pass ionate about what I was doing.

PAGE 91

91 I enrolled in the masters program in Applied Behavior Analysis at the University of North Texas (UNT), where I was given the opportunity to work in a variety of settings with a wide range of populations. I worked as an in-hom e therapist for children with autism, a unit psychologist in an institution for developmentally -disabled adults, an af ter-school facility for disadvantaged children, and a school for children w ith severe behavioral disorders. However, my favorite experiences at UNT were those I ha d with Sigrid Glenn, who taught me to teach undergraduate courses in behavior analysis, and Manish Vaidya, who chaired my thesis project on the delayed matching-to-sample performance of typically functioning adult humans. These experiences convinced me to pursue an academic car eer in the experimental analysis of behavior. I enrolled in the Ph.D. program at the University of Florida, and return ed to Gainesville in 2002 to work with Dr. Tim Hackenberg. While here I have spent most of my time studying basic behavioral processes in pigeons, and continuing to teach undergraduates about the experimental analysis of behavior. After graduating, I hope to continue studying basic behavioral processes in both humans and non-human animals and serve as faculty in a uni versity setting.


Permanent Link: http://ufdc.ufl.edu/UFE0019603/00001

Material Information

Title: Response-Independent Conditioned Reinforcement in an Observing Procedure
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0019603:00001

Permanent Link: http://ufdc.ufl.edu/UFE0019603/00001

Material Information

Title: Response-Independent Conditioned Reinforcement in an Observing Procedure
Physical Description: Mixed Material
Copyright Date: 2008

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0019603:00001


This item has the following downloads:


Full Text





RESPONSE-INDEPENDENT CONDITIONED REINFORCEMENT INT AN OBSERVING
PROCEDURE





















By

ANTHONY L. DEFULIO


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2007































O 2007 Anthony L. DeFulio




































To my best friend Kirsty, and to Andria, who knows me best.









ACKNOWLEDGMENTS

I thank my grandparents, Amelia and Ralph, without whom none of my accomplishments

would be possible. I thank my advisor, Tim Hackenberg, for his guidance and patience. I thank

Katie Saulsgiver, a great friend and wonderful person without whom several of the analyses in

the present manuscript would not be possible. I thank Rachel Cassidy and Stacy Strothers,

whose countless hours of data analysis were essential in completing the experiments described

here. I thank my surrogate siblings, Chris Bullock, Bethany Raiff, Rachelle Yankelevitz, Carla

Lagorio and Leo Andrade, who conducted many of the experimental sessions and provided me

with valuable discussion and feedback. I thank Raul Avilla, who provided suggestions that were

critical in the formulation of the experimental procedures. I thank my committee members, Marc

Branch, Jesse Dallery, Alan Spector, and David Daegling, each of whom was a critical ingredient

in my intellectual development. Lastly, I thank Obie for being a constant source of positive

reinforcers in a world of lean schedules.












TABLE OF CONTENTS


page

ACKNOWLEDGMENTS .............. ...............4.....


LIST OF TABLES ............. ....._. __ ...............7....


LIST OF FIGURES .............. ...............8.....


AB S TRAC T ............._. .......... ..............._ 10...


CHAPTER


1 GENERAL INTRODUCTION .............. ...............12....


Assessing Conditioned Reinforcement............... .............1

Response-Reinforcer Dependency ................. ...............16.................
Observing and Attention ................. ...............21........... ....
Uncertainty-Reduction Theory. .............. ...............22....
Conditioned Reinforcement ................. ...............24.................

The Present Study ................. ...............26................


2 EXPERIMENT 1 .............. ...............28....


M ethod ................. ...............29.................

Subj ects ......__................. .........__..........2
App aratu s................. ...............29...............
Preliminary Training .............. ...............29....

Experimental Procedure .............. ...............3 1....
R e sults................ ...............33........ ......
Discussion ................. ...............36........ ......


3 EXPERIMENT 2 ................. ...............48..............


M ethod ........._ ......... ....... ...............48.....

Subj ects and Apparatus ................. ...............48.._._._ ....
Procedure ........._..... ...._... ...............49.....
Re sults........._..... ..._ ... ...............51.....
Discussion ........._..... ...._... ...............54.....


4 DI SCUS SSION ........._..... ...._... ...............70....


Density of S+ and S-............. ...............70....

Response-Reinforcer Dependency ................. ...............73.................
Multiple Functions of Stimuli ................. ...............76................
Observing and Attending ................. ...............79........... ....
Future Directions ............. .............82......











LIST OF REFERENCES ................. ...............85................

BIOGRAPHICAL SKETCH .............. ...............90....


































































6










LIST OF TABLES


Table page

2-1 Order of conditions and number of sessions per condition for Experiment 1. ..................39

3-1 Order of conditions and number of sessions per condition for Experiment 2. ..................56










LIST OF FIGURES


Figure page

2-1 Rate of responses on the observing key as a function of rate of response-independent
stimulus presentations ................. ...............40......_._. .....

2-2 Rate of responses on the food key as a function of rate of response-independent
stimulus presentations ................. ...............41......_._. .....

2-3 Discrimination index of food-key responding in the presence of schedule-correlated
stimuli as a function of rate of response-independent stimulus presentations .................. .42

2-4 Discrimination index of food-key responding in the presence of the mixed stimulus
as a function of rate of response-independent stimulus presentations ............... .... ...........43

2-5 Discrimination index of observing-key responding in the presence of the mixed
stimulus as a function of rate of response-independent stimulus presentations ........._......44

2-6 Relative rate of responding on the food key as a function of rate of response-
independent stimulus presentations .............. ...............45....

2-7 Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation for three subjects (457, 250, 1774) in Experiment 1 ......................46

2-8 Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation for three subj ects (96, 289, 1770) in Experiment 1 .................. ......47

3-1 Observing responses per minute as a function of the duration of the schedule-
correlated stimulus ................. ...............57......__ .....

3-2 Rate of responses on the observing key as a function of dependency ratio. ................... ...58

3-3 Normalized group mean rate of responses on the observing key as a function of
dependency ratio. ............. ...............59.....

3-4 Rate of responses on the food key as a function of dependency ratio .............. ..... ........._.60

3-5 Discrimination index of food-key responding in the presence of schedule-correlated
stimuli as a function of dependency ratio .............. ...............61....

3-6 Discrimination index of food-key responding in the presence of the mixed stimulus
as a function of dependency ratio. ............. ...............62.....

3-7 Discrimination index of observing-key responding in the presence of the mixed
stimulus as a function dependency ratio ................ ...............63......__._...

3-8 Relative rate of responding on the food key as a function of rate of response-
independent stimulus presentations .............. ...............64....










3-9 Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation for three subjects (457, 250, 1774) in Experiment 2 ......................65

3-10 Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation for three subj ects (96, 289, 1770) in Experiment 2 ................... .....66

3-11 Cumulative record showing terminal performance for Pigeon 250 from the last
session of the 0/4 dependency ratio condition in Phase 1 of Experiment 2. ................... ...67

3-12 Cumulative record showing reacquisition of baseline performance in the first session
of a replication of the 4/0 dependency ratio condition in Phase 1 of Experiment 2..........68

3-13 Cumulative record showing stable baseline performance in the first session of a
replication of the 4/0 dependency ratio condition in Phase 1 of Experiment 2 .................69









Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

RESPONSE-INDEPENDENT CONDITIONED REINFORCEMENT INT AN OBSERVING
PROCEDURE

By

Anthony L. DeFulio

May 2007

Chair: Timothy D. Hackenberg
Major: Psychology

Pigeons' pecks on a response key produced food according to a variable-interval (VI)

schedule, in which responses produced food every 40 s, on average. These VI periods, or

components, alternated in irregular fashion, with extinction (EXT) components, in which food

was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli

(houselight flashes) correlated with the component schedule currently in effect. In Experiment 1,

these stimuli could be produced every 20 s, on average, by responses on the observing key, and

also occurred independently of responding. The rate of such response-independent stimulus

presentation did not produce systematic changes in observing behavior. Across phases in

Experiment 1, the proportion of session time spent in the EXT component was adjusted from 0.5

to 0.75 (Group A), and 0.25 (Group B). Increases in the proportion of session time spent in EXT

produced increases in the rate of observing. In Experiment 2, density of stimulus presentation

was held constant while the dependency between observing and presentation of the stimuli was

decreased across conditions. Results indicate that rate of observing decreased as the dependency

between responses and stimulus presentations was decreased. Furthermore, results indicate that

discriminative control by the schedule-correlated stimuli was systematically weakened as

dependency was decreased. Overall, these results show that conditioned reinforcers function









similarly to unconditioned reinforcers with respect to response-consequence dependencies, and

that stimulus control is greatly enhanced under conditions in which the relevant stimuli are

produced by an organism's behavior.









CHAPTER 1
GENERAL INTRODUCTION

Pavlov (1927) demonstrated that contiguous pairings of a neutral stimulus and an

unconditioned stimulus such as food produced a transfer of stimulus function, such that the

previously neutral stimulus came to elicit responses similar to those elicited by the unconditioned

stimulus. Pavlov's work emphasized the role of antecedent stimuli in the control of behavior,

but the pairing procedure was soon adapted for the study of operant conditioning. Thorndike

(1898) had already shown that consequences played a critical role in the acquisition and

maintenance of behavior. Thus, it was a simple next step to demonstrate that Pavlov' s pairing

procedure would produce similar results when the stimuli were used as consequences rather than

as antecedents to behavior.

Williams (1929) showed that exposing rats to a white discrimination box before delivering

food as a consequence for maze running was sufficient to transfer the reinforcing function of

food to the discrimination box. Similarly, several researchers showed that poker chips paired

with food could be effective in the maintenance of chimpanzee' s weight-lifting behavior

(Cowles, 1937; Wolfe, 1936).

With conditioned reinforcers it was also possible to build long sequential chains of

behavior in rats, pigeons, and chimpanzees with only minimal support by unconditioned

reinforcers (Kelleher, 1966). Thus, conditioned reinforcement might play an important role in

the explanation of similar behavior in humans:

People rarely engage in behavior which produces or avoids biologically significant stimuli
directly. Many people are virtually never deprived of food and water. If one is to account
for human behavior in terms of reinforcement principles one must look elsewhere for the
reinforcers ... the applicability of the principles of behavior control may depend upon our
understanding of conditioned reinforcement." (Schwartz, 1978, p. 281).









Assessing Conditioned Reinforcement

Although conditioned reinforcement is a valuable explanatory device, empirical evidence

suggests that cautious extrapolation of the principle is in order. Bugelski (1938) conducted an

experiment in which rats' lever presses produced a click followed by the delivery of food. He

then compared performance under conditions in which both the click and the food were

unavailable to conditions in which the click continued, but no food was delivered. Although

responding was more persistent when the click was presented, the effects were transient and

weak. Similar results have been obtained by many researchers (Miller, 1951).

Not only is continued pairing a requirement for continued efficacy of a conditioned

reinforcer, but when such stimuli are paired with a single kind of unconditioned reinforcer, the

effectiveness of the conditioned reinforcer is dependent on the same establishing operations as

the unconditioned reinforcer. For example, if a rat is not water-deprived, then stimuli paired

with the delivery of water will not serve as effective reinforcers (Estes, 1949). However, there is

no evidence to suggest that repeated presentation of a conditioned reinforcer produces satiation

of the kind produced when an animal is given extended access to food.

Immediacy is also an important determinant of the reinforcing value of both unconditioned

and conditioned reinforcers. It has been demonstrated that an unsignaled delay to the delivery

food lowers response rates whether responses during the delay increase the delay (Skinner, 193 8)

or not (Dews, 1960). Similarly, Wolfe (1936) showed that tokens paired with food maintained

chimpanzees' behavior when the tokens were delivered immediately, even when exchanges for

food were delayed. When the token delivery was delayed, however, responding deteriorated

considerably.

Another important piece of evidence that conditioned reinforcers are similar to

unconditioned reinforcers has been generated by the new-response procedure. This procedure










begins with repeated pairings of a neutral stimulus and a unconditioned reinforcer. The formerly

neutral stimulus is then used to train a new response. Kelleher (1961) showed that patterns of

behavior generated by conditioned reinforcers in the new response procedure were similar to

those generated by unconditioned reinforcers. Using food preceded by a brief click as the

consequence, he first trained pigeons to peck a key according to a Eixed-interval (FI) 5-min

schedule of reinforcement, in which food is produced by the first peck after 5 min has passed

from either the start of the session or the previous food delivery. This arrangement typically

generates a pattern of responding that is positively accelerated across the 5-min interval. Once

the characteristic pattern developed, Kelleher disconnected the food hopper and used the click

alone as the consequence for key pecking. The schedule of reinforcement was then changed to a

differential reinforcement of other behavior (DRO) schedule. Under this contingency, clicks

were produced whenever the pigeon did not peck the response key for a period of 10 s. Kelleher

found that pigeons pecked the response key according to the same low-rate pattern typically

generated by food reinforcement under similar conditions. During the same session, the

schedule of reinforcement was changed to a Eixed ratio (FR) 1. According to this schedule, every

peck to the response key produced a click. Throughout the remainder of the session the number

of pecks required to produce a click was gradually increased. This procedure produced high

rates of behavior, interrupted by brief pauses following the delivery of a click, a pattern of

behavior identical to that produced by FR schedules when food is used as the reinforcer. Thus,

Kelleher demonstrated that scheduling of conditioned reinforcers generated novel response

patterns similar to those generated by unconditioned reinforcers.

Additional evidence that conditioned reinforcers produce similar patterns of behavior to

those produced by unconditioned reinforcement comes from experiments employing









second-order and chained schedules. As in all chained schedule procedures, a sequence of

components, or links, each consisting of a simple schedule of reinforcement, are arranged such

that the consequence for completing the terminal link is a unconditioned reinforcer. The

consequence for the completion of early links in the chain is the production of the stimuli

associated with the next link, and thus greater temporal proximity to unconditioned

reinforcement. Second-order schedules are compound schedules in which, "the behavior

specified by a schedule contingency is treated as a unitary response that is itself reinforced

according to some schedule of unconditioned reinforcement" (Kelleher, 1966).

An understanding of this kind of schedule follows easily from a consideration of the

nomenclature used to describe them. For example, in a FR 4 (VI 30 s) second-order chained

schedule, each component requires a single response after a variable amount of time has passed,

averaging 30 s. Unconditioned reinforcement is produced after every 4th component schedule.

The consequence for completion of the other component schedules is an exteroceptive stimulus

change (e.g., a key color change) associated with greater temporal proximity to food. The effects

of these chained schedules are often compared to those of tandem schedules. Tandem schedules

are identical to chained schedules with the exception that no exteroceptive stimulus changes

occur. Comparisons of this sort are therefore useful for determining the function of the stimuli in

second-order schedules.

Gollub (1958) conducted this kind of comparison, and made several important

observations. First, responding in the chained schedule was stronger in the terminal link and

weaker in the initial link relative to responding in the tandem schedule. Second, the overall

pattern of responding generated by the chained schedule resembled the pattern generated by the

component schedule when unconditioned reinforcer was delivered after the completion of each









schedule. This was the case even when chains were composed of 5 components. Thus,

exteroceptive stimulus changes across early links in these chains were producing effects similar

to those produced by food as a consequence although they had never been directly paired with

food, or even paired with another stimulus that had been directly paired with food.

In summary, stimuli that are repeatedly paired with unconditioned reinforcers have many

similarities to unconditioned reinforcers. Patterns of behavior generated by schedules of

conditioned reinforcement are similar in character to those generated by schedules of

unconditioned reinforcement, and behavior is well maintained by the delivery of conditioned

reinforcers, as demonstrated by both second-order and chained schedules. Moreover, the

operations that modulate the effectiveness of unconditioned reinforcers, such as deprivation with

respect to the unconditioned reinforcer and immediacy of delivery, produce similar changes in

the effectiveness of conditioned reinforcers.

Response-Reinforcer Dependency

Another operation useful in comparing unconditioned and conditioned reinforcement is the

delivery of response-independent reinforcement. There has been substantial interest in the

effects of delivering response-independent unconditioned reinforcers, but very few studies have

examined the effects of delivering response-independent conditioned reinforcers.

The first study of response-independent delivery of reinforcers was conducted by Skinner

(1948). Skinner was interested in the patterns of behavior that developed when food was

delivered at regular intervals independent of behavior. He noted that pigeons exposed to such

fixed-time (FT) schedules eventually developed idiosyncratic stereotypic response patterns that

coincided with the delivery of food. Skinner termed such patterns "superstitious behavior," and

argued that such patterns were a function of accidental reinforcement of behavior at the moment

of food delivery. Although later more detailed analyses called Skinner' s interpretation into










question (cf. Timberlake & Lucas, 1985), the experiment provided important evidence that

reinforcement has powerful strengthening effects on behavior whether or not it is dependent on

responding.

Zeiler (1968) investigated the effects of response-independent reinforcers on pigeons' key

pecking. Pigeons were first trained to peck a response key by the method of successive

approximations using response-dependent reinforcers. In later conditions pigeons were exposed

to various response-independent schedules of food delivery. In addition to the FT schedules

used in previous studies, Zeiler investigated the effects of variable-time (VT) schedules of

response-independent reinforcement, in which food is delivered periodically and independent of

behavior. Zeiler found that FT and VT schedules of reinforcement maintained similar patterns of

behavior as FI and VI schedules of reinforcement, respectively. Rate of pecking was lower,

however, under FT and VT schedules than their response-dependent counterparts.

Of particular interest to the present line of research are studies that explore the combinative

effects of response-dependent and response-independent schedules of reinforcement.

Investigations of this sort involve FT or VT schedules superimposed on underlying response-

dependent schedules of reinforcement. Such arrangements are typically referred to as conj oint

schedules (Catania, Deegan, & Cook, 1966). The effects of such schedules depend on the

particular composite schedules used to construct the conj oint schedule.

Herrnstein and Morse (1957) reported increases in response rate when signaled FT food

deliveries were superimposed on a DRO schedule of reinforcement. Lattal and Bryan (1976)

found that response rates either increased or were unchanged when a VT 150 s schedule was

superimposed on responses maintained by a FI schedule. When FT schedules were later

superimposed on the FI schedule, unsystematic results were obtained. Zeiler (1977) also









obtained unsystematic results when FT schedules were superimposed on behavior maintained by

VI schedules. Despite the variation in these results, the most consistent effect of superimposing

response-independent schedules on response-dependent schedules of reinforcement is a decrease

in response rates (Burgess & Wearden, 1986).

Edwards, Peek, and Wolfe (1970) found that reinforcers delivered according to a FT

schedule lowered rates of behavior maintained on a FR schedule of reinforcement, and Zeiler

(1979) obtained similar results. The arrangements that have produced the most consistent

reduction in rate of behavior relative to response-dependent baseline performance, however, are

conjoint VI VT schedules. Rachlin and Baum (1972) found that response rates decreased

proportionally with the proportion of reinforcers delivered by the response-dependent schedule.

Other researchers have reported similar results (e.g., Imam & Lattal, 1988; Lattal, 1974).

Superimposing a response-independent schedule on a response-dependent schedule

changes both the temporal relations between responding and food (dependency), and the

discriminative control that can arise either as a function of changes in the dependency, or as a

function of differences in interfood intervals (Lattal & Abreu-Rodrigues, 1997). The effects of

conj oint schedule arrangements of this kind may therefore be a function of the interaction of

these variables. Imam and Lattal (1988) showed that decreases in response-reinforcer contiguity

accounted for observed decreases in rate of behavior under conjoint VI VT conditions. These

results are similar to those obtained when a delay to reinforcement is programmed on a simple

VI schedule (Sizemore & Lattal, 1977). Whether disparities in the results obtained when

arrangements other than conj oint VI VT can be explained by changes in response-rein-forcer

contiguity alone, or require a more complex analysis is still unclear. The consistency of the









results generated by conj oint VI VT arrangements, however, recommends them as a useful

starting point in the examination of response-independent delivery of conditioned reinforcement.

Marr and Zeiler (1974) investigated the effects of response-dependent and response-

independent brief-stimulus presentations on food-maintained key pecking of two pigeons. An FI

15-min schedule of food delivery was arranged throughout the experiment. In all conditions,

food was delivered 0.5 s after the response that satisfied the schedule requirement. In

experimental conditions, the brief stimulus (a 0.5-s key-color change) was produced according to

FI 3 min, VI 3 min, FT 3 min and VT 3 min schedules, respectively. Each of these arrangements

was examined under conditions in which the brief stimulus was paired with the food schedule,

and under conditions in which the brief stimulus was never paired with food. Early conditions in

which the brief stimulus was unpaired with food had little effect on behavior. As the experiment

progressed, however, the paired and unpaired conditions produced similar effects. With respect

to the paired conditions, both FI and FT schedules of brief stimulus presentation produced

positively accelerated patterns of responding similar to those typically produced by FI schedules

of food reinforcement. In contrast, schedules of VI and VT brief stimulus presentation produced

less post-stimulus pausing. This difference is similar to the difference in response patterning

obtained with VI schedules of food reinforcement compared to those obtained with FI schedules.

While the FI and VI schedules of brief-stimulus presentation enhanced rates of behavior early in

the FI-food interval, FT and VT brief stimulus presentations suppressed this responding. For one

pigeon, global rates were substantially reduced in paired FT and VT conditions relative to

response-dependent conditions. The other pigeon, however, showed little difference in overall

rates across these conditions. The authors concluded that response-independent delivery of









conditioned reinforcers (paired brief stimuli) has similar effects on the patterning of behavior as

the delivery of unconditioned reinforcers under similar circumstances.

One might also ask whether response-independent delivery of conditioned reinforcement

had similar effects on behavior maintained by conditioned reinforcement. To answer this

question, Lieving, Reilly, and Lattal (2006) used an observing-response procedure in which key

pecks produced food and treadle presses produced food-correlated stimuli. In Experiment 1,

treadle presses during VI food components produced 5-s presentations on the food key of stimuli

previously correlated with the availability of food. In the experimental condition, treadle presses

were rendered ineffective, and VI-correlated stimuli were presented according to a VT schedule

during VI food components. The VT schedule was yoked to each pigeons' own performance

during baseline, such that the density of VI-correlated stimulus presentations was held constant

across conditions. All pigeons showed a substantial decrease in rate of treadle pressing during

this condition. When the baseline conditions were reinstated (response-independent stimulus

presentations), responding recovered to levels similar to those observed during the original

baseline.

In Experiment 2, the effects of adding a delay to response-dependent VI-correlated

stimulus presentations were examined. Pressing the treadle during the delay reset the delay to its

maximum value, which was manipulated across conditions. The authors found that response

rates decreased as an inverse function of delay. Experiment 3 investigated the effects of

signaling the delay, and employed a second response key instead of a treadle as the observing

response. During signaled delay conditions, the observing key was darkened during the delay.

All subj ects pecked the observing key at slightly higher rates during the signaled-delay condition

compared to the unsignaled condition.









The results of each of these experiments are similar to those obtained when similar

manipulations are conducted on behavior maintained by unconditioned reinforcement. Thus, the

limited available literature suggests that the delivery of response-independent conditioned

reinforcers has similar effects as the response-independent delivery of unconditioned reinforcers.

The present study was designed to investigate the effects of response-independent delivery

of schedule-correlated stimuli in the context of an observing procedure. The present study was

also concerned with the discriminability of the background reinforcement context--variables

thought to modulate the reinforcing efficacy of the signals. This pertains to the broader issue of

stimulus control, and attention in particular. Interestingly, it was this issue of attention, rather

than conditioned reinforcement, that inspired the observing procedure.

Observing and Attention

The observing procedure was developed by Wyckoff (1952, 1969) to investigate attention.

The original procedure involved pigeons as subj ects and an experimental chamber equipped with

two manipulanda. Pecks on a lighted response key produced food on 50% of the trials; pressing

a treadle just above the floor changed the color of the food key such that a stimulus that was

uncorrelated with the trial outcome was replaced with a stimulus that was correlated either with

the presence (S ) or absence (S-) of food, depending on the programmed outcome for that trial.

This procedure was deemed relevant to questions concerning attention because the observing

response (treadle press) served only to produce correlated stimuli, but had no effect on the

availability of food. The procedure thus allowed for precise measurement of responses deemed

analogous to orienting responses such as head or whole body movements toward a stimulus.

For several decades following Wyckoff' s original report, the observing procedure was a

popular choice for researchers interested in investigating attention in non-human animals.

Research questions during this period were largely focused on determining what was maintaining









the observing response. While most researchers agreed that reinforcement was integral to an

explanation of observing behavior, there was considerable disagreement about its precise role.

Two general kinds of theories emerged in an attempt to explain the maintenance of observing

behavior, one of which appealed to the process of uncertainty reduction, and the other to the

process of conditioned reinforcement (Fantino, 1977).

Uncertainty-Reduction Theory

According to uncertainty-reduction theory, presentation of the S- should reinforce

observing to the extent that such a presentation reduces the uncertainty of a situation (Hendry,

1969). This account, which arose from information theory in applied mathematics and was

supported by cognitvely-oriented theorists, predicts that observing behavior should decrease as

the observing context becomes more predictable. For example, in observing procedures,

components in which food is available typically alternate with components in which food is

unavailable. In a given experimental session, contingencies are least predictable when 50% of

the session is spent in each component, and become more predictable with changes in the

proportion of session time in the presence of the two stimuli. Because the presentation of S+ and

S- reduce uncertainty to an equal extent, the function relating observing behavior to proportion of

time spent in one of the components should be a symmetrical inverted-U, with a peak at 50%.

Wilton and Clements (1971) tested this hypothesis in a trial-based observing procedure.

Pigeons produced outcome-correlated color changes on a lighted response key according to a

fixed-interval (FI) 15-s schedule. Completing the FI requirement produced a trial outcome (light

and food, or light only) after a 35-s delay, provided that no key pecks occurred in the 5-s prior to

the outcome. In one condition the probability of a food outcome was 0.2, and in a subsequent

condition this probability was changed to 0.8. Because these values deviate from 0.5 in equal









measure, uncertainty-reduction theory predicts that these values should generate the same rate of

response. However, the authors found that pigeons reliably pecked at higher rates during the FI

period in the 0.2 probability of food condition.

Kendall (1973) used a similar procedure, but allowed trials to progress in the absence of an

observing response. When the probability of food at the end of a trial was parametrically

manipulated across conditions (0.25, 0.5, and 0.75), the rate of observing was inversely related to

the probability that a trial terminated in food: observing increased as the probability of food

decreased.

Similarly, McMillan (1974) tested uncertainty-reduction theory, but used a wider range of

probability values. Pigeons produced food by pecking one response key according to a mixed

schedule with variable-interval (VI) 70-s and extinction (EXT) components (MIX VI EXT).

Mixed schedules are compound schedules of reinforcement in which no exteroceptive stimuli are

correlated with the respective component schedules. A single peck on a second key (the

observing key) changed the MIX schedule to a multiple (MULT) schedule, in which schedule-

correlated stimuli were available for 20 s. A probability gate that determined which of the

component schedules was in effect was sampled every 40 s. The probability of a VI component

was manipulated across seven conditions, including probabilities of 0.0 and 1.0. Contrary to the

predictions of uncertainty-reduction theory, McMillan found an asymmetrical inverted-U

function, with the highest rates of observing at moderately low probability values for 5 of 6

pigeons. Dinsmoor (1983) notes that such results are quite common and that the symmetrical

inverted-U function relating probability of observing to probability of a positive outcome

predicted by uncertainty-reduction has never been obtained.









Conditioned Reinforcement

The failure of uncertainty-reduction theory has led many researchers to favor explanations

of observing based instead on conditioned reinforcement. Although different versions of

conditioned reinforcement theory have been used to account for the data generated by

experiments employing the observing procedure, all are based on some type of stimulus-

reinforcer pairing. That is, observing is maintained by stimuli correlated with unconditioned

reinforcement (for reviews, see Dinsmoor, 1983; Fantino, 1977).

One apparent problem for explanations based on contiguous stimulus parings is that they

seem to require an additional assumption that the reinforcing value of the S+ is greater than the

punishing value of the S-. As Dinsmoor (1983) and others have pointed out, however, animals

will, to the extent that it is possible, maximize their exposure to S+ and minimize their exposure

to S-, for example, by turning away from the response key. This account, known as selective

observing, preserves a view based on contiguous stimulus-stimulus pairings.

The pairing view is further supported by data showing that presentations of S- do not

maintain observing behavior (Dinsmoor, Browne, & Lawrence, 1972). It has also been shown

that observing does not persist when component durations are very brief (Branch, 1973).

According to Dinsmoor, this fits well with a selective observing account because, "When

selectivity of observing is largely precluded...by restriction of the length of time for which the

S 'is available, artificial observing responses become difficult or impossible to maintain."

A more quantitatively sophisticated version of the pairing view of conditioned

reinforcement is Fantino's delay-reduction theory. In brief, the theory states that the value of a

conditioned reinforcer is a function of the relative reduction in delay to unconditioned

reinforcement signaled by its presentation (Fantino, 1977). Delay-reduction theory therefore










predicts that stimuli which signal an increase in the delay to unconditioned reinforcement should

not be effective conditioned reinforcers, and that behavior which produces them should therefore

not be maintained.

Initially developed to describe the effects of conditioned reinforcement on choice in

concurrent-chains procedures, delay-reduction theory has subsequently been extended to

observing procedures. For example, Case and Fantino (1981) exposed pigeons to MIX FI FI FI

schedules in which pecks on one response key produced food according to one of three

alternating FI schedules. Pecks on a second key converted the MIX to a MULT schedule,

replacing the uncorrelated stimulus with stimuli that were correlated with the FI schedule in

effect on the food key, and in some conditions produced a different (uncorrelated) stimulus.

Observing was well maintained by all stimuli that signaled a reduction in the delay to food

relative to the mixed stimulus. Contrary to the predictions of delay-reduction theory, observing

behavior was weakly maintained by stimuli that signaled an increase in the delay to food. The

results of a second experiment, however, showed that the rate of observing was higher when

such behavior produced a second, uncorrelated, stimulus than when it produced a stimulus

correlated with an increase in delay to food. The authors concluded that sensory reinforcement

was responsible for maintaining a low level of observing behavior, and that delay-reduction

theory therefore accounted well for the obtained data. However, uncertainty-reduction theory

could not account for the present data, because according to this theory, the reduction in

uncertainty provided by stimuli that signal an increase in the delay to food should be reinforcing.

The authors noted, however, that other theories of conditioned reinforcement could account

equally well for their data. Again, delay-reduction theory has not been shown to outperform

accounts of observing behavior based on contiguous pairing of stimuli. Delay-reduction theory









is only favored to the extent that it offers precise quantitative predictions that can be

demonstrated to apply across a range of conditioned reinforcement procedures, including the

observing and concurrent-chains procedures.

The Present Study

The purpose of the present investigation was to further explore the effects of response-

independent conditioned reinforcement on observing behavior. Unlike the Lieving et al. (2006)

study, the present study also manipulated the proportion of session time spent in either VI or

EXT components of the compound schedule operating on the food key. This manipulation was

conducted to examine the possibility that the effects of response-independent delivery of

schedule-correlated stimuli are modulated by the proportion of S' versus S- exposures. For

example, a greater proportion of S- exposures may produce greater proportional decreases in rate

of observing, or greater proportional decreases in measures of discriminative control by the

schedule-correlated stimuli. Furthermore, the effects of changing the proportion of session time

spent in each component of the compound schedule are well replicated and thus serve as solid

ground in what is otherwise a relatively novel procedure.

Another procedural feature adopted in the present study concerned preliminary training

with the component schedules. Mixed- and multiple-schedule training was begun

simultaneously, and training on the observing key did not begin until control by the mixed and

multiple stimuli was clearly evident. This training technique was suggested by Branch (1973),

because the common technique of introducing the mixed stimulus at the same time as the

observing response, and after multiple schedule training, may disrupt food key performance in

the presence of the mixed stimulus.

Another difference between the present study and Lieving et al. is that observing

responses were reinforced according to VI schedules, rather than a FR 1 schedule of









reinforcement. Variable-interval schedules are commonly chosen as baselines because they

minimize post-reinforcement pausing and engender steady intermediate rates of responding.

This is especially important in the present study because unlike the Lieving et al. study, which

examined response-dependent and response-independent schedules of conditioned reinforcement

in isolation, the present study attempted to demonstrate a more graded function relating response

rate to various combinations of conjoint VI VT schedules of conditioned reinforcement.

Perhaps the most unique procedural feature of the present study concerned the locus of

the correlated stimuli. In the maj ority of prior research on the topic, the mixed and multiple-

schedule stimuli are presented on a response key. In the present study, these stimuli were

presented using the houselight, with different flash rates corresponding to the different schedules.

This procedural feature is relevant to the issue of selective observing. Because the flashing of

the houselight is visible throughout the experimental space, these stimuli cannot be escaped,

thereby minimizing the possibility of selective observing.










CHAPTER 2
EXPERIMENT 1

Rachlin and Baum (1972, Experiment III-B) investigated the effects of conj oint VI VT

schedules of primary reinforcement on pigeons' key pecking. These authors found that

increasing the proportion of primary reinforcers per session delivered by the VT (response-

independent) schedule produced decreases in the rate of key pecking. The purpose of the present

experiment was to determine whether a similar relation is obtained when response-independent

deliveries of conditioned reinforcers are superimposed on behavior maintained by conditioned

reinforcement. Pecks on a response key produced food according to a mixed VI 40-s EXT

schedule of reinforcement. Pecks on a second (observing) key produced brief access to stimuli

correlated with the component currently in effect on the food key. During baseline conditions, a

VI 20-s schedule of stimulus presentations was arranged on the observing key. Across

experimental conditions, VT schedules of increasing density were superimposed on the

underlying VI contingency operating on the observing key.

Another variable of interest was the proportion of total session time spent in each of the

component schedules operating on the food key. After the manipulation described above was

conducted under conditions in which 50% of total session time was spent in the EXT component

of the mixed schedule on the food key, pigeons were exposed to a second series of conditions

identical to the first with the exception that either 25 or 75% of the total session time was spent

in the EXT component. This manipulation was accomplished by alternatively increasing or

decreasing the values of the distribution used to determine the duration of the EXT component.












Subj ects

One female (250) and five male (96, 289, 457, 1770, 1774) adult White Carneau pigeons

(Cohemba livia) served as subjects. All subjects were naive with respect to all experimental

procedures and maintained at approximately 83% of their free-feeding body weights. The

subj ects were housed in individual cages in a colony room where they had free access to fortified

water and grit. The colony was kept on a light/dark cycle with a light duration of 16.5 hours

beginning at 7AM.

Apparatus

Sessions were conducted in a standard three-key operant chamber. The chamber was light-

and sound-attenuating and located in a dark room. An exhaust fan in the chamber and a white

noise speaker outside the chamber served to further mask outside noise. The experimental

chamber measured 30.5 cm x 35 cm x 35 cm. Keys were arranged 23.5 cm from the floor of the

chamber and were spaced 5.7 cm apart. Only the left and center keys were operable. A

houselight provided general illumination in the chamber. A solenoid-operated food hopper was

used to deliver 3 s access to mixed grain. Response keys required a force of 0. 12 N to register a

response. The left key was illuminated red, and the center key white for all subj ects across all

training and experimental conditions. All experimental events were controlled and data collected

by a standard PC using MED-PC IV software and located in an adj acent room.

Preliminary Training

Subj ects were run in experimental sessions seven days per week at approximately the same

time every day. Each session began with a 5-min blackout period, during which all lights in the

experimental chamber were off and responses produced no scheduled consequences. The

pigeons were first trained to eat from the food hopper and then trained with food to peck the


Method









white center key via the method of successive approximations. Once key-pecking was

established, all pigeons were exposed to 21 sessions in which pecking produced food according

to a variable-interval (VI) 40-s schedule of reinforcement. Before the introduction of the left

(observing) key, all pigeons were exposed to training sessions in which the schedule of food

reinforcement on the center key alternated between VI 40 s and extinction (EXT) components.

The VI components terminated with food presentation. Duration of EXT components was

determined by drawing from a 20-element equal probability (Flesher-Hoffman) distribution of

values that were identical to but independent of the values of the VI 40-s distribution. The VI

and EXT components alternated pseudo-randomly such that the components could be presented

in either order, but with the restriction that an exposure to each component was completed before

a random determination of the next component type was made.

During these sessions and throughout the rest of the experiment, the houselight delineated

the S S-, and mixed conditions. Specifically, the houselight flashed at a higher rate (on and off

every 0.1 s) to signal the VI component (S ) and at a lower rate (every 0.66 s) to signal the

extinction component (S-). A steady-on houselight was used to signal the mixed schedule. All

stimuli used in the terminal procedure were introduced together in the training sessions.

Alternation between mixed and multiple (schedule-correlated) stimulus types occurred every 10-

min in the 60-min training sessions. Training sessions are therefore describable in standard

reinforcement-schedule nomenclature as MULT (MULT VI 40 s EXT/ MIX VI 40 s EXT).

Training continued until a clear discrimination developed between the 3 stimulus conditions, as

determined by visual inspection of cumulative records.

Following the initial training sessions, the left (observing) key was introduced. In the

absence of observing responses, the mixed stimulus was in place continuously. When the









observing requirement was satisfied the mixed stimulus was replaced for 3 s with the schedule-

correlated stimulus that corresponded to the schedule (VI or EXT) currently in effect on the food

key. The stimulus changed from S- to S+ immediately if components changed from EXT to VI

during the 3-s stimulus presentation. Because VI components terminated with food delivery, the

schedule-correlated stimulus did not change from S+ to S- during a single presentation. A

change-over delay (COD) that prevented the delivery of food for 1.5 s after an observing key

peck was in place throughout training and all experimental conditions.

Initially the observing-key schedule requirement was FR 1. That is, every peck on the

observing key changed the houselight from steady-on to flashing correlated with the contingency

of reinforcement in place on the food key. To facilitate acquisition of observing, initial exposure

to response-produced schedule-correlated stimuli was also controlled in such a way that the first

5 exposures to schedule-correlated stimuli in a session were S If the observing key was pecked

while an EXT component was in effect then the schedule-correlated stimulus was not produced

unless the pigeon had produced the S' at least Hyve times during that session. After several

sessions this restriction was lifted, and the observing schedule was leaned to a VI 10 s across

several sessions. Due to low observing rates, Pigeon 96 was exposed to an additional condition

in which schedule-correlated stimuli were produced only when observing responses were closely

(5 s or less) spaced. When observing rates were moderate and stable, the observing schedule was

changed to a VI 20-s schedule and Experiment 1 began.

Experimental Procedure

The schedule of food reinforcement arranged for pecks on the center key alternated

between VI 40 s and EXT, as described above. Sessions ended after 40 food presentations or 75

minutes, whichever came first. The schedule of reinforcement on the observing key was VI 20 s

throughout the experiment.









A variable-time (VT) schedule of response-independent presentation of schedule-

correlated stimuli was superimposed on the VI 20-s observing schedule across blocks of

sessions, or conditions, within a phase. Experimental phases were comprised of blocks of

conditions, and began with a baseline condition in which no response-independent presentations

of schedule-correlated stimuli occurred, followed by a series of conditions in which

response-independent presentations of schedule-correlated stimuli were presented with

increasing density (VT 40 s, VT 20 s, and VT 10 s, respectively). No COD was arranged

between these presentations and other experimental events. Each phase concluded with a

replication of the baseline condition. Thus, a phase consisted of 5 conditions. Conditions were

terminated after a minimum of 14 sessions had been conducted and a judgment of stability had

been made. Stability was determined by visual inspection of session rates of observing

responses for individual subj ects. Stability criteria required that no trend appear in the last five

sessions of a condition and that neither the highest or lowest session rates of observing responses

were contained in the last five sessions of a condition.

Across phases the proportion of session time spent in EXT was systematically

manipulated. This was accomplished by changing the values in the distribution comprising the

durations of EXT components. During Phase 1, pigeons spent approximately 50% of the session

in EXT. After completion of Phase 1 pigeons were split into two groups. Pigeons in Group A

(250, 457, 1774) were exposed to Phase 2, in which approximately 75% of total session time was

spent in EXT. Pigeons in Group B (96, 289, 1770) were exposed to Phase 3, in which

approximately 25% of total session time was spent in EXT. The original plan for this

experiment was for all pigeons to experience all phases. However, the within-phase changes in

the rate of response-independent stimulus presentation did not produce orderly changes in the









rate of observing behavior. Therefore, Experiment 2 began after 2 phases of Experiment 1 had

been completed. The order of conditions and number of sessions for each condition in

Experiment 1 appear in Table 2-1.

Results

Figure 2-1 shows mean rate of observing responses as a function of the average number of

response-independent stimulus presentations per min for the last five sessions of each condition

in Experiment 1. Each graph shows data for an individual pigeon. Subj ect number is shown in

the upper-left corner of each graph. Data for Group A pigeons are presented in the left column,

and data for Group B pigeons are presented in the right column. Each data path represents

results from a single phase. In general, response rates varied as a function of the proportion of

total session time spent in EXT. Specifically, for 25 of 29 within-subject across-phase

comparisons, rate of observing behavior increased as proportion of total session time spent in

EXT increased. Data were less orderly with respect to within-phase manipulations. Across

conditions within Phase 1 (50% EXT), 3 pigeons (250, 289, 457) showed no effect of average

number of response-independent presentations of the schedule-correlated stimuli. Data for two

of the remaining pigeons (1770, 1774) were characterized by a U-shaped function, whereas data

for a third (Pigeon 96) were characterized by a linearly decreasing function. Across conditions

within Phase 2 (75% EXT), the data were mixed: no effect for Pigeon 457, an inverted-U

function for Pigeon 1774, and a linearly decreasing function for Pigeon 250. Across conditions

within Phase 3 (25% EXT), the data were similarly mixed: an inverted-U function for Pigeon

289, and a decreasing function for Pigeon 1770. Observing behavior was not well maintained

across conditions in Phase 3 for Pigeon 96.

Figure 2-2 shows mean rate of responses on the food key as a function of the average

number of response-independent stimulus presentations per min for the last five sessions of each









condition in Experiment 1. Data are displayed similarly to Figure 2-1. For 23 of 29 within-

subj ect across-phase comparisons, rate of pecking on the food key varied as an inverse

proportion of total session time spent in EXT. Comparisons across conditions within Phase 1

indicate small idiosyncratic changes in rate of pecking on the food key as a function of the

density of response-independent stimulus presentations. For 3 pigeons (250, 289, 457), rates

were highest during baseline conditions, and then decreased but leveled off across conditions in

which response-independent stimuli were presented. For Pigeon 96, rates were lowest during

baseline conditions, but increased sharply when response-independent stimuli were introduced.

Data for Pigeon 1770 indicate an inverted-U shaped function, while data for Pigeon 1774

indicate no effect. Within Phases 2 and 3, the data indicate no effect for all pigeons across all

conditions. The lone exception to this was a decrease in rate of pecking for Pigeon 1770 during

the condition in which response-independent stimuli were presented at a rate of 6 per min.

To assess the extent to which the schedule-correlated stimuli controlled responses to the

food key, Figure 2-3 shows discrimination index (DI) as a function of the average number of

response-independent stimulus presentations per min for the last five sessions of each condition

in Experiment 1. Data are displayed similarly to Figure 2-1. The DI values shown in Figure 2-3

were calculated using Equation 2-1, where R1 is the key-pecking rate in the presence of S and

R2 the rate of key pecking in the presence of S-.


DI = (2-1)
Ri + R2

Figure 2-3 indicates strong control by the schedule-correlated stimuli across all subjects and

conditions, with mean DI above 0.9 in 35 of 59 cases, and above 0.7 in 58 of 59 cases. DI

remained relatively stable across all conditions for 5 of 6 pigeons. Pigeon 96 showed a slight









decrease in DI as the rate of response-independent deliveries of schedule-correlated stimuli was

increased.

A second DI analysis was conducted on food-key responses in the presence of the mixed

stimulus to determine the extent to which other events, such as food delivery, were serving a

discriminative function for responses to the food key. Equation 1 was used for this analysis, with

the rate of response during VI serving as R1, and the rate of response during EXT as R2.

Figure 2-4 shows mixed stimulus DI, and is arranged similarly to Figure 2-1. These data indicate

that all pigeons were more likely to peck the food key during VI than during EXT across all

conditions and phases. A slight increasing trend is apparent across conditions in 7 of 12 phases.

No effect is apparent across conditions in four phases. Phase 1 (50% EXT) data for Pigeon 96

show a high DI during baseline conditions, but a more moderate DI across conditions in which

stimuli were occasionally delivered independent of responding. A comparison of Figure 2-4 and

Figure 2-3 shows that DI was higher in the presence of schedule-correlated stimuli than in their

absence in all conditions for all subj ects.

Figure 2-5 shows a third DI analysis and is arranged similarly to the preceding figures.

Responses on the observing key were analyzed in identical fashion to the analysis of food key

responses in Figure 2-4. Data show DI values below 0.4 in 55 of 59 conditions across all

subjects. Thus, all pigeons were more likely to peck the observing key during EXT than during

VI in the presence of the mixed stimulus. Pigeon 96 was relatively more likely to peck the

observing key during VI than the other subjects across Phase 1 conditions. Across conditions a

slight decreasing trend is apparent in 9 of 12 phases.

Figure 2-6 shows relative rate of response on the food key (Pfood) as a function of the

average number of response-independent stimulus presentations per min. Data were analyzed










using Equation 2-2, where Rfood is the rate of pecking the food key in the presence of the mixed

stimulus, Robs is the rate of pecking the observing key in the presence of the mixed stimulus.

R~food
Pfood = (2-2)
Rfood + Robs

As in the preceding figures, each data path represents results from a single phase. Data indicate

that as proportion of total session time spent in EXT increased, the relative rate of responding on

the food key decreased. Relative rates were roughly similar across conditions within a phase for

all subjects. The only exception was for Phase 1 data for Pigeon 96. Relative rate of responding

on the food key was approximately 0.6 during baseline conditions, and increased sharply to

approximately 0.9 across conditions in which response-independent stimuli were presented.

Figures 2-7 and 2-8 show the contiguity between observing key pecks and subsequent

response-independent presentations of the schedule-correlated stimuli. Each dot shows a single

delay from an observing key peck to the next response-independent presentation of a stimulus,

and triangles indicate median values. If no observing key pecks occurred between two response-

independent stimulus presentations, then the latter presentation was excluded from the analysis.

For six of six pigeons, data show that as the rate of response-independent stimulus presentations

was increased, greater contiguity between observing key pecks and response-independent

stimulus presentations was obtained, both in terms of the median delay and the spread in the

distributions of obtained delays.

Discussion

Taken together, the results indicate that behavior was strongly controlled by the schedule-

correlated stimuli, and was systematically related to the proportion of total session time spent in

EXT. Increasing the density of response-independent stimulus presentations in the context of a









constant VI 20-s on the observing key, however, produced only relatively weak effects across all

dependent measures.

Although this latter effect appears to be at odds with comparable manipulations involving

unconditioned reinforcement, it was not due to lack of experimental control. First, the orderly

changes in observing behavior across phases, across which proportion of session time spend in

EXT components was manipulated, are similar to those obtained in previous experiments on

observing behavior (e.g., Kendall, 1973; McMillan, 1974; Wilton & Clements, 1971). Second,

the discrimination index for food-key responding in the presence of the multiple stimuli

remained high across all conditions of Experiment 1. These results indicate that S+ was serving

both discriminative and conditioned reinforcing functions, and that the reinforcing efficacy of the

S+ did not change appreciably across conditions.

As noted previously, changes in the response-reinforcer dependency have been implicated

as a critical controlling variable in the study of response-independent reinforcement (Lattal &

Abreu-Rodrigues, 1997). Across conditions in Experiment 1, the background VI schedule of

reinforcement for observing key pecks remained constant. Thus, it is possible that more extreme

disruption of the dependency between observing behavior and stimulus presentations could

produce more systematic changes in observing behavior. The relatively subtle changes in

dependency were accompanied by an increase in the probability of contiguous pairings of

observing responses and response-independent delivery of the schedule-correlated stimuli due to

the increasing density of response-independent stimulus presentations across conditions (see

Figures 2-7 and 2-8). This account is consistent with data showing that the pattern of key

pecking produced by a VI schedule of food presentation is relatively unchanged when the VI

schedule is replaced with a VT schedule (Marr & Zeiler, 1974; Zeiler, 1968). This possibility









was explored in Experiment 2 by holding the density of schedule-correlated stimulus

presentations constant across sessions, and manipulating the dependency via concomitant

changes in both the VI and VT schedules operating on the observing key.










Table 2-1. Order of conditions and number of sessions per condition for Experiment 1.
Group A Group B
Condition/ Phase 457 250 1774 96 289 1770


Phase 1 (50% EXT)
VI 20"
VI 20" + VT 40"
VI 20" + VT 20"
VI 20" + VT 10"
VI 20"

Phase 2 (75% EXT)
VI 20"
VI 20" + VT 40"
VI 20" + VT 20"
VI 20" + VT 10"
VI 20"

Phase 3 (25% EXT)
VI 20"
VI 20" + VT 40"
VI 20" + VT 20"
VI 20" + VT 10"
VI 20"


55 56
25 23
20 18
15 14
29 17


37 20
14 16
52 21
25 30
49 26


35 22
36 22
40 18
15 18
30 28


- 60 39 25 61
-41 17 16
-34 38 16
-20 63 22
20











25
S457


lo -
9 6


-t75%
20 8 -* 0
-A- 25%






5 2


0' 0
W 0 0 0.7 3.0 6.0 0.0 0.7 3.0 6.0

Z
5 0 250 14 289
UJ 12
S30 ~~_d

O w 20



4



UJ 0
CO0.0 0.7 3.0 6.0 0.0 0.7 3.0 6.0
O

301774 1770
14
25

20 12i -

15 -1cY8



10 -)-- 6


0.0 0.7 3.0 6.0 0.0 0.7 3.0 6.0

RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE


Figure 2-1. Rate of responses on the observing key as a function of rate of response-independent
stimulus presentations. Symbols indicate proportion of total session time spent in
EXT. Unconnected points are data from replicated conditions. Error bars indicate
standard deviations.











ou457




30 i--------

20--- _


60 1 96


-m 75%
S50%
~-25%


0 0 0.7


6 0 0.0 0.7


100 120


eo I 289


40

30

20

10

0


0.0 0.7


6.0 0.0 0.7


120 1 1774


1770


80


0.0 0.7


6.0 0.0 0.7


RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE


Figure 2-2. Rate of responses on the food key as a function of rate of response-independent
stimulus presentations. Symbols indicate proportion of total session time spent in
EXT. Unconnected points are data from replicated conditions. Error bars indicate
standard deviations.


I











457


-m 75%


6.0 0.0 0.7


0 0 0 7


250





0.8

0.7


289


0.0


0 0 0 7


1774


6.0 0.0 0.7


1770


0.0 l


0.0l


0.0 0.7 3.0 6.0 0.0 0.7 3.0 6.0
RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE


Figure 2-3. Discrimination index of food-key responding in the presence of schedule-correlated
stimuli as a function of rate of response-independent stimulus presentations. Symbols
indicate proportion of total session time spent in EXT. Unconnected points are data
from replicated conditions. Error bars indicate standard deviations.












1.0~ 457


-t75%


v


0 0 0.7


6.0 0.0 0.7


1.0~ 250


,to 289


~----,



5


0.8


0.6


0.0 0.



.01774


6.0 0.0 0.7


to1770


VY


f
0.6


0.0 0.7


6.0 0.0 0.7


RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE


Figure 2-4. Discrimination index of food-key responding in the presence of the mixed stimulus
as a function of rate of response-independent stimulus presentations. Symbols
indicate proportion of total session time spent in EXT. Unconnected points are data
from replicated conditions. Error bars indicate standard deviations.













75%
-* 50%


6.0


0.4



02



0.0
0.0 0.7


0 0 0 7


289


0.6 -


0 0 0 7



1774


6.0 0.0 0.7


1770


0.0
0.0 0.7 3.0 6.0 0.0 0.7 3.0 6.0

RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE


Figure 2-5. Discrimination index of observing-key responding in the presence of the mixed
stimulus as a function of rate of response-independent stimulus presentations.
Symbols indicate proportion of total session time spent in EXT. Unconnected points
are data from replicated conditions. Error bars indicate standard deviations.


I













96
1.0 .

0.8


457


0.8


0.4


0.2


0.0
6.0 0.0 0.7


-u 75%
-*- 50%
-A- 25%


0.0 0.7


1.0, 289


1.0 250


0.8


0.6 2~


0.6


0.4


0.2


0.0


0.0 0.7


6.0 0.0 0.7


1770
1.0

0.8


1774
1.0


0.8


0.0 0.7 3.0 6.0 0.0 0.7 3.0 6.0

RESPONSE-INDEPENDENT STIMULUS PRESENTATIONS PER MINUTE



Figure 2-6. Relative rate of responding on the food key as a function of rate of response-
independent stimulus presentations. Symbols indicate proportion of total session time
spent in EXT. Unconnected points are data from replicated conditions. Error bars
indicate standard deviations.










457 250 1774
160 160 1 160 :



140 : 14 140



120 120 120



100 j 1003I 1030 I i

lI I I

80 *80 80 .



60 60 I 60












Ill1 1 I o Io
0.7 3.0 6.0 0.7 3.0 6.0 0.7 3.0 6.0

RESPONSE-INDEPEN DENT STIMU LUS PRESENTATIONS PER MINUTE


Figure 2-7. Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation as a function of the rate of response-independent stimulus
presentations for three subjects (457, 250, 1774) in Experiment 1. Points show
individual obtained delays from the last five sessions of each condition. Open
triangles indicate median values.










96 289 1770
160 160 160



140 ( 4 140



120 120 120



100 1003 1(00







40 40 I I 40 i




20 2 0 2 0



0 0 40




I l l 0 I I I
0.7 3.0 6.0 0.7 3.0 6.0 0.7 3.0 6.0

RESPONSE-INDEPEN DENT STIMU LUS PRESENTATIONS PER MINUTE



Figure 2-8. Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation as a function of the rate of response-independent stimulus
presentations for three subjects (96, 289, 1770) in Experiment 1. Points show
individual obtained delays from the last five sessions of each condition. Open
triangles indicate median values.










CHAPTER 3
EXPERIMENT 2

To manipulate the dependency between observing responses and the presentation of

schedule-correlated stimuli, a procedure was employed that mimicked a previous study designed

to investigate the relationship between dependency and unconditioned reinforcement. Lattal

(1974) arranged a procedure in which the rate of reinforcement was held constant while the

dependency between pigeons' key pecks and food delivery was manipulated in a graded fashion

across conditions. This was accomplished by first establishing baseline performance according

to a VI schedule of reinforcement, and following this with a condition in which a similarly

valued VT schedule arranged food deliveries independent of responding. Following these

conditions, a series of intermediate dependency conditions were conducted in which every third

or every tenth reinforcer delivery was contingent on a response. In addition, one pigeon was

exposed to a condition in which every third reinforcer was delivered independent of responding,

but the remaining reinforcers were contingent on a response. Thus, conditions were conducted in

which 100, 66, 33, 10, and 0% of the reinforcers were response-dependent. Results indicated

that the rate of key pecking decreased as the proportion of response-dependent reinforcers

decreased. In the present experiment a similar manipulation was conducted, but the technique

used to accomplish this was slightly different, involving conjoint VI VT schedules in various

combinations, as described below.

Method

Subjects and Apparatus

Subj ects and apparatus were identical to those used in Experiment 1.









Procedure

All procedural details were identical to those of Experiment 1, unless otherwise noted.

Experiment 2 began immediately after Experiment 1 without a break. Prior to Phase 1 a

condition was conducted in which the duration of schedule-correlated stimulus presentation was

10 s. All other procedural details were identical to those of the baseline condition of Phase 1 of

Experiment 2. This condition was conducted to determine if increasing the stimulus duration

would produce higher rates of pecking on the observing key. After the completion of this

condition the duration of schedule-correlated stimulus presentation was reduced to 3-s. Because

most pigeons pecked on the observing key more frequently when the stimulus duration was 3-s,

this stimulus duration was used throughout Experiment 2.

Across conditions within a phase, contingencies were changed such that the programmed

density of schedule-correlated stimuli was held constant, while the level of dependency between

observing responses and schedule-correlated stimulus presentations was manipulated. During

baseline conditions, schedule-correlated stimuli were produced according to a VI 15-s schedule,

for an average of 4 response-dependent stimulus presentations per min, and 0 response-

independent stimulus presentations (4, 0). Across conditions within Phase 1, the average number

of response-dependent stimulus presentations per min was decreased by one, and the average

number of response-independent stimulus presentations was increased by one, until all stimulus

presentations occurred independent of responding. Thus, the schedule of stimulus production

operating on the observing key during the second condition of Phase I was a conj oint VI 20 s VT

60 s (3, 1). This condition was followed by conditions in which the schedule was conjoint VI 30

s VT 30 s (2, 2), conjoint VI 60 s VT 20 s (1, 3), and then VT 15 s (0, 4). The phase was

concluded with a replication of the VI 15 s baseline (4, 0).










Manipulation of the other maj or independent variable (proportion of session time in

EXT) followed the pattern established in Experiment 1, except that all subj ects received all

levels of the independent variable: 50% (Phase 1), 75% (Phase 2), and 25% (Phase 3). This was

accomplished by changing the mean of the rectangular distribution tied to EXT component

duration to 180 s (Phase 2) or 20 s (Phase 3).

Also unlike Experiment 1, component durations in Experiment 2 were determined by two

identical independent 11i-element rectangular distributions averaging 60 s. In this way, the

delivery of food on the VI schedule would not signal the absence of food (EXT).

The order of conditions and number of sessions for each condition in Experiment 2 appear

in Table 3-1. Conditions were terminated after a minimum of 14 sessions had been conducted

and a judgment of stability had been made. Stability was determined by visual inspection of

session rates of observing responses for individual subj ects. Stability criteria required that no

trend appear in the last five sessions of a condition and that neither the highest or lowest session

rates of observing responses were contained in the last five sessions of a condition.

When Phase 3 began rates of observing behavior dropped substantially. Thus, guidelines

were established to determine the sequence of conditions in Phase 3 dependent on individual

pigeons' performance. If the mean rate of observing key pecks across the last five sessions of

the Phase 3 baseline condition was less than one per minute, then no conditions involving

response-independent stimulus presentation were conducted. Instead, an A-B-A reversal was

conducted in which the proportion of total session time spent in EXT was changed from 25% to

50%, and then back to 25%. Only Pigeon 250 followed this sequence of conditions. If the mean

rate of observing key pecks across the last five sessions of the Phase 3 baseline was greater than

one per minute, but less than 1/3 of the rate observed in the Phase 2 baseline replication









condition, then an A-B-A reversal was conducted in which the dependency ratio was changed

from 4/0 to 0/4, and then back to 4/0. Only Pigeon 457 followed this sequence of conditions.

The sequence of Phase 3 conditions for Pigeons 1774, 96, and 289 was identical to the sequence

of conditions conducted in Phase 2. Pigeon 1770 was not exposed to Phase 3 conditions because

this subj ect required substantially more time to complete the first 2 phases.

Results

Figure 3-1 is a comparison of the rates of observing key pecks in the presence of the mixed

stimulus across conditions in which the duration of schedule-correlated stimulus presentations

was manipulated from 10 s (black bars) to 3 s (grey bars). Each set of bars show data for an

individual subj ect, as indicated on the X-axis. For 4 of 6 pigeons observing rates were higher

when the stimulus duration was 3 s than when it was 10 s. Rates were similar under both

stimulus durations for Pigeon 457 and were higher under the 10-s duration for Pigeon 1774.

Figure 3-2 shows for each pigeon the mean rates of observing responses for the last five

sessions of each condition as a function of the dependency ratio, defined as the average number

of schedule-correlated stimulus presentations available per min from VI and VT schedules,

respectively. Within-phase functions are characterized by an inverse relationship between the

dependency of observing key pecks on the presentation of schedule-correlated stimuli and the

rate of pecking the observing key. A decreasing trend in rate of observing key pecks across

conditions is apparent in all 16 phases across all subjects. Comparisons between conditions of

maximal and minimal dependency (4/0 and 0/4 dependency ratios, respectively) show lower

rates in the latter for 16 of 16 comparisons. Comparisons between Phases 1 and 2 indicate that

observing rates did not change substantially when the proportion of session time spent in EXT

was increased from 50% to 75%, although rates were slightly higher under Phase 2 conditions










(75% EXT) in 14 of 21 cases. Phase 3 conditions (25% EXT) generated lower rates of observing

than comparable conditions in Phases 1 and 2.

Figure 3-3 shows normalized group mean observing key pecks for all conditions of

Experiment 2. This analysis was conducted to assess the extent to which changes in the

proportion of session time spent in EXT interacted with changes in the dependency ratio. The

extensive overlap of the three functions on this figure indicate that these variables produced

additive, rather than interactive effects on the rate of observing key pecks.

Figure 3-4 shows mean response rate on the food key as a function of dependency ratio.

None of the pigeons showed systematic changes in response rate across conditions within a

phase. Across-phase comparisons indicate food-key response rates were inversely related to the

proportion of session time spent in EXT for four pigeons (457, 250, 289, 1770). Pigeons 96 and

1774 showed the highest rates of responding to the food key when the proportion of session time

spent in EXT was 50%. These results are similar to those obtained in Experiment 1.

Figure 3-5 shows the discrimination index for food-key responding in the presence of the

schedule-correlated stimuli. Discrimination index was calculated with Equation 1, using the

same measures as in Figure 2-3. As in Experiment 1, this measure indicates strong control of

food-key responding by schedule-correlated stimuli for all pigeons across all conditions. A clear

decreasing trend is apparent in 16 of 16 cases across conditions within a phase. Between-phase

comparisons indicate that proportion of session time spent in EXT did not affect discriminability

of the schedule-correlated stimuli, as indicated by the overlapping curves for 5 of 6 subj ects.

Figure 3-6 shows the discrimination index for food-key responding in the presence of the

mixed stimulus, calculated as in Figure 2-4. For all pigeons the discrimination-index values

were approximately 0.6 across all conditions and phases.










Figure 3-7 shows the discrimination index for observing responses in the presence of the

mixed stimulus. The discrimination-index values were roughly stable across conditions, with

values generally ranging from 0.3 to 0.5. This is consistent with the stability observed in the

discrimination index for food-key responding shown in Figure 3-6, and indicates that, in the

presence of the mixed stimulus, pigeons were most likely to peck the observing key during EXT,

and most likely to peck the food key during VI.

Figure 3-8 shows the relative rate of responding on the food key in the presence of the

mixed stimulus, calculated according to Equation 2. Relative rates were above 0.6 for all

pigeons under all conditions, and above 0.8 in 54 of 73 cases. Comparisons across conditions

within a phase indicate an increase in the relative rate of food key responding as dependency

ratio decreased. Across-phase comparisons show that the highest relative rates were obtained in

Phase 3 (25% EXT) in 5 of 5 cases. Comparison of relative rates obtained in Phases 1 and 2

show no difference in 5 of 6 cases. Pigeon 457 showed slightly higher relative rates during

Phase 1 (50% EXT) than Phase 2 (75% EXT).

The contiguities between observing key pecks and subsequent response-independent

presentations of the schedule-correlated stimuli are shown in Figures 3-9 and 3-10. These

figures are arranged similarly to Figures 2-7 and 2-8. The data show that temporal contiguity

between these events decreased as the dependency ratio was decreased for all pigeons, as

evidenced by increases in the median delay and the spread in the distributions of obtained delays.

Figures 3-11, 3-12, and 3-13 are representative cumulative records from two consecutive

sessions for Pigeon 250. Figure 3-11 shows the final few minutes of the last session of the 0/4

dependency ratio condition in Phase 1 (50% EXT). All schedule-correlated stimulus

presentations were response independent in this condition. The observing key record for this










Eigure is generally flat, and is punctuated by two brief periods of activity during EXT

components. Figure 3-12 shows the beginning of the following session, in which the

dependency ratio was changed to 4/0. No pecks on the observing key occurred for the first three

minutes of this session. The first peck on the observing key produced S-, after which rate of

observing key pecks briefly increased. Figure 3-13 shows the typical pattern of responding

during baseline conditions. Note that this record shows performance after approximately 3.5

minutes had passed from the end of Figure 3-12. Thus, reacquisition of terminal response

patterns was rapid. Note the bursts of responding to the observing key during EXT components,

as well as the higher rates of response on the food key during VI components.

Discussion

The results of the present experiment show that observing responses were controlled by the

dependency between observing responses and the presentation of schedule-correlated stimuli.

Specifically, decreases in the dependency ratio decreased rates of responding on the observing

key. This finding extends the Eindings of Lattal (1974) to behavior maintained by conditioned

reinforcement and is consistent with the findings of Lieving et al. (2006).

It is interesting to note that the discrimination index for food-key responding in the

presence of the multiple stimuli also decreased as the dependency between observing key pecks

and schedule-correlated stimulus presentations decreased. This was a product of increases in rate

of responding on the food key during S- presentations (data not shown). As shown in Figure 3-7,

responses on the observing key were more likely during EXT components throughout the

experiment. Thus, decreases in the rate of observing reduced competition between the observing

and food keys, especially during EXT components, as response-independent S- presentations

became more likely. This pattern of behavior also indicates that the pigeons did not move away

from the intelligence panel during S- presentations, as suggested by a selective observing










account of observing behavior. Personal observations of the experimental sessions further

substantiate that the pigeons in this experiment typically remained close to the intelligence panel

throughout the sessions. It seems likely that delivering the schedule-correlated stimuli via the

houselight prevented the development of selective observing.

Lastly, increasing the duration of schedule-correlated stimulus presentations generally

reduced rate of responding on the observing key in the present experiment. This finding is

limited because no reversal conditions were conducted. Dinsmoor, Mulvaney, & Jwaideh (1981)

showed that pigeons prefer long-duration S' presentations over short duration S' presentations.

However, measures of preference do not necessarily correspond to measures of rate of response.

For example, it is possible that one schedule of reinforcement would produce higher rates of

response than another schedule of reinforcement when exposure to both schedules is forced, but

that the schedule that engendered the lower rates of behavior would be chosen more frequently in

a preference assessment. Further experimentation is required to determine the precise relations

that controlled this outcome.











Table 3-1. Order of conditions and number of sessions per condition for Experiment 2.
Condition/ Phase 457 250 1774 96 289 1770
Phase 1 (50% E>D]


Vll15"
Vll15"
VI 20" +VT 60"
VI 30" +VT 30"
VI 60" +VT 20"
VT 15"
Vll15"
Phase 2(75% E>D]
Vll15"
VI 30" +VT 30"
VT 15"
Vll15"
Phase 3(25% E>D]
Vll15"
VI 30" +VT 30"
VT 15"
Vll15"b
Vll15"


20 20 20
42 36 16
14 22 20
14 19 58
14 23 19
17 16 41
25 24 24

25 14 15
38 17 14
18 37 16
29 40 18

41 28 32
22
16 13
-14
16 25


15 29 20
72 24 49
28 18 21
21 35 57
28 19 34
15 22 18
25 32 57

22 14 16
15 14 36
17 17 21
15 27

25 24
19 27
16 16


Note: a Schedule-correlated stimulus duration was 10 s during this condition, b 50% of
session time spent in EXT during this condition.























TT


D'


289 1770


PIGEON


Figure 3-1. Observing responses per minute as a function of the duration of the schedule-
correlated stimulus. Error bars indicate standard deviations.


M 3s


457 250 1774 96











lo I 96


-t75%

-A- 25%


4/0 3/1 2/2 1/3 0/4


4/O 3/1 2/2 1/3 0/4


1289


250


4/O 3/1 2/2 1/3 0/4


4/O 3/1 2/2 1/3 014


16-1770
14-

12i -


1774


4/0 3/1 2/2 1/3 0/4 4/0 3/1 2/2 1/3 0/4
DEPENDENCY RATIO


Figure 3-2. Rate of responses on the observing key as a function of dependency ratio. Symbols
indicate proportion of total session time spent in EXT. Error bars indicate standard
deviations. Unconnected points are data from replicated conditions. The open circle
indicates a replication of the 50% EXT baseline condition during Phase 3.




































U.U '
4/O 3/1 2/2 1/3 0/4

DEPENDENCY RATIO




Figure 3-3. Normalized group mean rate of responses on the observing key as a function of
dependency ratio. Symbols indicate proportion of total session time spent in EXT.


1.0 -


0.8 -




0.6 -




0.4 -




0.2 -


-c75%
S50%
- 25%











so I 96


Ir f


S75%
-* 50%
-A- 25%


4/0 3/1 2/2 1/3 0/4


250


4/O 3/1 2/2 1/3 0/4


so-289


4/O 3/1 2/2 1/3 0/4


4/O 3/1 2/2 1/3 014


so I 1770


1774


4/0 3/1 2/2 1/3 0/4


4/0 3/1 2/2 1/3 0/4


DEPENDENCY RATIO


Figure 3-4. Rate of responses on the food key as a function of dependency ratio. Symbols
indicate proportion of total session time spent in EXT. Unconnected points are data
from replicated conditions. Error bars indicate standard deviation. The open circle
indicates a replication of the 50% EXT baseline condition during Phase 3.


40 -1 h
A,
30














-t75%
S50%
~-25%


1.0


0.9


0.8


0.7


0.9


0.8


0.7


4/0 3/1 2/2 1/3 0/4


4/0 3/1 2/2 1/3 0/4


289


1.0


0.9


0.8


0.7



0.0~
4/0 3/1 2/2 1/3 0/4



1774


4/0 3/1 2/2 1/3 0/4



1770


0.9


0.8


0.7


0.9


0.8


07-


4/0 3/1 2/2 1/3 0/4 4/0 3/1 2/2 1/3 0/4

DEPENDENCY RATIO



Figure 3-5. Discrimination index of food-key responding in the presence of schedule-correlated
stimuli as a function of dependency ratio. Symbols indicate proportion of total
session time spent in EXT. Unconnected points are data from replicated conditions.
Error bars indicate standard deviations. The open circle indicates a replication of the
50% EXT baseline condition during Phase 3.











1.0 96


1.0~ 457


~-25%


4/0 3/1 2/2 1/3 0/4


4/D 3/1 2/2 1/3 0/4


1.0- 289


1.0- 250


t


0.6


4/0 3/1 2/2 1/3 0/4


4/0 3/1 2/2 1/3 0/4


.0 1770


1.0~ 1774


4/0 3/1 2/2 1/3 0/4


4/0 3/1 2/2 1/3 0/4


DEPENDENCY RATIO



Figure 3-6. Discrimination index of food-key responding in the presence of the mixed stimulus
as a function of dependency ratio. Symbols indicate proportion of total session time
spent in EXT. Unconnected points are data from replicated conditions. Error bars
indicate standard deviations. The open circle indicates a replication of the 50% EXT
baseline condition during Phase 3.


r
r


0.6 r~


0.6











S75%

~-25%


4/0 3/1 2/2 1/3 0/4


4/D 3/1 2/2 1/3 0/4


289


02


4/0 3/1 2/2 1/3 0/4


4/0 3/1 2/2 1/3 0/4


1774


1770


4/0 3/1 2/2 1/3 0/4 4/0 3/1 2/2 1/3 0/4
DEPENDENCY RATIO


Figure 3-7. Discrimination index of observing-key responding in the presence of the mixed
stimulus as a function dependency ratio. Symbols indicate proportion of total session
time spent in EXT. Unconnected points are data from replicated conditions. Error
bars indicate standard deviations. The open circle indicates a replication of the 50%
EXT baseline condition during Phase 3.











1 o 1 96


-t75%

~-25%


4/0 3/1 2/2 1/3 0/4


250s


4/0 3/1 2/2 1/3 0/4


289


4/0 3/1 2/2 1/3 0/4


1774


4/0 3/1 2/2 1/3 0/4


I 1770




0.6

04


4/0 3/1 2/2 1/3 0/4 4/0 3/1 2/2 1/3 0/4
DEPENDENCY RATIO


Figure 3-8. Relative rate of responding on the food key as a function of rate of response-
independent stimulus presentations. Symbols indicate proportion of total session time
spent in EXT. Unconnected points are data from replicated conditions. Error bars
indicate standard deviations. The open circle indicates a replication of the 50% EXT
baseline condition during Phase 3.










457


250


160 I 160 160



140 I 140 140



120 120 120



100 100 100*




S80 80 1 80



60 60 60



40 40 40



20 20 20



0' 0' 0
3/1 2/2 1/3 014 3/1 2/2 1/3 0/4 3/1 2/2 1/3 0/4

DEPENDENCY RATIO



Figure 3-9. Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation as a function of the dependency ratio for three subj ects (457,
250, 1774) in Experiment 2. Points show individual obtained delays from the last
five sessions of each condition. Open triangles indicate median values.





65


1774











289


160 16 160



140 140 140



120 120 120



100 100 100.








60 60 60



40 40 40



20 i 20 20





3/1 2/2 1/3 0/4 3/1 2/2 1/3 0/4 3/1 2/2 1/3 014


DEPENDENCY RATIO

Figure 3-10. Delay from an observing key peck to a response-independent schedule-correlated
stimulus presentation as a function of the dependency ratio for three subj ects (96,
289, 1770) in Experiment 2. Points show individual obtained delays from the last
five sessions of each condition. Open triangles indicate median values.


1770










30-


250


2500 i


I I


SECONDS


Figure 3-11. Cumulative record showing terminal performance for Pigeon 250 from the last
session of the 0/4 dependency ratio condition in Phase 1 of Experiment 2. The upper
line shows food key pecks and the lower line shows observing key pecks. Tick marks
at the zero point of the Y-axis show food deliveries. The upper event record shows
the schedule in operation on the food key, with an upward displacement of the record
indicating EXT. The lower event record indicates when schedule-correlated stimuli
were presented, with a downward displacement of the record indicating the presence
of schedule-correlated stimuli. Note the break on the Y-axis and corresponding
change in scale.











soo. 250


200-
75 a


SECONDS


Figure 3-12. Cumulative record showing reacquisition of baseline performance in the first
session of a replication of the 4/0 dependency ratio condition in Phase 1 of
Experiment 2. Data are arranged as in Figure 3-11. Note the break on the Y-axis and
corresponding change in scale.











250





Ow







EXT

60 0080 0
SECND

Fiue31.Cmltv rcr hwn tbebaeiepromnei tefrtssino
relcto fte40dpnec ai oniini hs fEprmn .Dt
ar ragda nFgr -1 oete ra nteYai n orsodn
chng inscl









CHAPTER 4
DISCUSSION

In Experiments 1 and 2, across-phase changes in the proportion of session time spent in

EXT components of a mixed schedule of food delivery produced orderly changes in rate of

observing key pecks. In both experiments, pigeons observed at lowest rates in the 25% EXT

Phases. In Experiment 1, the 75% EXT Phase produced the highest rates of observing, while in

Experiment 2 similar observing rates were produced by the 50% EXT and 75% EXT Phases. In

the presence of the mixed stimulus, observing responses were most likely during EXT

components for all pigeons throughout both experiments.

In Experiment 1, the rate of response-independent schedule-correlated stimulus

presentations was increased across sessions in the context of a constant response-dependent

schedule of conditioned reinforcement. This manipulation produced unsystematic changes in the

rate of observing key pecks, and did not produce appreciable changes in the discriminative

control by the schedule-correlated stimuli. In Experiment 2, density of schedule-correlated

stimulus presentations was held constant across conditions and the composite schedules of the

conj oint VI VT schedule operating on the observing key were varied to manipulate the

dependency between responses and schedule-correlated stimulus presentations across conditions.

Results showed orderly decreases in the rate of observing key pecks and the discriminative

control by the schedule-correlated stimuli as the dependency between pecks and stimulus

presentations was decreased. The implications of these results for theories of conditioned

reinforcement, observing, and attention will be discussed in this chapter.

Density of S+ and S-

The effects of the across-phase manipulation in both experiments, in which the proportion

of session time spent in the respective component schedules operating on the food key was









varied, provides a point of comparison to prior research (e.g., McMillan, 1974; Kendall, 1973).

Comparisons across the baseline conditions of each phase in Experiment 1 show results very

similar to those reported by Kendall. In general, when a larger proportion of session time was

spent in EXT components, rates of observing behavior were higher.

Results from baseline conditions in Experiment 2 show a slightly different relationship,

however. Wilton and Clements (1971) described a theoretical function relating the reinforcing

value of S+ presentations as a function of the probability of their presentation. According to this

function, the 50% and 75% EXT Phases should produce identical rates of response on the

observing key, but the 25% EXT Phase should produce considerably lower rates of observing

behavior. The data obtained in Experiment 2 are broadly consistent with this prediction; rate of

observing behavior was similar across the 50% and 75% EXT Phases, but dropped considerably

during the 25% EXT Phase.

An additional comparison relevant to the present study was conducted by McMillan

(1974). Two pigeons spent more time in the presence of schedule-correlated stimuli under

conditions in which the probability of a positive outcome was 0.2 (roughly similar to the 75%

EXT Phase in the present experiment) than when this probability was 0.5 (comparable to the

50% conditions). Another 2 pigeons showed similar performance under these conditions and the

remaining 2 pigeons spent more time in the presence of the schedule-correlated stimuli during

the 0.5 probability condition.

The reason for the difference between the results of the across-phase manipulations in

Experiments 1 and 2 is easily explained. During baseline conditions in these experiments, no

response-independent stimuli were presented. Thus, the only difference between these

conditions was the method used for switching between components operating on the food key.









In Experiment 1, VI components terminated with food delivery, while in Experiment 2

component alternations were unrelated to other experimental events. The method used in

Experiment 1 enhances component discriminability in the presence of the mixed stimulus. A

comparison of the baseline discriminability indices for food-key responding in the presence of

the mixed stimulus indicates that discriminability was roughly similar during baseline 50% EXT

conditions across the two experiments, but that the respective components were more

discriminable during 75% EXT baseline in Experiment 1 than in the same condition in

Experiment 2. Why this would produce the difference in baseline observing rates across the two

experiments requires a consideration of the typical pattern of behavior displayed by the pigeons

in both experiments. The discrimination indices of observing key responding in the presence of

the mixed stimulus indicate that observing pecks were more likely during EXT components

under all conditions in both experiments (see Figures 2-5 and 3-7).

Kelleher, Riddle, and Cook (1962) provided the following account of the pattern of

behavior they observed under baseline conditions similar to those used in the present study:

When observing responses produced the negative stimulus, the bird paused on both keys.
When the negative stimulus went off, the bird responded on the observing key until the
negative or positive stimulus appeared. If the positive stimulus appeared, the bird
responded at a high rate on the food producing key. When the positive stimulus went off,
the bird continued to respond on the food-producing key in the mixed stimulus (p. 6).

Similar patterns of behavior were evident in the present study, as shown in Figure 3-13.

As in the present experiments, Kelleher et al. reported that most of the observing key pecks

occurred during EXT components. Thus, it could be expected that extended bouts of observing

key pecks would follow food deliveries during the Experiment 1 75% EXT baseline condition,

when such deliveries signaled periods of food unavailability three times as long as under 50%









EXT. In contrast, food deliveries in Experiment 2 could be followed by another food delivery

only seconds later in both 75% and 50% EXT conditions.

The present data are also consistent with delay-reduction theory. Consider the most

consistent finding from the across-phase manipulations in the present experiments--that rate of

observing behavior was lower in the 25% EXT Phases than in the 50% EXT Phases. The inter-

reinforcement interval in the presence of the mixed stimulus was 60 s in the former case and 80 s

in the latter case. In both cases presentation of the S+ signaled a reduction in the average delay

to food delivery (40 s). This indicates a far greater reduction in the delay to food in the 50%

EXT condition. Delay-reduction theory therefore predicts that this condition would produce

higher response rates. This account matches the obtained result in the present experiments.

These results are also consistent, however, with modern pairing theories, because these theories

predict that the strength of a conditioned response is in part a function of the conditional

probability of an unconditioned stimulus in the presence of a conditioned stimulus relative to the

conditional probability of an unconditioned stimulus in the absence of a conditioned stimulus

(Rescorla, 1967). Thus, the results of the across-phase manipulation support conditioned-

reinforcement based theories of observing behavior in general, but do not differentially support a

particular version of such theories. The results do not support any version of uncertainty-

reduction theories, however, because such theories predict that the 25% and 75% EXT phases

should produce similar rates of observing behavior because these situations are equally

uncertain.

Response-Reinforcer Dependency

Rachlin and Baum (1972, Experiment III-B) compromised the dependency between

pecking and access to food by superimposing various VT schedules of food presentation on a VI

3-min schedule that was held constant across conditions. In comparison, Lattal (1974) held the









density of food presentations constant across conditions and varied the proportion of food

deliveries that were dependent upon a response. Similar results were obtained in both

experiments; as the number of response-independent reinforcer deliveries increased the rate of

food key pecking decreased. The present experiments suggest that whether similar functions are

obtained for behavior maintained by conditioned reinforcement depends on which of these

methods are used to compromise the dependency between responding and the presentation of

conditioned reinforcers. Specifically, increasing the density of response-independent

presentations of schedule-correlated stimuli in the context of a constant underlying response-

dependent schedule did not produce systematic effects on observing behavior in Experiment 1.

However, holding the number of schedule-correlated stimulus presentations per min constant and

reducing the proportion of these presentations delivered according to the response-dependent

schedule of stimulus delivery systematically reduced rate of response to the observing key in

Experiment 2. Thus, Experiment 2 extends the findings ofLattal to situations in which the

response-independent delivery of conditioned reinforcers is superimposed on behavior

maintained by conditioned reinforcement.

It is not immediately obvious why the manipulations conducted in Experiment 1 failed to

replicate the effect reported by Rachlin and Baum (1972). One possibility concerns the overall

density of reinforcement. Rachlin and Baum conducted a control condition in which a conj oint

VI 45 s VT 180 s schedule of reinforcement operated on the food key. Although this schedule

delivered the same proportion of response-independent reinforcement as another condition with

the same subj ects (conj oint VI 180 s VT 720 s), rate of key pecking was considerably higher in

the condition with the higher overall density of reinforcement for 3 of 4 pigeons. Thus, it seems

reasonable to conclude that increasing the density of reinforcement compresses the temporal









relations between response-independent events and behavior maintained by response-dependent

presentation of those same events. This interpretation is corroborated by data from the present

experiments showing that the different methods used to disrupt the dependency between

observing key pecks and stimulus presentations in Experiments 1 and 2 changed the contiguity

between these events in different ways. Specifically, increasing the rate of response-independent

stimulus presentations in the context of a continuous background schedule of response-

dependent stimulus presentations increased the contiguity between response and putative

conditioned reinforcer in Experiment 1. Thus, within-phase manipulations in Experiment 1

produced conflicting changes with respect to dependency and contiguity of events. In contrast,

decreases in the dependency ratio across conditions in Experiment 2 produced concomitant

decreases in the contiguity between responses and response-independent stimulus changes.

Given these outcomes, it seems likely that discriminative control by the response-independent

schedule was disrupted by the contiguity between responding and conditioned reinforcement

during Experiment 1.

It is also possible that conditioned reinforcers are fundamentally different from primary

reinforcers with respect to the effects of changes in the response-reinforcer dependency. This

seems very unlikely, given the extent to which the voluminous literature on conditioned

reinforcement supports the position that conditioned reinforcers function similarly to

unconditioned reinforcers in most respects, as reviewed in the introduction. The results of both

Marr and Zeiler (1974) and Lieving, et al. (2006), the only previous studies involving response-

independent conditioned reinforcement, also argue against this possibility.

Perhaps the strongest evidence for rej ecting the possibility that conditioned reinforcers are

somehow fundamentally different from primary reinforcers with respect to response-independent









reinforcement is Experiment 2 of the present investigation. As discussed by Burgess and

Wearden (1986), the manipulation of response-reinforcer dependency conducted in Experiment 2

confounds the effects of reducing the rate of response-dependent reinforcement and increasing

the rate of response-independent reinforcement. However, the endpoint conditions, in which

only response-dependent or response-independent presentations of schedule-correlated stimuli

were available, clearly illustrate the importance of dependency in conditioned reinforcement.

Further experimentation designed to isolate the relative contributions of the confounded

operations in Experiment 2 is warranted, but the graded function relating dependency to rate of

observing behavior suggests dependency, as formulated in Experiment 2, as an important

controlling variable.

Multiple Functions of Stimuli

There is one other alternative explanation for the discrepancy between the results of

Experiment 1 and the results of Rachlin and Baum (1973). Some theorists have argued the

conditioned reinforcing function of any stimulus depends on its establishment as a discriminative

stimulus--the so-called discrimination hypothesis (Keller & Schoenfeld, 1950). Keller and

Schoenfeld cited two experiments in support of the discrimination hypothesis. The first of these

showed that when a light was repeatedly paired with food after a rat had already collected the

food from the food cup, the light was ineffective as a reinforcer in later sessions in which it was

applied as a consequence for a new response (Schoenfeld, Antonitis, & Bersh, 1950). The

second experiment began by establishing a light as a discriminative stimulus by reinforcing rats'

lever-presses in its presence but not in its absence. Then, under extinction conditions in which

food was never delivered, the light was presented either as an antecedent for some rats and as a

consequence for others. No differences in performance during extinction were obtained. In later

extinction sessions, the temporal relation between lever presses and the light was reversed; rats









formerly exposed to the light as an antecedent were now exposed to the light as a consequence

for lever pressing, and vice-versa. No changes in performance were produced by changes in the

temporal relation between responding and stimulus presentation (Dinsmoor, 1950). Based on the

results of these experiments, Keller and Schoenfeld concluded that a discriminative function is a

necessary precondition for a conditioned reinforcing function, and that discriminative and

conditioned reinforcing functions are interchangeable.

Other authors, notably Kelleher (1966), have argued against the discrimination hypothesis

of conditioned reinforcement. Kelleher noted that the failure to obtain a reinforcing function in

the experiment by Schoenfeld et al. was similar to failures to produce an eliciting function using

similar arrangements in Pavlovian conditioning procedures. More importantly, Kelleher cited an

experiment by Stein (1958), which showed that rats developed a preference for a lever that

produced a tone after being exposed to a Pavlovian conditioning procedure in which the tone was

paired with intracranial brain stimulation. In this arrangement the tone never served a

discriminative function, yet the results indicate that it was an effective conditioned reinforcer.

In a similar vein, Kelleher (1961) used a pairing procedure to establish a click as a

reinforcer, and then used the click alone to generate patterns of behavior similar to those

generated by unconditioned reinforcer (as described earlier in the General Introduction). As in

the experiment by Stein, the putative conditioned reinforcer never served a discriminative

function, yet produced patterns of behavior consistent with a reinforcing function. As noted in

several extensive reviews on conditioned reinforcement, the notion that stimuli can have multiple

functions is not new, and does not j eopardize an account of the results of experiments involving

stimulus-response chains, second-order schedules, or observing procedures in terms of

conditioned reinforcement (Gollub, 1970; Kelleher, 1966; Kelleher & Gollub, 1962).









Other researchers have taken a more extreme position than that proposed by Keller and

Schoenfeld (1950). While Keller and Schoenfeld argued that a conditioned reinforcing function

depends on a discriminative function, Baum (1973), and Staddon (1983), for example, have

advocated for the abandonment of conditioned reinforcement as an explanation for performance

in chained schedules. An experiment by Schuster (1969, Experiment 3) is sometimes cited to

support the claim that conditioned reinforcement is an unnecessary concept. In this experiment

pigeons were exposed to a concurrent-chains procedure. Concurrent-chain schedules are

schedules in which satisfying the response requirement on one of two concurrently available

response keys (the initial link) produces another schedule of reinforcement (the terminal link) on

the chosen response key, and darkens the other key. Food is delivered when the response

requirement is completed in the terminal link. In Schuster' s experiment, identical VI 60-s

schedules operated during the initial links. This procedure differed slightly from typical

concurrent-chains procedures, however, in that each peck produced a 1-s blackout on both keys,

during which responses were ineffective. Completion of the response requirement on either key

produced the terminal-link stimulus on that key and darkened the other key. Each terminal link

provided brief access to food according to a VI 30-s schedule of reinforcement, but one key also

provided presentations according to an FR 11 schedule of stimuli paired with food delivery.

Pigeons in this experiment reliably preferred the response key that did not produce additional

stimuli, although for 2 of 5 subjects the preferences were weak. Additionally, all pigeons pecked

at a higher rate in the terminal link that included additional stimulus presentations than in the

terminal link that did not. Schuster reasoned that if the stimuli were conditioned reinforcers,

then the pigeons would prefer the response key that increased the number of stimulus

presentations, and thus concluded that the stimuli in chained schedules do not serve a reinforcing









function. He attributed the higher rates in the less preferred terminal link to the discriminative

function of the stimuli paired with access to food.

Gollub (1970) argued that instead of invalidating theories of conditioned reinforcement,

Schuster' s results simply corroborate earlier work by Fantino (1968), showing that higher rates

of behavior in one terminal-link produces preference for the alternative link. Baum (1973)

suggested that Gollub's interpretation was incorrect, because Fantino's results are restricted to

situations in which higher response rates are required, and so on.

Schuster' s results remain controversial, and experiments employing a variety of procedures

outside the scope of the present discussion have been conducted to illustrate that the concept of

conditioned reinforcement outperforms available alternatives (e.g., Dunn & Williams, 1991;

Dunn, Williams, & Royalty, 1987; Royalty, Williams, & Fantino, 1987). These experiments, in

concert with the cogent reviews of the literature by Gollub (1977, 1970) and Kelleher (1966)

provide strong evidence for the viability of the concept of conditioned reinforcement. In

contrast, there is relatively little evidence that supports extreme versions of the discrimination

hypothesis, as advanced by Baum, Staddon, and, more recently, Davison and Baum (2006). That

conditioned reinforcers also function as discriminative stimuli in many, and perhaps most cases

is uncontroversial, however.

Observing and Attending

Returning to the issue of observing, correlated stimuli undoubtedly serve multiple

functions, discriminative as well as reinforcing. Indeed, the observing procedure, as developed

by Wyckoff (1952, 1969), was designed to investigate aspects of stimulus control concerned

with attention. The observing response is an experimental analogue of responses such as turning

toward a stimulus, which presumably increases contact between receptor cells and the stimulus.

This has led some researchers to distinguish between "natural" and "artificial" observing










responses, with the stimulus-producing responses in observing procedures in the latter category

(e.g. Dinsmoor, 1983). Although this "natural/artificial" distinction may be intuitively

reasonable, there are many exceptions, in which naturally occurring conditions involve

"artificial" stimulus-producing responses. Imagine a situation in which a child is playing a video

game while her father is giving her important instructions concerning things she must do later in

the day. If she continues to press buttons on the game controller and stare at the screen, the

father is likely to command, "Pay attention to me!" This command is essentially equivalent to,

"Pause the game and look at me," which specifies a response quite like an observing response in

Wyckoff' s procedure.

The concept of attention is more commonly invoked under circumstances that are

analogous to those pertaining to stimulus control more broadly. For example, consider the

phrase, "He wouldn't have been hit by the bus if he had been paying attention." Note that in this

example attention is used as an explanation for the lack of behavior change in the presence of a

relevant stimulus. While such comments are convenient in everyday discourse, they are

unsatisfactory as explanations in a scientific account because they appeal to a concept that was

inferred from the behavior it is said to explain. As noted by Terrace (1966):

The use of attention as an explanatory principle in these instances is begging the question,
and seems to be nothing more than a mask for our ignorance concerning the establishment
of stimulus control (p. 289).

The distinction between attention as an explanation for behavior and attention as a

description of behavior has led researchers to address the concept of attention by determining the

conditions under which a stimulus comes to control behavior. Heinemann, Chase, and Mandell

(1968), for example, showed that whether a pigeon's behavior is altered by changes in a stimulus

depends on how stimulus changes relate to contingencies of reinforcement. In this experiment,










responses on one of two concurrently available response keys produced food. When the keys

were red, pecks to the left key produced access to food in the presence of a 300-Hz tone, and

pecks to the right key produced access to food in the presence of a 1000-Hz tone. When the keys

were green, only pecks to the right key produced access to food. Pecks on incorrect keys

produced an inter-trial interval of variable duration, during which the response keys were turned

off. The inter-trial interval followed food deliveries as well. After highly accurate performance

was obtained, the probability of reinforcement for correct responses was leaned to 0.5. In the

final condition, tones ranging in frequency from 450 Hz to 3200 Hz (but not 1000 Hz) were

presented across the 50% of trials in which food delivery was not programmed. The relationship

between the frequency of the tone on the unreinforced trials and the probability of a right-key

peck was examined. When the response keys were red the pigeons' behavior was controlled by

the frequency of the tone. Frequencies nearer to 1000 Hz produced a higher probability of right

key pecks. When the response keys were green, however, pigeons pecked the right key almost

exclusively, regardless of the frequency of the tone. In colloquial language, the pigeons learned

to attend to the tone when the keys were red, and ignore the tone when the keys were green. This

result shows that contingencies of reinforcement are critical determinants of stimulus control,

and therefore "attention". For example, our hypothetical bus accident victim's behavior may

have been under the control of stimuli signaling the availability of other kinds of reinforcement,

such as the smell of food from a nearby restaurant. Thus, the observing procedure addresses

attention in terms of orienting responses, and the discrimination literature addresses attention in

terms of stimulus control.

In Experiment 2, a function relating decreases in the dependency between observing

responses and the production of schedule-correlated stimuli to decreases in rate of observing









behavior was shown. The discrimination index used to measure control of food-key responding

by the schedule-correlated stimuli shows concomitant decreases in stimulus control as a function

of decreases in the dependency. Because the schedule-correlated stimuli were presented

diffusely throughout the experimental space, this result cannot be a function of the pigeons' lack

of contact with the stimulus. Instead, the decrease in stimulus control is a direct function of

changes in the dependency, as evidenced by the graded function relating these variables.

Although this finding is rather intuitive (as further demonstrated by the use of the same word to

describe these relations in everyday language), I know of no other study employing an observing

procedure that provides direct evidence that these two kinds of attention are related, although

theorists interested in these procedures have presumed as much (e.g., Dinsmoor, 1995).

That this function has not been demonstrated previously is probably a function of the

decades of controversy regarding why observing responses were maintained in Wyckoff' s

procedure. Although Wyckoff developed the observing procedure to address attentional

processes, in which antecedent-behavior relations are critical, most studies employing the

observing procedure emphasize behavior-consequence relations. The present study shows that

the observing procedure has not yet been fully exploited as a method for the study of attention.

Future Directions

Future experiments could employ variants of the present procedure to pit the level of

dependency between observing responses and schedule-correlated stimulus presentations (or the

response requirement on the observing key) against the strength of the correlation between the

stimuli and the schedule operating on the food key. Such a manipulation would facilitate a

comparison of the effects of effort and discriminative control on observing behavior.

Studies combining the observing procedure and discriminated timeout-avoidance

procedures would also be interesting. For example, responses on one key might postpone










periods of timeout from response-independent reinforcement, while responses on a second key

might produce stimuli correlated with temporal proximity to timeout. Another possibility is an

experiment in which response-independent presentations of schedule-correlated stimuli would be

prevented by pecks on an avoidance key. The proportion of session time spent in EXT

components might then be manipulated. This procedure could be used to determine whether the

rate of avoidance behavior is proportional to the number of S- presentations. Arrangements such

as this would be useful for the study of attention in the context of aversive control.

Any of these arrangements, including the present procedure, could be employed as

terminal links in choice procedures designed to assess preference in relation to, for example,

response-dependent versus response-independent presentations of schedule correlated stimuli. It

would also be informative to offer choices between response-independent presentations of S-

only, S+ only, or both schedule-correlated stimuli. Clearly, there is much work to be done if we

are to improve upon conventional accounts of attention (most of which have not progressed

much beyond James, 1890), and the possibilities offered by the observing procedure are almost

endless.

Another direction for future studies it to extend quantitative models of choice to situations

involving response-independent conditioned reinforcement. For example, an experiment could

be arranged in which the schedule requirement of the VI component of a conj oint VI VT

schedule operating on the observing key could be varied across conditions within an

experimental phase. The VT component could then be manipulated across phases, with the same

series of changes to the VI schedule conducted in each phase of the experiment. A parametric

manipulation of this kind would produce data amenable to modern quantitative analysis,









allowing for a determination of the extent to which currently available models can account for

the effects of response-independent delivery of conditioned reinforcement.

In closing, it may be worthwhile to consider some possible applications of the present

results. The relation between the dependency of schedule-correlated stimulus presentations on

observing behavior and the control of food-key pecking shown in the present procedure has clear

relevance to instructional design. The present results suggest, for example, that requiring a

response to produce instructions would enhance the discriminative control exerted by an

instruction. For example, if a company was interested in providing employees with training in

behavioral safety in the workplace, an interactive computer-based program in which the

employee controlled the delivery and removal of the instructions throughout the lesson should

produce considerably better results that even the best-made instructional video.












LIST OF REFERENCES


Baum, W. M. (1973). The correlation-based law of effect. The Journal of the Experimental
Analysis of behavior, 20, 137-153.

Branch, M. N. (1973). Observing responses in pigeons: Effects of schedule component duration
and schedule value. The Journal of the Experimental Analysis ofBehavior, 20, 417-428.

Bugelski, R. (1938). Extinction with and without sub-goal reinforcement. Journal of
Comparative Psychology, 26, 121-134.

Burgess, I. S., & Wearden, J. H. (1986). Superimposition of response-independent
reinforcement. The Journal of the Experimental Analysis of Behavior, 45, 75-82.

Case, D. A., & Fantino, E. (1981). The delay-reduction hypothesis of conditioned reinforcement
and punishment: Observing behavior. The Journal of the Experimental Analysis of
Behavior, 35, 93-108.

Catania, A. C., Deegan, J. F., & Cook, L. (1966). Concurrent fixed-ratio and avoidance
responding in the squirrel monkey. Journal of the Experimental Analysis ofBehavior, 9,
227-231.

Chomsky, N. (1959). A review of Skinner's Verbal Behavior. Language, 35, 26-58.

Cowles, J. T. (1937). Food-tokens as incentives for learning by chimpanzees. Comparative
Psychology M~onographs, 14, 1-96.

Davison, M., & Baum, W. M. (2006). Do conditional reinforcers count? Journal of the
ExperimentalAnalysis ofBehavior, 86, 269-283.

Dews, P. B. (1960). Free-operant behavior under conditions of delayed reinforcement: I. CRF-
type schedules. Journal of the Experimental Analysis of Behavior, 3, 221-234.

Dinsmoor, J. A. (1950). A quantitative comparison of the discriminative and reinforcing
functions of a stimulus. Journal ofExperimental Psychology, 40, 458-472.

Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioraland'Brain
Sciences, 6, 693-728.

Dinsmoor, J. A. (1995). Stimulus Control: I. The Behavior Analyst, 18, 51-68.

Dinsmoor, J. A., Browne, M. P., & Lawrence, C. E. (1972). A test of the negative discriminative
stimulus as a reinforcer of observing. Journal of the Experimental Analysis ofBehavior,
18, 79-85.










Dunn, R., Williams, B., & Royalty, P. (1987). Devaluation of stimuli contingent on choice:
Evidence for conditioned reinforcement. Journal of the Experimental Analysis of
Behavior, 48, 117-13 1.

Estes, W. K. (1949). A study of motivating conditions necessary for secondary reinforcement.
Journal ofExperimental Psychology, 39, 3 06-3 10.

Fantino, E. (1968). Effects of required rates of responding upon choice. Journal of the
ExperimentalAnalysis ofBehavior, 11, 15-22.

Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of
Behavior, 12, 723-730.

Gollub, L. R. (1958). The chaining offixed-interval schedules. Unpublished doctoral
dissertation, Harvard University.

Gollub, L. R. (1970). Information on conditioned reinforcement: A review of Conditioned
Reinforcement, edited by Derek P. Hendry. Journal of the Experimental Analysis of
Behavior, 14, 361-372.

Heineman, E. G., Chase, S., & Mandell, C. (1968). Discriminative control of attention. Science,
160, 553.

Hendry, D. P. (1969). Reinforcing value of information: Fixed-ratio schedules. In D. P. Hendry
(Ed.), Conditioned reinforcement (pp. 300-341). Homewood, 1L: Dorsey Press.

Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of
reinforcement. Journal of the Experimental Analysis ofBehavior, 4, 267-272.

Herrnstein, R. J., & Morse, W. H. (1957). Some effects of response-independent reinforcement
on maintained operant behavior. Journal of Comparative and Physiological Psychology,
50, 461-467.

Imam, A. A., & Lattal, K. A. (1988). Effects of alternative reinforcement sources: A
reevaluation. Journal of the Experimental Analysis of Behavior, 50, 261-271.

James, W. (1890). The principles of psychology. New York, NY: Henry Holt and Company.

Kelleher, R. T. (1961). Schedules of reinforcement during experimental extinction. Journal of
the Experimental Analysis of Behavior, 4, 1-5.

Kelleher, R. T, (1966). Chaining and conditioned reinforcement. In W. K. Honig (Ed.), Operant
behavior: Areas ofresearch and application (pp. 160-212). Englewood Cliffs, NJ:
Prentice-Hall.










Kelleher, R. T., & Gollub L. R. (1962). A review of positive conditioned reinforcement.
Journal of the Experimental Analysis ofBehavior, 5(4, Suppl.), 543-597.

Kelleher, R. T., Riddle, W. C., & Cook, L. (1962). Observing responses in pigeons. Journal of
the Experimental Analysis ofBehavior, 5, 3-13.

Keller, F. S., & Schoenfeld, W. N. (1950). Principles ofpsychology: A systematic text in the
science of behavior. East Norwalk, CT: Appleton-Century-Crofts, Inc.

Kendall, S. B. (1973). Effects of two procedures for varying information transmission on
observing responses. Journal of the Experimental Analysis of Behavior, 20, 73-83.

Lattal, K. A. (1974). Combinations of response-rein-forcer dependence and independence.
Journal of the Experimental Analysis ofBehavior, 22, 3 57-362.

Lattal, K. A., & Abreu-Rodrigues, J. (1997). Response-independent events in the behavior
stream. Journal of the Experimental Analysis ofBehavior, 68, 375-398.

Lattal, K. A., & Bryan, A. J. (1972). Effects of concurrent response-independent reinforcement
on fixed-interval schedule performance. Journal of the Experimental Analysis of
Behavior, 26, 495-504.

Lieving, G. A., Reilly, M. P., & Lattal, K. A. (2006). Disruption of responding maintained by
conditioned reinforcement: Alterations in response-conditioned-reinforcer relations.
Journal of the Experimental Analysis ofBehavior, 86, 197-209.

Marr, M. J., & Zeiler, M. D. (1974). Schedules of response-independent conditioned
reinforcement. Journal of the Experimental Analysis ofBehavior, 21, 433-444.

Mazur, J. E. (1997). Learning and behavior (4 ed.). Upper Saddle River, NJ: Prentice-Hall.

McMillan, J. C. (1974). Average uncertainty as a determinant of observing behavior. Journal of
the Experimental Analysis of Behavior, 22, 401-408.

Miller, N. E. (1951). Learnable drives and rewards. In S. S. Stevens (Ed.), Handbook of
experimentalpsychology (435-472). Oxford, England: Wiley.

Pavlov, I. (1927). Conditioned reflexes. New York, NY: Oxford University Press.

Pierce, W. D., & Cheney, C. D. (2004). Behavior analysis and learning (3rd ed.). Mahwah, NJ:
Lawrence Erlbaum Associates.

Prokasy, W. F. Jr. (1956). The acquisition of observing responses in the absence of differential
external reinforcement. Journal ofConsparative and Physiological Psychology, 49, 13 1-
134.










Rachlin, H., & Baum, W. M. (1972). Alternative reinforcement: Does the source matter?
Journal of the Experimental Analysis ofBehavior, 18, 23 1 -241 .

Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological
Review, 74, 71-80.

Royalty, P., Williams, B. A., & Fantino, E. (1987). Effects of delayed conditioned reinforcement
in chain schedules. Journal of the Experimental Analysis ofBehavior, 47, 41-56.

Schoenfeld, W. N., Antonitis, J. J., & Bersh, P. J. (1950). A preliminary study of the training
conditions necessary for conditioned reinforcement. Journal ofExperintental
Psychology, 40, 40-45.

Schuster, R. H. (1969). A functional analysis of conditioned reinforcement. In D. P. Hendry
(Ed.), Conditionedreinforcentent (pp. 192-234). Homewood, 1L: Dorsey Press.

Schwartz, B. (1978). Psychology of learning and' behavior. Oxford, England: W. W. Norton &
Co.

Sizemore, O. J., & Lattal, K. A. (1977). Dependency, temporal contiguity, and response-
independent reinforcement. Journal of the Experimental Analysis ofBehavior, 27, 119-
125.

Skinner, B. F. (1938). The behavior oforgan2isnas: An experimental analysis. Oxford, England:
Appl eton-C century.

Skinner, B. F. (1948). 'Superstition' in the pigeon. Journal ofExperintentalPsychology, 38,
168-172.

Staddon, J. E. R. (1983). Aninzal learning and' behavior. Cambridge, England: Cambridge
University Press.

Stein, L. (1958). Secondary reinforcement established with subcortical stimulation. Science,
127, 466-467.

Terrace, H. S. (1966). Stimulus Control. In W. K. Honig (Ed.), Operant behavior: Areas of
research and' application (pp. 271-344). Englewood Cliffs, NJ: Prentice-Hall.

Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative
processes in animal s. Psychological M~onoguraphs, 2, 1-109.

Timberlake, W., & Lucas, G. A. (1985). The basis of superstitious behavior: Chance
contingency, stimulus substitution, or appetitive behavior? Journal of the Experimental
Analysis ofBehavior, 44, 279-299.










Williams, K. A. (1929). The reward value of a conditioned stimulus. thriversity of Cahfornia
Publications in Psychology, 4, 3 1-55.

Williams, B. A., & Dunn, R. (1991). Preference for conditioned reinforcement. Journal of the
ExperimentalAnalysis ofBehavior, 55, 37-46.

Wilton, R. N., & Clements, R. O. (1971). The role of information in the emission of observing
responses: A test of two hypotheses. Journal of the Experimental Analysis ofBehavior,
16, 161-166.

Wolfe, J. B. (1936). Effectiveness of token rewards in chimpanzees. Comparative Psychology
M~onoguraphs, 12, 1-72.

Wycoff, L. B., Jr. (1952). The role of observing responses in discrimination learning.
Psychological Review, 59, 68-78.

Wycoff, L. B., Jr. (1969). The role of observing responses in discrimination learning. In D. P.
Hendry (Ed.), Conditioned reinforcement (pp. 237-260). Homewood, 1L: Dorsey Press.

Zeiler, M. D. (1968). Fixed and variable schedules of response-independent reinforcement.
Journal of the Experimental Analysis ofBehavior, 11, 405-4 14.

Zeiler, M. D. (1977). Elimination of reinforced behavior: Intermittent schedules of not-
responding. Journal of the Experimental Analysis ofBehavior, 27, 23-32.









BIOGRAPHICAL SKETCH

Born in New Jersey, and raised in Philadelphia, PA by my grandparents, I first arrived in

Florida in 1987 at the age of 13. In my first few years in Florida I attended Largo High School

and learned to play the guitar. After earning a Florida Undergraduates' Scholarship I decided to

attend the University of Florida, and arrived in Gainesville in 1992. I put myself through school

with a combination of grants, loans, and part-time j obs. When part-time j obs became scarce, I

donated plasma to have money for food. One night I was robbed at gunpoint while working in a

Subway restaurant on NW 13th Street, just north of University Avenue. After another robbery

about a month later, I decided to quit that j ob and work at Toys R Us. Although there was no

shortage of guns in Toys R Us, they were all made of plastic and shot foam bullets. It was about

that time that I decided to become a psychology major, more as a default option rather than out

of any kind of real passion. After sitting through classes listening to endless serious

presentations of ridiculous theories, volunteering as a counselor for the local suicide hotline

where the principle technique was to paraphrase what the caller said and repeat it back to them,

and as an assistant in a rat lab where no one ever talked about why what they were doing was

relevant, I was seriously entertaining the idea of changing maj ors. It was then that I met Dr.

Hank Pennypacker, and things began to change quickly. I began to understand the practicality

and power of the study of behavior in its own right. The silly theories were all gone, and

replaced with a simple, elegant set of descriptions about the relationship between behavior and

environment. The techniques were easy to apply in everyday life, and I thought I might be onto

something. I enrolled in a course called "Behaviorism in Contemporary Society" taught by Ed

Malagodi, who's insightful guidance led me to an understanding that what I had stumbled upon

was not a simple collection of techniques for changing behavior, but a thoroughgoing world-

view. It had become easy to be passionate about what I was doing.









I enrolled in the masters' program in Applied Behavior Analysis at the University of North

Texas (UNT), where I was given the opportunity to work in a variety of settings with a wide

range of populations. I worked as an in-home therapist for children with autism, a unit

psychologist in an institution for developmentally-disabled adults, an after-school facility for

disadvantaged children, and a school for children with severe behavioral disorders. However,

my favorite experiences at UNT were those I had with Sigrid Glenn, who taught me to teach

undergraduate courses in behavior analysis, and Manish Vaidya, who chaired my thesis proj ect

on the delayed matching-to-sample performance of typically functioning adult humans. These

experiences convinced me to pursue an academic career in the experimental analysis of behavior.

I enrolled in the Ph.D. program at the University of Florida, and returned to Gainesville in 2002

to work with Dr. Tim Hackenberg. While here I have spent most of my time studying basic

behavioral processes in pigeons, and continuing to teach undergraduates about the experimental

analysis of behavior. After graduating, I hope to continue studying basic behavioral processes in

both humans and non-human animals and serve as faculty in a university setting.