Analogies Between Occasion Setting and Pavlovian Conditioning Ralph R. Miller and Philippe Oberling 0 ccasion setting refers to the potential of a stimulus to clarify the predictive value of an ambiguous cue. More technically speaking, occasion setting refers to the ability of a punctate stimulus (or diffuse context), sometimes called an occasion setter or feature, to modulate re- sponding to a partially reinforced conditioned stimulus (CS) when the occasion setter and the target CS are presented in close temporal proxim- ity. In other words, the occasion setter (X) “sets the occasion” for respond- ing (or not responding) to the CS (A). Specifically, positive occasion setters signal that a CS will be reinforced, and negative occasion setters signal that a CS not be reinforced (see Exhibit 1). The term occasion setter will originated with Skinner (1938), who applied it to operant situations (see Bonardi, 1988, 1989; Colwill & Rescorla, 1990; Davidson, Aparicio, & Rescorla, 1988; for contemporary examples of occasion setting in instru- Support for this research was provided by National Institute of Mental Health Grant 33881, Universite Louis Pasteur-Strasbourg, Fondation Bettencourt-Schueller,S andoz Laboratories (France), and Fondation Guy Weisweiller. The authors would like to thank Moore Arnold, Robert Barnet, Aaron Blaisdell, Daniel Burger, Robert Cole, James Denniston, Lisa Gunther, James Esposito, Haney Mallemat, Hernan Savastano, and Russell Wishtart for their assistance in gathering the data from our laboratory that we present here and for their comments on an early version of this chapter. 3 MILLER AND OBERLING Defining Occasion Setting Train Test Result No occasion setting A-USIA A CR Positive occasion setting X+A+US/A X+A CR A - X - Negative occasion setting A+US/X+A A CR X+A - X - NOTE: Partial reinforcement without an occasion setter is ambiguous. Occasion setters can remove this ambiguity. X and A represent initially neutral punctate stimuli, which as a conse- quence of training has resulted in A becoming an excitatory conditioned stimulus and X becoming an occasion setter. + denotes “followed by”; different trial types that were given interspersed are separated by I; the approximate magnitude of the observed conditioned responding is indicated by the presence or absence of a conditioned response (CR) in the result column. mental situations with a discriminative stimulus playing the role of an occasion setter). Although interest in occasion setting was dormant for several decades, there has been a recent resurgence of interest in the topic, particularly in Pavlovian situations. This resurgence is a result in large part to the fact that occasion setters have been found to modulate simple associations, an unexpected phenomenon in the framework of reflexive Pavlovian re- sponding in which the acquired potential of a stimulus is limited to being able to directly elicit a conditioned response (see Pavlov, 1927). This find- ing encourages the use of occasion-setting principles to augment associa- tive theory in providing explanations of complex acquired behavior, par- ticularly choice behavior. It should be emphasized that this renewed interest in occasion setting relies mainly on the monumental work of Peter Holland (e.g., 1992) and colleagues in illuminating the nature of occasion setting. Central to our understanding of occasion setting are two early findings: First, a single stimulus can simultaneously possess both excitatory (i.e., Pavlovian) and modulatory (i.e., occasion-setting) properties, and these two properties are (almost) independent of each other (e.g., Holland, 4 ANALOGIES: OCCASION SETTING AND CONDITIONING 1983, 1985; Rescorla, 1985, 1986, 1987; Ross, 1983). In actuality, positive occasion-setting training often gives the occasion-setting stimulus both occasion-setting properties and Pavlovian excitatory properties. Nonrein- forced presentations of the occasion setter alone during or after training, however, commonly eliminate (i.e., extinguish) the Pavlovian excitatory properties of the stimulus without influencing its occasion-setting proper- ties (see “Extinction,” later in this chapter). Second, Holland and other researchers have determined that occasion setters affect behavior through some means other than their associations to the unconditioned stimulus (US) summating with CS-US associations (e.g., Holland, 1983, 1985, 1989). These two important findings themselves do not illuminate the way in which occasion setters act to modulate Pavlovian responding or the conditions under which a stimulus will come or not come to serve as an occasion setter. This chapter primarily addresses the latter question. Like many researchers, our interest in occasion setting was kindled primarily by its potential use as a means of accounting for complex learn- ing phenomena within the framework of the simple and relatively well understood principles of Pavlovian conditioning. Our presentation in this chapter will be relatively nontheoretical. In our view, models at the associa- tive or cognitive levels rarely reflect actual psychological mechanisms (which potentially provide explanations of behavior at a more molecular level of explanation), but instead serve primarily as heuristic devices to guide us toward discovering important new behavioral phenomena. Our heuristic with respect to this chapter is the potential analogy between serial occasion-setting phenomena and the phenomena discovered during 80 years of research in Pavlovian conditioning. EXTINCTION The earliest phenomenon that led us to think about a set of analogies between serial occasion setting and excitatory Pavlovian conditioning was extinction. In Pavlovian conditioning, extinction is often described opera- tionally as repeated nonreinforced presentations of the CS, which results in a decrement in conditioned responding (see top part of Exhibit 2). 5 MILLER A N D OBERLING ~ ~~ ~ Group Train Extinguish Test Result Extinction in Pavlovian conditioning (Pavlov, 1927) Extinction A+US A A cr Control A+US B A CR Extinction in occasion setting (Rescorla, 1986; see also 1983) Ross, OSing X+A+US/AN+B+US/B Context only X+B CR B cr Exp. 1 X+A+US/AN+B-+US/B X X+B CR B cr Exp. 2 X+A+US/A/Y+B+USIB X+A/A+US X+B cr B cr Exp. 3 X+A+US/AN+B+US/B X+A X+B cr B Cr NOTE X, Y, A, and B were initially neutral stimuli. --f denotes "followed by"; different trial types that were given interspersed are separated by /; the approximate magnitude of the observed conditioned responding is indicated by the case, size, and boldness of the font used in the result column. Only select groups are represented here. Seeking an analogy to Pavlovian extinction within occasion setting, Res- corla (1986; see also Ross, 1983) initially examined the effects of presenting the occasion setter alone (i.e., group Exp. 1 in the bottom part of Exhibit 2). (Note that throughout this chapter, we ignore procedural details and the fine grain of the data of individual studies. For this information, readers should consult the primary sources.) As indicated in Exhibit 2, Rescorla found that simple, nonreinforced presentation of occasion setter X did not attenuate occasion setting by X (group Exp. 1). However, duplication of the training treatment with the reinforcement contingencies reversed from those of training did attenuate occasion setting. That is, reinforce- ment on trial types that lacked reinforcement during initial occasion- setting training and nonreinforcement on trial types that were reinforced during initial training sufficed to degrade the initial occasion-setting value of a stimulus. Moreover, this outcome was observed with or without 6 ANALOGIES: OCCASION SETTING AND CONDITIONING A+US trials (groups Exp. 2 and Exp. 3, respectively). Thus, with regard to behavioral consequences, presentation of X alone is actually a poor analogy to Pavlovian extinction. If extinction treatment is operationalized as exposure to the training conditions (either for all trials or only for those trials involving the feature) with the trial-by-trial reinforcement contingencies reversed, however, then a good analogy of the extinction of Pavlovian excitation exists within occasion setting. This analogy follows the same operational rule for decrementing stimulus control by an occa- sion setter as by a Pavlovian excitor (i.e., training conditions with reversal .' of the reinforcement contingencies) TEMPORAL CODING In recent years, considerable attention in Pavlovian conditioning has been directed toward the role of timing, beyond the long-recognized benefits for behavioral control of close temporal proximity. Those espousing the traditional view assume that temporal proximity is a catalyst for Pavlovian conditioning. In contrast, recent research has suggested that (a) Pavlovian associations include information about the temporal relationship of the paired events, and (b) this temporal information is used by subjects in determining the form and timing of conditioned responding (see Miller & Barnet, 1993, for a brief review). One example of such a demonstration is provided by Barnet, Cole, and Miller (1997). In this study (see Exhibit 3), group 1st-ORDER dis- ~ ~~~ 'Although it is a digression from occasion setting, consider for a moment the implications of this finding for our conceptualization of extinction in general-that is, rather than defining extinction in the abstract as nonreinforced presentation ofa stimulus that previously had acquired control over behavior, extinction might be viewed more generally duplication of the conditions of training with the trial-by-trial rein- as forcement contingencies reversed. Thus, our seeking analogies to Pavlovian conditioning in order to better understand occasion setting not only might tell us something about occasion setting but also about the real essence of the phenomenon in question (e.g., extinction) in a variety of situations, thereby freeing us from conceptualizations that are specific to a single situation, such simple Pavlovian excitatory as conditioning. This approach immediately gives greater generality to the phenomenon of extinction. For example, extinction, operationally defined simply nonreinforced exposure to a stimulus, is not applica- as ble to a stimulus following Pavlovian inhibition training (i.e., A+US/AB), because this treatment does not yield a loss of inhibitory control by stimulus B. However, extinction, operationally defined as a reversal of the reinforcement conditions of Pavlovian inhibition training (i.e., A/AB+US treatment following A+US/AB training), does degrade B si nhibitory value. Thus, in this framework, the conclusion that inhibition is not subject to extinction is incorrect. 7 MILLER A N D OBERLING Temporal Coding in Pavlovian Conditioning (As Demonstrated in Sensory Preconditioning, Barnet, Cole, & Miller, 1997) Group Phase L Phase 2 Test Result EXP s3+s1/s4-+s2 s1+us/us+s2 s3 CR s4 CR CONTROL s1/s2/s3/s4 s1-+us/us+s2 s3 - s4 - 1s t-ORDER S3+SlIS4-+S2 s1+us/us+s2 s1 CR s2 cr NOTE: S1, S2, S3, and S4 were initially neutral stimuli. + denotes “followed by”; different trial types that were given interspersed are separated by I;t he approximate magnitude of the observed conditioned responding (CR) is indicated by the case, size, and boldness of the font used in the result column. played the well-known superiority of forward (Sl) over backward (S2) CSs with regard to subsequent behavioral control. But when second-order cues were paired with the first-order forward and backward CSs (i.e., group EXP), stronger excitatory responding to the second-order CS (S4), which had been paired with the backward first-order CS (S2), was seen than to the second-order CS (S3), which had been paired with the forward first-order CS (Sl). This observation is surprising in light of the prevailing view that responding to the second-order CS is mediated by the first-order CS. In that framework, more vigorous responding to a first-order CS (in this case the forward paired CS) should produce more vigorous responding to the second-order CS that had been paired with it. Barnet et al. explained the observed data in terms of an integration of temporal maps from each of the two phases of training, so that, in the integrated maps, onset of S4 had better temporal contiguity with onset of the US than did onset of S3. This and many other studies (e.g., Matzel, Held, h Miller, 1988) strongly indicate that CS-US temporal relationships are an inexorable part of the content of Pavlovian associations. Now consider a recent demonstration of temporal encoding in occa- sion setting reported in a study by Holland, Hamlin, and Parsons (1997). 8 ANALOGIES: OCCASION SETTING AND CONDITIONING ~ ~~~ ~~ Temporal Coding in Occasion Setting (Holland, Hamlin, & Parsons, 1997) Train Test Result os.L~cs.L+us/cs.L/os.s+cs.s+us/cs.s OS.L+CS.L CR CS.L cr os.s+cs.s CR cs.s cr 0S.L-cS.L CR CS.L cr os.s+cs.s CR cs.s cr OS.L--+CS,S CR cs.s cr OS.S+CS.L CR CS.L cr OS.L+CS.S CR cs.s cr OS.S----+CS.L CR CS.L cr NOTE: 0S.S was the occasion setter (0s) trained with a short (S) occasion setter onset to CS onset interval; 0S.L was the occasion setter (0s)t rained with a long (L) occasion setter onset to CS onset interval, + denotes “followed by” with a short interval; +d enotes “followed by” with a long interval; CS.S was the CS trained with a short (S) occasion setter to CS interval; CS.L was the CS trained with a long (L) occasion setter to CS interval; different trial types that were given interspersed are separated by /; the approximate magnitude of the observed conditioned responding (CR) is indicated by the case, size, and boldness of the font used in the result column. In this study, occasion setters were trained with short or long intervals between their onset and that of their accompanying target CSs. Each fea- ture was then tested for transfer to both its own training target CS and the target CS used in training the other occasion setter. Moreover, at test, intervals between termination of the feature and onset of the test CS included both the short and long intervals of training for all combinations of features and target CSs (see Exhibit Holland and his colleagues 4). 9 MILLER AND OBERLING Overshadowing Pavlovian Excitation in (Kaufman & Bolles, 1981; Matzel, Schachtman, & Miller, 1985) Group Train Extinguish A Test Result ov BA-tUS Context only A cr CONTROL A-tUS Context only A CR RECOV BA-tlJS B A CR ~~ ~ NOTE: B and A were initially neutral stimuli, with B being more salient than A. The approximate magnitude of the observed conditioned responding (CR) is indicated by the case, size, and bold- ness of the font used in the result column. Only select groups are represented here. found that occasion setters retained their modulatory potential best on these transfer tests when they were conducted with an interval between their onset and the onset of the test target CS that was the same as the interval that prevailed during training of the feature. However, the modu- latory potential of the occasion setter was unaffected by the occasion setter-target CS interval that prevailed during training of the target CS. Thus, the information encoded about a stimulus that allows it to serve as an occasion setter appears to include the interval from the onset of the occasion setter to the onset of a target CS-that is, feature-target inter- vals-are encoded as part of the content of occasion setting. This property of occasion setters is analogous to the previously discussed propensity of conventional CS representations to be encoded to include information concerning the interval between CS and US onset. OVERSHADOWING Let us consider overshadowing. When two stimuli are presented in com- pound and reinforced (e.g., BA-US), each cue (particularly the less salient cue) accrues less potential to elicit conditioned responding than does the same cue when reinforced alone (i-e., A+US). Exhibit 5 illustrates over- shadowing of CS A by CS B in group OV relative to group CONTROL. Notably, Kaufman and Bolles (1981) and Matzel, Schachtman, and Miller (1985) found that recovery of responding to overshadowed cue (A) can 10 ANALOGIES: OCCASION SETTING AND CONDITIONING be obtained through massive posttraining extinction of the overshadowing cue (B, see group RECOV in Exhibit 5). This recovery from overshadowing without further training strongly suggests that overshadowing is a result at least in part of a failure to express an A-US association that was acquired during training, rather than a failure to acquire the A-US association as assumed by most models of learning (e.g., Rescorla & Wagner, 1972). We recently conducted a series of studies that examined overshadow- ing of potential occasion setters (Gunther, Cole, & Miller, 1997). In this and all subsequent studies from our laboratory that we will describe, the basic procedure was conditioned suppression of licking by water-deprived rats. Lights, tones, and clicks served as the controlling stimuli, and a brief, mild foot shock served as the unconditioned stimulus. Duration of condi- tioned suppression of ongoing drinking behavior was our basic dependent variable. In this first series of studies, the occasion setters were local context cues that cycled 5 min on, then 5 min off. More specifically, the overshad- owing occasion setter was a bright flashing light, and the overshadowed occasion setter was a soft click train. The three positive occasion-setting studies of this series are represented in Exhibit 6. Experiment 1 merely documented in our preparation occasion setting by stimulus X (as opposed to the possibility that X came to simply serve as another excitatory Pavlov- ian CS that associatively summated with A). This was done by taking advantage of the knowledge that occasion setters do not transfer their modulation to unambiguous CSs (e.g., Holland, 1985; Rescorla, 1985) such as CS B, which was reinforced during training both in the presence and absence of X. As can be seen in Exhibit 6, X modulated responding to A but not to B. Notably, responding to B was not at ceiling, so if X had had Pavlovian excitatory potential, it would have been expected to aug- ment responding to CS B as well as CS A. The modulation of conditioned responding of CS A, but not CS B, indicates that our preparation estab- lished X as an occasion setter when no competing occasion setter was present. In Experiment 2 (see Exhibit 6), group CONTROL was trained with stimulus X as an occasion setter for target CS A using exactly the same parameters as in Experiment 1. For group OV, however, X was com- 11 MILLER AND OBERLING Overshadowing in Occasion Setting (Gunther, Cole, & Miller, 1997) ~~ ~~ Group Train Extinction Test Result Exp. 1 X -A~jU S/A/XjB+US/B+ US X+A CR A cr X+B CR B CR Exp. 2 OV YX+ AAJSIA X+A CR A cr CONTROL X-+A-+US/A Y+A CR X+A CR A cr Exp. 3 OV XY+A+US/A Context X+A CR A CT EXTINCTION XY+A+US/A Y-tA X+A CR A cr NOTE. X, Y, A, and B were initially neutral stimuli. X+A+US denotes X onsetting before the A+US trial with X staying on for some time after the completion of the trial (i.e., X was a local context that served as an occasion setter); different trial types that were given interspersed are separated by /; the approximate magnitude of the observed conditioned responding (CR) is indicated by the case, size, and boldness of the font used in the result columns. pounded (simultaneous onset, simultaneous termination) with the more salient stimulus Y. As indicated in Exhibit 6, responding to A in the pres- ence of X was greater for group CONTROL than for group OV. This demonstrates overshadowing of X as an occasion setter by Y. In Exp. 3, group OV received the same treatment as group OV in Exp. 2 and yielded similar weak occasion setting. However, group EXTINCTION had the occasion-setting value of Y degraded following overshadowing treatment. Enhanced modulatory control of responding to A by X was observed as a result of this degradation of Y. In fact, X exhibited modulatory control in group EXTINCTION comparable to that observed in group CONTROL of Exp. 2. Thus, we see that, analogous to Pavlovian conditioned excitation, both overshadowing and recovery from overshadowing as a result of de- 12