Schedules of Reinforcement

Chapter One
INTRODUCTION
W h e n a n o r g a n i s m acts upon the environm ent in which it lives, it changes th a t en
vironm ent in ways w hich often affect the organism itself. Some of these changes are
w hat the laym an calls rew ards, or w hat are now generally referred to technically as
reinforcers: w hen they follow behavior in this w ay, they increase the likelihood th at
the organism will behave in the same way again. M ost events which function as re
inforcers are related to biological processes im portant to the survival of the organism.
Thus, food is reinforcing to a h ungry organism . T he capacity to be reinforced by
food substances has presum ably been acquired as p a rt of the evolutionary develop
m ent of the species. Because of the strengthening of behavior which follows, the b e
havior of the organism is shaped u p so th a t it is m axim ally effective in any partic
ular environm ent. T he shaping process includes the differentiation of new forms of
response, including the subtle refinem ents of form called skill. It also includes the
developm ent of a p p ro p riate stim ulus control, so th a t a given response is generally
em itted only upon an appropriate occasion. T he traditional study of reinforcement
in the field of learning has been alm ost exclusively concerned w ith the acquisition and
retention of behavior in this sense.
A nother im portant function of a reinforcem ent is to maintain an active repertoire of
behavior. O nce a response has been acquired and brought under appropriate stim
ulus control, the question rem ains w hether it will continue to be em itted u n d er a p
propriate stim ulating conditions. In classical studies of learning this question is usu
ally assigned to the field of m otivation. T h e subject of an experim ent acquires a
motor skill or learns a list of nonsense syllables. T he processes through which this is
achieved are considered a p a rt from the question of w hether the subject will a t any
given m om ent actually oblige the experim enter by displaying his skill or testing his
memory. But the m aintenance of behavior in strength after it has been acquired is
an equally im p o rtan t function of reinforcem ent. It is only indirectly related to the
acquisition of the form of a response or of the control exerted by the stim ulating e n
vironment.
T he effect of reinforcem ent in m aintaining behavior in the repertoire of an organ
ism has been neglected p a rtly because the contingencies of reinforcem ent actually
studied have usually been of an all-or-nothing nature. An act has been reinforced,
l
SCHEDULES OF REINFORCEMENT
or it has not. For example, in the traditional study of learning, right responses
are always rew arded a n d w rong responses are always allowed to go unrew arded.
But the most casual observation of the norm al environm ent of an organism will show
th a t these conditions are not typical. Behavior of a given form seldom has precisely
th e same effect upon the environm ent in two instances, an d the kind of effect called
a reinforcem ent is seldom inevitable. M ost reinforcem ents, in other words, are in
term ittent.
In 1933 one of the present authors (Skinner, 1933) reported experiments in which
reinforcem ents were in te rm itte n t, a n d late r (S kinner, 1938) pointed out th a t they
m ay be scheduled in m any ways. A lthough only two types of schedules were reported
in detail at th a t time, they clearly showed th at subtle differences in scheduling m ight
generate dram atic differences in behavior. T he present book is an exhaustive exten
sion of this earlier work. It is now clear th a t far from being a m ere falling-short of the
ideal of inevitable reinforcem ent, interm ittent reinforcem ent constitutes an im portant
condition of action. M any significant features of behavior can be explained only by
reference to the properties of schedules. By the m anipulation of schedules, a wide
range of changes in behavior can be produced, most of which would previously have
been attributed to m otivational or em otional variables.
A schedule of reinforcem ent m ay be defined w ithout reference to its effect upon
behavior. T hus, a response m ay be reinforced on the basis of the tim e w hich has
elapsed since the preceding reinforcem ent, or on the basis of the num ber of responses
which have been em itted since the preceding reinforcem ent. A given schedule m ay
be fixed, or it m ay vary either at random or according to some plan. These two possi
bilities yield four basic schedules: fixed-interval, variable-interval, fixed-ratio, and
variable-ratio. B ut other possibilities exist, as well as m any com binations of such
schedules. T he first step in our analysis of the field is therefore a purely form al state
m ent of these possibilities, such as th a t given a t the beginning of C hapter Two.
A second step is a description of the performances generated by such schedules. For
each schedule in our logical classification, we present the typical perform ance under
standard conditions of a representative organism. Such d a ta m ay be collected w ith
out respect to any theoretical form ulation of the results.
A m ore general analysis is also possible which answers the question of why a given
schedule generates a given perform ance. It is in one sense a theoretical analysis; but
it is not theoretical in the sense of speculating about corresponding events in some other
universe of discourse. It simply reduces a large num ber of performances generated by
a large num ber of schedules to a form ulation in term s of certain comm on features. It
does this by a closer analysis of the actual contingencies of reinforcement prevailing u n
der any given schedule.
A schedule of reinforcem ent is represented by a certain arrangem ent of timers,
counters, and relay circuits. T he only contact betw een this system and the organism
occurs at the m om ent of reinforcement. W e can specify the stimuli then present in
IN TR O D U C TIO N
purely physical terms. These m ust include a description of the recent behavior of the
organism itself. T he extent to which features of the present or im m ediately past envi
ronm ent actually enter into the control of behavior is an experim ental question. U n
der a given schedule of reinforcement, it can be shown th at at the m om ent of reinforce
m ent a given set of stimuli will usually prevail. A schedule is simply a convenient w ay
of arranging this. Reinforcement occurs in the presence of such stimuli, and the future
behavior of the organism is in p a rt controlled by them or by sim ilar stimuli according
to a well-established principle of operant discrimination.
Some features of the current and recent behavior of the organism generated by
schedules a t the m om ent of reinforcem ent are com m on to m ore th an one schedule.
Therefore, we can simplify the em pirical description of the results. T he behavior of
the organism under any schedule is expressed as a function of the conditions prevailing
u n d er the schedule, including the behavior of the organism itself. Some schedules
lead to steady states, in which repeated reinforcem ents m erely em phasize the control
being exerted by cu rren t conditions. U n d er other schedules, reinforcem ent u n d er
one set of conditions generates a change in perform ance leading to a new condition at
the time of reinforcement. T he result m ay be a progressive change or an oscillation.
W e deal w ith conditions at the m om ent of reinforcem ent in two ways: (1) inferen
tially, by com p aring the effects of different schedules a n d p articularly of schedules
designed prim arily to m ake such inferences most plausible; and (2) by direct m an ip
ulation or determ ination. T he prim ary purpose of the present book is to present a
series of experim ents designed to evaluate the extent to which the organism s own b e
havior enters into the determ ination of its subsequent behavior. From a form ulation
of such results we should be able to predict the effect of any schedule.
Such a theoretical analysis is only one result, however, and possibly the least im por
tant. T he experim ental analysis of schedules now perm its the experim enter to achieve
a degree of control over the organism w hich is of an entirely new order. H igh levels of
activity m ay be generated for long periods of time. Interm ediate and low levels of ac
tivity m ay also be generated. Changes in level which have hitherto seemed capricious
m ay be m ore readily understood. T h ro u g h an application of scheduling, extrem ely
com plicated exam ples of behavior can be set up, an d behavior can be brought u n d er
subtle and complex stimulus control. As a result, complex processes can be studied in
the lower organism at a new level of rigor.
T he technological use of schedules of reinforcem ent is rapidly expanding. In re
search in psychophysics, problem solving, and motor skills, lower organisms m ay now
be used as conveniently as h u m an subjects a n d w ith m any advantages arising from
the greater possibility of m any types of control. Performances generated by particular
schedules have proved to be useful in the study of m otivation (e.g., in the analysis of ingestive an d sexual behavior), of em otion (e.g., in the study of anxiety), of punish
ment, of escape and avoidance behavior, and of the effects of drugs. Techniques in
volving schedules have been ad ap ted to a wide range of species. Surprisingly sim ilar
perform ances, p articu larly u n d er com plex schedules, have been dem onstrated in o r
ganisms as diverse as the pigeon, mouse, rat, dog, cat, and monkey. A t the hum an
level the analysis of schedules has proved useful in the study of psychotic behavior an d
in the design of educational techniques for norm al h u m an subjects in the classroom.
O th e r applications to the problem of the control of hu m an behavior, as in law and
penology, religion, industry, and commerce, offer considerable promise (Skinner, 1953).
C hapter Two
PLAN OF ANALYSIS
SCHEDULES OF REINFORCEMENT STUDIED
T w o NONiNTERM iTTENT s c h e d u l e s of reinforcem ent are:

Continuous reinforcement (erf), in which every response em itted is reinforced; and
Extinction (ext), in which no responses are reinforced.
Schedules of interm ittent reinforcem ent analyzed here include the following:
A. Fixed-ratio (FR ), in w hich a response is reinforced upon com pletion of a fixed
n um ber of responses counted from the preceding reinforcem ent. (The word r a
tio refers to the ratio of responses to reinforcem ents.) A given ratio is indicated
by th e ad d itio n of a n u m b er to the letters FR . T hus, in F R 100 the o n e -h u n
dredth response after the preceding reinforcem ent is reinforced.
B. Variable-ratio (V R ), sim ilar to fixed-ratio except th at reinforcements are sched
uled according to a random series of ratios having a given m ean an d lying betw een
arb itrary values. T he m ean m ay be noted by a num ber, as in V R 100.
C. Fixed-interval (FI), in w hich the first response occurring after a given interval
of tim e m easured from the preceding reinforcem ent is reinforced. A given inter
val is indicated by the addition of a num ber to the letters FI. (Unless otherwise
noted, the num ber always indicates minutes.) T hus, in FI 5 the first response which
occurs 5 m inutes or more after the preceding reinforcem ent is reinforced.
D. Variable-interval (V I), sim ilar to fixed-interval except th at reinforcements are
scheduled according to a random series of intervals having a given m ean and lying
betw een a rb itra ry values. T h e average in terv al of reinforcem ent (in m inutes) is
indicated by the addition of a num ber to the letters V I, e.g. V I 3.
T he three m ain schedules involving both num bers of responses a n d intervals of time
are:
E. Alternative (alt),1 in which reinforcem ent is program m ed by either a ratio or
an interval schedule, whichever is satisfied first. T hus, in alt F I 5 F R 300 the first
response is reinforced: (1) after a period of 5 m inutes provided 300 responses have
not been m ade; or (2) upon com pletion of 300 responses provided 5 m inutes has
not elapsed.
1 T h is schedule was first used by W . H M orse a n d R . J . H errnstein.
5
F. Conjunctive (conj),1 in which reinforcem ent occurs w hen both a ratio and an
interval schedule have been satisfied. For exam ple, a response is reinforced when
at least 5 m inutes has elapsed since the preceding reinforcem ent and after at least
300 responses.
G. Interlocking (interlock),2 in which the organism is reinforced upon completion
of a num ber of responses; but this n um b er changes during the interval which fol
lows the previous reinforcem ent. For exam ple, the n u m b er m ay be set at 300 im
m ediately after reinforcem ent, b u t it is reduced linearly, reaching 1 after 10
m inutes. If the organism responds very rapidly, it will have to em it nearly 300
responses for reinforcem ent. If it responds at an interm ediate rate, it will be rein
forced after a sm aller n um ber say, 150. If it does not respond at all during 10
m inutes, the first response thereafter will be reinforced. M any different cases are
possible, depending upon the way in which the num ber changes with time.
O n two im portant schedules a single reinforcem ent is received after two conditions
have been satisfied in tandem order:
H. Tandem (tand), in which a single reinforcem ent is program m ed by two sched
ules, the second of which begins when the first has been completed, with no corre
lated change in stimuli. In tan d F I 10 F R 5, for exam ple, a reinforcement occurs
upon com pletion of 5 responses, counted only after a response after expiration of the
10-m inute fixed interval. In ta n d F R 300 FI 5 a response is reinforced after the
lapse of 5 seconds, tim ed from the com pletion of 300 responses. It is often im p o r
ta n t to specify which of the two schedules is the m ore substantial p art of the
schedule. This m ay be done by italicizing the im portant member. For example,
tan d F R F l describes the case tan d F R 200 F I 5 sec while tan d F R F I describes the
case F R 1 F I 10. In the first the ratio of 200 is the m ore im p o rta n t p a rt of the
schedule. In the second the 10-m inute fixed interval is the m ajor part, except th at
the tim ing begins only after the execution of a single response. (The F R 1 is by no
m eans trivial; for exam ple, it has a m arked effect in opposing the developm ent of a
pause after reinforcement.)
I. Chained (chain), sim ilar to tan d em schedules except th a t a conspicuous change
in stimuli occurs upon completion of the first com ponent of the schedule. T he sec
ond stim ulus eventually controls the perform ance ap p ro p riate to the second sched
ule, and, as a conditioned reinforcer, reinforces a response to the first stim ulus.
This schedule is usually studied when both contributing schedules are substantial.
T he values in any schedule m ay be systematically changed as the experim ent pro
gresses in term s of the performances generated. W e have, then, another schedule:
J . Adjusting (adj), in which the value of the interval or ratio is changed in some
system atic way after reinforcem ent as a function of the im m ediately preceding p er
form ance. (In an interlocking schedule the change in value of the ratio occurs
1 Ibid.
2 Some early u n published experim ents on this schedule w ere done by N o rm an G uttm an.
PLAN OF ANALYSIS
between reinforcements.) For example, a fixed ratio is increased or decreased by a

small am ount after each reinforcem ent, depending upon w hether the tim e from the
preceding reinforcem ent to the first response is less th an or greater th a n an a rb i
trary value.
A complex program m ay be composed of two or m ore of these schedules arranged

in any given order. D epending upon the presence or absence of correlated stimuli,
we m ay distinguish:
K. Multiple (m ult), in which reinforcem ent is program m ed by two or more
schedules altern atin g usually a t random . Each schedule is accom panied by a dif
ferent stimulus, which is present as long as the schedule is in force. For example,
in m ult F I 5 F R 100 the key is sometimes red (when reinforcem ent occurs after an
interval of 5 m inutes) an d sometimes green (when reinforcem ent occurs after 100
responses). T h e key colors a n d their corresponding schedules m ay occur either at
random or according to a given program in any determ ined proportions.
L. M ixed (mix), sim ilar to m ultiple except th a t no stimuli are correlated with the
schedules. For exam ple, m ix FI 5 F R 5 0 represents a schedule in w hich a rein
forcement sometimes occurs after an interval of 5 m inutes and sometimes after the
completion of 50 responses. These possibilities occur either at random or according
to a given program in any determ ined proportion.
A small block of reinforcem ents on one schedule m ay be introduced into a back
ground of another schedule. Such cases are referred to as:
M. Interpolated (interpol). For example, a block of 10 reinforcements on a fixed
ratio of 50 is inserted into a 6-hour period of reinforcem ent on F I 10 w ithout change
of stimulus.
THE BASIC DATUM
M ost of the following experiments are concerned with pigeons and with the behav
ior of pecking a key on the wall of an experim ental cham ber. In some of the experi
m ents rats were studied, a n d the response th en consisted of depressing a small
horizontal b ar projecting from the wall of the experim ental cham ber. Such responses
are not wholly arbitrary. They are chosen because they can be easily executed, and
because they can be rep eated quickly a n d over long periods of tim e w ithout fatigue.
In such a bird as the pigeon, pecking has a certain genetic unity; it is a c h aracter
istic bit of behavior w hich appears w ith w ell-defined topography. Its features m ay
nevertheless be m odified by differential reinforcem ent; a n d in m ore detailed analyses,
such particu lar features as speed and direction m ust sometimes be specified.
O u r basic d a tu m is the rate at which such a response is em itted by a freely m oving
organism. This is recorded in a cum ulative curve showing the num ber of responses
plotted against tim e. T h e curve perm its im m ediate inspection of rate and change in
rate. Such a datu m is closely associated w ith the notion of probability of action.
A m ong th e in d ep en d en t variables w hich m odify this ra te or probability are some

which are not prim arily at issue in a study of interm ittent reinforcement. Levels of
deprivation, for example, although occasionally varied in a systematic fashion, are gen
erally held constant. T he principal variables to be considered are those arising from
schedules of reinforcem ent and from the m om entary stim ulus conditions which they
generate.
PROCESSES ASSUMED
Reinforcement and extinction
In m ost of the following experim ents th e organism is m ain tain ed a t less th a n its
norm al body-weight, an d food is used as reinforcem ent. T he process of operant con
ditioning describes the fact th a t any behavior im m ediately followed by the presenta
tio n of food tends to occur m ore frequently thereafter. A p articu la r contingency
betw een th e response of pecking th e key a n d the p resen tatio n of food is a rra n g e d by
certain reinforcing circuits; b u t any contingency, even w hen not specified by the a p
paratu s, is assum ed to be effective. In p a rticu la r, any behavior w hich im m ediately
precedes a specified response is subject to operant conditioning. C oncurrent behav
ior, such as th e assum ption of a p a rticu la r posture a t the tim e of pecking the key, is
also likely to be reinforced.
W henever behavior which has been reinforced in this way is not followed by a rein
forcement, it tends to occur w ith a reduced frequency thereafter. This change is the
process of extinction.
Control exerted by stimuli present a t the time o f reinforcem ent
T he effect of reinforcem ent is m axim ally felt w hen precisely the same conditions
prevail. Thus, if a response is reinforced in the presence of stimulus A, any increase
in frequency will be m axim al in the presence of stim ulus A; b u t a sm aller increase in
frequency m ay nevertheless occur in the presence of stim ulus B, which differs from A
in some particular. A com parable process is assum ed for extinction. Thus, a given
frequency of responding m ight prevail u n d e r stim uli A, B, an d C because of earlier
conditioning. If the response now goes unreinforced in the presence of B, the d e
crease in frequency is m axim al in the presence of B, but less in A and C.1
In the process of stimulus discrimination a response is reinforced in the presence of one
stim ulus a n d extinguished in th e presence of a second stim ulus. T his intensifies
the degree of stim ulus control, the frequency being m axim al in the presence of the one
stim ulus a n d possibly zero in the other. W h e th e r the two procedures of (1) re in
forcem ent in one stim ulus a n d (2) extinction in another are always necessary in estab
lishing a differential stimulus control is not easy to decide. T he experim ental
c h am b er is usually only p a rt of the daily environm ent of the organism ; an d unless
1 E xcellent dem onstrations of these gradients of stim ulus control are afforded by recent experim ents by N o r
m an G u ttm an (1956).
PLAN OF ANALYSIS
th e experim ent is ru n continuously, it m ay be argued th a t some instances of the se

lected response are likely to go unreinforced outside the cham ber. T he effect of novel
stimuli m ust also be considered in an evaluation of stimulus control.
A discriminative stimulus functioning as a reinforcer
A stim ulus w hich has acq u ired th e stim ulus control described in the preceding
section m ay also be a reinforcer; th a t is, it m ay be used to condition further behavior.
Such a stim ulus is an exam ple of a conditioned reinforcer. T he reinforcing stim uli
which are m ade contingent upon behavior in the present experim ent are all condi
tioned. T h e stim uli accom panying the presentation of food are m ade conditioned
reinforcers th rough an explicit process described in C h ap ter T hree. U nconditioned
stim uli alm ost in v ariab ly follow. T h e conditioned reinforcers, such as a light or
buzzer, are useful because they can be m ade contingent on a p a rtic u la r form of a
response.
A discrim inative stim ulus as a reinforcer assumes special im portance in the chain
ing o f behavior w hen a response produces a stim ulus which is the discrim inative stimulus
for an o th er response. T his condition is explicitly arra n g e d in some of the experi
ments which follow, and is indirectly arranged by m any schedules of reinforcement, as
will be shown in later chapters.
Differentiation o f the form o f response
W hen separate instances of a given response can be shown to differ with respect to
some property, reinforcem ent m ay be contingent upon one value of the property alone.
T hus, the response of pecking the key m ay be reinforced w hen the beak passes from
rig h t to left across the key, b u t not w hen it passes from left to right. If the original
responses to th e key include instances of b o th types, differentiation is achieved by
reinforcing responses of one form a n d extinguishing responses of the other form. T h e
stereotyped posture a d o p ted by the organism in executing a response often results
from accidental differential reinforcem ent of this sort.
D ifferentiation of form is particularly im p o rtan t in extremes of frequency. W ith
a very high rate of responding the extent of the excursion of the head is greatly reduced.
Special postures m ay arise, an d m ovem ents of the beak alone m ay be em phasized.
U nder such circum stances the original response of pecking the key can no longer be
identified. T h e speed w ith w hich a response is executed m ay also be differentially
reinforced. T h e actu al speed of m ovem ent of the head m ay be differentially rein
forced if it can be recorded in a form perm itting the a p p a ra tu s to distinguish betw een
two speeds. Speed is indirectly differentiated when the contingencies are expressed
in term s of tim es elapsing betw een successive responses, or betw een a given stim ulus
an d the first following response. T he differentiation of low rates of responding usu
ally produces the chaining of m ediating behavior, such as pacing in the experim ental
cham ber, which does not directly affect the reinforcing circuits. Both cases illustrate
10
th e present point: an ap p aratu s which differentially reinforces rate of responding m ay

function through contingencies betw een the form of response and the reinforcement.
C O NDITIO NS PREVAILING UNDER A SCHEDULE OF REINFORCEMENT
W hen a more-or-less stable perform ance has been well-established under a given
schedule, the organism is being reinforced under certain stimulus conditions. T he ex
perim enter arranges some of these, such as the details of the experim ental cham ber
an d special stimuli added for explicit purposes. But am ong the physical events occur
ring in the experim ental cham ber are the activities of the organism itself. These e n
ter into the contingencies an d m ust be specified as p a rt of the a n im a ls environm ent.
(N ote: W e are not interested here in identifying the specific end-organs involved.
J u s t as we specify the color of the light on th e key as a stim ulus w ithout investigating
the mode of operation upon the organism, so we define certain typical features of the
o rganism s own behavior w ithout determ in in g how the organism receives them .
T h e procedure for finding w hether the organism is sensitive to given kinds of stimuli is
the same in both cases: we determ ine a sensitivity to color by dem onstrating a differ
ential reaction to colored stimuli, and we dem onstrate a sensitivity to some aspect of the
organism s behavior by dem onstrating a differential reaction to th a t behavior.)
In certain cases the topography of behavior can serve as a discrim inative stimulus
controlling other behavior. In general, however, the im portant properties of such
stim uli are precisely those of our dependent variable, nam ely, the rate. A given rate
of responding m ay be both a dependent variable (a description of the b ird s behavior
a t the m om ent of reinforcem ent) and an independent variable (a stimulus upon which
reinforcem ent is contingent). This distinction m ay be the source of considerable con
fusion. W e deal w ith a response both as an activity of the organism and as part of
a series of events affecting the organism as a stimulus.
It can be shown th a t the organism reacts not only to the im m ediate rate a t which
it is responding at the m om ent of reinforcem ent, but also to recent changes in th at rate.
T h e ways in w hich reinforcem ents are scheduled lead us to em phasize certain fea
tures of this com plex variable: we find it convenient to point to the number o f responses
w hich th e organism has em itted since the preceding reinforcem ent or some other
m arking point. This n u m b er will not be a useful d a tu m if rate of responding varies
widely; b u t w here the rate is reasonably constant, it m ay be a useful aspect of the
stim ulus situation. T he time which elapses from the preceding reinforcem ent, or from
some other m arking point, m ay also be a useful aspect, although again it will be sig
nificant only if rate of responding is held constant or is changed according to some
specified pattern.
The effect o f a schedule in m aintaining a set o f conditions a t reinforcem ent
W hen an organism is first reinforced on a given schedule, the actu al conditions

prevailing a t reinforcem ent depend upon th e behavior w hich the organism brings to
th e experim ent. W hen a different schedule has previously been in effect, certain
PLAN OF ANALYSIS
11
fairly predictable relations m ay prevail betw een the rate of responding an d reinforce
m ent. Sometimes, however, the current perform ance is quickly disrupted, and u n i
form contingencies do not arise for some tim e. W hen a schedule is based upon n u m
ber of responses, the behavior m ay be lost ; th a t is, the preceding schedules have
not provided for a sufficient num ber of responses to produce the reinforcements
needed for further m aintenance of the behavior. T his occurs, for exam ple, w hen an
organism is placed on a fairly large fixed ratio after a history of only continuous re
inforcem ent.
As a schedule rem ains in effect, how ever, the behavior which it generates begins to
establish fairly consistent contingencies at the m om ent of reinforcem ent. Early re
inforcem ents lead to a perform ance which, in com bination with the reinforcing circuit,
causes reinforcem ents to occur in a specified relation to the behavior of the o rg an
ism. T he three characteristic states are:
Steady state. U nder some conditions the perform ance generated by a given sched
ule produces just those conditions a t the m om ent of reinforcement required to m aintain
th a t perform ance. T h e perform ance u n d er the schedule m ay show no appreciable
change for hundreds of hours. Such a schedule is a small fixed-ratio.
Oscillation. W hen a schedule generates a tem porary steady state, the uniform ity
of the behavior m ay produce contingencies of reinforcem ent not specified in the sched
ule. These disrupt the perform ance; a n d d u rin g the period of disruption, the
schedule again begins to reconstruct the steady state. U nder reinforcement on a small
fixed interval, for exam ple, there is a tem porary stage w hen the rate is fairly constant
an d w hen the num ber of responses at reinforcem ent, counted from the preceding re
inforcem ent, also approaches constancy. This condition is not specified by the
scheduling apparatus, and it. has the effect of introducing features appropriate to ratio
schedules. These features m ay disrupt uniform perform ance under the fixed inter
val and thus destroy the constancy of the num ber a t reinforcement. T he perform ance
later returns to th a t appropriate to the fixed interval alone, w hereupon another cycle
begins.
Slow drift. Probably no perform ance obtained u n d er a schedule rem ains u n
changed indefinitely. A fixed interval of m oderate size m ay produce a perform ance
w hich is fairly stable for h u ndreds of hours; yet, slight changes m ay occur (for ex
am ple, in the pause before the first response after reinforcem ent) w hich are traceable
to the very large num ber of instances perm itting a m ore sensitive tem poral discrim
ination. A variable-interval schedule w hich produces a constant rate of responding
for m any hours m ay eventually be learned by the organism if the arrangem ent of
intervals is not entirely ran d o m , b u t this effect is very subtle an d m ay not be felt for
hundreds of hours. These slight changes need not disrupt the perform ance a p p ro
priate to the schedule, a n d a retu rn to an earlier state m ay never occur.
12
ANALYTICAL PROCEDURES
In order to achieve a generalized account of the effects of schedules, the stim uli
w hich e n te r into reinforcing contingencies m ust be identified a n d evaluated. This
can be done in several ways.
Inference from perform ance
Assum ptions about conditions at the tim e of reinforcem ent m ay be checked by an

analysis of performances appropriate to various schedules.
Direct control o f contingencies
1.
Differential rate reinforcement. A schedule w hich tends to a rra n g e a p a rtic u la r
contingency a t reinforcem ent m ay be m ad e to do so w ith ad d itio n al controlling c ir
cuits. For example, a device which arranges a schedule under which reinforcements
usually occur at a given rate of responding m ay be extended w ith an additional de
vice w hich w ithholds reinforcem ent u ntil precisely th a t rate is achieved. This m ay
often be done w ithout appreciable changes in other characteristics of the schedule.
3. Supplementary correlated stimuli. T he im portance of the organism s own behavior
as a stim ulus m ay be dem onstrated by the introduction of additional stim uli entering
into the sam e contingencies. For exam ple, if the effect of a schedule seems to de
pen d upon the differential stim ulus control exerted by the behavior of pausing for dif
ferent lengths of tim e, a clearly discrim inable external stim ulus m ay be introduced
w hich varies w ith the length of pause. S upplem entary stim uli have also been added
w hich vary w ith the following: (1) the n u m b er of responses since the preceding
reinforcem ent ( added counter ); (2) the tim e since the preceding reinforcem ent
( added clock); or (3) the m om entary rate of responding ( added speedom eter ). If
the added stimuli have a large effect upon the perform ance, the stimuli supplied by the
behavior itself probably have not been particularly im portant or have different c h a r
acteristics from the added stimulus.
3.
Time out. A period of tim e o u t is injected into an ex p erim ent by the a r
ran g e m en t of a n y condition u n d e r w hich th e b ird does not respond. (See C h a p te r
T h ree, Technical Procedures.) T h e assum ption is th a t stim uli arising from the b ird s
ow n behavior ju st before such a tim e out will no longer be effective. For exam ple, if
th e behavior of the organism d u rin g a fixed interval is suspected of being affected
by the behavior in the preceding interval, a tim e out m ay be used to elim inate these
stimuli.
NATURE OF THE FO LLO W ING EXPERIMENTS
A reasonably satisfactory form ulation of schedules of reinforcem ent em erged slowly

as the experim ents to be described here were conducted. N ot all the experim ents are
crucial tests of the form ulation, however; m any belong to an earlier stage in the research
w here we were m ainly interested in collecting representative perform ances under rep
PLAN OF ANALYSIS
13
resentative schedules. Since these early experim ents yielded inform ation w hich is of
value in itself, a fairly full report is given. Points leading to an d supporting the pres
ent form ulation are emphasized.
T he experiments also show different levels of technical development. T he a rt of
program m ing an d recording the perform ances generated by schedules has greatly im
proved during this research. Since im portant characteristics of the perform ance were
often impossible to predict, instrum entation was frequently inadequate. Nevertheless,
we have reported those results w hich have appeared to be im portant for other reasons.
For exam ple, a schedule now recognized to be most effective when a period of tim e out
follows each reinforcem ent m ay have been studied before the im portance of the tim e
out was discovered. A lthough the perform ance on the schedule in the absence of tim e
out is less significant for our present analysis, it is nevertheless w orth recording and, of
course, ultim ately dem ands explanation.
Chapter Three
TECHNICAL PROCEDURES
APPARATUS
The experim ental box
A t y p i c a l e x p e r i m e n t a l b o x is m ade from a picnic icebox approxim ately 30 by 12

by 15 inches. A rem ovable panel divides the box into two com partm ents. T he b ird s
com p artm en t (Fig. 1) is 12 by 12 by 15 inches. A sm all blower, m ounted on an o u t
side wall, exhausts air from the box th rough a filter, w hich collects the feather dust
which w ould otherwise blow about the room. T he pigeon stands on a heavy No. 5
wire m esh floor, u n d er which pap er towels are spread. T he mesh gives the bird a
clean, dry surface on which it can stand a n d walk m ore easily th an on a sm ooth floor.
Fresh w ater is supplied in the cup a t the rear of the cage. A hood above the cup is
sometimes used to prevent pollution of the w ater by feces. T he com partm ent is illum i
n ated by a light near the ceiling. T h e m agazine, the key, the lights, and w hatever
14
15
electrical equipm ent and wiring are most conveniently placed in the box rather th an
with the rest of the program m ing equipm ent are m ounted on the opposite side of the
panel (Fig. 2). T he key and the m agazine are described below. A cable (not shown)
enters the box through the wall and ends a t a connector on the panel. Connections to
the blower and the light in the bird s com partm ent lead to a second connector. W hen
these leads are disconnected, the panel w ith all the working parts of the apparatus can
be removed for repairing or for cleaning the box.
Sound insulation. Such a box attenuates outside noises roughly from 20 to 30 db. A
m asking w hite noise is provided in the box from an earphone (not shown) driven by a
white-noise generator. Because of this insulation an d m asking, norm al noises occur
ring in the laboratory do not produce effects upon behavior observable against existing
baselines. C are is taken, however, to keep critical relays in the program m ing circuits
as quiet as possible. Such relays are lightw eight an d shock-m ounted. U nder these
conditions, no discrim inations based on relay operation have been observed even after
m any hundreds of hours on a given schedule. T he recorder can be heard inside the
16
box, but this sound always accompanies the norm al noise of the key and probably helps
to m aintain a stable form of the pecking response. Indeed, it is sometimes ad v a n ta
geous to emphasize the auditory feedback from a response by the addition of other stim
uli generated by the operation of the key.
Four to eight such apparatuses are operated concurrently in a single experim ental
room. Each apparatus has its separate program m ing equipm ent, so th at any auditory
stim uli from the activity of one bird has no relevance to the schedule in another a p p a
ratus.
Key. Figure 3 shows the key used in most of these experim ents. Its position in the
a p p a ra tu s is indicated in Fig. 2. T h e a rm a tu re consists of a piece of frosted Plexiglas
2 by 3 by 1/16 inches. A n axle passes through two Plexiglas blocks cem ented to oppo
site edges. A silver contact is riveted on the Plexiglas above the axle and is in contact
w ith a strip of phosphor bronze w hich runs halfw ay dow n the center of the arm ature,
w here it is riveted. A n electrical connection to the m oving contact is m ade from this
strip th rough the spring w hich supplies the a rm a tu re tension. T h e arm atu re is
m ounted in a fram e consisting of a ^-inch piece of alum inum 2 \ by 3 inches. Two p il
low blocks on the side of the fram e provide bearings for the a rm a tu re axle. T he bird
pecks through a 1-inch hole in this fram e. A piece of brass fastened to one of the p il
low blocks overhangs the upper p a rt of the key. A thum bscrew passes through a hole
tap p e d in the brass directly above the contact on the arm ature, an d a small silver con
tact soldered to the end of the screw provides the fixed contact for the key. Pecking
th e Plexiglas arm ature opens this switch. Tension on the arm ature is adjusted by tu rn
ing a second screw th ro u g h the overhang w hich presses a phosphor bronze spring.
17
A sponge-rubber block on the bottom of the key lim its the excursion. Six m illion re
sponses per m onth over periods of from 1 to 2 years have been recorded from such a
key w ith only a rare failure.
T h e height of the apertu re w hich exposes the surface of the key to the pigeon is im
portant. T he rate of responding depends in p a rt u pon the energy required; if the key
is too high or too low, the pigeon m ust stretch its neck excessively or assume aw kw ard
postures, so th a t a lowered rate results. T he average hom ing pigeon operates most
conveniently a key centered a b o u t
inches from the floor; larger pigeons, such as
W hite C arneaux, work best on a key approxim ately 8 | inches from the floor.
T he Plexiglas arm ature has a low mass, so th at high n a tu ra l frequencies are possi
ble. It also has a certain elasticity, w hich perm its m any thousands of responses to be
m ade each day w ithout injury to the beak or undue fatigue. A n atu ral frequency suf
ficient to follow all rates of responding can be achieved w ith such a key by the p lace
m ent of a lim it on the excursion of the a rm a tu re a n d by the proper adjustm ent of the
tension. T h e advantage of a break contact is felt here. A very light touch on the key,
moving the a rm a tu re through only a very short distance, is sufficient to open th e con
tact an d thus to operate the controlling circuit. T h e tension is a compromise betw een
too stiff a key a n d too slow an operation: if the spring is too tense, some responses m ay
not break the circuit; on the other h a n d , if the spring is too light, the contact m ay not
close fast enough after each response a n d the frequency of operation will be reduced.
T he key is designed to give the pigeon a m echanical advantage. In the lever system
composed of the a x le , contact, a n d the exposed area of the arm ature, the contact m ay
be held closed w ith a substantial force yet opened by a fairly light touch.
T he response which the pigeon m akes in pecking such a key has the following desir
able properties:
1. It uses an effector system (the beak, the head, a n d the neck of the bird) which oc
cupies an im portant place in the norm al activity of the organism.
2. T h e effector system has a relatively small mass a n d is capable of high rates of ac
tivity.
3. Successive responses to the key resem ble each other closely. T he topography of
examples of this class of responses can be fairly narrow ly specified.
4. Responses can be reliably recorded a n d reinforced through their action upon the
key.
T he p a rt of the Plexiglas arm atu re exposed through the aperture is illum inated from
behind by 6-w att Christm as tree lam ps, each w ith a 300-ohm resistor in series in or
der to prolong its life. W hen an experim ent does not require discrim inative stim uli,
the key is usually illum inated w ith a w hite light. In establishing discrim inations, sev
eral key colors m ay be program m ed by m ounting different lam ps behind the key a n d
changing the illum ination th ro u g h the controlling circuits. In critical experim ents,
two lam ps of each color m ay be provided in case one burns out. In nearly all the ex
perim ents to be reported the stim uli are presented in this way. T he fact th a t the bird
pecks directly a t the surface carrying the stim ulus color is im portant; a weaker control
18
follows w hen changes in color occur, for exam ple, in the general illum ination of the ex
perim ental box.
No effort has been m ade to specify such stim uli precisely. In any given experim ent
th e only im p o rta n t consideration is th a t stim uli are grossly different. W hen we say,
for example, th a t the bird is reinforced when the key is red and not reinforced when the
key is blue, it should be understood th a t the two colors also undoubtedly differ in satu
ration, intensity, an d possibly the distribution of light on the key.
In some instances (e.g., during a tim e out), all lights in the box are turned off. T he
bird is not in total darkness, however, since some light enters through the edges of the
lid and the blower and cable holes.
Food magazine. Tw o types of food m agazine have com m only been used. O ne type
gives free access to food for a fixed period of tim e, while the o ther delivers a fixed
am ount of food. T he latter has several disadvantages. Food m ay be left in the m ag
azine a n d m ay be found d u ring a late r p a rt of the experim ental session w hen a re in
forcem ent is not scheduled. Such reinforcem ents break dow n the stim ulus control of
th e m agazine-approach behavior, a n d lead to responses ( looking for food ) which
com pete w ith the behavior generated by the independent variables in the experim ent.
W ith such a type of m agazine, m oreover, the am ount of tim e the anim al spends in e a t
ing varies from reinforcem ent to reinforcem ent. Recorded pauses following reinforce
m ent are therefore difficult to interpret.
These objections are avoided by using a m agazine which presents a supply of food for
a fixed period of tim e. All rem aining food, as well as the container in which the food
is presented, is w ithdraw n a t the end of the period. A lthough some cleaning or groom
ing behavior m ay follow, the total tim e occupied by eating usually assumes a constant
value. This type of m agazine also helps to bring responses to the m agazine u n d er
com plete stim ulus control. A pproach to the m agazine in the absence of the norm al
stim ulus (spontaneous looking for food ) quickly disappears during m agazine tra in
ing. T h e am ount of food actually eaten from reinforcem ent to reinforcement varies
only slightly after m agazine train in g has taken place. A m agazine which presents
food for a fixed tim e is m ore successful for a pigeon th an for such an organism as a rat,
since the pigeon has no way of picking u p m ore food th a n it can im m ediately eat.
W ith the rat, pow dered or liquid food generally m ust be used, to avoid small-scale
hoarding operations.
T h e food m agazine used in most of the following experim ents is shown in Fig. 2.
G rain is stored in the hopper (to the right of the draw ing), from w hich it feeds into the
shallow pivoted tray. A solenoid draw s the tray into horizontal position, and the bird
reaches the grain through a f-inch-square aperture in the hood a t the front of the m ag
azine. Tw o w hite, 6-w att lam ps are w ired in parallel w ith the solenoid, so th a t the
grain is well-lighted during the operation of the m agazine. T h e second lam p is pres
ent in case one burns out d u ring an experim ent. T h e pivoted tray is occasionally
cleaned out to remove inedible debris which slowly accum ulates in it. W hen the m ag-
19
azine is not operating, the grain is not only out of reach hut unlighted and scarcely vis
ible to the bird.
T he actual operation of such a m agazine generates the all-im portant conditioned re
inforcing stim uli which supply a precise contingency betw een behavior an d reinforce
m ent. In this case the stim uli are both auditory an d visual. T he operation of the
tray and the solenoid can easily be heard. T he m agazine lights illum inate the grain
an d usually throw a white light on the key. Such stimuli are usually am plified by a r
ranging th a t all key lights and the house light go off sim ultaneously w ith the operation
of the m agazine.
In all experim ents reported here, reinforcem ent consists of free access to grain for
from 3.5 to 4 seconds. U nder good conditions of accessibility of the grain and uniform
size of individual grains, the am ount consumed per reinforcement is approxim ately 0.25
gram . A daily ration of 60 reinforcem ents therefore equals 15 grams. This exposure
to grain, together w ith the resulting am ount eaten, was determ ined em pirically, as an
am ount which sustained perform ance u n d e r the types of schedules considered here.
In order to reinforce frequently during a daily session, however, the am ount is kept
close to the m inim um w hich will produce this result. U nfortunately, it is not always
easy to detect the fact th a t a reinforcem ent is too small. T h e resulting low rates and
irregularity do not provide an im m ediate indicator. In work w ith a new organism
and a new food, the am ount of reinforcem ent th a t should be used is probably impos
sible to determ ine in advance. A rule of thum b for a prelim inary trial value, however,
is 0.5 gram per 1000 gram s body-weight.
Autom atic program m ing
W ork on schedules of reinforcem ent is all b u t impossible w ithout autom atic pro
gram m ing of the contingencies of reinforcement. A large num ber of events need to be
arranged during long experim ental sessions. M ore th an a million responses per m onth
are often recorded from a single subject, w ith daily experim ental sessions ranging up
to 15 hours. In such cases m an u al program m ing is unthinkable. M oreover, the re
action tim e and variability of the h u m an operator introduce errors in program m ing
w hich defeat th e purpose of such research. T h e req u ired level of precision can be
achieved only through the use of m echanical or electrical devices.
W ith au tom atic program m ing a single experim enter can ru n a large n u m b er of
experim ents concurrently an d alm ost continuously. Long experim ental sessions m ay
extend through the night w ithout the presence of laboratory personnel. Relatively
unskilled personnel can start and stop p articular experiments, while the experim enters
tim e is free for the design of new equipm ent a n d experim ents and the treatm ent and
evaluation of results.
Typical circuit. In the present experim ent, program m ing of contingencies is c a r
ried o u t largely by various types of relays a n d electric a n d electronic tim in g a n d
counting devices. Figure 4 shows a circuit containing m any comm only occurring ele
20
ments. This circuit is designed to program an d record a fixed-interval schedule of

reinforcem ent. C ircuit elem ents w hich are draw n to the right of the vertical broken
line are custom arily wired into the pigeons com partm ent. These control the m ag a
zine tim ing cycle, th e lights, a n d th e key, w hich usually do not change from experi
m ent to experim ent. T he circuit elem ents to the left are wired outside and are con
nected to th e b ird s c o m p a rtm e n t by a cable. T his p a rt of the circuit is changed
frequently a n d m ust be flexible.
In Fig. 4, S 2 is th e norm ally closed sw itch on the key a t w hich the b ird pecks.
Fig. 4.
D iagram o f a typical circuit
Every tim e the bird pecks, S 2 opens a n d R elay 1 releases. T h e first pole on R elay 1
operates the recorder each tim e the bird pecks.
T h e m agazine cycle is program m ed by Relays 4 an d 5. A pulse to the coil of R elay
4 locks up R elay 4 th rough its own coil, its first p a ir of contacts, a n d the norm ally
closed contact of R elay 5. W hile R elay 4 is locked up, the norm ally open contact of
th e second pole supplies pow er to: L 1 a n d L 2, the lights above the m agazine w hich
illum inate the grain; M 3, the tim ing m otor by which the duration of the m agazine
cycle is determ ined; an d the solenoid w hich draw s up the tray so th a t the bird can
rea c h the grain. T h e end of the m agazine cycle is determ ined by a cam on the shaft
of th e m otor, M 3, w hich operates S 1, a microswitch. T he microswitch operates R e
lay 5. Since R elay 4 is locked up by the fact th a t it draw s its pow er th ro u g h the
21
norm ally closed contact on the second pole of R elay 5, the operation of R elay 5 causes
R elay 4 to release. T he first pole of R elay 5, however, is connected to the light, m o
tor, a n d m agazine solenoid, w hich continue to operate until the m otor drives the cam
bevond the point w here m icrosw itch S 1 releases. A t this point a single pulse long
enough to close R elay 4 can produce a second m agazine cycle. T h e norm ally open
contact on the second pole of R elay 5 is connected to the pen m arker on the recorder,
w hich m akes a diagonal m ark on the record w henever reinforcem ent occurs. T he
norm ally closed contact on the second pole of R elay 4 is connected to the paper feed on
the recorder so th a t the p ap er does not feed d u rin g the operation of the m agazine.
Box light L 3 a n d key light L 4 are pow ered continuously through switches S 3 and S 4.
R 1, R 2, a n d R 3 are 300-ohm , 5-w att resistors, w hich keep the voltage across the
lam p filam ent low in order to prolong the life of the lam p. T 2 is a tim ing device con
sisting of a m o to r w hich actu ates a sw itch once every 10 m inutes. T h e switch on
T 2 operates R elay 3. T h e second pole on R elay 3 operates R elay 2, w hich locks
u p th ro u g h its first pole an d th ro u g h the norm ally closed contact of R elay 5. W hen
R elay 2 is locked up, the norm ally open contact of the second pole is grounded and
supplies a ground connection to the second pole of R elay 1, the key relay. T h e other
contact on the second pole of R elay 1 is connected to the m agazine; thus, whenever
R elay 2 is locked up an d R elay 1 (the key) operates, a pulse reaches R elay 4 which ini
tiates the m agazine cycle. T he m otor on T 2 is pow ered by a norm ally closed con
tact of the second pole of R elay 2 a n d by the norm ally open contact of the first pole of
R elay 3. T h e conditions of operation of the m otor are: (1) R elay 2, the reinforce
m en t lock-up, is not o p e ra ted ; a n d (2) R elay 3, th e relay o p e ra ted by the switch on
T 2, is operated. T h u s, the m otor on T 2 is so w ired th a t the tim er can never stop
w ith th e switch closed; b u t once a reinforcem ent has been set u p (R elay 2 locked up),
the m otor will ru n only long enough for the switch on T 2 to open again. This cir
cuit program s a fixed-interval schedule in w hich the interval of reinforcem ent is d e
fined from the reinforcem ent. If one reinforcem ent is delayed because of a low rate of
responding, the following interval is not shortened.
T 1 is a 12-hour program m er of the type used in kitchen appliances. It starts the
experim ent a n d stops it after a preset period of tim e.
Circuit elements. W herever possible, d-c relays are used.
These are usually of
higher quality, have m ore dependable action, a n d are sm aller th a n a-c relays. E ach
relay has a capacitor w ith a resistor in series connected betw een the ground and the
coil for spark suppression. S park suppression m aterially increases the life of the con
tacts a n d is especially im p o rta n t w here relays m ust operate m any millions of times.
(A rough rule for designing spark suppression for coil is to begin w ith a small capacitor
[0.01 m icrofarad, 400 volts] a n d to v ary th e size of the resistor by a p otentiom eter
w hile o perating the coil in a slightly d arkened room. T h a t value of resistance is
chosen w hich gives the least am o u n t of spark w hen the coil is
operated or released.
T h e size o f th e c ap acito r is th en increased to a larg er value a n d the process repeated.
T h e most efficient spark suppression occurs w hen further increases in the size of the
22
capacitor do not reduce the am ount of sparking when the relay is operated or released.)
T he keying relay operates m ore often th an any other elem ent in the circuit except
the switch on the key, an d is therefore m ore subject to wear. A good-quality, fast-act
ing d-c relay is used. Because the relay acts directly through the contacts of the key,
it is chosen for low current drain, and spark suppression is carefully arranged to prevent
pitting of the key contacts. T he relay handles rates as high as 15 responses per sec
ond. (A slow -acting or slow-releasing relay can produce an erroneous record of the
b ird s actu al behavior, especially in schedules involving differential reinforcem ent at
high rates.)
T he keying relay m ust deliver a pulse sufficiently long to operate the recording and
controlling circuits. In most of the present experim ents a special type of relay was
used w hich g u aran teed a pulse of 0.040 second provided the key contact was broken
for 0.005 second or m ore. T he relay has a tw o-sectioned coil on a com m on core,
a n d is o p erated by one coil a n d sustained by th e induced c u rren t in the other. T he
0.040-second pulse guarantees the operation of the recorder, and particularly of the
m agazine lock-up relay, an d of any counting circuits currently in effect. Since the re
leasing tim e of the relay is usually shorter th an the operating time, a norm ally closed
key contact is generally advantageous. T he keying relay also should control the cir
cuit on release.
It is particularly im portant th a t the sound of the operation of the relay which sets up
a reinforcem ent should not affect the organism. T he insulation and m asking noises
already described ordinarily insulate the bird from the controlling circuits. However,
th e additional precaution is taken of using a lightw eight, high-im pedance, sensitive
relay which is shock-m ounted an d makes very little noise relative to the clicks of stan d
a rd relays. (This is the relay which is closed by the tim ing circuit so th a t the next
response will be reinforced. Poor sound insulation or a noisy lock-up relay would p er
m it the bird to form a discrim ination and to respond only after a reinforcem ent had
been set up.)
Complex circuits. T he circuit shown in Fig. 4 is basic to nearly all simple schedules
of reinforcem ent. C hanges from one schedule to a n o th e r are m ade a t position T 2.
F or exam ple, w hen the schedule is a rra n g e d by a clock, T 2 is a tim er m easuring
fixed or variable intervals. B ut w hen the schedule of reinforcem ent is based upon the
num ber of responses, T 2 is a counter, counting a fixed or variable num ber of responses.
A lthough a counter a t T 2 could operate the m agazine upon com pletion of a preset
num ber of responses, this practice could introduce a slight delay between response a n d
reinforcem ent. It is better to set the counter for one less th an the num ber of responses
in the ratio a n d to let it lock u p R elay 2, in w hich case the next response is reinforced.
C om plex schedules are arra n g e d by stepping switches or selector switches which
change scheduling devices a n d relevant stim uli. Figure 5 is a block circuit-diagram
of a p rogram of altern atin g fixed-interval an d fixed-ratio schedules u n d e r stim ulus
control (m ult F IF R ).
Flexible circuit-building equipment. In research of this sort the controlling circuits are
23
F IX E D IN TE R V A L
M AGAZINE
K EY
S T IM U L U S S T IM U L U S
OF
R E IN FO R C E M E N T
PAPER
FEED
S TEP
RECORDER
P EN
MARKER
SCH ED U LE S E L E C TO R
(S TE P P IN G S W IT C H )
FR
F I
D
R E IN F O R C E M E N T
L O C K -U P
Fig. 5.
Block d ia g ra m o f a circuit fo r mult FIFR
frequently changed, an d changes m ust often be m ade quickly. T h e relays and tim ing
an d counting devices are therefore a ttach ed to panels a n d prew ired to a p a tte rn of
snap leads. T he panels can be clipped into place on brass rods w hich carry a lte rn a t
ing or direct current. U nits are wired together with snap leads in a functional
wiring diagram . Changes in a circuit m ay be m ade very quickly, an d a given a p p a
ratus redistributed for further use after it has been discontinued. A circuit such as
th a t shown in Fig. 4, for exam ple, can be assem bled a n d ready for use in less th a n 10
minutes.
TREATMENT OF DATA
Recording
Cumulative curves. A graph showing the num ber of responses on the ordinate
against tim e on the abscissa has proved to be the most convenient representation of the
behavior observed in this research. F o rtu n a te ly , such a cum ulative record m ay
be m ade directly a t the tim e of the experim ent. T he record is raw data, but it also
perm its a direct inspection of rate and changes in rate not possible w hen the behavior
is observed directly. Figure 6 shows how a cum ulative record is taken. Each tim e
th e b ird responds, th e pen moves one step across th e p ap er. A t th e sam e tim e, the
p a p e r feeds continuously. If th e b ird does not respond a t all, a horizontal line is
d raw n in the direction of th e p a p e r feed. T h e faster the b ird pecks, the steeper the
line. A lthough th e ra te of responding is directly pro p o rtio n al to th e slope of the
curve, a t slopes above 80 degrees small differences in angle represent very large differ
ences in rate; a n d although these can be m easured accurately, they cannot be eval
24
SCHEDULES OF REINFORCEM ENT
EACH RESPONSE MOVES
u ated easily by inspection. T he speed of the paper m ovem ent and the scale of the pen
m ovem ent are chosen w ith respect to rates to be observed. These depend in tu rn
upon the organism used, the response used, an d the schedule of reinforcement.
By proportionately increasing both the rate of the paper feed an d the distance the
pen travels per response, such a graph can be m agnified or reduced w ithout changing
th e slope. T he values to be used are determ ined by a compromise between the p re
cision of a large-scale record a n d the com pactness a n d convenience of a sm all-scale
record. Occasionally, two recorders are used: one to provide m easurem ents and easy
inspection of details, a n d the other to provide a com pact sum m ary of the whole ses
sion.
In the standard recorder the pen returns to its starting point after it crosses the paper.
Since the p a p e r itself is essentially endless, very long experim ental sessions can be
continuously recorded.
In m ultiple schedules two or m ore recorders m ay be used, only one of which operates
at any given time. T hus, the behaviors appropriate to several conditions m ay be au to
m atically separated an d cum ulated for study.
Cumulative recorders. Several types of cum ulative recorders were used in the research
described here. Im provem ents m ade d u rin g the research have been incorporated
in a sta n d a rd cum ulative recorder m an u factu red by R a lp h G erbrands, 96 R o nald
R oad, Arlington, M assachusetts. Figure 7 shows one version of a cum ulative recorder.
T he pen is stepped across the paper by the actuating m echanism of a telephone-type
stepping switch capable of 250,000,000 operations. T he stepping switch is linked to
th e pen-driving m echanism w ith a w orm an d gear. T he m axim um speed of opera-
Fig. 7.
25
Photograph o f a cum ulative recorder
tion is achieved by o perating th e coil for th e first in sta n t of operation a t a voltage

higher th an th at for which it is rated. After the arm atu re of the stepping switch begins
to draw in, an in te rru p ter spring opens; an d from th en on the coil draw s its power
th ro u g h a voltage-dropping resistor. T h e pen is m o u n te d on a carriage w hich is
free to slide across the paper. It is m oved by a pin soldered to a continuous chain
driven by the stepping switch. W hen the carriage reaches the top of the paper, a cam
disengages it from the pin, an d a small weight returns the carriage to the bottom of
the paper, w here it is engaged by the next pin on the chain. In resetting, the pen
draws a vertical line w hich is useful in the m easurem ent of slopes or w hen the figures
are cut for reproduction. T he pen m ay be m oved w ith respect to the pen carriage
by a sm all relay coil. W hen the relay is operated briefly, the pen draw s a small p ip ,
which m ay be used to indicate the occurrence of a reinforcem ent or any other event.
In one type of recorder used, this pip m oved horizontally to the left of the record. In
a later m odel, it m oved to the right an d dow nw ard. (See Fig. 8.) T h e pip draw n
Fig. 8.
Reinforcement marks
26
by this m odel is preferable, since it c an n o t coincide w ith any portion of the recorded
behavior.
A nother type of recorder used in some of these experim ents can be reset to zero from
the controlling circuit (and not, as in the above case, only w hen the pen reaches the
to p of the paper). If the pen is reset at each reinforcem ent, the records representing
th e perform ances between successive reinforcem ents are collected in a form conven
ient for inspection and m easurem ent.1 Both the stepping m echanism and the paper
feed m ay be a d a p te d for use in tim ing certain schedules. Unless the operation of
th e recorder is quite reliable, however, there is a certain danger in this practice. For
exam ple, in using the stepping m echanism to schedule a ratio, the possibility th a t
th e stepper is not reporting the behavior accurately m ay not be detected in the re
corded perform ance.
In all experim ents the paper feed of the recorder is stopped during the operation of
th e m agazine. A ny pause after reinforcem ent is thus n ot confused w ith the tim e
when the bird is eating. T he recorder is also stopped during periods of time out. T he
stepping m echanism rem ains connected, a n d any responding occurring d u ring this
tim e is recorded as a vertical line th a t is easily distinguishable from other features.
W henever the recorder is stopped for a substantial period, additional ink from the pen
produces a darkened spot which is useful in reading the records.
Processing o f data
T he records for a given bird during an experim ent are pieced

together a n d accordion-folded for storage. T he length of fold represents 1 hour of
behavior. T h e reports of the experim ents reproduced here are representative portions
of such records. Because im p o rtan t features are usually not related to a zero-point
in either tim e or num ber, standard coordinates would not be very useful. In ste a d , we
indicate the scale of each record by attach in g a sm all set of coordinates containing
some representative slopes. Since full coordinates need not be indicated, the records
m ay be com pressed by the rem oval of unused parts of the paper. Figure 9A shows
a cum ulative curve as recorded. In Fig. 9B coordinates have been added a n d the
record telescoped by cutting out diagonal strips. In cutting an d pasting the cor
rect angle of each portion of the record w ith respect to the vertical is carefully m ain
tained. If a vertical line is draw n w hen the recorder resets, parts are left a t each end
to indicate continuity.
W here the actual rate of responding a t a given point is im portant, careful m easure
m ents are m ad e w ith a special type of p ro tracto r an d are reported in the text. A
rough evaluation of the absolute rate m ay be m ade from the coordinates supplied. In
general, we are m ore concerned w ith changes in rate th an w ith absolute values.
Description o f rates. In the description of records of this sort, the w ord r a te can
have several interpretations. O ccasionally, responses occur w ith a relatively con
sta n t inter-response tim e, so th a t uniform segments such as those in Fig. 10A are proConstruction o f graphs.
1 T h e latest m odel supplied by G erb ran d s has this resetting feature.
Fig. 9.
27
P reparation o f a telescoped figure
duced. A lthough the actual record is stepwise, a single constant slope m ay be

identified; and when records are greatly reduced, as in most of those reproduced in this
book, the record has the character of a straight line. Inter-response times m ay vary,
however, even though we m ay still distinguish a m ean rate as in Fig. 10B. In m any
experim ents we m ust distinguish betw een a local rate, sustained for short periods of
tim e, and the over-all m ean rate. Thus, Fig. 1OD indicates short bursts of responding
a t a constant local rate a lte rn a tin g w ith periods of no responding. Both of these
combine to establish a single, interm ediate over-all rate.
These distinctions m ay also be m ade w hen the rate changes in a reasonably u n i
form fashion. Thus, Fig. IOC is a sm ooth curve resulting from the continuous length
ening of inter-response tim es; to the rig h t are two cases of roughly the sam e curve
produced by m uch less uniform changes in inter-response times. Sometimes, consid
erable local changes m ay occur, although the record tends to approxim ate a constant
over-all ra te as a t D. A t E a fairly uniform local ra te is m ain ta in e d despite
tw o local deviations, one caused by a te m p o rary increase in ra te followed by a com
pensatory decrease, an d the other by a decrease in rate followed by a com pensatory
increase. T he la tte r deviation is m ore com m on, b u t both occur. At F a sm ooth
curve is evident in spite of a m arked local deviation. W hen longer portions of an
28
experim ent are considered, it m ay be necessary to distinguish between three rates.

Thus, under certain conditions of fixed-ratio reinforcem ent, the ratio m ay be run off
a t (1) a constant high rate. But successive ratios separated by pauses produce (2)
a w ell-m arked over-all ra te of in term ed iate value. Segm ents a t this over-all rate
m ay alte rn a te w ith prolonged pauses, to yield (3) a total rate for the experim ent of a
still lower over-all value.
Recording special features. A dditional recording instrum ents are used w hen some
Fig. 10.
Cum ulative curves showing v a ri

eties o f rate change
special aspect of the d ep en d en t variable is of p a rtic u la r interest. R ecording all be

havior in detail at all times would be uneconom ical. T he recording selected in any
given experim ent is determ ined by exploratory work. For exam ple, in a study of the
fixed-interval schedule of rein fo rcem en t, the c u rv atu re shown by the perform ance
d u rin g an interval m ay a p p e a r to need careful analysis. M easurem ents taken from
th e cum ulative records m ay not be precise enough or convenient. A polygraph can
th en be added to the recording equipm ent to provide m ore accurate measures, or a set
of counters can be arranged to count responses during specified fractions of the interval.
29
W ith the latte r procedure, m easuring single cases or com puting the m ean result for
an experim ent w ould be unnecessary. Again, if a study of a fixed-ratio schedule sug
gests th a t the tim e which elapses before the first response is of special im portance, a
polygraph or other type of recorder can be arranged to provide a record of just this p a rt
of the b ird s performance.
EXPERIMENTAL PROCEDURES
Care o f pigeons
In most of these experim ents, we used W hite C arn eau x males, 1 to 2 years old a t
the start of the experim ent, from the stock of the Palm etto Pigeon Plant, Sum ter, South
Carolina. These pigeons have the advantage of being uniform in body structure an d
in their m etabolic reactions to schedules of feeding. A few hom ing pigeons of mis
cellaneous origin were also used. W e have not discovered any species differences,
except size, relevant to the variables we have explored.
As soon as a pigeon is received in the laboratory, an identifying band is put on the
right leg. T h e first flight feathers are cut off along the line form ed by the tips of the
second row of feathers, a n d the long tail feathers are clipped along the line form ed
by the second set of tail feathers. T h e toe nails are inspected; and if they are unduly
long, they are clipped w ith sm all w ire-cutters. T h e birds are housed in individual
cages, about 1 cubic foot in size. Fresh w ater and a standard health grit are available
at all times in cups at the front of the cage. T he floor is covered with a heavy grade of
w rapping paper, which is changed weekly.
T h e food is a special m ixture of 40% vetch, 50% K affir corn, and 10% hem p seed.
T he grains of this m ixture are all approxim ately the sam e size. If a stan d ard pigeon
m ixture containing whole corn as well as very small kernels is used, the pigeons will
hunt for large grains or preferred grains, and unpreferred grains quickly accum ulate in
the m agazine. To lessen changes in deprivation or reinforcing effect, the same grain
m ixture is fed both in the living cages an d in the food m agazine. T he grain is kept in
uncovered m etal containers.
Control o f body-w eight
Percent body-w eight is a useful m ethod of controlling deprivation. Tw o pigeons

of the sam e percent body-w eight m ay not show the sam e rate of responding on com
parable schedules, but they will differ less th an when deprivation level is m aintained
in term s of a schedule of feeding or the am ount fed per day.
T he a d lib body-w eight is first o b tain e d by giving the bird continuous access to
food for 2 or 3 days, or until the weight shows no appreciable increase. (This practice
is reliable only if m ature birds are used.) All food is then rem oved for 2 days; a n d
beginning w ith the th ird day, 5 gram s is given daily u ntil the weight of the bird falls to
80% of its a d lib weight. F u rth er changes m ay be advisable. If the bird continues
to show em otional behavior in th e exp erim en tal box a n d does n ot eat read ily w hen
30
m agazine training is begun, the weight m ay be reduced below 80% tem porarily.
R ates of responding later observed in the experim ents are used as the basis for a final
adjustm ent of the percent body-weight used. W henever possible, the adjustm ent is
upw ard, since the birds rem ain healthier a t higher weights. T he body-weight can
often be raised to 85% or even 90%, and the birds m ay still show a satisfactory rein
forcing effect of food a n d a high stability of the behavior so reinforced. Sometimes, a
retu rn to practically 100% of body-weight is possible after the bird has worked in the
experim ental a p p a ra tu s for some tim e. T h e actu al percentage should not be taken
too seriously, since some progressive change in body-w eight m ay occur w ith aging.
Birds are sometimes m atch ed in deprivation level as m easured by the rate of re
sponding u n d e r a variable-interval schedule. E ach b ird is ru n for a session of fixed
length each day. A criterion rate of responding is selected to which each bird is to
be adjusted. This is expressed as a n u m b er of responses per session. T h e weight to
w hich each bird is fed after the session is adjusted upw ards or dow nw ards in term s
of th e difference betw een the n u m b er of responses em itted on th a t day a n d the cri
terion. A scale of adjustm ent is chosen as the num ber-of-gram s added or subtracted
from th e w eight for each 1000 responses shown in th a t difference. W hen the two
birds are to be m atched a t a value which is otherwise unim portant, the average b e
tw een the two rates shown by the birds before adjustm ent m ay be selected as a cri
terion.
A protocol card is p rep ared for each bird, showing its leg-band num ber, descrip
tion, a n d source of supply. D aily entries of body-w eight are m ade on the card, and a
ru n n in g curve of the body-weights m ay be plotted. In the final experim ental pro
cedure, the bird is weighed im m ediately before an experim ent and again im m ediately
after the session. S upplem entary food is th en given if necessary to bring the b ird s
w eight to a chosen value. A b ird whose experim ental weight is set a t 425 gram s, for
exam ple, m ay be found to weigh 412 gram s before a given session. D uring the ex
p erim en t, it ingests some food a n d is found to weigh 422 gram s w hen taken from the
box. T hree grams is therefore given to bring its weight up to the level of 425 grams.
O n the following day, the bird is again weighed before the session, and the same proce
dure is followed, all weights being recorded. Such m easurem ents are subject to fluc
tu atio n due to the drinking of w ater, the eating of g r it, and defecation. W hen a daily
routine has been well-established, however, the times a t which the pigeon drinks, eats
grit, a n d defecates become fairly stable, provided w ater an d grit are always available.
If a bird is heavier th an its designated weight, it is not used or fed th a t day.
Conditioning the pecking response
Adapting the pigeon to the apparatus. T he pigeon m ay be ad ap ted to the experim ental
box concurrently with adjustm ent of the body-weight. T he bird is placed in the a p
p aratus for several hours w ith its daily ration. A daptation of em otional responses a p
pears to occur m ore quickly w hen the pigeon eats its daily ration in its new
environm ent.
31
T he process of em otional adaptation continues during m agazine tra in
ing.
T he pigeon is conditioned to respond to the presentation of food
in three stages. In stage 1 the bird has reached a suitable percentage of its ad lib
weight a n d is placed in the box w ith the m agazine in position for eating. It is allowed
to eat for 1 or 2 m inutes from a m agazine held in place so th a t it has continuous a c
cess to grain. If a bird does not eat w ithin 10 m inutes, it is returned to the living cage
w ithout food until the following day. This procedure is used until the bird eats rea d
ily from the open m agazine. In stage 2 the m agazine is operated repeatedly until
the bird begins to eat during the brief presentation of food. T h e first response of the
b ird to the o peration of the m agazine is usually an em otional p a tte rn w hich m ust
adapt out before the bird will eat. T he bird usually comes to stand near the m agazine
because of the accidental contingencies favorable to the reinforcem ent of such beh av
ior, an d a th ird stage is necessary to break u p this behavior. T h e m agazine is opened
(by h a n d ) only w hen the bird has m oved ab o u t the box. T his stage is continued
until the bird will tu rn and move tow ard the food m agazine quickly from any position
in the box. T h e strong conditioned reinforcing stim uli generated by the operation
of the m agazine are im portant here. T h e superstitious conditioning of stereotyped
responses m ust be avoided a t this stage. If the m agazine is repeatedly presented by
a clock device, an irregular schedule should be used; b u t stage 3 is best accom plished
by a n experim enter who can observe the b ird s behavior.
T h e pigeon is ready for final conditioning w hen it tu rn s prom ptly, approaches the
m agazine, a n d eats readily as soon as the m agazine is presented, b u t does not a p
proach the m agazine in the absence of m agazine stimuli.
Differentiating the peck. A pigeon can be conditioned in several ways to peck the
key:
1. If the key is adjusted so th a t a very light contact will operate the m agazine (a
voice key can be a d a p te d for this purpose d u rin g the initial stages of training), a n d if
the key is lighted brightly from behind, a w ell-adapted pigeon will usually peck a t
the key d u ring a 1-hour experim ental session. A sm all spot on the key will increase
the probability of such a response. If the bird has already been m agazine-trained,
conditioning alm ost always occurs a t the first peck. Some birds m ay require several
sessions to produce the response. T h e topography is th a t of a light exploratory peck,
but it shifts u n d er continuous reinforcem ent to a m ore vigorous and stable form.
2. W here the process of acquisition is not im portant, a small grain of corn can be a t
tach ed to th e key w ith Scotch tape. It is lighted from the pigeons side so th a t it
can be seen in its n orm al color. T h e pigeon will quickly peck a t the grain, a n d the
m agazine will open. W hen conditioning to the key has taken place, the grain can
then be rem oved, either a t once, or by a progressive reduction in size until it disappears
altogether. T h e resulting response has the topography of the type of peck w ith which
a grain is picked off a surface.
Magazine training.
32
3. T h e response of pecking the key can be shaped u p by hand. T he experi

m enter holds a switch which operates the food m agazine. Any response m ade by the
bird in the direction of the key is reinforced by the presentation of food. Later, only
responses w hich bring the b ird s h ead close to the key and, still later, only responses in
w hich the head moves tow ard the key are reinforced. E ventually, an exploratory
peck occurs and is, of course, im m ediately reinforced. If the experim enter can be seen
by the bird, he m ust be below the b ird s horizon, since a bird is relatively undisturbed
by objects below it, but adapts only w ith difficulty to a person appearing above. A l
though this procedure is a useful dem onstration of the technique of the shaping of a
response th ro u g h op eran t reinforcem ent, it is likely to produce a poor topography.
T h e experim enter m ust be quite skilled in an ticip atin g a n d reinforcing the correct
m ovement. For example, the bird will often be reinforced w hen it moves its head past
th e key. A pendulum -like oscillation n ear the key often results. This m ovem ent
m ay persist as superstitious behavior for a long tim e, a n d it requires so m uch tim e for
execution th a t high rates are impossible.
4. A fourth m ethod is useful especially w hen tim e in the experim ental box is at a
prem ium . A p u n ch b o ard is constructed of a block of plywood, 8 by 8 by f inches,
in which several f-in ch holes have been drilled to a depth of about inch. A sheet
of alum inum , also 8 by 8 inches, is drilled w ith f-in c h holes in the same places. T he
board is placed flat on a table; the holes are filled with grain; a sheet of tissue paper,
8 by 8 inches, is placed over the board; and the alum inum tem plate is fastened in place.
T he result is several exposed disks of paper, f inch in d iam e te r, behind each of which
is a small pocket of grain. A few of these exposed segments are cut lightly with a knife
blade so th a t some of the grain m ay be seen. A board so prepared is placed in the
living cage of the bird. M ost pigeons quickly learn to eat the grain behind the paper
an d to peck through the uncut disks. Slightly tougher papers are used on successive
days. Since the w hite disks closely resem ble the lighted key on the experim ental box,
w ell-adapted birds will th en peck a t the key alm ost im m ediately w hen confronted
w ith it for th e first tim e in the experim ental box. If they have been well m agazinetrained, conditioning usually takes place instantly. T he topography of the resulting
response resembles the behavior of p uncturing and tearing a disk of paper.
In most of the following experiments, the process of acquisition is unim portant, ex
cept as it bears upon the topography of the resulting behavior. In most of the experi
m ents reported here, the birds were conditioned with the punchboard technique.
T he rest were shaped by hand.
Im m ediately after conditioning, the response is continuously reinforced. At least
three sessions of continuous reinforcem ent, containing 60 reinforcem ents each, are a r
ranged.
33
SPECIAL TECHNIQUES
D ifferential reinforcem ent o f rates
D rl is conveniently a rran g ed w ith a

tim er w hich is reset by each response. If the tim er reaches a given setting before a
response occurs, it sets up a reinforcem ent. T h e device can be added to other sched
uling circuits; for exam ple, the reinforcem ent thus set up is effective only if a v ariable-interval tim er has also set u p a reinforcement.
The differential reinforcement o f high rates (drh ). D rh is m ore difficult. If we rein
force a response w hen the inter-response tim e falls below a small fraction of a second,
the characteristics of the key an d the keying relay becom e crucial. Any chattering of
the contacts will produce an unscheduled reinforcement. Even w ith reliable contacts,
this m ethod tends to reinforce a topography of response which results in vibration of
th e key. It is better to take several responses into account in m easuring the rate.
This appears to be justified by the probability th a t high rates which serve as stimuli in
the contingencies of reinforcem ent generated by schedules are rates sustained for a
num ber of responses. A great m any different possibilities can be investigated, depend
ing upon the n u m b er of responses considered a n d the extent to which m ore rem ote
responses are perm itted to contribute to the result. W e have studied only a few a rb i
trary cases in which electrical and m echanical devices have provided the crucial con
tingencies.
In some cases differential reinforcem ent of high rates is achieved by a condenser
which is charged a given am ount by each response b u t is constantly discharging
through a high resistance. W hen the rate is above a given value, the charge reaches
a point a t which the condenser energizes another circuit, operating the food m agazine.
Figure 11 shows a schem atic design of a sim ilar arrangem ent with a m echanical over
The differential reinforcement o f low rates (drI).
take. An arm (A) driven clockwise by a tim ing m otor carries a contact (B). A sec
ond a rm (G) pivoting ab o u t the same center is driven by a ratch et operated by the
keying relay. A th ird arm (E), rigidly fastened to A, carries a break contact (F). If
the bird is not responding, arm E overtakes arm C, an d the break contact at F dis
connects the tim ing motor. T he whole system is then motionless. As soon as the bird
begins to respond, arm C is driven tow ard A by the ratchet, the contact a t F is closed,
an d the arm s E an d A begin to move. If the rate of responding is sufficiently high
an d sufficiently sustained, arm G will overtake arm A, an d contacts D and B will oper
ate the m agazine. T h e angle betw een E an d A m ay be changed from experim ent
to experim ent so th a t the gap is altered through which arm C moves. This angle p a r
tially determ ines the num ber of responses which m ust be executed a t a given rate in
order to overtake arm A beginning at rest. O th e r a rb itrary features are, of course,
the speed of m ovem ent of A and the distance w hich C moves p er response. Some ac
tu al values and the corresponding perform ances are described in C hapter Ten.
34
A dded stimuli
T he design of an external stimulus which varies with the stimuli autom atically en
tering into the reinforcing contingencies under various schedules raises m any problems,
most of w hich we have not attem p ted to solve here. T h e stim ulus most frequently
used is a sm all spot of light projected on the key which the bird pecks. Tw o strips of
opaque tape are applied to the back of the key, leaving a lighted horizontal slit 1/16
inch wide extending across the key. Tw o vertical shutters behind the key determ ine
th e length of the slit illum inated. These shutters m eet in the m iddle of the key; and
as they separate, the spot of light lengthens. In general, we have used a spot no
sm aller th an 1/16 inch square; the m axim um length is the w idth of the key, or 1 inch.
W hen the shutters are driven w ith cams cut in particular ways, the rate of growth m ay
have any character desired. If the cams are driven by a clock, the stimulus represents
Fig. 11.
A device fo r d iffe re n tia lly re in fo rc

ing high rates
added tim e ; w hen driven by a counter, added n u m b er ; and when changed by the
rate of responding, added speedom eter.
Time out
Tim e o u t (abbreviated as T O ) is any period of tim e during which the organism

is prevented from em itting the behavior under observation. A T O can be arranged in
several ways.
1. T he anim al is rem oved from the apparatus. This is difficult to accomplish a u
tom atically and m ay generate unnecessary em otional behavior.
2. T he key or other m anipulandum is removed. This m ust be done quickly a n d
in such a w ay th a t th e organism is not in ju red if it is responding a t the tim e. Some
clearly discrim inable stim ulus correlated w ith the presence or absence of the key is
advisable. O therw ise, the organism will continue to a p p ro ach the place w here the
35
m an ipulandum norm ally appears; or if this extinguishes, it will fail to approach when
the m anipulandum is in place. But if a conspicuous stimulus is to be arranged, pro
cedure 4 is preferable.
3. For most birds a n d some m am m als, all light in the a p p a ra tu s m ay simply be
tu rned off. A pigeon m ay be conditioned to respond in total darkness, b u t it norm ally
does not do so. Such a blackout is the sim plest m ethod to use w ith the pigeon,
since it is easy to arrange electrically and is effective immediately.
4. A stimulus previously correlated w ith no reinforcem ent is introduced. This
m ethod requires some prep aratio n but has m any advantages, particularly in working
w ith anim als such as rats, which continue to respond in the dark. For example, a dis
crim ination is established between silence, in which responses are reinforced, and the
sound of a buzzer, in w hich no response is ever reinforced. W hen discrim ination
training is well-advanced, the organism im m ediately stops responding when the buzzer
sounds. It has been found experim entally, w ith organisms which do not respond in
the dark, th a t such a stim ulus has the same effect as turning out the lights.
A tim e out m ay have other effects th an those for which it is prim arily used:
A. U n d e r some circum stances it functions as an aversive stim ulus, which m ay be
used to generate avoidance or escape behavior, to develop a conditioned anxiety sup
pression, or as a punishm ent. T im e out has been used, for exam ple, to im prove p er
form ance under a complex discrim ination where some responses are followed by
reinforcem ent and others by tim e out. As H errnstein (1955) has shown, a time out
is aversive only w hen the schedule in w hich it occurs is positively reinforcing. W hen
the schedule itself is aversive, the tim e out m ay serve as a positive reinforcement.
B. As a novel stim ulus a T O m ay have an em otional effect in the form of a pause
a n d /o r a reduced rate. This eventually adapts out.
C. D uring a T O the bird m ay move about the box; and at the end of the T O it m ay
be farth er from the key th a n usual. A slight pause will then be introduced as the
bird returns to the key.
D. D uring tim e aw ay from the key, com peting behavior drinking, cleaning, etc.
m ay develop; once in progress, it m ay not term in ate im m ediately upon cessation of
the T O . T he result m ay be longer pauses th an in C.
E. N either C nor D should produce a depressed rate once the b ird has begun to
respond. However, when a T O is used against the background of a very high rate, re
turning from other positions in the box m ay leave the bird in an unfavorable posture
for executing a high rate. A brief reduction in rate m ay then appear.
F. W hen darkness is used as a T O , the bird m ay roost. Little is known of this
effect; but it m ight produce a pause and perhaps even a lowered rate after the term ina
tion of T O . This result is not inevitable, however, because m any instances hve been
observed of an im m ediate start a t a m axim al rate after fairly long T O s.
G. W hen a T O follows a reinforcem ent, it m ay give the bird tim e to com plete such
behavior as cleaning its beak or grooming. T he T O will elim inate any slight pausing
characteristically observed after reinforcem ent because of such behavior.
36
H.
A T O could possibly have side effects from incidental conditioning if it is also
used to start and stop experiments. (See below.)
Yoked boxes
In one of the following experim ents a control technique is used which m ay have a
m ore general application. For exam ple, when a schedule is changed from an interval
to a ratio, residual effects from the earlier schedule m ay have to be considered. A l
though these intervals are not specially scheduled, they still naturally elapse between
successive reinforcements. Since such intervals m ay be quite irregular, it is difficult
to do a control experim ent w ith the sam e irreg u lar interval schedule. This problem
m ay be solved w ith yoked boxes. Tw o standard experim ents are run completely
independently except th a t reinforcements are set up in both boxes by a single relay. If
this relay is operated by an interval tim er, the same contingencies prevail in both boxes.
B ut if it is operated by a ratio schedule involving the bird in the first box, the p e r
form ance of this bird will be responsible for the scheduling of reinforcements for the
other bird. Except for an occasional reinforcem ent which is missed before another is
set up, the same tem poral program obtains in both boxes. Nevertheless, the first b ird
receives reinforcem ents on a ratio schedule, while the second is on a variable-interval
schedule. Differences in the end effect of the two schedules m ay be com pared while
the tem poral pattern of reinforcements rem ains constant.
Probe
A probe is some m om entary change of conditions against a background of a stable

or slowly changing perform ance used only once or so rarely an d unpredictably th a t it
cannot begin to function as a discrim inative stimulus. O ne effective probe is the tim e
out noted above. O thers include: (a) an unscheduled reinforcem ent of a response; (b)
a free presentation of the food m agazine w hen no response is m ade; a n d (c) a brief
in tro d u ctio n of a discrim inative stim ulus w hich, because of o ther contingencies or
other conditions of deprivation or aversive stim ulation, controls a different rate of re
sponding. These probes m om entarily change some of the variables operating in the
stable perform ance being studied. Thus, they m ay reduce the effectiveness of recent
behavior (as w ith a tim e out), produce stim uli generated by responding at another
rate, introduce stim uli generated by a reinforcem ent at some other tim e in a schedule,
etc.
Length o f experimental session
T h e length of a daily session m ay be expressed either as a period of tim e or as a

n um ber of reinforcements. W here the schedule involves the num ber of responses per
reinforcem ent, the session is usually term in ated on com pletion of a given num ber of
reinforcements. W here the reinforcements are scheduled in term s of time, the experi
m ent is usually term inated by a clock. T h e lim iting factor is the am ount of food the
anim al ingests during the experim ent. This am ount m ust be equal to or less than th a t
37
needed to bring the anim al back to its stan d ard w eight if experim entation is to con
tinue daily. Suitable lengths of experim ental sessions can be chosen by observing the
body-weight of the bird from day to day. T he most efficient experim ental session is
one in w hich the bird receives its whole daily ration during the experim ent. Because
this ration varies from bird to bird, experim ental sessions are not always of the same
length in a given experim ent. In general, the experim ental session is m ade as long as
possible, since under these circumstances a process is least likely to be affected by in
cidental changes in the cu rren t history of the organism . T h e periods used in the fol
lowing research vary from a few m inutes to 15 hours.
Starting and stopping an experimental session
In experim ents which are autom atically controlled, the organism usually spends
some tim e in the experim ental box both before a n d after the actual experim ental ses
sion. It m ust be prevented from responding during these periods. T h e same solu
tions are available as with the tim e out. In experiments with pigeons the experim ental
box is d ark both before and after the experim ental session.
M istakes are reduced if some such routine as the following is observed. T he a p p a
ratus is first tested to ensure th a t the proper circuits are in effect and operating, th at
the key has the proper sensitivity, th a t the m agazine is full of grain, th a t w ater is avail
able, a n d th a t all lights are operating. T h e an im al is then placed in the box with
tim e-out conditions in effect. T he anim als num ber and the experim ental procedures
are entered on the cum ulative record and in a perm anent-records book. T he recorder
pen is cleaned and filled, and the paper supply checked. T he experim ent proper is
started a few m inutes later, either by hand or by a tim ing device. It is discontinued at
the end of a pre-arranged tim e or num ber of reinforcements, and tim e-out conditions
prevail until the organism is rem oved from the box. W hen a single experim ent of only
a few hours d uration is ru n during the night, these tim e-out intervals before a n d after
the experim ent m ay last for hours. T he bird is fed up to weight im m ediately after
being rem oved from the apparatus an d returned to its living cage.
Procedural irregularities and mistakes
In general, an effort is m ade to experim ent on a given organism day after day w ith
out interru p tio n for the du ratio n of the experim ent. T im e spent in the living cage
usually can be shown to have some effect upon the results, an d missing a day from an
over-all schedule m ay be an im p o rtan t disturbance. This happens w hen for some
reason the weight of the bird exceeds the set weight. (The food m agazine m ay ja m in
an open position, the bird m ay be fed by m istake, an d so on.) Occasionally, also, the
ap p aratu s fails. T he organism m ay undergo extinction or m ay be reinforced on an
unscheduled program . A lthough these difficulties are serious, they need not result in
the total loss of an experim ent. T he prevailing perform ance can usually be recap
tu red w ith a day or two on the proper procedure. N o im portant change in procedure
is ever m ade until a stable perform ance has again been observed.
38
Length o f exposure to schedules
T he effect of a schedule often depends u pon m om entary accidental contingencies

resulting from the interaction of the schedule a n d the existing perform ance of the o r
ganism. W e do not expect two organisms to reach either a stable or an oscillating
state at the same rate. T he experim ents are therefore not designed in advance. T he
pigeon is placed on a given schedule, an d its perform ance is w atched from day to day
until it is stable or until several cycles have revealed the nature of any oscillatory re
sults. Especially in the later experim ents, generous exposures to a schedule were a r
ranged.
Change o f conditions
So far as possible, a change in conditions is m ade in the m iddle of an experim ental

period in order to detect first effects independently of the special variations produced by
the beginning of an experim ental session.
Reversibility
W ith some exceptions, the conditions to be described here are reversible; th a t is,
the perform ance characteristic of one schedule m ay be recovered after the organism has
been exposed to other schedules or conditions. This usually makes it possible to es
tablish a baseline, introduce a variable, and retu rn again to the baseline in such a way
th a t the organism serves as its own control.
Num ber o f subjects
Because of the very large num ber of hours of behavior recorded w ith this technique,
a n d because we m ay use the reversibility of processes to provide control curves, the
use of large groups of organism s was not necessary. Tw o pigeons are usually studied
on a given schedule, b u t in some cases as m any as four or five are used. If differ
ences in performance are conspicuous and cannot be im m ediately explained in terms of
differences in the behavior w hich the organism s bring to the schedules, the experi
m ental conditions are fu rth er analyzed, an d the experim ent m ay be repeated with
o ther pigeons. But this is rarely done. A given experim ent is often repeated m any
tim es in later, m ore complex experim ents. For exam ple, the total num ber of pigeons
studied on fixed-ratio schedules in this research is of the order of fifty, although no
p articu lar experim ent involves m ore th an three.
T he research as a whole covers approxim ately 70,000 recorded hours, during which
the experim ental organisms em itted approxim ately one-quarter of a billion responses.
Chapter Four
FIXED RATIO
INTRODUCTION
I n a f i x e d - r a t i o s c h e d u l e of reinforcem ent, every rath response produces a reinforc

ing stim ulus. O n a cum ulative plot of responses as a function of tim e, reinforcem ent
occurs when the curve crosses a line parallel with the base of the graph. T he height
of this line is the num ber of responses in the ratio. T h e tim e to reinforcement varies,
depending on how rapidly the required responses are em itted. A t least five relevant
conditions prevailing at reinforcem ent can be specified. These are m anipulable con
ditions which m ay be used in the analysis of fixed-ratio schedules.
CONTINGENCIES RESULTING FROM FIXED-RATIO REINFORCEMENT
Frequency o f reinforcement
T h e higher the rate at which the responses required for reinforcem ent are em itted,
the shorter the tim e to reinforcem ent and, hence, the higher the frequency of reinforce
m ent. T he high rates of responding which occur u n d er small fixed-ratio schedules,
for exam ple, m ay simply be due to a relatively high frequency of reinforcem ent. E x
perim ents described in C hapter Eight isolate the contribution of frequency of reinforce
m ent to behavior under a fixed ratio.
The reinforcem ent as a discrim inative stimulus
A response is never reinforced ju st after a previous reinforcement. T he operation

of the m agazine and the ingestion of food is therefore a stim ulus which bears a constant
relation to (the next) reinforcement. Like any other stimulus, its effect m ay extend
beyond its term ination. O ne com m on effect is a low rate of responding ju st after re
inforcement.
Later effects o f time since reinforcem ent
A response cannot be reinforced w ithin a shorter period of tim e than th at required

to count out the ratio at the highest possible rate. But since the actual rates vary, there
is only a rough correlation between tim e since reinforcement and the probability th a t
39
40
a response will be reinforced. C ontrol from this source is m uch less precise th a n in a
fixed-interval schedule. However, tim e since reinforcem ent and the n um ber of re
sponses since reinforcement vary together when the rate is constant, as in most fixedratio performances. Hence, some allowance m ust be m ade for such a factor.
Num ber o f responses since reinforcement
Evidence presented in C hapter Eleven shows th a t the num ber of responses em itted
since reinforcem ent or some other a rb itrary event is a discrim inable stimulus. It is an
im p o rtan t factor in a fixed-ratio schedule because the n um ber is constant at reinforce
ment. T here are two principal effects:
(A) Number as a discriminative stimulus. P robability of reinforcem ent is m axim al
when the num ber of responses em itted equals the ratio. T he probability of response
is therefore eventually m axim al a t th a t point. Sm aller num bers of em itted responses
exert proportionately less control. In p a rticu lar, a count of zero responses ju st after
reinforcem ent is the occasion upon which a response is never reinforced. Zero and N
(the ratio) are thus end points on a stim ulus con tin u u m w hich exerts a gradient of
control because of its correlation w ith probability of reinforcement.
(B) Number o f responses since reinforcement as a conditioned reinforcer. As a discrim inative
stimulus, the num ber of responses em itted a t reinforcem ent is also a reinforcing stim
ulus. M ore generally, at any point during a fixed ratio, a response m ay be reinforced
because it increases the num ber a n d advances this stim ulus tow ard the reinforced end
of the continuum .
A chain of responses develops as follows:
S_i-R->SL.
(l )
H ere, the next to the last response is a discrim inative stim ulus in the presence of which
a response opens the m agazine. Then, N -l responses since reinforcement would re
inforce any behavior which produces it, as in:
SS-2 R -> S_! R -> S ag
(2)
In (2) the second from the last response in the fixed ratio is the occasion on which a re
sponse produces the occasion for the last response, w hich is in tu rn followed by the
appearance of the magazine. T he chain can extend backwards to the preceding r e
inforcem ent.
SrDeint R -> S? R -> S ? ........... S 3-! R - Sag
(3)
H ere, the reinforcem ent is the occasion on which the first response produces a count
of one, w hich is the occasion on w hich a second response produces a count of two, and
so on.
T he preceding reinforcement, then, m ay have 2 opposed effects: (1) It m ay control
a low or zero rate because a response is never reinforced when the bird has eaten re
cently or w hen only a small num ber of responses has been emitted. (2) It m ay control
FIXED RATIO
41
a substantial rate as the occasion on w hich responses are reinforced because they con
spicuously increase the count or rem ove a zero count. (A discrim inable increase in
n um ber m ight be a function of the n u m b er of responses already em itted. A psycho
physics of the num ber of responses is possible here, ju st as w ith any stim ulus dim en
sion. Experim ents bearing on these points are reported in C hapter Eleven, p a rticu
larly in the sections on mix FR FR , mix F IF R , and mix F R ext.) T he two types of con
trol are usually present simultaneously. However, the first function can be separated
from the second u n d e r conditions in w hich the conditioned-reinforcing properties of
num ber are slight or lacking.
T he im portance of num ber of responses as a conditioned reinforcem ent in fixed-ratio
behavior can be settled m ore generally by an analysis of how reinforcing properties
of a conditioned reinforcer are established, m aintained, and changed (C hapter Twelve).
Differential reinforcement o f high rates
In interval schedules the probability th a t a response will produce a reinforcem ent

increases w ith tim e after any unreinforced response has been m ade. After a long
pause, the probability of reinforcem ent is high (the clock having ru n continuously d u r
ing the pause). In a fixed ratio, however, the probability th a t a response will p ro
duce a reinforcem ent is independent of the tim e since the last response, the reinforce
m ent being determ ined by a counter w hich operates only as a function of the b ird s
behavior. If the elapsed tim e betw een responses differs th a t is, if responses are
groupeda reinforced response is m ore likely to follow a short inter-response time on
a ratio schedule an d a long inter-response tim e on an interval schedule (Skinner, 1938,
p. 274).
T he relative im portance of these five properties of the reinforcing contingencies gen
erated by fixed-ratio schedules will be evaluated here in experim ents involving (1) a
tim e out after reinforcem ent, (2) a tim e-out probe, (3) the num ber of reinforcements
in the ratio, (4) drugs, (5) level of food deprivation, (6) novel stimuli, (7) brain dam age,
and (8) added counter.
TECHNICAL PROBLEMS
Fixed-ratio schedules generate high rates of responding; in the pigeon they often
exceed 10 responses per second. T h e key, the recorder, a n d the device w hich de
term ines the ratio m ust be designed to follow these rates. Differences betw een the
responses of these elem ents to high rates m ay lead to poor counting of the ratio not de
tected in the records, incom plete records, an d so on. T h e conditioned reinforcers
(such as the noises which accom pany the operation of the m agazine) m ust also be p re
sented very quickly. Even good-quality, telephone-type stepping switches, although
otherwise satisfactory for program m ing fixed-ratio schedules, are usually back-acting;
th a t is, they advance a t the end of a pulse rath e r th an a t the beginning. If the m ag a
zine is arran g ed to be operated by a pulse from the stepper, the slight delay m ay have
th e effect of reinforcing some later stage in the act of pecking the keyfor exam ple,
42
th e m ovem ent of w ithdraw ing the head. If this contingency rem ains in force for a
long tim e, it will alter the topography and possibly affect the rate. T he solution noted
in C h ap ter T hree is to perm it the next to the last response in the ratio to close another
relay, so th at the final response to the key operates the m agazine directly.
THE TRANSITION FROM CO NTINUO US REINFORCEMENT
TO SHORT FIXED RATIOS
A fixed-ratio schedule will take control directly after continuous reinforcement if the
ratio is not too large. U n d er the new schedule a short period a t a high b ut declin
ing rate usually occurs, representing extinction of the effect of the previous continuous
reinforcem ent. This is followed by a period of low rate w ith subsequent acceleration.
T he rate either continues to increase until the term inal rate of the fixed ratio is reached,
or it enters a further slight decline before such an increase. Figure 12 is a stylized
Fig. 1 2.
Stylized p lo t o f the transition from erf to FR
FIXED RATIO
43
graph illustrating the m ain features of the transition. Occasionally, a third slight in
flection occurs. T he special contingencies im posed by a fixed ratio begin to operate
during the first decline in rate; in particular, the pauses during the low rate are not
likely to be followed by reinforcem ent as in interval schedules. Reinforcem ents tend
to occur after responses which are m em bers of small groups.
Figure 13 shows the transition from erf to F R 22 w here a perform ance very close to
th e term inal perform ance of the schedule is reached during the 1st session. T h e over
all rate begins at about 1.5 responses per second a t a and declines slowly. T he fact
th a t some of the pauses in the region b follow reinforcem ents shows a possibly early de
velopm ent of a discrim ination based on the reinforcement. A low rate occurs at the
inflection point a t c, after which the rate increases continuously until a term inal rate is
reached, beginning at d. T he over-all rate is then approxim ately 2.5 responses per
second, and pauses following reinforcem ent are alm ost wholly eliminated.
Figure 14A shows the transition to a larger ratio (erf to FR 40). A high rate at a
appears after a slight w arm -up. T he rate declines fairly smoothly until a t b it is a l
most zero. T he reinforcement following the pause a t b follows a brief rapid run; this
characteristic contrasts sharply w ith the long inter-response times ju st preceding it
w hich were unreinforced. T he result is the highest rate yet seen. A slight decline
sets in quickly and continues through c. T h e rate increases again, and the bird con
tinues for the rem ainder of the session a t an over-all rate of approxim ately 3 responses
per second, w ith no pauses following reinforcem ent. T ow ard the end of the session,
44
some curvature tends to ap p ear at the start of the ratio, as a t d. In the 2nd daily ses
sion (Fig. 14B) a characteristic stable perform ance for this value of fixed ratio develops.
Some irregularity occurs at the start of the session, where the rate falls after the m ax
im u m rate of the ratio is reached, as a t e , f a n d g. T h ereafter, the perform ance is
sta n d a rd for a fixed ratio of this value a pause after reinforcem ent of only a few sec
onds, possibly a brief period of acceleration, and a term inal rate of from 4 to 6 responses
per second.
Figures 15 and 16 show two cases of the transition from erf to F R 4 5 in which the
final perform ance develops more slowly. In Fig. 15 the characteristic high rate follow
ing continuous reinforcem ent at a declines to a lower rate a t b, and a m oderate rate
of responding is resum ed in the region c. A higher, fairly stable rate then prevails for
some tim e ( d). T he grain of the record is rough, however, and this bird shows large
changes in over-all rate from reinforcem ent to reinforcem ent. By the end of the ses
sion the bird is responding at a sustained rate during the last p a rt of each ratio. H ow
ever, the over-all rate does not exceed 1.5 responses per second, which is considerably
below the norm al rate on this size of ratio.
Figure 16 shows a second instance w here the term inal perform ance develops slowly.
As in Fig. 15, the initial portion of the curve is negatively accelerated, reaching its
5 M IN .
Slow developm ent o f FR 4 5 a fte r erf
300
RESPONSES
Fig. 15.
Fig. 16.
Slow developm ent o f FR 4 5 a fte r erf (second bird)
46
lowest rate at a. This negative acceleration is repeated and reaches an interm ediate
rate at b. Although the grain is rough, the over-all rate increases steadily for the re
m ainder of the period. T he over-all rate does not exceed 0.8 response per second by
the end of the session. But some runs are at 2.5 and 3.5 responses per second, as at
c and d, respectively.
Figure 17 shows a transition from erf to F R 5 0 in which the first negatively accel
erated portion of the curve is considerably sm aller th an usual. T h e preceding erf has
created only a slight tendency to respond in extinction. H ow ever, the rate declines
as usual through the first 4 ratios to b. T h e rate is so low th a t the reinforcem ents at a
an d b are separated by about 15 m inutes. T he over-all rate increases during the next
Fig. 17.
Transition from erf to FR 5 0
3 ratios to approxim ately 1 response per second, w hich is m ain tain ed to the end of
th e session. T he grain rem ains rough, and m arked fluctuations in over-all rate occur,
as a t c, d, and e.
Figure 18 shows a transition from erf to F R 50 w hich does not conform to the usual
pattern. An initial low rate accelerates slowly a n d continuously through the first 2
excursions of the figure. By the 3rd segm ent th e rate has reached 1 response per sec
ond, which is m aintained for the rem ainder of the session with some slight oscillation.
T h e grain rem ains rough throughout the session. As in Fig. 15, 16, and 17, the fixedratio perform ance develops slowly.
Figure 19 shows examples of a rapid transition to a final F R 20 perform ance. In
R ecord A the term inal rate is reached alm ost im m ediately, and no pauses are above 1
or 2 seconds. At a, how ever, 2 ratios are ru n off a t a rate of only 1.3 responses per
RESPONSES
300
Fig. 1 8.
Unusual transition from erf to FR 50
Fig. 1 9.
Transition from erf to FR 20
48
second, instead of a prevailing rate of from 1.8 to 1.9 responses per second. T h e sec
ond bird, R ecord B, shows a trace of the typical pattern during the first 9 ratios, b ut
a perform ance very n ear the final form begins abru p tly at b. V ariations in the early
fixed-ratio curves after erf are to be expected, since the transition depends upon
w hether the preceding erf has generated not only the stable rate but the topography
of response necessary to m ake the fixed-ratio contingencies effective. T he actual con
tingencies are not precisely controlled. Nevertheless, we m ay say th a t fixed ratios of
the order of 50 or 60 m ay be program m ed directly following continuous reinforcement,
assum ing a stan d ard level of deprivation represented by 80% body-weight. M ore
drastic levels of deprivation would probably perm it the sustaining of higher ratios. In
any case, longer fixed ratios, as we shall see, can be set up by first establishing stable
performances on shorter ratios.
FINAL PERFORMANCE UNDER SHORT FIXED RATIOS
Records A and B in Fig. 20 show final performances under F R 20 after 650 reinforce
m ents. (Figure 19 shows the birds transitions from erf.) T h e over-all rates are 2
a n d 2.5 responses per second, respectively, while the local rates (ignoring the slight
pauses after reinforcem ent) are slightly above 3 per second in R ecord A and approxi
m ately 4 per second in R ecord B. Records C and D show the performances of 2 other
birds on FR 31 after 1300 reinforcements. H ere, the local rates are 3 per second for
both birds, the over-all rate being 2.4 in R ecord C a n d 2.6 in R ecord D. T he slightly
lower rate in R ecord C is due to very slight pauses following reinforcements.
These curves represent the order of m agnitude of rates to be expected under fixed
ratios of this size as well as the order of consistency an d uniform ity. These perform
ances are not likely to change with further exposure to the schedule.
W ith fixed ratios of 40 or more, the final perform ance occasionally develops m uch
slower. Figure 21 shows the m iddle segments of 9 consecutive daily sessions on F R 45
after erf. (H ere, the recorder resets after each reinforcem ent.) R ates of the order of
1.25 responses per second during the first 2 sessions increase to almost 4 responses per
second by the 8th session (Record H ). Pauses following reinforcem ent are comm on
an d m ay last up to 20 to 30 seconds. A cceleration frequently occurs in the early p a rt
of each ratio, and in R ecord H the rate falls abruptly to zero for approxim ately 30 sec
onds after the bird has begun to respond at the term inal rate. Such pauses and rate
changes indicate th a t the schedule has not yet h ad its final effect, an d th a t further ex
posure to the schedule will produce m ore uniform curves.
Figure 22 shows a typical perform ance after prolonged exposure to F R 60. T he 2
records are for consecutive sessions occurring after more th an 4000 reinforcements on
this schedule. Occasionally, a pause follows the reinforcem ent, as at a and b, and the
shift to the term inal rate is characteristically abrupt. In nearly all other cases, how
ever, the bird begins pecking w ithin a few seconds after reinforcem ent, and attains the
term inal rate immediately.
This standard perform ance can be generated uniformly from subject to subject, al-
Fig. 2 0 .
Final perform ance on FR 20
A /AA/MAAAAAAA
b
4 AAAAJuWS/j
.
mmumjmm
JMAMjmmm.
e MMM
mm
F
G
5 MIN.
Fig. 21.
Early developm ent on FR 45
50
though m erely program m ing a fixed ratio of this size does not autom atically guarantee
such a result. A fixed-ratio schedule acts in concert w ith the p a rticu la r behavior
w hich the organism brings to the experim ent, a n d ap p ro p riate m anipulation m ay be
required to achieve a perform ance similar to Fig. 22 in every case. If some other re
sult is observed, the size of the ratio is dropped until no pauses occur after reinforce
m ent a n d until the term inal rate is of the order of 3 responses per second. T he ratio is
th en increased by 5 or 10 responses and held at the new value for several sessions in
ord er to ensure th a t pauses or irreg u lar features will n ot develop. If m arked curva-
FIXED RATIO
51
ture, low term inal rates, or long pauses after reinforcem ent appear, the ratio is again
decreased tem porarily. (It is assum ed of course th a t the am o u n t of reinforcem ent is
adequate, th at the m anipulandum and all circuit elements can follow high rates, a n d
th a t the level of deprivation of the anim als is satisfactory.)
INTERMEDIATE FIXED RATIOS
As the num ber of responses in the fixed ratio is increased, the pauses following re
inforcem ent grow longer. Figure 23 shows a segm ent from a final perform ance on
F R 120, after alm ost 6000 reinforcem ents on fixed ratios from 20 to 120. T he pauses
following reinforcem ent m ay be as long as 10 m inutes (a t b), alth o u g h very short
pauses still occur, as well as in te rm e d ia te values (at a). O nce the bird begins to re
spond, however, it reaches the term inal rate of the fixed ratio im m ediately a n d m a in
tains it until the ratio is completed.
Figure 24A shows a n exam ple of a stable perform ance on F R 200 after a history of
alm ost 4000 reinforcem ents on fixed ratios from 50 to 180. T he b ird pauses after
each reinforcem ent for a period varying from a few seconds to m ore th an a m inute; the
rate th en shifts (usually ab ruptly'' to 3.5 to 4 responses p er second, which is m a in
tain ed (though possibly w ith a slight decline) u n til reinforcem ent. A second bird
(R ecord B) in the same experim ent on F R 120 after a sim ilar history shows som ew hat
longer pauses a n d an instantaneous rate change from zero to the term inal rate.
A w ell-m arked, b u t rare, exception is the deviation called a knee. An exam ple
is show n in Fig. 25 at b. After a long pause, the pigeon begins a t nearly th e te r
m inal rate b u t decelerates smoothly to zero before returning to complete the ratio run.
This bird showed slight negative curvature (especially a t d) and irregular positive ac
celerations a t c and e.
10 MINUTES
perform ance on FR 2 0 0 and FR 1 20
FIXED RATIO
53
T h e over-all p a tte rn for a daily session a t a ratio w hich generates occasional long
pauses is shown in Fig. 26, w here the record has been pieced together to m ake th e co
ordinates continuous in tim e a n d num ber. T he bird h a d begun w ith a short fixed
ratio after erf, a n d the ratio h a d been slowly increased over a long period of tim e to a
size large enough to produce pauses following reinforcem ent in most cases. T he av
erage rate of responding is less th an 1 response per second, b u t the actual ru nning rate
is about 5 responses per second. T h e pauses following reinforcem ent vary from p rac
tically zero (as a t a and b) to over 10 m inutes (as a t c). Tw o other common features
of th e curve should be noticed. First, ratios beginning w ith short pauses generally
ten d to group. Second, a slight over-all curv atu re develops during the session w hich
is fairly characteristic of perform ance u n d e r larger ratios. Even a t the end of the
session, w here th e over-all rate has fallen, responding occasionally begins w ith alm ost
no pausing after reinforcem ent (as a t d).
54
LARGE CHANGES IN THE FIXED RATIO
W hen a large fixed ratio which is producing pauses following reinforcem ent is re
duced, pauses do not im m ediately disappear. Figure 27 shows a transition from
F R 180 (R ecord A) to F R 6 5 (R ecord B). In the m iddle of the session on F R 180,
after a history of over 6000 reinforcem ents on fixed ratios as high as 240, pauses range
from only a few seconds (as a t b) to alm ost 1 m inute (as a t a). At the beginning of
the following session (R ecord B) the ratio was reduced to 65. T he absence of a pause
after the 1st reinforcem ent (at c) is characteristic of this bird a t the start of a session,
even u n d e r larger ratios w hich show considerable pausing elsewhere. T h e 2nd rein
forcem ent (at d) shows a pause of slightly less th an 1 m inute. T h e next 2 reinforce
m ents are followed by progressively shorter pauses until (at e) a characteristic
perform ance on F R 65 has been established. T his perform ance is m ain tain ed for
th e rest of the session.
Figure 28 shows a sim ilar result. R ecord A is from the m iddle of the session on
F R 185, after a history of over 7000 reinforcem ents on fixed ratios of from 40 to 170.
(N ote the examples of negative curvature in single segments a t a a n d b.) W hen the
ratio is dropped to 65 (R ecord B), the pauses following reinforcem ent do not disappear
im m ediately, alth o u g h the reduced size of th e fixed ratio shortens them consider-
FIXED RATIO
55
ably. T he bird ultim ately reached a stable perform ance on F R 65 sim ilar to th a t in
Fig. 27B.
Figure 29 illustrates a m ore extrem e change in the size of the fixed ratio. T h e b ird
had had a history of tan d F K F l before exposure to the fixed-ratio schedule. R ecord
A shows 3 segments from a stable perform ance on F R 160, w ith only brief pauses after
reinforcem ent. R ecord B shows the beginning of the following session, when the r a
tio was reduced to 35. T he pauses following the reinforcem ent becam e progressively
shorter a n d essentially disappeared by the 5th reinforcem ent. In the following ses
sion the size of the ratio was increased to 140; a n d R ecord C, containing 3 segm ents
from th e early p a rt of th e session, shows the com plete absence of any pausing follow
ing reinforcem ent. C urvature an d pausing again develop by the m iddle of the session
(R ecord D).
Rs
150
Fig. 2 9 .
Transition fro m FR 1 6 0 to FR 35
FIXED RATIO
57
In an o th er experim ent the ratio was changed from 60 to 10 and then back to 70.
Figure 30 shows the transition for 2 birds. Little effect is felt a t R ecord A, because the
higher ratio h ad not produced pauses. A m arked effect is shown a t R ecord B, how
ever. R ecord B1 shows a segment from the final perform ance on F R 60, where pauses
up to 60 seconds occur. W hen the ratio is reduced to 10 in the following session, a
stable perform ance develops after a short period of acceleration. T h e re tu rn to a
larger fixed ratio (B3 a n d B4) yields a result sim ilar to th a t in Fig. 29. (B oth birds
show a low er ra te u n d e r F R 10 th a n u n d e r F R 6 0 . T his decline is p a rtly due to
Fig. 30.
Transition from FR 6 0 to FR 10 to FR 70
slight pausing inevitably involved in reinforcem ent, b u t the running rates during the
ratios are also different. At very small ratios the running rate tends to be relatively
low. (See Fig. 20.)
In sum m ary, the pauses generated by large ratios do not disappear im m ediately
when the ratio is reduced, nor do they retu rn again im m ediately w hen the higher ratio
is in effect following a period a t a sm aller ratio.
EXTINCTION AFTER FIXED RATIO
E xtinction after fixed ratio shows th e effects of th e controlling contingencies a r

ranged by th a t schedule. M ost o f the behavior occurs a t the high running rate of the
ratio. A decline in over-all rate develops because of increasingly longer periods of
58
no responding.
appear.
Subjects vary considerably in the extent to which interm ediate rates
Figure 31 shows an extinction curve taken in a single session following 700 reinforce
m ents at F R 6 0 (after erf). N early all the responding in this 3.5-hour session occurs
a t rates above 5 responses per second. Even a t the end of the session (at i, for exam
ple), the rate is approxim ately 11 responses per second. M ost of the transitions from
a high rate to pausing occur abruptly for exam ple, a t c, d, e , f and g. Some examples
of b rief periods of responding a t interm ed iate rates a n d w ith rough grain occur a t
the start of the session (as at a a n d b) a n d tow ard the end of the session (as a t h and j ) .
Figure 32 is a curve showing roughly the sam e p a tte rn b u t after a very different his
tory. T he bird h ad received 14,000 reinforcem ents on F R 60 with a total history of
35,000 reinforcements on various fixed ratios, including adjusting ratios. D uring the
im m ediately preceding session only, the bird h ad been reinforced on F R 4 0 w ith p e r
centage reinforcem ent. (See below.) T he over-all features of the curve resemble
Fig. 31. T h e usual term in al rate continues for only a few h u n d red responses before
negative acceleration sets in at a. T he over-all rate then falls off, as a result of in-
FIXED RATIO
59
creasingly longer pauses between bursts. W hile instances of a b ru p t shifts to zero rates
an d a b ru p t shifts from pausing to ru n n in g at high rates occur (as a t c, d; g, a n d j ) , fre
quent instances ap p ear of slight negative curvature (as a t b , f and h ) and desultory re
sponding at low rates (as a t e and ).
T he bird whose extinction curve is shown in Fig. 33 h ad received more th an 8000
reinforcements on various fixed-ratio schedules, with tim e out and added stimuli. T h e
im m ediately preceding schedule was F R 170 for approxim ately 400 reinforcem ents.
At the beginning of the extinction session, approxim ately 1000 responses occur a t the
usual ratio rate. A short pause appears at a and is followed by a brief acceleration
to another ru n of 800 or 900 responses. A pause of 2 hours an d 50 m inutes occurs a t
the first break in the record. A 2nd ru n of approxim ately 650 responses at b is fol
lowed by another sustained pause of m ore th an 1.5 hours. Except for the brief period
of acceleration at a, the curve shows no interm ediate rates of responding. T he record

shows the result of a n especially well-developed contingency betw een high rate a n d
reinforcement.
A t th e o th er extrem e, Fig. 34 shows an extinction curve in w hich frequent a n d
sustained interm ediate rates occur in spite of a history of over 5000 reinforcem ents on
fixed ratios of from 40 to 120. A segment of the perform ance on F R 120 im m ediately
preceding extinction is illustrated. T h e term inal fixed-ratio rate shows some grain
(as a t a), a n d the running rate is 3 responses per second, which is m inim al for a ratio
of this size. W hen extinction is begun a t b, a lower rate is quickly reached and the rest
of the record shows considerable curvature and interm ediate rates of responding.
After a period of 2 hours an d 35 m inutes in w hich no responses occurred, the rate re
turns very close to th a t observed during the ratio perform ance (at c). Instead of an
a b ru p t break to a zero rate, how ever, a period of negative acceleration (a t' d) follows.
A nother pause (at e) is again followed by negative acceleration. Even a t interm e-
60
Fig. 34
Extinction a fte r FR 120 showing sustained interm ediate rates
diate ra te s, however, the fine grain of the record indicates th a t groups of responses are
still occurring at the ratio rate. T h e m arked an d frequent curvature in this record is
seldom observed after a fixed-ratio schedule. Even so, the long pauses an d the return
to the high rates 3 hours after extinction began are characteristic.
In general, a shorter history of reinforcem ent brings quicker extinction. Figure 35
shows a curve in which extinction is virtually com plete after 20 m inutes of rapid re
sponding. T he bird had a history of 360 reinforcements on F R 60 after erf. T he final
perform ance on F R 60 is shown a t the beginning of the record. A single tim e-out
FIXED RATIO
61
probe (see below) occurs a t a. W hen extinction is begun (at b), a ru n of over 500 re
sponses is sustained until a short break occurs a t c. Shorter runs and longer pauses
th en appear, until a t d extinction is virtually complete. A short burst appears later a t
e, b u t further extinction during a 14-hour session produced less th an 50 responses.
Figure 36 shows an o th er extinction curve after a fixed-ratio schedule w ith a history
sim ilar to th a t of Fig. 35. A sam ple of th e perform ance on F R 60, 350 reinforce
m ents after continuous reinforcem ent, ap p ears a t the sta rt of the figure, w ith th e re
corder resetting after each reinforcem ent. E xtinction (beginning a t a) shows a rough
over-all decline com posed of runs a t th e fixed-ratio rate a n d increasingly m ore fre
q u e n t a n d longer pauses. T here is no sustained interm ediate rate, although some
cu rv atu re follows runs a t high rates, as a t b a n d c. Even near the end of the curve (at
d), sustained runs of m ore th an 150 responses occur a t the fixed-ratio rate.
A ratio-type extinction curve requires a w ell-developed fixed-ratio perform ance.
In Fig. 37 extinction curves were taken on 2 successive sessions before a stable fixed-ra
tio perform ance had developed. R ecord A begins w ith a brief perform ance on F R 60,
200 reinforcem ents after the b ird was first exposed to this schedule after erf. T he
g rain a n d th e wide variations in rate show th a t a stable fixed-ratio perform ance has
not yet been reached. Extinction proceeds at an interm ediate rate. Some effect of
F R has been felt: pauses as long as those a t a a n d b are unlikely in the early stages of
extinction after interval reinforcem ent. In the following daily session (R ecord B ),
reinforcem ent was reinstituted a t F R 60, a n d an o th er extinction curve then taken.
Both the ratio perform ance an d the extinction curve show an advance over the preced-
Fig. 37.
Extinction a fte r FR 6 0 before developm ent o f a norm al FR perform ance
FIXED RATIO
63
ing day. T h e extinction curve consists m ainly of 2 sustained runs a t rates higher
th a n any occurring in the previous session. Because of the longer pauses, the over-all
size of this curve is nevertheless not so great as th a t in R ecord A.
Summary
In extinction after fixed-ratio reinforcem ent, the organism responds predom inantly
a t the fixed-ratio rate. An over-all decline in rate is due to increasingly longer pauses
alternating betw een increasingly shorter runs. T he curves are in some cases almost
square, w ith most of the responding appearing very early in the session. O thers, as in
Fig. 31 and 32, show a fairly smooth decline in rate throughout the experim ental ses
sion.
THE EFFECT OF INSUFFICIENT REINFORCEMENT O N FIXED RATIO
In an experim ent on the effect of short tim e-out probes during the developm ent of
F R after erf, the m agazine ho p p er becam e p artially blocked. E ventually, th e b ird
h a d access to only a few grains a t each reinforcem ent. This condition was discov
ered a t the end of the 5th session, w hen atypical curves were recorded. T he blocking
of the m agazine h ad probably been progressive, beginning w ith norm al operation on
the first day of the experim ent.
Figure 38 shows th e 5th session on F R 4 0 after erf. (Arrow s indicate 1-m inute
tim e-out probes, w hich will be discussed in a later section.) Before being affected by
th e g radually reduced reinforcem ent, the bird h ad reached a good fixed-ratio p er
form anceno pausing after reinforcement, a high rate, and good grain. T he reduced
reinforcem ent produces a lower term inal rate of responding an d an over-all decline
in rate d uring the session. Interm ed iate rates an d a rough grain appear. Pauses
after reinforcem ent do not change greatly, however. T h e curve is therefore quite dif
ferent from one in which too high a ratio has produced a sim ilar decline in over-all
rate.
Figure 39 shows the 3rd session for an o th er bird in the sam e apparatus. H ere, the
reduced reinforcem ent produces sustained periods of responding a t a low rate after
reinforcem ent. C haracteristic fixed-ratio perform ances occur a t a an d at e (following
a probe). In general, however, the responding is som ew hat scattered, beginning soon
after each reinforcement. Interm ediate rates appear a t b, c, and d.
Figure 40 shows the 4th session of F R after erf for a third bird. Here, the fixed-ratio
character of the behavior is m aintained under reduced reinforcem ent despite a severe
decline in over-all rate. T ow ard the end of the session, a t c for exam ple, 4 ratios are
ru n off norm ally. A low over-all tendency to respond is indicated by the fact th a t
only 8 m ore reinforcem ents occurred w hen the session was extended for 13 hours be
yond the portion shown in the figure. Occasional disruptions of the fixed-ratio per
form ance ap p ear a t a, b, a n d d.
T he same kind of m agazine failure occurred in another experim ent on F R 20 after
erf. Figure 41 shows the beginning portions of the 2nd, 3rd, 4 th , 5th, and 6th sessions.
64
66
U n d er this shorter ratio the effect of the reduced reinforcem ent is felt almost entirely
as a lengthening of the pause after reinforcem ent. Despite the reduced over-all ten d
ency to respond, continued exposure to F R 2 0 through 5 sessions produces an in
crease in the actu al ru n n in g rate. P re c e d in g /in R ecord E the b ird pauses after
reinforcem ent; b u t once it begins to respond, the rate is above 3 responses per second.
R u n n in g rates of approxim ately 2 per second prevail in R ecord A. Occasional d e
p artu res from a characteristic fixed-ratio perform ance sometim es ap p ear as interm e
diate rates (as at a, b, and e) and as broken segments (as a t c and d).
T h e apparatus was repaired, and the ratio was reduced to 12 and then progressively
increased over a period of 20 days to F R 50. D uring this period the m agazine failed
again. Figure 42 gives samples of the perform ance during this period. Figure 43
shows the m arked inflections in the over-all rate in 2 later sessions, when the birds were
still u n d er inadequate reinforcement. M ost of the changes in over-all rate occur as
a result of pauses after reinforcement.
FIXED RATIO
67
PERCENTAGE REINFORCEMENT
A nother kind of interm ittency can be introduced into any schedule of reinforcem ent
by substituting some other event for a certain percentage of the reinforcements. This
can be done in a t least two ways:
1. T h e stim uli w hich have been present w henever food was available in the m ag a
zine can be presented for the same length of tim e, although the m agazine hopper is
either em pty of grain or does not rise w ithin reach. For the type of pellet dispenser
commonly used with rats, a certain percentage of the pellet com partm ents is left em pty;
the m agazine sounds b u t does not deliver a pellet. This m ethod of percentage rein
forcem ent is of interest as a n instance of chaining. T h e reinforcing effect of the dis
crim inative stim ulus for ap proach to the m agazine (say, the sound of the operation
of the m agazine) is studied as a function of the schedule according to which approach
to the m agazine is reinforced by food. This kind of percentage reinforcement in tro
duces a type of complication into the study of fixed-ratio schedules. Since reinforce
m ent is a stim ulus w hich m ay in p a rt control the subsequent rate of responding, a
fixed-ratio perform ance after reinforcem ent w ith a n em pty m agazine will begin
under slightly novel circumstances.
2. If each reinforcem ent is norm ally followed by a tim e out, reinforcem ents
w ith an em pty m agazine m ay also be followed by a tim e out, which m ay cover the pe
riod during which the preceding reinforcem ent rem ains an effective stimulus. T he
behavior which follows food plus tim e out a n d th a t which follows tim e out alone occur
under nearly similar circumstances.
This m ethod of percentage reinforcem ent also raises its own problems. O n a r a
tio schedule th e tim e out is produced by a response w henever it is substituted for the
usual reinforcem ent and m ight have a punishing effect, which could reduce the rate.
In a first experim ent on percentage reinforcem ent a 3-m inute T O rep laced the
operation of the food m agazine w henever a reinforcem ent was om itted. W hen a re
inforcem ent occurred, it was not followed by a T O . Figure 44 shows the first effect.
T he points a t which T O s were substituted for reinforcem ents are indicated by the dots
at the left of the record. Reinforcem ents are m arked as usual. R ecord A is a seg
m ent of the final performance on F R 40 after m ore th an a year on fixed-ratio schedules.
W hen T O s were substituted for 50% of the reinforcem ents in the following session,
the over-all rate im m ediately increased (Record B). Record C shows the perform ance
7 sessions after Record B where the size of the fixed ratio has been reduced to 30 an d
a further increase takes place. (This increase in rate is an effect of tim e out per se r a
th er th a n percentage reinforcem ent, as will be seen in a later section on tim e out.)
N either the omission of the reinforcem ent or the T O produces any disruptive effect a t
this stage. T h e procedure was then changed so th at a 3-m inute T O followed all r a
tios, b u t a percentage of these w ere preceded by th e usual reinforcem ent. If the
m agazine operated, a 3-m inute T O followed; if a reinforcem ent was om itted, a 3-m in
ute T O followed the last response in the ratio.
68
Fig. 4 4 .
First e ffe ct o f percentage

reinforcem ent
T h e 2 birds in the experim ent reacted a t different speeds to these conditions. O ne

showed a lower tendency to respond while m aintaining the general properties of the
fixed-ratio perform ance. Figure 45 shows the 10th session on F R 30 for the first bird,
in w hich 50% of the ratios are reinforced a n d th en followed by a 3-m inute T O , while
50% are simply followed by a 3-m inute T O . Except for the size of the ratio and
om itted reinforcem ent m arks, this curve could be m istaken for one produced by too
large a fixed ratio. T he 3 longest pauses (at a, b, an d c) follow om itted reinforcements.
If this factor is significant, the length of pause m ust be influenced by the presence or
absence of reinforcem ent w hich occurred 3 m inutes earlier.
T h e second bird continued to respond a t a high rate until the percentage of rein
forcem ent was reduced below 50%. Figure 46 shows segm ents from the 20th to the
FIXED RATIO
69
37th sessions of th e experim ent for this b ird, in w hich th e percentage of reinforce
m ent om itted ranged from 50% to 85%. R ecord A shows the 20th session an d R ecord
B shows the 23rd session under F R 30, w ith 50% of the reinforcem ents om itted. It
is sim ilar to Fig. 44C, although some sign of disturbance is evident. Responses occur
during T O (producing vertical segments, as a t b), a n d the rate changes slightly w ithin
a ratio (as a t a), so th a t the over-all record looks m uch less linear. R ecord C shows
70
th e 4th session, w here 75% of the reinforcem ents are om itted. T he local rate changes
of R ecord B have largely disappeared, b u t slight pauses tend to occur at the start of
th e ratio (as at c a n d d). T h ree sessions later, w ith 85% om itted (in R ecord D ), the
pauses become m ore pronounced u n d er this same procedure, and changes in the lo
cal rate are beginning to ap p e ar (as a t e). Records E, F, a n d G, a t the same percen
tage, were taken 8, 13, and 16 sessions, respectively, after R ecord D. T h e fixed-ratio perform ance deteriorates progressively, w ith increased pauses following the
reinforcem ent (as a t i a n d j) ; an d m arked rate changes occur w ithin a ratio (as at f , g,
a n d h) as well as rough grain. Figure 47B illustrates in detail the perform ance in
R ecord G. T he punishing effect of the T O appears as a decline in rate ju st before re
inforcement. R ecord A in the figure shows a later stage in the experim ent in which
the schedule is F R 7 0 w ith 50% reinforcement. T he negative curvature has disap
peared, but responding still occurs in bursts separated by slight pauses. T he first
bird (Fig. 45) eventually showed a sim ilar effect of the punishm ent w hen the schedule
was changed to F R 6 0 with 75% reinforcement.
Fig. 4 7 .
D etail o f n egative curvature under

percentage reinforcem ent
T h e reduction in ra te tow ards the ends of the ratios in Fig. 47B is equivalent to the
lowered rate observed during a pre-aversive stim ulus, which is characteristically fol
lowed by a shock (Estes and Skinner, 1941). T he num ber of responses already
em itted constitutes a pre-aversive event w hich is functionally equivalent to the buzzer
used in m ost such experim ents, a n d th e T O is eq u iv alen t to the shock. In m ore
general term s, th e rem oval of a positively reinforcing stim ulus (the group of stim uli
under which the key is pecked) is negatively reinforcing. By m aking a period of tim e
out contingent upon a response, we punish a response by rem oving the occasion
on which the bird is reinforced.
W e have used T O in other experim ents as a punishm ent in enhancing stimulus con
trol. In some discrim inative situations involving 2 or m ore responses (e.g., in m atch
ing experim ents), the frequency of reinforcem ent is high enough to sustain behavior
even in the absence of any stim ulus control. A bird will usually behave w ithout re
gard to available stimuli. By occasionally allowing an unreinforced response to p ro
duce a short tim e out, however, a m uch more precise stim ulus control can be
developed.
FIXED RATIO
71
Summary
Little evidence of the effect of percentage reinforcem ent per se is forthcoming w ith
these m ethods. T h e results are confounded by effects of the tim e out, including its ef
fect as punishm ent. Before the punishing effects of the T O influence the perform
ance, however, the percentage reinforcem ent increases the pause after reinforcem ent
w ithout d isrupting the norm al fixed-ratio p a tte rn . T his is sim ilar to th e effect of
lessened deprivation.
THE EFFECT OF DEPRIVATION LEVEL O N FIXED-RATIO PERFORMANCE
T h e effect of deprivation on a stable fixed-ratio perform ance was studied in the

following way. Tw o pigeons were m aintained on FR. 110 after m ore th an 70 sessions
on various values of fixed ratio w ith occasional T O probes after reinforcement. T he
schedule was then held at F R 110 for several m onths, during which the body-weights of
th e birds were varied over a wide range.1 T h e principal effect was upon the pause
after reinforcem ent. T h e local rates of responding show very little sensitivity to even
wide ranges in deprivation. T his finding confirms a published experim ent by Sidm an an d Stebbins (1954).
T h e results are sum m arized in Fig. 48, which presents the average length of pause
following reinforcem ent. Since each daily session showed a decline in over-all rate,
pauses a t the beginning a n d end of each session are presented. T h e top graphs show
the m ean pause for the last 2 excursions of the recorder pen, and the m iddle graphs
show the m ean pause for the first 2 excursions. W hen the over-all rate of responding
was so low th a t 23 fixed-ratio segm ents did not occur d u ring the 4-hour session a l
lowed for the experim ent, the value of the pause is indicated on the graph by X .
Pauses for Bird 129 R P over 300 seconds a n d for Bird 130 R P over 200 seconds are
indicated by an X. T he weight changes in the experim ent are shown in the graph
at the bottom of the figures.
Bird 129 RP
D uring the first 40 days of this experim ent, Bird 129 R P s weight increased from
slightly over 400 gram s to alm ost 550 grams. But this increase h ad little effect on the
F R perform ance, except during the higher values a t about 40 days and a single peak
weight a t 10 days. A slight increase in pausing is m ore m arked tow ard the end of the
session, as th e top graph indicates. As the w eight is increased further, beyond the
40th session, to 600 gram s (approxim ately 100% of the a d lib w eight for this bird),
pauses increase to a m axim um . In th e last 10 days of the experim ent, w hen the
weight fell to 550 grams, the pauses becam e shorter a t the beginning of the session but
decreased only slightly d u ring the last part. A differential effect is also present at
th e beginning an d end of the session betw een 30 a n d 50 days.
Bird 130 RP
Bird 130 R P confirms the results of the first bird, although its weight changed in a
more com plicated way. Bird 130 R P was a sm aller bird, whose lowest weight was 370
1 See com m ents on change in body-w eight in C h a p te r Six, p. 293.
72
OVER
300
300 -
129 RP
L A S T 2 EXCURSIONS
250 \
X s 4 HOUR S ES SIO N IN WHICH

L E S S TH A N 36 FIXED RATIOS
A R E C O M P L E TE D .
( X
XXXXXXXXXX XX X
300
X 4 HOUR
200
LU
in
CD
250 [
SES SIO N IN WHICH

L E S S TH A N 36 FIX E D RATIO S
ARE C O M PLETED .
o
ij
130 RP
L A S T 2 EXCURSIONS
150
xwoooooco XX x m a
200
x x x
150
UJ
100
5
I- 100
50
50
_J\___ J
o0
to
20
30
40
DAYS
50
60
70
80
1/
10
20
30
40
DAYS
DAYS
DAYS
DAYS
DAYS
Fig. 4 8 .
50
60
TO
80
Summary o f the effect o f deprivation on FR
gram s co m p ared w ith m ore th a n 400 gram s for Bird 129 R P . D uring the first 15
sessions of the experim ent, Bird 130 R P s weight increased rapidly to 540 grams, which
was above 100% of the ad lib w eight determ ined before the start of the experim ent
some m onths before. C orresponding w ith the increase in weight, the length of the
pause following the reinforcem ent increases, too. Between the 15th and 30th day, the
w eight fell to a b o u t 460 gram s, a n d th e pause following reinforcem ent was corre
spondingly shorter. From the 35th to about the 50th day, the weight reached a m axi
m um v a lu e , an d both the top a n d m iddle curves show extrem e pausing. T h e weight
curve then fell slowly, and the length of the pause decreased correspondingly. H ere,
FIXED RATIO
73
again, a given level of deprivation produces m ore pausing a t the end of the session th an
at the beginning.
A n ad lib w eight determ ined m ore th a n 6 m onths before the experim ent was begun
is probably not very m eaningful. However, a body-weight could be found a t which
very little pecking occurred on the fixed-ratio schedule, and this was used as a reference
level. W e will call the reference level the inactive weight. For 129 R P it was a p
proxim ately 600 gram s; an d for 130 R P , 520 gram s. O th e r w eight levels in this ex
perim ent can be expressed as the percentage of the inactive weight. In the following
section, figures will be presented w hich characterize the perform ance a t 4 general
ranges of body-w eight in reference to each b ird s inactive weight. These are sam ple
daily sessions taken at a, b, c, a n d d in the weight curve for 129 R P , and a t a, b, c, d,
an d e for Bird 130 R P in Fig. 48.
Figure 49A represents point a in Fig. 48 (left g rap h for Bird 129 R P). T h e bird
Fig. 4 9 .
FR a t 6 8 % and 8 2 % o f the inactive w e ig h t (1 2 9 RP)
74
SCHEDULES OF REINFORCEM ENT

,_____r*
Fig. 50.
O ver-all rate changes under FR a t 95% o f the inactive w e ig h t (1 2 9 RP)
is at 68% of its inactive weight. This bird characteristically shows brief positive accel
e ratio n to a term in al rate of over 7 responses per second a n d a n over-all rate of 4
responses per second. Pauses occur occasionally after reinforcem ent. R ecord B is
for point b at 491 grams, or 82% of the inactive weight. At this weight level the fixedratio perform ance is stable, and both the over-all rate and the local rates are approx
im ately the same as those in R ecord A.
Figure 50 shows a com plete daily session for the still higher body-weight at c. This
weight is 95% of the inactive weight, or 575 grams. At this weight level the over-all
rate falls off during the period, m ainly because of increasing occurrences of long pauses
following reinforcement. (In order to show this, the curve has not been collapsed in
th e usual m anner.) T he ratios are still being executed a t approxim ately the same
ru n n in g rate as in Fig. 49, although some positive acceleration occurs which cannot be
seen at this reduction, and also occasional irregularities in rate. Figure 51 contains
Fig. 51.
10 M IN U T E S
Detail o f a record a t an FR a t 105% o f the inactive w eight
FIXED RATIO
75
2 segments from point d w hich are both a t 105% of the inactive weight, or 609 grams.
This is essentially a free-feeding weight, and the curves represent at most a few hours
of deprivation. R ecord A shows considerable disruption of the fixed-ratio perform
ance a n d little evidence of the norm al fixed-ratio rate a t any tim e. Low or interm e
diate rates are usual, w ith only an occasional exception, as a t a. R ecord B, a segment
from the following session at the same ad lib feeding level, shows alm ost no disruption
of the sta n d a rd fixed-ratio perform ance. T h e pause following reinforcem ent is often
Fig. 52.
FR a t 71 % and 8 8% o f the inactive w eight (1 3 0 RP)
extended (as, for exam ple, the final pause shown in the record), an d a b rief accelera
tion from a near-term inal to a term inal rate of responding occurs. But once the bird
begins to respond, the acceleration is rapid to a term inal rate of over 7 responses per
second, as in Fig. 49.
Figure 52 shows 2 curves taken a t a in the weight curve for Bird 130R P at 71% of
th e b ird s inactive weight, or 375 gram s, an d a t b a t 88% of this b ird s inactive weight,
or 465 grams. This bird showed generally lower rates th an Bird 129 R P and larger
pauses after reinforcem ent. No curvature is evident a t the start of the ratio, however.
In R ecord A, the session at the lowest body-w eight, local rates are 3.5 to 4.5 responses
76
per second, w ith a n average over-all rate of approxim ately 3 responses per second.
In R ecord B, a t 88% of the inactive weight, instances of higher local rates th an those in
R ecord A occur, an d the value of the local rates of responding varies widely. T h e
over-all rate rem ains 3 per second, however, a n d the general form of the behavior does
not change.
Figure 53 shows a complete daily session a t an interm ediate body-weight represented
a t d in the right curve in Fig. 48 a t 97% of the inactive weight, or 502 grams. T h e
perform ance here is sim ilar to th a t in Fig. 50 a t 95% of the inactive weight for Bird
129 R P. An over-all decline in rate during the period is due to a progressive increase
in pauses following reinforcement. Local running rates are indicated in several cases,
a n d are of the same order as those given in Fig. 52 a t m uch lower body-weights.
Figure 54 shows samples from a daily session a t 99% of the inactive w eight, or 520
grams, for Bird 130 R P (c in the weight curve). T h e over-all rate is so low th a t only
10 fixed ratios are com pleted in the 4 hours allotted for the experim ental session. A l
though pauses as long as 2 hours occur following reinforcem ents, the ratios are gen
erally com pleted a t rates co m parable w ith those a t lower body-w eights if they are
FIXED RATIO
77
started a t all. A t this body-weight, several responses occasionally occur after a pause
w ithout im m ediately leading into the term inal rate, as a t a an d b.
T h e perform ance u n d e r the highest w eight recorded in this experim ent is shown
in Fig. 55 at 112% of the m ean inactive w eight, or 577 gram s (e in the weight curve for
1 3 0 R P in Fig. 48). T h e segments shown in Fig. 55 show a range of local running
rates from 4.5 responses per second a t a to 2.5 a t c. T h ere is a som ew hat higher dis
tribution of running rates a t lower body-weights in Fig. 52 an d Fig. 54. T he curves
show frequent irregularities, including a w ell-m arked knee at e, a smaller knee at b, a n d
slight pauses after the term inal rate has been reached a t c a n d d, w here some grain
is beginning to appear.
Summary
A fairly wide range exists betw een approxim ately 90% of the inactive weight a n d
any lower weight a t w hich the anim al will rem ain healthy w hen perform ance under a
fixed-ratio schedule is invariant. A t higher weights, pauses after reinforcem ent b e
come extreme, b u t responding occurs at the fixed-ratio rate whenever the bird responds
a t all. T he characteristic perform ance of the individual ratio ru n is disturbed only
a t very high body-weights.
NOVEL STIMULI
In cid en tal stim uli in the experim ental ch am b er often exercise considerable control
over the behavior of an organism even though no explicit contingencies are arranged
w ith respect to them . T his control is seen w hen a novel stim ulus is added or w hen a
feature of the experim ental cham ber is altered. A characteristic effect, probably to
be classified as em otional, is a general depression of the whole repertoire o f the organ
ism. T h e effect of a given stim ulus depends upon the species a n d upon the stim uli
to w hich the subject has already been exposed. A novel stim ulus involving a change
in the experim ental cham ber m ight conceivably function as a change in a discrim ina
tive stimulus, since the box is a situation w here pecking is reinforced, com pared w ith
th e hom e cage w here it is not.
78
Some incidental effects of novel stim uli upon perform ance under various schedules
have been observed. These were m ainly the result of accident; a few of these will be
reported.
Figure 56
A bird h a d received m ore th a n 5000 reinforcem ents on F R 50. By accident the

key was changed to a novel color a t the beginning of a session. T he heretofore norm al
fixed-ratio perform ance was severely disrupted (Fig. 56A). R ough grain during the
ratio, interm ediate rates of responding (as a t b), a n d pauses before the com pletion of a
FIXED RATIO
79
ratio (as a t a a n d c) appear. Even tow ard the end o f the session, some disturbance
is still evident, in th a t the ratio is ru n off in small bursts of responses (as a t g and h).
T h e long pauses after reinforcem ent a t d, e, a n d / a r e not typical of a fixed ratio of this
size. Tw o sessions later, still w ith the new key-color, a norm al fixed-ratio perform ance
is recovered (R ecord B), although some sign of the disturbance still rem ains in the
grain of the record a t i an d k an d in the pause in the m iddle of the ratio a t j.
Figure 57
H ere, the effect of a novel stim ulus is confounded w ith the passage of tim e since
th e bird was last ru n in the experim ent.
After a history of approxim ately 15,000 reinforcements on an adjusting fixed ratio,
followed by 6 m onths of no experim entation, 2 birds were ru n for the first time in a new
apparatus, which differed in dimensions a n d other details from the earlier apparatus.
B oth birds w ere slow in responding to th e key, a n d the first 2 responses of each were
reinforced. T he records in Fig. 57 begin im m ediately thereafter. Each bird failed to
show its previous norm al fixed-ratio behavior. In R ecord A, following the 1st rein
forcem ent a t a, the over-all rate declines continuously. This decline was produced
by a reduction in ru n n in g rate ra th e r th a n by pauses (cf. b a n d c). Sm all deviations
ap p ear a t d and e. A fairly standard perform ance u n d er fixed-ratio reinforcem ent re
tu rn s before the end of the session. T h e o ther bird (R ecord B) also shows a lower
over-all rate, w ith a m ore m arked negative acceleration during th e ratios (as a t f g, h,
a n d i). Even by the end of the session, R ecord B shows a low term inal ru n n in g rate,
considerable grain, an d pauses th a t are too long for this size of ratio.
Fig. 5 7 .
FR 65 w ith novel stimuli
80
Figure 58
A long history of ta n d F K F I followed by ab o u t 1000 reinforcem ents on F R 100 pro

duced the stable perform ance on F R 90 shown in Fig. 58A. T he experim ent was then
transferred to a new experim ental cham ber w ith novel dimensions a n d details. R ec
ord B shows the 1st session complete. A t a a reinforcem ent was set up so th a t a single
response w ould open th e m agazine. A second reinforcem ent was set up a t b after
fewer th a n 90 responses. T he program then rem ained F R 90 for the rem ainder of the
session. T he novel surroundings produced a m uch lower term inal running rate,
m arked curvature, irregular grain, and occasional negative curvature. This effect is
p robably p artly a function of the previous ta n d F K F I, w hich has also produced ex
ten d e d periods of low or zero rate im m ediately following reinforcem ent an d the slow
a n d fairly sm ooth acceleration to a term inal rate in each ratio observed tow ard the
end of the session shown. (See Fig. 541, ta n d F R F I.)
Figure 59
A history of m ore th a n 5000 reinforcem ents w ith F R varying betw een 40 and 110
yielded the stable perform ance on F R 90 seen a t R ecord A. A slight negative acceler
ation (as at a a n d b) is characteristic. (Cf. Fig. 24B for the same bird.) T h e bird
was then ru n w ith an uncorrelated block counter, in which the key-color changed
from w hite to green to yellow to blue, etc., every 10 responses. (See later section.)
R ecord B shows the first effect of the changing key stim ulus. Pauses after reinforce
m ent are briefer, a n d slight pauses occur at m any of the color changes; b u t these ef
fects are not pronounced.
Figure 60
A nother bird h ad developed a stable perform ance on F R 2 1 0 (shown in Fig. 24A).

H ere, again, the novel stim ulus consisted of a change in key-color every 20 responses.
Record A is the 1st segment under this novel stimulus; and thereafter every other ex
cursion of the pen is shown. T he novel stim ulus produces a severe drop in both over
FIXED RATIO
81
all and local rates. T he rate a t b is slightly over 1.5 responses per second, com pared
w ith a term in al rate of over 3 responses per second in Fig. 24. T he rate increases
progressively th rough the session, however; a n d by R ecord F the ru nning rate has re
tu rn e d to nearly 3 responses per second. Some g rain is still evident in the record,
caused by slight pauses every 20 responses at the color changes. Note the unusual sud
den changes at a, c, d, and e which occur only very rarely in a stable fixed-ratio p er
formance.
Figure 61
R ecord A shows a final perform ance on F R 105 following 54 sessions of F R after

erf, w hen the fixed ratio was slowly increased. T h e key was blue. A t the beginning
of the following session the key-light h a d b u rn ed out. R ecord B shows the entire fol
lowing session with no light behind the key. T he adjustm ent to the new stimulus is
continuous b u t slow. By the end of the session, the form er stable fixed-ratio perform -
Fig. 6 1 .
FR 105 w ith o u t a key light
FIXED RATIO
83
ance has not yet been recovered. T h e norm al pattern is especially disrupted during
the first 8 ratios. Pecks occur in bursts, w ith negative curvature, very few sustained
ru n s , and long pauses. T he ratios at a an d b show the beginning of a return to norm al ;
b u t pauses still ap p ear (as at c) after the term inal ru nning rate has been reached. By
the end of the session, m arked pauses still occur after reinforcem ent, responses are in
small bursts, a n d the rate varies widely w ithin the fixed-ratio segment.
THE EFFECTS OF DRUGS O N FIXED RATIO
Some prelim inary experim ents have been un d ertak en to sam ple the changes which
drugs pro d u ce in fixed-ratio behavior. T h e drugs w ere adm inistered in collabora-
Fig. 6 2 .
Segments sum m arizing the e ffect o f brom ide on FR 60
tion w ith D r. P. B. Dews of the D epartm ent of Pharm acology, H a rv a rd M edical

School.
Two birds were put on a sodium chloride-free diet, and desired brom ide blood-levels
w ere established by the feeding of sodium brom ide. T o reverse the condition, the
birds were given free sodium chloride, an d the excretion of brom ide produced an expo
nential decline in the brom ide level. Figure 62 shows segments from the 2 birds over a
period of 2 m onths, in w hich the brom ide blood-level was raised by the feeding of 100
m illigram s of sodium brom ide orally each day. R ecord A is the final perform ance for
one bird u n d er F R 60 before any brom ide was given. Records B, C , D, a n d E are
the 3rd, 10th, 12th, and 14th sessions under bromide, during which the brom ide level
increased progressively.
Between segm ents B an d C (3rd a n d 10th days) th e over-all rate increased; b u t
84
this increase was not well-represented in the small segment shown in the figure. J u s t
before R ecord D the brom ide level reached 35 m illiequivalents, a n d R ecord D shows
th e first depressive effect. R ecord E, 2 sessions later, shows a further decline in rate as
well as m arked shifts in local rate from one ratio to the next. In Record F, which is
from the session following Record E, the brom ide level had reached 47 milliequivalents.
T he fixed-ratio perform ance is severely disrupted. Segm ent G, for the following
session, shows sim ilar disturbances in augm ented form. M arked negative accelera
tion during the ratio, rough grain, and long pauses are evident. A t this high concen
tratio n the birds constantly staggered an d fell easily if pushed. Bromide was then dis
continued, a n d 100 m illigram s of sodium chloride was adm inistered orally each day.
R ecord H shows the fixed-ratio perform ance alm ost 2 m onths later, after the blood
level h a d returned to norm al.
R ecord I shows the perform ance for the second bird before bromide. Three sessions
later, in R ecord J , a total of 300 m illigram s of brom ide has produced an increase in
both over-all a n d ru n n in g rates; these increases are m ore easily seen in the samples.
T h e high rate is m aintained 7 sessions later (10 sessions after the first adm inistration of
brom ide), w ith a blood level of 35 m illiequivalents (R ecord K). A t this tim e the bird
was staggering badly u n d er the effects of the brom ide a n d lost its balance if lightly
pushed. T he 13th, 14th, and 15th sessions (Records L, M , and N) show a m arked and
progressive disruption, w ith long pauses, rough grain, interm ediate rates, an d nega
tive acceleration during the ratio segments. T he brom ide blood-level has reached 51
m illiequivalents at R ecord N. A pproxim ately 1 m onth later, after daily feedings of
100 m illigram s of sodium chloride, a norm al fixed-ratio perform ance em erged (R ec
ord O).
Figure 63 illustrates fu rth er details of the progress of the brom ide effect betw een
Records K and L. O n these 2 days an 8-m inute T O was program m ed following every
10th reinforcem ent. (See later section.) In R ecord A (11th session) a blood level
of slightly over 35 m illiequivalents of brom ide has little effect on the fixed-ratio p er
form ance, alth o u g h th e bird is grossly staggering. T h e T O s after reinforcem ent, at
th e arrow , produce no effect except a very slight decline in rate, w hich can be seen if
the curves are foreshortened (for example, a t a). By the end of the session, rate
changes are occurring (as a t b) an d pauses several seconds long are beginning to a p
pear during the ratio (not shown at this reduction). O n the following day shown (R ec
ord B), the brom ide level reached 37 m illiequivalents a n d produced a large fall in
over-all rate as well as deviations from the norm al ru nning rate. T he first 2 T O s (at c
a n d d) have no effect, b u t the 3rd (at e) is followed by a fall in the ra te extending
th ro u g h several reinforcem ents. T h e last 2 T O s (at f a n d g ), w here the fixed-ratio
perform ance has deteriorated w ith wide fluctuations in rate, grain, and negative cu r
vature, produce pauses of approxim ately 50 a n d 30 seconds, respectively.
B oth birds show very strong resistance to any disruptive effect of brom ide on F R
co m p ared w ith the effect on F I (H errn stein a n d M orse, 1956) (Dews, 1955). Be
sides a general over-all decline in the rate, brom ide produced a severe disruption of the
FIXED RATIO
85
norm al fixed-ratio pattern. A m ore sensitive effect of central nervous system drugs is
possible by m odifications of the fixed-ratio schedule (M orse an d H errnstein, 1956).
FIXED-RATIO PERFORMANCE AFTER ABLATION OF BRAIN TISSUE
W e are indebted to Dr. J . L aw rence Pool for suggesting some exploratory experi
m ents on the effects of various brain lesions upon the perform ance of pigeons under v a r
ious schedules of reinforcem ent. A t the C olum bia P resbyterian M edical C enter in
New York, D r. Pool operated on 6 pigeons selected from various experim ents in prog
ress at the tim e. W hen he retu rn ed them to our laboratories, a rough exam ination
of their perform ances u n d e r various schedules was u n d ertak en . F our of these
birds w hich h a d been on schedules involving fixed ratios were tested under fixed-ratio
reinforcem ent after operation . O th er results will be reported in appropriate places.
T he circum stances of the experim ent do not perm it a precise comparison of perform
ance before a n d after b rain operation. T h e lesions w ere few in n u m b er a n d only
roughly localized in post-m ortem exam ination. T herefore, no atte m p t will be m ade
to ascribe an y p a rticu la r behavioral effect to a specific lesion. T h e experim ents will
m erely serve to supply exam ples o f the kinds of disturbances w hich m ay be gener
ated in interm ittently reinforced behavior by ablation of cerebral tissue.
(It should be noted th a t the h e a d is a n im p o rta n t m echanical detail in these ex
perim ents. If, as the result of the operation, quick m ovem ents of the head produce
86
painful stim ulation, some disturbance in perform ance w ould be expected a p a rt from
any brain dam age. C ontrol experim ents suggest th at this difficulty is minim al. [See
C hapter Ten.])
O ne bird (74G) had an extended history of F R , m ult F IF I, a n d then F I 1. Parts
of the cerebral hem ispheres were then rem oved. Post-m ortem sections showed a cyst
in the right posterior aspect of the cerebrum and considerable atrophy in the left h alf
o f th e brain. T h e perform ances were not n orm al a n d will be described in C h a p
ter Five. T he schedule was th en changed to F R 60. Figure 64A shows the
10th session, w here the schedule is still not producing a norm al fixed-ratio perform
ance. Pauses occur after reinforcem ent despite the small ratio, a n d most segments
show considerable curvature. T he ratio was reduced to 20 in the following session;

R ecord B shows a segm ent of th e perform ance. T h e sm aller fixed ratio elim inated
the pause after reinforcement; but the low rate, oscillation in over-all rate, and frequent
acceleration after reinforcem ent are still not typical of a fixed ratio of this size. (Com
pare, for exam ple, Fig. 20.) R ecord C shows a segm ent on the same F R 2 0 ,9 sessions
later. H ere, the perform ance is even m ore disturbed, w ith pauses after reinforcem ent
a n d breaks after responding has begun. T he ratio was then increased to 40; Record D
shows a segment after 5 sessions at this value. F u rth er deterioration of perform ance
beyond th a t of R ecord A is evident in the disappearance of the high term inal rate, even
though the fixed ratio is only 40 here com pared w ith 60 in R ecord A. T he extrem e
deterioration in Figure 64D did not appear im m ediately. Figure 65 contains the com
plete 3rd session on F R 4 0 , which occurred 2 sessions before Fig. 64D. T he over-all
curve is negatively accelerated, alth o u g h some recovery can be observed to w ard the
FIXED RATIO
87
end of the session. Parts of the perform ance resem ble R ecord A for F R 6 0 : a pause
after the reinforcem ent is followed by sm ooth acceleration to a term inal rate. T h e
term in al rate varies, however (cf. b w ith c). Substantial pauses or changes in rate
after the term inal rate has been reached occur at a an d d. A lthough this bird pecks
the key at a fairly sustained rate, the fixed-ratio schedule does not produce a norm al
performance.
A second bird, whose post-m ortem exam ination revealed dam age in both h em i
spheres an d the cerebellum , was reinforced on F R after pre-operative an d post
operative performances on m ult F IF R . T he m ult F IF R perform ance was severely dis
rupted by the operation. After 5 sessions on th at schedule, the bird was given 1 session
A B
Fig. 6 7 .
C D E
D evelopm ent on FR 4 0 a fte r b rain lesion
FIXED RATIO
89
of erf a n d th en placed on F R 4 0 . Figure 66 illustrates the 1st session on F R 4 0 (37

days after the operation). T he curve shows m any features of a first exposure to F R
after erf (cf. Fig. 16). But although the initial segment of the curve (at a) shows the
negative acceleration generally seen in the transition erf to F R , the small bursts of re
sponses followed by pauses are unusual. F u rth e r developm ent of the perform ance
on F R 40 is shown in Fig. 67, in segm ents chosen from the last parts of the next 6 ses
sions. T he 2nd session on F R 4 0 (R ecord A) shows a slight increase in over-all rate.
Short bursts of responses separated by pauses show in the grain. T he 3rd and 4th ses
sions (Records B and C) show a progressive fall in over-all rate, and the rough grain
continues. In the following session (R ecord D) the rough grain rem ains. T h e seg
m ents becom e m ore sharply scalloped; nevertheless, the over-all rate of responding
increases. In the 6th session (Record E) the bird responds im m ediately after reinforce
m ent at a higher term inal rate and w ith a smooth grain. In the 7th session (R ecord
F) the ratio is being held fairly well; however, unlike a norm al F R , breaks occur in the
ratio as a t a, b. c, a n d d. Figure 67 represents a very slow developm ent for a fixed
ratio of this size.
ADDED STIMULI
A dded counter Experiment I
An added stim ulus is a stim ulus some dim ension of which is designed to rem ain pro
portional to some feature of a schedule of reinforcem ent or of the perform ance gen
e ra ted by a schedule. In a fixed-ratio schedule the m ost im p o rta n t feature is the
num ber of responses since the reinforcement. H ere, the added stimulus m ay be called
an ad d ed co unter. If such a stim ulus is clear-cut, it m ay acquire a m ore effective
control th an the b ird s own behavior. W e can then m anipulate the num ber of re
sponses in a way which is impossible when the b ird s own behavior is the stimulus.
T he first step in the analysis of an added counter is to dem onstrate th at it can achieve
control. In the experiments described in this section, the key was opaque except for
a small slit of light w hich could be m ade to grow as a function of the n u m b er of times
the bird pecked the key. (See C hapter T hree.) T he m axim um length of slit in this
case was 3 /4 inch. T h e slit returned to the m inim um size, 1/16 by 1/10 inch, follow
ing reinforcem ent, or during a short T O after reinforcem ent. As a m odest effort to
com pensate for the fact th at more responses were necessary to m ake the slit wider a t the
end of the ratio th a n earlier, the rate of grow th was m ade roughly proportional to the
size.
Development o f FR with added counter
A long history of ta n d FRF1 (C hapter E ight) was followed by 300 reinforcem ents
on F R 190; during the last 100 reinforcem ents the slit which was to become the added
counter was present b u t stationary a t the smallest size. T h e first segment in Fig. 68A
is the last perform ance for one bird w ith th e slit sm all (the recorder resetting after
each reinforcement). Beginning at the arrow , the slit grew as the bird pecked the key.
150
R 's
Fig. 6 8 .
First e ffe ct o f a counter on FR 190
FIXED RATIO
91
T h e novel stim ulus disrupts the norm al fixed-ratio p attern . (See section above on
Novel Stim uli.) It also produces a rough grain w ith an interm ediate rate of respond
ing w hich falls to zero before the ratio is com pleted. T h e disruption produced by
th e (now) large slit is so severe th a t the food m agazine was operated {ad lib ) to restore
the behavior. T he second bird in the experim ent showed less pausing after reinforce
m ent both in the previous experim ent on ta n d F R F I a n d in the later stages of this
experim ent. T h e first 3 segments of Fig. 68B are the last ratios d uring which the slit
rem ains small throughout. A t the arrow the slit begins to grow with the num ber o f
responses. H ere, the only im m ediate effects of the novel stim ulus are a slower acceler
ation to the also slightly lower term inal rate.
Reversing the added counter
Reversing the direction of growth of the slit easily shows the control acquired by the
ad d ed counter as a result of the reinforcing contingencies in w hich it plays a part.
Figure 69A shows the perform ance on F R plus counter for the bird in Fig. 68A after 90
reinforcem ents. A pause after reinforcem ent is followed by fairly slow acceleration
to a term inal rate which varies rath er widely (cf. a with b). A t the arrow the direction
of the added counter was reversed. U pon retu rn in g to the key after reinforcem ent,
the bird finds the slit large. A high rate of responding follows im m ediately. T he slit
now shrinks w ith each response until it reaches its sm allest size as the fixed ratio is
completed. A rough negative acceleration of rate parallels this change. T he curve a t
c under the reversed direction of growth is m erely a 180-degree rotation of the earlier
prevailing curve. Negative acceleration persists for several ratios, and the curves then
pass through a linear phase w ith no pause after reinforcement. T he com bination of
th e ad d ed stim ulus a n d the b ird s behavior here has no differential effect on the rate.
T he new counter begins to take effect in the second line of R ecord A, however, and the
segments begin to resemble those before the reversal.
R ecord B shows a sim ilar result for the second bird. T h e curve a t rfjust before the
arrow is the last ratio during which the slit grows from small to large. At the arrow
the slit was largest ju st after reinforcem ent b u t shrank as the bird com pleted the ratio.
T h e bird begins to respond im m ediately in the presence of the large size of slit, an d it
shows only slight negative acceleration as the segm ent is com pleted. (This pattern
confirms the less effective control shown in Fig. 68B for the same bird.) T he next 2
segments show a low er-than-norm al term inal ratio rate, b u t by e the curves have be
come positively accelerated, a n d by f (later in the same session) a slight pause rea p
pears after reinforcem ent. Again, the reversal of the direction of grow th a t the 2nd
arrow in R ecord B shows th a t the new counter has now taken control. T h e last seg
m ent before this 2nd reversal (g) shows a brief pause after reinforcem ent and a rapid
acceleration to a high term inal rate w hich is m aintained u n til reinforcem ent. In the
1st segm ent following the reversal the b ird begins responding a t a high rate im m edi
ately. However, this ra te does not fall off as the slit reaches its largest size a t the end
o f the ratio. N egative curvature is clear in the 2nd segm ent after reversal, and for
92
some tim e thereafter. A linear phase th en sets in; an d by the end of the session, posi
tively accelerated curves, w ith a pause following the reinforcement, appear again, a l
though several instances of negative acceleration still occur, as a t h and i.
T he direction of grow th of the slit was also reversed w hen the same 2 birds h a d sta
bilized a t larger FRs w ith added counter. Records A and B in Fig. 70 show parts of
2 successive sessions on F R 3 8 0 after approxim ately 1900 reinforcem ents on various
sizes of F R w ith added counter. R ecord A begins w ith the final perform ance on
F R 380, w ith the slit growing from small to large.
Fig. 7 0 .
Reversal o f the counter on FR 3 8 0
T h e curves show m arked negative acceleration w ith rates of approxim ately 10 re

sponses per second for the first 100 responses of the fixed ratio, followed by an often
a b ru p t shift to about 5 responses per second (as a t a). This curvature occurs occasion
ally in F R w ithout added counter. It suggests a nonlinearity of counter readings,
possibly even w hen the bird s own behavior is the only stim ulus influencing the contin
gencies. Pauses following reinforcem ent are either absent or only a few seconds long.
A t th e arrow th e direction of grow th of the slit was reversed. D uring the 1st seg
m ent thereafter ( b), the rate is constant at the value prevailing previously a t the larg-
FIXED RATIO
93
est size. Some positive acceleration, which can be seen best w hen the curves are fore
shortened, becomes m arked by Segm ent c. H ere, the rate continues at slightly over
3 responses per second until about 100 responses before the com pletion of the ratio,
w hen it shifts abru p tly to 8 responses per second. This positive acceleration, which
is again alm ost a 180-degree inversion of some of the curves before the arrow , appears
frequently d u ring the rem ainder of the session, not all o f w hich is shown in R ecord A.
At the beginning of the following session the slit is still large ju st after reinforcem ent
b u t shrinks during the ratio. R ecord B shows the m iddle p a rt of the session. T he
perform ance before the arrow (where the direction of growth of the slit was again re
versed to grow ing from small to large, as at the start of the session in R ecord A) shows
a continuation of the perform ance at the end of the previous session, with the slit chang
ing from large to small during the fixed-ratio segment. T h e rate does not vary be
tween such wide extremes, however. Following reversal (at the arrow ), the original
perform ance returns im m ediately to th a t seen before the arrow in R ecord A, where
th e slit grew from small to large. In the segm ent a t d the rate is approxim ately 10
responses per second for the first 100 responses, w ith a shift to slightly less than 6 per
second during the rest of the ratio.
T he other bird could not m ain tain so high a n over-all rate, even w ith the help of an
added counter. After approxim ately 1300 reinforcements a t various sizes of ratio, long
pauses continued to occur after reinforcem ent and were followed by slow acceleration
to the term inal rate. T he first two lines of Fig. 71A show a characteristic performance.
O ne pause is m ore th a n 1.5 hours long, a n d all segments show extended curvature.
At the arrow , d u rin g a pause, the size of the slit was changed from sm all to large and
the direction of the growth reversed. (T he slit now changes from large to small as the
bird completes the fixed ratio.) T he bird begins to respond w ithin 5 seconds of the a p
pearance of the large slit, although nearly 30 m inutes had elapsed since the last rein
forcem ent w ithout a response. T h e decline in rate of responding as the spot becomes
sm aller again shows the powerful control exercised by the spot of light. T he first 4
segments show m arked negative acceleration throughout the whole ratio. Except for
the absence of a long pause, which is prevented by the occurrence of reinforcem ent,
these curves are again 180-degree rotations of the curves before the arrow. C ontinued
reinforcem ent on the reversed counter produces a linear phase (at a), and then p ro
duces a pause after reinforcem ent, w ith subsequent positive curvature. Fifteen ses
sions later, ratios are being run off in blocks separated by long pauses. O ne such block
is shown in R ecord B, where the pause was 3 hours long and the rate preceding the
1st segm ent was low. T h e 1st segm ent a t b is ru n off a t a constant rate, b u t the next
segm ents show a progressive increase in c u rv atu re u ntil an o th er long pause follows
the last segment in the record.
T h e long pauses after reinforcem ent at larger fixed ratios are clearly not due to a fac
to r such as physical exhaustion or fatigue, b u t to the extrem ely unfavorable stim uli
then present.
94
Stopping the added counter at its unoptim al reading
T h e beginning of Fig. 72 shows a final perform ance on F R 4 1 0 w ith ad d ed counter

after approxim ately 2100 reinforcements on various fixed ratios w ith added counter.
(T his is the same bird as th a t in Fig. 70.) C onsiderable negative curvature occurs,
w ith rates falling from 10 to 5 responses per second during the last 100 responses of the
ratio. Following the reinforcem ent a t the arrow , the slit becam e small as usual, but
was allow ed to rem ain small for the rest of the session. As a result the rate rem ained
n e a r 10 per second thro u g h o u t each segm ent. A t a the slit failed to reset to small
for a few seconds. T he large slit im m ediately after the reinforcem ent produced an im
m ediate burst of responding a t approxim ately 10 per second. T he effect of the large
FIXED RATIO
Fig. 72.
95
S topping the added counter a t its unoptim al reading: F R 410
slit at a an d the lower rate of responding at the end of the ratio segments are contradic
tory. Some interaction between the added counter and the bird s behavior m ust
produce the lower rate a t the end of the norm al segment.
(An incidental effect is recorded a t b, where the experim ental apparatus was acci
dentally jarred . This jarrin g produced a lower rate, which then accelerated in roughly
the same way as at the beginning of a segment.)
Increasing the size o f the fixed ratio
T he rate of grow th of the slit has to be reduced to accom m odate larger ratios if it
is to reach its largest value upon com pletion of the ratio. Such a change is m ade for
the first tim e in Fig. 73. T h e schedule a t the beginning of the record was F R 190,
Fig. 73.
Transition from FR 190 + counter to FR 3 6 0 + counter
Fig. 7 4 .
Increased negative curvature on
FR 4 1 0 b y stopping the counter a fte r 2 5 0 re
sponses
m m m
mmmmm
5 MIN.
Fig. 75.
First transition from slit large to counter: FR 7 0
FIXED RATIO
97
with the slit growing to the full width. At the arrow the ratio is increased to 360, w ith
th e slit correspondingly reaching the sam e m axim um size in 360 responses. W hile
th e m ethod of recording does not perm it a precise com parison of the 2 curves, no very
gross differences are apparent. T he pause after reinforcem ent appears less commonly
in th e larger ratio, a n d the negative curv atu re seen a t F R 190 becomes clearer and
m ore extensive at F R 360. A possible explanation for the reduced pause could be the
novel stimuli from the b ird s own counter at the new ratio.
M ore negative curvature in larger fixed ratios m ay occur because the necessarily
sm aller increase in the size of the slit per response is not so reinforcing. T he correc
tion for relative change in size was probably inadequate. T he increase in count,
however, would be weakest as a reinforcer tow ard the end of the ratio. W e checked on
this effect in a further experim ent. T he slit grew to a m axim um size in 250 responses;
th en , a blue light ap p eared on the key, th e reinforcem ent occurring 160 responses
later. If the decline in' the rate of responding a t the end of the' fixed-ratio segment is
due to in ad eq u ate reinforcem ent by changes in the counter reading, this procedure
should produce m ore extended negative curvature, because only the bird s own counter
operates during the blue light. Figure 74A shows the first effect of this procedure.
At the arrow the blue light appears for the first tim e. T he rate during the last 160 re
sponses in the 1st fixed-ratio segment is approxim ately 2.5 responses per second com
pared w ith about 10 responses per second during the sustained ru n w hen the added
counter operates. By the m iddle of the same session (Record B) the rate during the
last 160 responses has increased to 3 responses per second, b u t the rate during the e ar
lier sustained ru n with added counter has increased to over 12 responses per second.
By the end of the session (R ecord C) the final rate has reached 4 per second. Several
sessions later, in the segm ent shown in R ecord D, the blue light cam e on after 300 re
sponses instead of 250. By this tim e negative curv atu re also developed during the
early p a rt of the ratio in which the added counter is operative.
This is essentially an experim ent in chaining. A response is reinforced on F R 250
w ith a d d ed counter by a blue light. U n d e r this stim ulus a response is reinforced on
F R 50.
A dded counter Experiment II
T he 1st portion of Fig. 75A shows the perform

ance (700 reinforcem ents after erf) on F R 7 0 , w ith the slit rem aining large. At the
arrow the slit becam e small for the first tim e; it grew from 1/16 inch to 3 /8 inch, each
response producing the same change in size of slit regardless of the size of the ratio.
W hen the slit changes to small for the first tim e a t the arrow , responding begins in
stantly a t a very high rate despite the novel stim ulus dim ensions. It then tapers off,
u n til a t the 1st reinforcem ent, a t a, it is very low. T h e 2nd reinforcem ent, a t b, is
reached at a m uch lower rate , w ith rough grain and some positive acceleration.
W ithin a few reinforcements the perform ance shows little change from the fixed ratio
w ithout a counter. T h e ad d ed counter eventually produces a very high rate. RecDevelopment o f F R with added counter.
98
ords B and C show the 2nd and 3rd sessions. Pauses after reinforcement largely disap
pear, an d the ru n n in g rate characteristically reaches values of the order of 9 or 10
responses per second. A lthough perform ances of this order can be generated without
added counter, the rapidity with which the perform ance shifts from that of Record A
indicates one effect of the counter. T h e high rate of responding in the presence of
th e novel stim ulus m ay be related to some innate tendencies of birds to peck small ob
jects rath er th an the effect of the novel stim ulus per se.
A second bird shows less clearly the tendency for a small spot to evoke more rapid
responding a p a rt from its place in reinforcing contingencies. T h e bird received a p
proxim ately 700 reinforcements on F R 110 w ith the slit continuously at the larger size.
Records A an d B of Fig. 76 are segments from the last 2 sessions under these condi-
)\M A A A A A /m \A A /
Fig. 76.
Early perform ance on FR 110 +
counter a fte r FR 1 1 0 w ith slit a t large
FIXED RATIO
99
tions. Note the scale of this record. T he paper was moving twice as fast as in preced
ing figures. D uring the reinforcem ent at the arrow in R ecord C, the slit was changed
to small for the first tim e. This change has no im m ediate effect on the rate; b u t after
approxim ately 15 seconds, there is a brief burst a t 10 responses per second (at a). T he
rate th en returns to the usual value. T he next 4 ratio segments show the effect evi
dent in Fig. 75. In the next daily session (R ecord D) the perform ance resembles th at
on F R alone. Records E through K show segm ents from the 2nd to 11th sessions
following the addition of a counter. Note th a t most of the segments are S-shaped: they
begin w ith a pause, and the rate is highest in the m iddle of the segment b ut declines
before the end of the ratio. E ventually, a m uch higher over-all rate developed, pre
sum ably because of the added counter.
Tw o other birds received essentially the same treatm en t; b u t instead of being ex
posed to the largest size of the slit, they were exposed to a growing and resetting slit
not correlated w ith the ratio. All sizes of slit h a d been equally correlated w ith rein
forcement. Record A in Fig. 77 shows the final perform ance on F R 7 0 after approx
im ately 900 reinforcements on F R 7 0 after erf, with the slit varying at random . T he
counter was added at the start of R ecord B. T h e pauses are reduced som ew hat after
reinforcem ent, and the over-all rate increases. R ecord A was recorded a t standard
10 MINUTES
Fig. 77.
Early developm ent on FR + counter a fte r FR w ith uncorrelated counter
Fig. 7 8 .
E arly d e ve lo p m e n t on FR +
co u n te r a fte r FR w ith unco rre la te d counter
(second bird)
F IX E D R A T IO
101
speed (top coordinate). T h e o th e r records in the figure were m ad e a t double speed

(lower coordinate). R ecords C th ro u g h I show segments from 11 sessions, d u rin g
which the influence of an a d d e d co un ter is felt in a n increase in the rate a n d the elim i
nation of cu rv atu re after reinforcement. Eventually, a final perform ance (R ecord I)
shows a v e rag e rates of the o rd e r of 6 per second a n d re spo ndin g im m e d ia tely after
reinforcement.
T h e o th er bird previously exposed to all slit sizes was able to sustain a fixed ratio
of 130. Records A, B, a n d G of Fig. 78 show segments from the 3 days before the a d d i
tion of a counter, with the slit v ary ing un system atically d u rin g the session. A t the
start of R ecord D the slit was small following reinforcement a n d grew to the largest size
as th e ratio was co u n ted out. Little or no im m e d ia te effect is a p p a re n t, no r a n y p ro
gressive change, th roug h Records E, F, a n d G, which show segments from the 2nd, 3rd,
Fig. 79.
Change in the scale of the added counter
a n d 4th sessions after the co u n te r was ad d e d .
T h e fact th a t the grow ing spot was
a c q u ir in g control as a stim ulus was shown at the b eg in n in g of the 5th session (R eco rd
H ), w h e re th e slit was large after reinfo rcem en t a n d r e m a in e d large for the rest of the
session. T h e first 5 or 6 segments show th e elim ination of the pause after reinforce
m ent. T h e p erfo rm an c e th e n begins to resem ble earlier records w ith or w ith o u t the
counter. (Since sustained rates of responding of app ro x im a te ly 7 responses per sec
o n d o ccur before the co u n te r is ad d e d , it is p e rh a p s not surprising th a t the co u n ter
has no d ra m a tic effect.)
C h a n g e in s c a le o f a d d e d c o u n te r.
In the experim ents ju st described the counter reached
a m a x i m u m size of f inch.
It was later c h a n g e d to grow to f inch a t th e com pletion
of the sam e size of ratio. F igure 79 shows th e session im m ediately following Fig. 77 I.
T h e b ird begins im m ediately; b u t as the slit grows to its novel size, the rate declines
sharply. T h e 1st reinforcem ent (at a ) occurs only after 4 minutes. T h e rate rem ains
lower th a n norm al, although before the end of the session (second line of the figure), i n
stances a p p e a r (at b a n d c ) where ratios are ru n off at rates com parable with those in
Fig. 77.
'
102
W ith the slit reaching m axim um size (f inch), the ratio was increased to 125. Fig
ure 80A shows the final performance. T he curves are predom inantly S-shaped, begin
ning im m ediately after reinforcem ent (or after a short pause) a n d accelerating quickly
to rates of approxim ately 10 per second. These are m aintained for varying num bers
of responses; they finally give way to m uch lower rates of the order of 3 to 4 per sec
ond tow ard the end of the ratio. This is an unusual am ount o f curvature in a fixed
ratio. T hree sessions later, in R ecord B, the session begins on F R 110 w ith the slit
large. A t the arrow the counter is added, the slit again growing to f inch a t the end
of the ratio. T h e over-all rate rem ains the sam e as before, b u t the curves become
m ore nearly linear. In R ecord C, taken 4 sessions after R ecord B on the same sched-
1
Fig. 80.
Transition from FR 125 w ith large slit to FR 125 + counter
ule, the over-all rate has increased; rates of 10 per second in the m iddle p a rt of the
ratio give way to rates of 4 per second near the end. Occasionally, a low rate is m ain
tain e d th ro u g h o u t the ratio, as a t a. T h e highest sustained rates of responding are of
the order of 8 responses per second, com pared w ith over 10 responses per second in Fig.
77 on F R 6 0 , w here the counter was first added.
Transition from added counter to slit at optimal reading. Performances at large FRs w ith
ad d ed counter were probed occasionally by letting the counter rem ain a t the optim al
size. T ransitions in the reverse direction were also observed.
Tw o birds on F R 120 w ith ad d ed co u n ter gave the curves at the left of Fig. 81.
B eginning a t the vertical dashed line, the slit rem ain ed large. In A the curves w ith
th e growing spot are S-shaped w ith m arked negative curvature at the end of the seg-
FIXED RATIO
Fig. 81.
103
Transition from FR 120 + counter to FR 120 w ith slit a t large
ments. W hen the slit rem ains large, the slight pausing after reinforcem ent is tem p o
rarily dropped a n d the bird responds a t a constant rate eq u al to th a t occurring a t
the end of the segm ent w ith the added counter. T he old higher rate reappears in the
m iddle of the fixed-ratio segment (as at a and b) as a norm al fixed-ratio perform ance
develops. In R ecord B (the second bird) the perform ance on the fixed ratio w ith the
added counter is linear after a rough start. W hen the slit rem ained large, the ru n
ning rate did not change; but the pause after the reinforcem ent disappeared for several
ratios. Shortly after the transition in Fig. 81 A, the ratio was increased to F R 2 2 0
w ith the slit rem aining a t large. Figure 82 shows segments from 10 consecutive ses
sions. (Records J and K show the beginning and end of the last session.) Instances
of runs at very high rates early in the ratio appear, as a t a, b, c, an d d in R ecord A.
These disappear in later sessions w ith the slit still a t large, although S-shaped curves
reappear tow ard the end of the session in R ecord K.
Figure 83 shows later transitions from F R with added counter to F R at optim al rea d
ing for 4 birds on FR 220. At this late stage in the experim ent, after approxim ately
4400 reinforcem ents on various values of fixed ratio w ith a n d w ithout added counter,
a fairly high rate of responding prevails in the final perform ance on the fixed-ratio
schedule w ith ad d ed counter (left of dashed line). W hen the slit rem ains large, the
next segments begin a t the term inal ra te of responding, in the absence of the c h a r
acteristic pause. In R ecord A the effect of th e large slit lasts for ab o u t 10 segments,
after which the perform ance becomes sim ilar to th a t occurring with the counter. In
R ecord B a short b reak appears n e a r the beginning of the 3rd segm ent, a n d a pause
appears after th e 4th reinforcem ent. T o w ard the end o f the record, the negative
curvature becomes less frequent. In R ecord C pauses are elim inated only in the first
2 segments after the slit rem ains at large, although a group of 3 quick starts appears
a t a. T h e rest of the session shows a perform ance sim ilar to th a t w ith the ad d ed
counter. In R ecord D the optim al counter reading has the most prolonged effect.
W ith the added counter, a pause of the order of 60 seconds consistently appeared after
reinforcem ent. W hen the slit rem ains large, th e pause is absent for the first 12 seg
ments.
104
large
Figure 84 shows transitions from a stable perform ance, w ith the slit stationary a t
large, to th e ad d ed counter, w here the slit grows from sm all to large. These are for
the sam e b ird b u t tak en a t early, interm ed iate, a n d late stages of the experim ent,
u n d e r 3 values of fixed ratio. R ecord A shows an early transition on F R 110. T he
am ount of pausing after reinforcem ent is tem porarily less. T he pause increases,
however, until by the end of the session it is of the sam e order of m agnitude as before.
R ecord B, taken about 1 m onth later u n d er F R 220, shows a sim ilar effect. W hen
Fig. 83.
Transition from FR 2 2 0 + counter to FR 2 2 0 w ith slit a t large
Fig. 84.
Transition from FR w ith slit a t large to FR + counter
106
th e counter is ad d ed at the arrow , the pause disappears for a few reinforcem ents and
th en increases until the original perform ance is recovered. Record C shows a perform
ance on F R 420 m uch later in the experim ent. Before the addition of the counter,
the bird pauses up to 1 m inute or more, w ith a ru n n in g rate of slightly less th an 6
responses per second. T he added counter, a t the arrow, reduces the pause after the
reinforcem ent for at least 4 segments, an d increases the term inal rate from less th an
5 responses per second to 8 responses per second. This rate change is not ap parent in
Records A and B.
Figure 85 shows 3 records, separated by 2 or 3 sessions w here the slit rem ains con
tinuously a t large u n til the arrow , after w hich it grows as w ith an ad d ed counter.
W hile the spot is large (left of th e arrow ), a pause characteristically follows reinforce
m ent, a n d the curves are roughly linear w ith an occasional burst at a higher rate, as
at a, b, a n d c.
Rate o f growth o f the added counter. A n S-shaped counter is ad d ed in R ecord A of
Fig. 85.
Transition from FR 2 2 0 w ith slit large to FR 2 2 0 - f counter
Fig. 85; th a t is, it grew most rapidly in the m iddle range. It has little im m ediate ef
fect, although the running rate progressively increases and the pause after reinforce
m ent progressively decreases. In Record B the counter grows most rapidly after
reinforcem ent, and proportionally slower as the size of the slit increases. It im m e
diately reduces the pause following the reinforcem ent and produces negatively accel
erated curves. This p a tte rn is tem porary, however; a n d by the end of the period the
negative c u rv atu re disappears, leaving a term in a l rate sim ilar to th a t w ithout the
added counter. In Record C the ratio is being held with short pauses. T he S-shaped
counter im m ediately reduces the pause after reinforcement. An increase in the ru n
n ing ra te early in the ratio gives a sigm oid a p p earan ce, w hich is m ore obvious if the
curves are foreshortened. T h e rate a t the end of the ratio is not increased.
An added counter m ay help in m aintaining a high ratio. Figure 86 shows 3
changes to an ad d ed counter for a b ird w hich h a d difficulty in m ain tain in g F R 220.
(A linear counter was used in R ecord A a n d an S-shaped counter in R ecord B. In
R ecord C the slit grew m axim ally a t the smallest size of slit an d proportionally less as
th e size of the slit increased. T h e day-to-day v ariability in these perform ances is
such th a t we cannot safely ascribe any effect to the shape of the added counter.)
T he advantage of a counter in holding the ratio is clear. T he performances to the left
FIXED RATIO
Fig. 86.
107
Transition from FR 2 2 0 to FR 2 2 0 + counter reducing the pause a fte r reinforcem ent
of the arrow (w ith the counter reading steadily m axim um ) show rough grain, long
pauses, an d occasional knees, as a t a, b, c, an d d. W hen the counter is added, all of
these characteristics disappear. As the session continues, the over-all rate tends to
fall an d the pause after reinforcem ent increases. T his condition can best be seen in
th e longer R ecord C, w hen by the end of the session a pause or period of low rate
consistently follows reinforcem ent. Even here, however, the form of the curve differs
from th a t w ithout counter. T he rate of responding is high in the m iddle range of
th e segm ent, after w hich the rate shifts, som etim es ab ru p tly , to a lower value for the
rem ainder of the ratio.
T he effect of added counter in m aking it possible to sustain a high ratio was felt m ore
slowly in an o th er b ird , samples of whose records are given in Fig. 87. In this ex
perim ent the ra te of grow th of the slit was most rapid a t the beginning an d end of the
ratio. (This counter was chosen in an effort to counteract 2 features of the usual seg
m ent: the reinforcing effect of the rapid grow th at the beginning of the ratio should
Fig. 87.
Transition from FR 4 2 0 to F R 4 2 0 + counter reducing the pause a fte r reinforcem ent
108
affect the stim ulus-controlled pause, while th a t at the end should correct the frequently
observed negative acceleration late in the ratio segm ent.) Figure 87A shows an
early perform ance a t F R 4 2 0 w ith this counter. T here is considerable straining
w ith ragged knees. In general, inverted Ss are suggested. A t R ecord B, 5 sessions
late r, ratios are com pleted m uch m ore rapidly. T h e sh ap e of the ad d ed counter
fails to prevent some negative curvature (as a t a). A t C, 17 sessions (and 200 rein
forcem ents) after A, the ratios are ru n off a t a n alm ost constant rate, an d pauses after
reinforcem ent have become fairly short. (Note th at records are still a t the faster p a
per speed.)
A second bird in this experim ent showed rates of responding which corresponded
m uch m ore closely to the rates of grow th of the added counter. Figure 88 was taken
Fig. 88.
Shape o f the FR segment corresponding to the rate o f grow th o f the counter
a t a stage of developm ent com parable w ith th a t ab o u t halfw ay betw een Records B
a n d C in Fig. 87. T h e early, rap id grow th of the slit has not elim inated the pause a t
the start of each segment, b u t the rate begins high. A relatively sm ooth decelera
tio n to a n in term ed iate rate is evident, a n d an acceleration to a high term inal rate of
about the same value as the starting rate. T he second ratio a t the bottom line is a
good exam ple of this effect. After a pause of less th a n 1 m inute, the rate accelerates
briefly to 7.5 responses per second at a. A continuous decline in rate leads to an in
term ed iate rate of slightly less th a n 2 responses per second halfw ay th ro u g h the ratio
( b). This rate is followed by a smooth acceleration to about 6 responses per second
at the end of the ratio. T he high rates occur w hen the slit on the key changes by the
largest am ount per response, and the lowest rate where the slit changes least.
At the a rro w an injection of 0.5 cubic cen tim eter of saline was given in tra m u sc u
larly. T h e a p p a ra tu s was stopped, a n d the b ird was taken o ut of the box, injected,
FIXED RATIO
109
and replaced. T he brief interval required for the injection does not show in the rec
ord. T h e form of the segm ents following th e arrow shows little, if any, effect of this
disturbance.
The effect o f sodium pentobarbital on F R with added counter. T h e bird shown in Fig. 88
h a d been injected w ith 5 m illigram s of sodium p e n to b a rb ita l 2 sessions before Fig.
88 in a sim ilar procedure. Figure 89 shows the result. All segments before the pento
b a rb ital injection show the inverted S (for exam ple, a t a, b, an d c). Im m ediately
after injection (at the arrow ) a pause of about 1 m inute occurs, as after the controlled
saline injection in Fig. 88. T he bird begins responding again; but w ith the onset of
the d ru g less th a n 1 m inute later, the rate a b ru p tly drops to zero (d ). N o responses
occur for the next 1.5 hours, alth o u g h the bird is aw ake a n d m oving ab o u t in the
ex p erim ental a p p aratu s. W hen the b ird resum es responding a b ru p tly a t e, it com
pletes the ratio a n d begins the next w ithin a few seconds. D uring the rem aining
ratios of the session, the control previously exercised by the grow th of the slit is alm ost
com pletely lost. T h e first 4 ratios after injection show very short pauses after rein
forcem ent a n d a sustained interm ediate rate sim ilar to the earlier rate in the m iddle of
the ratio. At f an unusually long pause is followed by an ab ru p t shift to a very high
rate, w hich is m aintained until reinforcem ent.
This effect probably corresponds to the excitatory phase of the pentobarbital which
is com m only observed in clinical work and in fixed-interval schedules w ith this drug,
an d m ay also show some loss of the com plicated stim ulus control by the slit.
Block counter
In the preceding experim ents the slit on the key w hich serves as a counter increases
in size continuously as the bird pecks the key. A nother type of added stim ulus is pos-
110
sible in w hich the counter advances in discrete steps. For exam ple, the color of the
key qould change from red to orange after 30 responses, to green after an o th er 30
responses, a n d to blue after another 30 responses, w here a response to the key is rein
forced 30 responses later (at F R 120). Such a block counter corresponds to m any
schedules of reinforcem ent in education w here a certain am ount of work is required
to achieve a level of proficiency which is then the occasion w hen the next level of diffi
culty can be attacked. Such a counter is essentially a case of chaining (C hapter F our
teen). Each color is an occasion on which a response is reinforced by the next color
in the sequence on F R 30. T he final color is an occasion on w hich a response is rein
forced on F R 30 by the m agazine.
A nother case of the block counter is possible w here the color changes are not corre
lated consistently w ith the reinforcement. This is called an uncorrelated block
co u n te r. T h e sequence of colors occurs in a constant p a tte rn , b u t the cycle has no
relation to the appearance of the m agazine. A change in the counter reading m ight
be reinforcing because it shows progress tow ard the reinforcem ent even though it does
not indicate the distance still to be covered. If the num ber of responses required to
produce a change in color is a factor of the fixed ratio, one of the color changes will oc
cur simultaneously w ith reinforcement, and the first response after reinforcement will
always occur at a new reading of the block counter. In the experim ent to be described
here, the num ber of responses required to change the block counter was not a factor
of the fixed ratio. N o relation exists betw een any given stim ulus value and reinforce
ment.
Uncorrelated block counter. T he stable perform ance on F R 120 w ith a constant blue
key-light was first established, and the uncorrelated block counter was then added.
Figure 90 shows a badly strained perform ance on F R 120, w ith pauses after rein-
Fig. 9 0 .
Transitions from FR 1 2 0 to FR 120 + uncorrelated block counter
FIXED RATIO
111
forcem ent as long as 2 hours, occasional knees (as a t a ), an d curvature preceding the
term inal ru n (as a t b). T h e key-color was blue. O n the following day on a sched
ule of F R 140, a block counter was ad d ed consisting of a key-color change every 10
responses in a sequence of yellow, blue, white, green, yellow, etc. R ecord B shows the
resulting reduction in the am o u n t of pausing following the reinforcem ent. T h e over
all rate of responding is sharply increased; the rate difference is due to the elim ina
tion of the pause after reinforcem ent ra th e r th a n to an increase in the local rate.
Slight disturbances are evident as the key-color changes, a n d especially as they be
come m ore m arked in the 2nd excursion of the pen. R ough grain appears a t c and d,
ju st after color changes.
T h e effect of an uncorrelated block counter in elim inating the pause is not necessar
ily due to the reinforcing effect of a counter. Pauses a p p e ar to result from a very
unfavorable stim ulus ju st after reinforcem ent. T h e present counter tem porarily
breaks u p any contingencies betw een key-color a n d nonreinforcem ent.
Figure 91A shows a stable perform ance on F R 90 (steady blue key) for a second
Fig. 91.
Transitions from FR 9 0 to FR 9 0 + uncorrelated block counter (second bird)
112
bird in the experim ent, in w hich a consistent pause after reinforcem ent is followed by
an a b ru p t assum ption of the term inal rate. T h e uncorrelated block counter added
a t the beginning of the following session (R ecord B) produces a few brief pauses at color
changes (as a t a) a n d a tem porary reduction in pauses after reinforcem ent (as at b).
T he effect of the block counter disappears by the 2nd session, however. Pauses after
reinforcem ent increase, a n d the grain a n d term inal rate of the ratio retu rn to norm al.
No fu rth er effect of the uncorrelated block counter was observed in 5 further sessions.
R einstating the steady blue key h a d no effect. Presum ably, the uncorrelated color
changes now have no discrim inative power.
Correlated block counter. T h e fixed ratio was increased to 120 for both birds in the
preceding experim ent, the key rem aining blue. A block counter was then added; a
yellow key after reinforcem ent changed to blue after 35 responses, to w hite after 70
responses, a n d to green after 105 responses, the m agazine operatin g a t the 125th re
sponse. T he schedule m ay be regarded as 4 chained fixed ratios.
Figure 92 shows the early developm ent w ith a correlated block counter for the bird
Fig. 92.
Early developm ent o f correlated block counter
in Fig. 91. T h e correlated counter causes no im m ediate effect because of the history
of the uncorrelated block counter. T h e 1st session (R ecord A) produces no change in
perform ance. T he 2nd session (R ecord B) begins to show some effect o f the counter.
Color changes show in the fine g rain of th e record (as a t a, b, a n d c). R ecords C
a n d D are segments from the beginning and end of the 3rd session. T he correlation
between readings on the block counter and the schedule of reinforcement now produces
FIXED RATIO
113
a m ark ed increase in th e pause after reinforcem ent. (T h ere is now a least favor
able key-color.) T he running rate increases to approxim ately 6 responses per second
a t the end of R ecord D com pared w ith 4 per second in R ecord A, even though the
over-all rate falls.
This result is confirm ed by the second bird in the experim ent. Figure 93 gives seg
m ents from the first 3 sessions w ith the correlated block counter. T h e block counter
(R e c o rd A) causes no im m ediate effect, because o f the history of the uncorrelated

block counter. In Records B an d C, segments from the 2nd an d 3rd sessions, there
is a progressive increase in the pause after reinforcem ent, as well as the com pensating
increase in the ru n n in g rate. T he increase in the term inal rate is not sufficient to
m aintain the over-all rate, however.
In la te r sessions th e pausing produced by the correlated block co u n ter tends to
disappear during the session, returning again on the following day. Figure 94A shows
a sample for the bird in the preceding figure, taken after 720 reinforcements with the
block counter. N ot only do th e pauses becom e short as the session proceeds, b u t the
term inal rate increases. This p a tte rn is the reverse of the usual order of events in a
fixed ratio of substantial size, w here the session custom arily begins w ith very short
pauses an d a high over-all rate, which then declines during the session. U pon return-
RESPONSES
770
Fig. 95.
D aily sessions showing progressive decrease in the pause a fte r reinforcem ent
FIXED RATIO
115
ing to the steady blue key, the birds show an im m ediate reduction in the pauses after
reinforcem ent a n d some increase in ru n n in g rate. In Fig. 94B the key is blue for the
first tim e following the history of the block counter ju st discussed. T he running rate
is of the order of 6 responses per second com pared w ith from 4 to 5 responses per second
in R ecord A. This reduction is tem porary, how ever; pausing after reinforcem ent
and a lower running rate return. Record C shows a segment from the 6th session w ith
th e steady blue key.
O ne of these birds showed a continuing tendency to reduce the length of pause after
reinforcem ent throughout the session. Figure 95 presents 4 com plete sessions taken
approxim ately 3 m onths after the perform ance shown in Fig. 94, during which the bird
h ad been on F R 110 w ith a steady blue key. T h e other bird returned to a very sta
ble perform ance under the same circumstances, w ith a slight pause after reinforcem ent
an d curvature at the start of the fixed-ratio segment. T he last 8 sessions of this pro
cedure for this bird are shown complete in Fig. 96, indicating very uniform perform
ance from session to session. (N ote th a t in Fig. 96 a faster p ap er speed is used to
reduce the over-all slopes of the curve and to am plify local features.)
Fig. 9 6 .
Eight successive d a ily sessions on FR 110
116
Summary
T h e correlated block counter tends to accen tu ate the pause after reinforcem ent,
while at the same tim e it increases the term inal rate of responding. In Fig. 96, in p a r
ticular, the pause becomes rem arkably uniform . Even w here the size of the pause
changes progressively through the session (Fig. 95), any local portion shows an unusual
degree of uniform ity. This characteristic m ay be explained as the effect of the first
key-color in the ratio a n d its unfavorable position in the reinforcing contingencies.
T he block counter differs from the continuous counter in th a t it shows less stim ulus in
duction betw een extrem e readings.
TIME OUT IN THE ANALYSIS OF FIXED RATIO
A tim e out (T O ) as an analytical tool has been described in C h ap ter T hree. Its
relevance to the study of fixed-ratio schedules is obvious. W hen a T O of sufficient d u
ration occurs after reinforcem ent, it reduces the stim ulus control from the preceding
ratio perform ance, p articu la rly the influence of the n u m b e r of responses a n d rate as
well as the reinforcem ent itself, which can be shown to be an effective stimulus for a t
least 30 seconds after it has occurred. W hen a T O is used as a probe d u rin g a fixed
ratio, it interrupts current controlling stim uli in the ratio behavior. If a T O is pro
gram m ed regularly, it will acquire controlling properties from its relation to reinforc
ing contingencies. W hen used as an occasional probe, however, this is unlikely.
T im e out was carried out in all of the experim ents w ith the pigeons by tu rn in g out all
th e lights in the experim ental cham ber. T h e T O has been applied to the problem
of F R in a series of experiments.
Time out a fte r reinforcement
Early development after erf Figure 97A shows a sam ple o f a perform ance on F R 50
during the 2nd session after erf. T he over-all rate is approxim ately 3 responses per
FIXED RATIO
117
second; slight pauses follow reinforcem ent, b u t responding is otherwise a t a single high
rate. O n th e following day (R eco rd B) a 1-m in u te T O followed every reinforce
m ent. (T he recorder was stopped during the T O .) T he pause after reinforcem ent is
im m ediately elim inated, an d the term inal rate is slightly increased. T h e over-all rate
is now 4 per second, a 33% increase over R ecord A. T h e next session (R ecord C)
shows approxim ately the same rate; b u t in the following sessions (Records D and E) the
over-all rate increases to alm ost 8 responses per second. (T he counter used to p ro
gram the ratio does not follow the rate accurately.)
A second bird showed the sam e result, though not so clearly (Records F, G , H , I,
a n d J ). Pauses still occur at the start of the fixed-ratio segment in spite of the T O , b u t
th e over-all ra te nevertheless increases from slightly over 2.5 responses p er second
(R ecord F) to almost 4 per second (R ecord J ). A sim ilar result for a third bird is
show n in R ecords K , L, a n d M , in segm ents from th e 2nd, 3 rd , a n d 4th sessions after
erf, w here th e ad d itio n of a T O 1 after reinforcem ent in R ecord M elim inates the
rough grain.
Time out after blocks o f reinforcements. In an o th er experim ent, perform ances under F R
a n d F R T O were com pared w ithin a single experim ental session. Blocks of 10 ratios
in w hich reinforcem ent was followed by a 3-m inute T O altern ated w ith blocks of 10
ratios w ithout T O . A stable perform ance developed after several sessions in birds w ith
a history (following erf) of 42 sessions d u rin g w hich th e value of F R was increased
from 20 to 60. Figures 98A a n d 98B show com plete sessions for 2 birds w ith alternate
blocks of T O . T h e T O s are indicated by small dots to the left of the reinforcements
they followed.
T he T O after reinforcem ent consistently produces a higher rate of responding.
T his m ay best be seen if th e g ra p h is foreshortened. T h e b ird in R ecord A shows a
higher rate of responding u n d e r T O . A sim ilar increase in over-all rate in R ecord
B is due p a rtly to the elim ination of the low rate after reinforcem ent.
A t larger fixed ratios, T O after reinforcem ent m ay increase the pause a t the start
o f th e fixed ratio ra th e r th a n decrease it as in Fig. 98. T h e ru n n in g rate, how ever,
m ay still rem ain higher following T O . R ecord A of Fig. 99 shows a perform ance on
F R 85, 11 sessions after th e perform ance show n in Fig. 98A. T h e over-all ra te is
low er for those segm ents w ith T O after reinforcem ent, b u t instances m ay be found
w here for b rief periods the ru n n in g rate is higher th a n in any segm ent w ithout T O .
(N ote th a t over-all and local rates of responding are unusually low for a fixed-ratio p e r
form ance after so m uch exposure to these schedules.) T h e second bird (R ecord B)
showed a considerably higher over-all rate of responding a n d frequent pauses following
th e reinforcem ent, w here there was no tim e out. T h e effect of T O here is both to
increase the local rate of responding a n d decrease the am ount of pausing a t the start of
th e fixed ratio.
F u rth er exposure to F R 110 T O after reinforcem ent produces pauses after the rein
forcem ent, both w ith and w ithout T O . Figure 100 shows a com plete session on
F R 110, 2 sessions after th e perform ance show n in Fig. 99A. P ausing is m ost ex-
Fig. 9 9 .
TO a fte r blocks o f FR 85
FIXED RATIO
Fig. 100.
119
TO a fte r blocks o f FR 110 showing marked pausing
trem e w hen T O s follow reinforcem ents, a n d no pauses occur in the first few segments
in each block of F R w ithout T O . T h e ru n n in g ra te rem ains higher in segm ents
preceded by T O .
Perform ances w ith a n d w ithout T O were studied further by the scheduling of com
plete experim ental sessions under each condition. Figure 101B shows the 5th session
^M U M O TMMP.
10 MINUTES
Fig. 101.
A lte rn a te d a ily sessions on FR w ith and w ith o u t TO a fte r reinforcem ent
120
w ith a 3-m inute T O after every reinforcem ent. (T h e sam e bird was used as in Fig.
100.) T h e over-all rate of responding declines progressively during the session,
m ainly because of the increasing pauses after reinforcem ent. T he session continued
for m ore th a n 1 h o u r beyond th e perform ance show n in th e record, d u rin g w hich
only 1 additional reinforcem ent was received. M arked negative curvature appears a t
c, and an interm ediate rate at d. In the following session, which is shown in its entirety
in R ecord A, T O was withheld. N egative curvature (as a t a a n d b) appears m ore
frequently th a n in R ecord B. Some pausing occurs after nearly every reinforcem ent,
b u t never for m ore th a n a few m inutes.
T he other bird, which showed a generally higher over-all rate of responding, yielded
sim ilar results. Figure 10ID shows the com plete 8th session on F R 170, in which a
3-m inute T O followed every reinforcem ent. T he result is a high over-all rate, which
declines during the session because of a n increasing tendency to pause after reinforce
m ent. A 30-m inute period at the end of the session is not shown, in w hich only 7
ratios were completed. Some positive acceleration during the early p art of a ratio b e
comes pronounced tow ard the end of the session (as a t a). W hen T O was w ithheld in
th e following session (R ecord C), pausing develops early in the session; b u t it con
tinues a t ap p ro x im ately the sam e level w ithout progressive increase, as in R ecord B.
T h e term inal rate of responding is approxim ately 6 responses per second in R ecord
A a n d 8 per second in Record B.
W here T O after reinforcem ent produces m arked pausing on a short ratio, elim i
natin g th e T O often reduces the pause dram atically. Figure 102 shows this effect.
Record A is the final p art of a session showing m arked pausing with a T O after all rein
forcements. Ju st after the start of the following session in R ecord B, the elim ination
of the T O a t the arrow elim inated the pause except in the 1st segm ent, w hen it was a
few seconds long.
The effect o f an extraneous stimulus on an F R performance with time out after blocks o f rein
forcements. In 1 experim ental session in w hich T O s occurred in alternate blocks of 10
reinforcem ents (see Fig. 97, 98, a n d 99), a loud buzzer was sounded continuously.
This extraneous (and possibly aversive) stim ulus disrupted the perform ance during the
segm ents preceded by T O , b u t it h a d little effect u p o n the perform ance w hen T O
was om itted. C om plete sessions for the 2 birds are shown in Fig. 103. Both records
are from the 8th session on the alternate T O procedure (2 sessions after Fig. 98). T he
over-all rate falls by a very large factor w hen T O s follow reinforcem ents, b u t it is very
largely unaffected in the absence of T O . T h e only deviations from a norm al fixedratio perform ance occur a t c an d d, w here the ratios begin in a norm al m an n er b u t
show a m arked decline in rate before the end. This is the same kind of deviation p ro
duced after T O . Segments tend to begin w ith a high rate. T he rate then sharply
declines to near zero until reinforcem ent (as a t a). Segments which begin with a pause
or low rate m ay show an S-shaped curve, as a t b. In general, the perform ance d u r
ing each fixed-ratio segment suggests an extinction curve, b u t the over-all p attern is ex
trem ely ragged an d the grain rough. R ecord B also shows a very gross difference in
300 RESPONSES
Fig. 103.
The e ffe ct o f a loud buzzer on FR w ith TO a fte r blocks o f reinforcements
122
the over-all rate betw een fixed-ratio perform ances w ith a n d w ithout T O after rein
forcem ent, although the character of the disturbance is not so uniform.
Time-out probes
Occasional probes. T h e effect of occasional T O probes on fixed-ratio behavior is not
well enough understood for them to be classified. T he following figures, however, p re
sent some examples of the kinds of effects.
Figure 104 shows the effect of three 8-m inute T O probes on a well-developed p er
form ance on F R 60. This bird had a history of 33 sessions on fixed ratios of from 20
to 60. Probes had been used in the preceding session. T he T O at b followed rein
forcement; those at a and c occurred a few responses after the start of the fixed-ratio seg-
Fig. 104.
TO 8 probe o f FR 6 0 producing
pausing
FIXED RATIO
123
m ent. T he fixed ratio is being held well, w ith no pause after reinforcem ent and good
grain. T h e T O produces a pause in each case, varying from 1.5 m inutes a t a to 4
m inutes at b. After the pause, responding begins at the term inal fixed-ratio rate, and
the segment is ru n off in a norm al m anner.
Figure 105 shows 5 segments from the 2nd session on F R 90 after 190 hours on tan d
F R F I, a n d shows a sim ilar effect of a 2-m inute T O after reinforcem ent. T he T O fol
lowed every 10th reinforcement. Tw o ratios preceding the T O and one following it
are shown for all T O s in the session. In the 1st segm ent the T O is followed by a 30second period a t a low rate, w ith acceleration to the term inal rate. T he other T O s
produce a pause of from 15 to 30 seconds.
Figure 106 shows the behavior of a second b ird in the early stages of F R 160 fol
lowing ta n d FZ?FI. Records A, B, C, a n d D are sam ple segments from several ses
sions in which 2-m inute T O probes followed occasional reinforcements. In Records
A, C, a n d D the T O elim inates the pause completely, while in R ecord B it shortens
it. In Records E, F, a n d G the T O s occur during fixed-ratio segments. In Records
124
E and G the result is a drop to a lower rate followed by acceleration to the term inal
rate. In R ecord F a short burst a t the term inal rate im m ediately follows the T O ,
but the rate falls to an interm ediate level for a brief period.
Figure 107 shows 2 successive sessions on F R 4 0 , beginning 12 sessions after con
tinuous reinforcem ent, in which a 2-m inute T O followed every 11th reinforcem ent.
T he only effect in R ecord A is a slight lowering of the rate after the 3rd and 5th TO s, a t
a a n d b. In the following session (R ecord B) all T O s except the 2nd have a m arked
effect. At c a pause occurs n ear the end of the ratio; at d the rate drops from 3.75
responses per second to approxim ately 2 per second for the whole ratio; a t a cusp
follows a substantial pause and acceleration after T O ; and a t / a slight pause appears.
T h e perform ance then deteriorates.
Figure 108 shows the complete 8th session on F R 5 0 after erf, in which every 11th re
inforcem ent is followed by a 2-m inute T O (indicated by vertical dashed lines above
reinforcements). T he considerable pausing after reinforcement and the low term inal
rate show th a t this is not a final perform ance for this size of fixed ratio. It is neverthe
less fairly stable at this stage. In every case except a t d the T O reduces the pause after
reinforcem ent, an d in most cases (as, for exam ple, at a, b, a n d c) elim inates it.
Figure 109 shows 2 fixed-ratio segments before and 3 segments after each T O in a
125
FIXED RATIO
10 M I N U T E S
Fig. 108.
Effect o f TO 2 on FR 5 0 showing pausing
later series w ith the bird in Fig. 108. T h e ch aracter of the fixed-ratio behavior is now
quite stable a n d is fairly typical, w ith either a pause of several seconds followed by a
very slight a n d rap id acceleration to the term in al rate, or an im m ediate start after
reinforcem ent w ith im m ediate or very rap id assum ption of the term inal rate. A l
most w ithout exception, an 8-m inute T O elim inated the pause following reinforce
m ent. It often reduced the am ount of pausing after the next reinforcem ent as well.
T he term inal rate rem ained unchanged.
Continuous sample o f time-out probes after reinforcement and during the fixed ratio. A series
of 25 sessions w ith 2 pigeons was arranged as follows: an 8-m inute T O followed the
5th reinforcem ent of the session; a 30-second T O , the 25th response after the 13th re
inforcem ent; and a n 8-m inute T O , 25 responses before the 20th reinforcement of the
session. This arrangem ent was a n attem p t to com pare the effect of duration of T O
both after reinforcem ent a n d during the ratio.
Fig. 109.
TO 8 probe o f FR 5 0 e lim inating pausing a fte r reinforcem ent
126
Figure 110 shows the perform ance of one bird in the 11th, 16th, and 25th sessions
u nder this procedure on FR s of 85, 125, and 125, respectively. This bird had been
on a schedule sim ilar to th a t in Fig. 107, 108, a n d 109, in which a short F R was probed
by a T O after every 11th reinforcem ent for 22 sessions. T he 8-m inute T O following
the 5th reinforcem ent of each session elim inates or reduces the pause after reinforce
m ent (a, d, a n d g) a t the large arrows. T h e 30-second T O occurring 25 responses b e
fore the 12th reinforcem ent produces a slight cusp a t b, a n d a m ore pronounced pause
a n d acceleration at e and h.
A t c, f an d i the 8-m inute T O , 25 responses after the
20th reinforcem ent, produces a slight shift in rate a t c, has almost no effect at f , and
produces a pause and abrupt shift to a higher rate at i.
Figure 111 shows the second b ird on this procedure. Tw o excursions of the re
corder are shown for the 6th session, an d segments from 3 other adjacent sessions were
added. In R ecord A the 8-m inute T O after the 5th reinforcem ent eliminates the pause
after reinforcem ent which occurs a t this size of fixed ratio. T here is no effect on the
subsequent ratio. At R ecord B the 30-second T O occurring 25 responses before the
13th reinforcem ent produces a slight cusp a n d a decline to a lower rate, sim ilar to the
effect occurring in Fig. 110 at c, e, an d h. R ecord C shows the 8-m inute T O occur
ring 25 responses before the 20th reinforcement. A cusp and lower term inal rate of re-
FIXED RATIO
127
sponding occur in 2 cases, while only a slight shift to a lower rate occurs in the others.
Figure 112 shows a complete series of T O s on the same procedure (for the same bird
as th a t in Fig. I l l ) 13 sessions after the size of the ratio was increased to 120. T h e
fixed-ratio segm ents im m ediately preceding a n d following the T O s are presented for
th e following session also. In R ecord A the 8-m inute T O following the 5th reinforce
m ent still characteristically reduces or elim inates the pause, as a t b ; b u t occasional in
stances occur in which there is little if any effect, as at a. In R ecord B the 30-second
T O d u ring the fixed ratio produces a slight cusp a t / o r a m ore m arked effect at g.
T he 8-m inute T O in Record C, however, frequently has no effect, as a t h, or produces a
pause, as at k. T h e large pause at k is from a session during which there was great
difficulty in m aintaining a perform ance on this size of fixed ratio a n d in which long
pauses occurred following reinforcem ent. A t this stage in the schedule, moreover, in
term ediate rates occur (for exam ple, a t i a n d j ) , as well as instances of smooth posi-
128
tive acceleration (at c, d, and e). This deterioration in the fixed-ratio performance is
probably due to the punishing effects of T O , sim ilar to the effect in percentage rein
forcem ent w ith T O .
A n 8-m inute T O after reinforcem ent elim inated or considerably reduced the pause
in all of the cases studied. T h e 30-second T O occurring before the end of the 13th
fixed ratio alm ost always produced a cusp and decline to a lower rate; b u t an 8-m inute
T O in a sim ilar position later in the session frequently h ad little effect.
Summary
U n d e r some conditions a T O after reinforcem ent will consistently elim inate the
pause characteristically observed in a ratio perform ance. This effect will obviously
not be observed if the perform ance shows no pausing; it is likely to be obscured by
pauses due to other causes which m ay resemble the characteristic ratio pause.
U n d er some conditions, however, a T O produces pauses in a fixed-ratio performance.
Introduced a t times other than after reinforcement, it m ay produce some slight pausing
and scalloping before returning to the norm al high rate of responding. Some collat
eral effects of T O which m ay explain these are discussed in C hapter Three.
W hen the T O reduces a characteristic pause, the effect is clear and unam biguous.
Failure to get this result, however, does not m ean very m uch in our present state of
knowledge.
The e ffe c t o f time out on FR 875
C h a p te r E ight shows th a t after a history of ta n d F I 45 F R , in w hich the tan d em

ratio is continuously increased from 10 to 400, 2 pigeons were able to respond in a
sustained fashion on F R 875. E arly perform ances on F R 875 w ith T O 15 after rein-
FIXED RATIO
Fig. 11 3.
Transition from FR 8 7 5 TO 2 0 to FR 8 7 5
129
113
forcem ent are described in C h a p te r E ight in Fig. 522. T h e beginning of Fig. 113
shows a stable perform ance in one bird after a total of 80 reinforcements on F R 875.
(T he recorder resets during reinforcement.) A term inal rate of 3 responses per second
is reached after an acceleration extending roughly over the first 200 responses.
M easurem ent of the pause before the 1st response in the last 51 fixed-ratio segments
w ith T O after each reinforcem ent showed 21 cases of a 1st response w ithin 2 m inutes,
11 cases of a response w ithin 4 m inutes, a n d 17 cases in w hich the 1st response oc
curred m ore th an 4 m inutes after the term ination of the T O . T he tim e required to
com plete the entire fixed-ratio segment follows the same pattern. O u t of the 51 cases,
20 ratios were ru n off w ithin 10 m inutes; 16, w ithin 15 m inutes, and 10, between 15
and 25 minutes. A t the arrow in Fig. 113 the T O was discontinued. T he bird begins
to respond a t the term inal rate im m ediately after reinforcem ent and m aintains th at
rate until the next reinforcement. Twenty-five further reinforcements were received in
the rem ainder of the session not shown in the figure. In each case, responding be
gan almost im m ediately after reinforcem ent a n d continued a t a constant rate until the
next reinforcem ent. At a later stage, long pauses w ith prolonged acceleration to the
term inal rate occasionally occur. Figure 114 shows a segment from the following ses-
Fig. 114.
Later perform ance on FR 8 7 5 a fte r FR 8 7 5 TO 2 0
130
sion, 75 reinforcem ents after the last reinforcem ent shown in Fig. 113. A long pause
appears at b; b ut frequent instances still occur in neighboring segments w here the bird
assumes the term inal rate im m ediately after reinforcem ent a n d m aintains it th ro u g h
out the ratio, as at a a n d c. O f 170 reinforcem ents recorded under F R 8 7 5 w ithout
T O for this bird, only 13 were followed by pauses and acceleration, as at b.
T he final perform ance for the second bird u n d er F R 8 7 5 w ith a T O 15 after rein
forcem ent is also shown in Fig. 523, discussed in C h a p te r Eight. T he pause after re
inforcem ent was m easured for the last 35 fixed ratios w ith T O after reinforcem ent.
In 32 instances it was m ore th an 20 m inutes long, and in the rem aining 3 cases was over
10 m inutes. T he first omission of T O occurred at the beginning of the session shown
in Fig. 115. T he first fixed-ratio segm ent has been om itted from the record because
Fig. 115.
Later perform ance on FR 8 7 5 a fte r FR 8 7 5 TO 2 0 (second bird)
the start of the session functions as a T O . It begins after the usual pause an d slow
acceleration to the term inal rate. After the first reinforcem ent not followed by T O , a
pause of less th an 1 m inute (at a) occurs. R esponding begins almost at once at the
term inal rate a n d continues until the next reinforcem ent. T h e 4th segment, begin
ning at b, shows interm ediate rates of responding and a m arked knee at c. T he te r
m inal rate is sustained for only the last 200 responses of the ratio. D uring the rest
of the session (not shown) the pausing increases progressively, although very few in
stances occur where it reaches the order of m agnitude observed with T O after rein
forcement.
T he rem ainder of this perform ance for this bird on F R 8 7 5 w ithout T O was c h a r
acterized by groups of ratios ru n off w ith little or no pause, a n d very rap id assum p
tion of the term inal rate interspersed w ith instances w here pauses as long as 6 hours oc
FIXED RATIO
131
curred, followed by prolonged acceleration. W hen 15-m inute T O s were again

introduced after reinforcem ent, the perform ance retu rn ed to norm al as in Fig. 523
(C hapter Eight).
Summary o f time out on FR875
Both birds confirm the effect of the tim e out. W ith the tim e out after every rein
forcem ent, consistent pausing a n d a period of acceleration to the term inal rate occur.
W ithout the tim e out one bird shows groups of ratios run off with no pause or
acceleration, interspersed w ith long pauses a n d slow, rough grain extending through
most of the ratio. T he other bird shows a sim ilar effect of the rem oval of the tim e
out, except th at instead of the groups of ratios being run off w ith no pause following
the reinforcem ent, the am ount of pausing is considerably less th an w ith tim e out.
Also, the long pauses a n d slow accelerations to the term inal rates, which are in te r
spersed in the m ore rapidly run-off ratios, follow the longer pauses (as long as 6 hours).
T he omission of the tim e out in the large fixed ratio is sim ilar to the effect in large
fixed-interval schedules. H ere, the carryover from the preceding interval produces
im m ediate responding after the reinforcem ent instead of the appropriate pause.
The effect o f time out on short fixed ratios with an insufficient amount
o f reinforcement
Figure 116 shows instances in which T O probes of an early fixed-ratio perform ance
were followed by long pauses. Records A and C are for one bird, 5 sessions after erf;
an d Records B, D, an d E are for another bird, 3 sessions after erf. An undeveloped
132
fixed-ratio perform ance an d the insufficient reinforcem ent are suggested by the in ter
m ediate rates a n d pauses th a t occur even after the bird begins to respond a t the te r
m inal rate.
A sim ilar result has already been described in Fig. 39, w here the first and last T O
produced a long pause. O th e r T O probes in Fig. 38 a n d 40 had, if anything, the op
posite effect.
Chapter Five
FIXED INTERVAL1
INTRODUCTION
I n a f ix e d -in t e r v a l s c h e d u l e
of reinforcem ent (FI) the first response after a desig

n ated interval of tim e is followed by a reinforcing stimulus. It is program m ed by a
tim er w hich starts from zero after each reinforcem ent (or from the start of the session)
an d closes a circuit ( sets u p a reinforcem ent) a t the end of a designated tim e. T he
first response following this period operates the m agazine.
Since the reinforced response m ay occur some tim e after the reinforcem ent is set up,
we have the option of tim ing the next interval from the reinforcem ent or from the end
of the preceding interval. By tim ing from the end of the previous interval, we m ain
tain the designated fixed interval as an average about which the actual intervals will be
distributed. If the interval is tim ed from the last reinforcem ent, none of the intervals
will be less th a n the designated fixed interval. Some will be larger, however, and the
average interval will exceed the designated interval. In p rac tic e , there is little differ
ence betw een these procedures, since fixed-interval schedules norm ally generate a sub
stantial rate of responding at the tim e of reinforcem ent, and the reinforced response
usually occurs w ithin a second or two of the designated fixed interval. In all the ex
perim ents reported here the fixed interval is tim ed from the reinforced response, so
th a t no instance occurs of a response being reinforced after less th an the designated
fixed interval.
CONTINGENCIES RESULTING FROM FIXED-INTERVAL REINFORCEMENT
Differential reinforcement o f low rates
In C hapter Four we pointed out th a t fixed-ratio schedules increase the num ber of
instances w here a reinforced response is im m ediately preceded by another response or
group of responses rath e r th an by a pause. This indirect contingency results in a dif
ferential reinforcem ent of high rates. A fixed-interval schedule has the opposite effect
of differentially reinforcing responses following pauses th at is, lower rates of respond
ing. In the fixed-interval schedule, reinforcem ent is program m ed by elapsed time;
1 T h is is the periodic reinforcem ent of The Behavior o f Organisms.
133
134
therefore, the longer the interval since the last response, the m ore likely the next re
sponse is to be reinforced. This contingency operates only when responses occur in
groups. In both fixed-ratio a n d fixed-interval schedules the indirect contingency
would not be felt if the rate were constant. A nd in both schedules, as we shall see, the
stable perform ance shows a nearly constant rate in the region of the reinforced re
sponse; thus, this indirect contingency can be appealed to only during the developm ent
of such a perform ance or in transitions from one schedule to another.
Rate o f responding a t the moment o f reinforcement
As will be seen later, a fixed-interval schedule norm ally generates a stable state in
which a pause follows each reinforcement, after which the rate accelerates to a term inal
(usually m oderate) value. In the stable state a given rate of responding is correlated
w ith reinforcem ent. M oreover, another (low) rate is correlated w ith nonreinforce
m ent at and near the beginning of the interval. A lthough reinforcem ent does not a l
ways occur whenever the rate reaches its term inal value, th at rate of responding
nearly always accom panies reinforcem ent. O ccasional instances w here the bird does
not respond th roughout the interval a n d is reinforced for a single response w eaken
this correlation. T he extent to w hich this contingency is a controlling factor in the
fixed-interval perform ance depends on the stage of developm ent of the performance.
Number o f responses emitted in the fixed interval
In a quite stable performance, in which the rate is low or zero after reinforcem ent
b u t increases in a standard fashion as the interval elapses, the n um ber of responses
em itted at the reinforcem ent should be fairly constant. Since this condition depends
in a sensitive way upon the triggering of the acceleration to a higher rate, the num ber
of responses at reinforcem ent will usually vary considerably. If either the num ber
of responses em itted since reinforcem ent or the rate of responding a t reinforcem ent is
an im p o rtan t factor in fixed-interval contingencies, we m ust allow for the chaining
of responses. T hus, if a given n u m b er of responses characteristically precedes rein
forcem ent, any responding which brings the actual n um ber closer to this value should
be reinforcing. N ote th at any such effect should produce an oscillating state, since
the increase in rate due to chaining should upset the num ber contingencies responsible
for chaining. A similar argum ent m ay be m ade for rate.
Time since the preceding reinforcement
This is constant in a fixed-interval schedule, b ut it probably acts through m ediat

ing behavior. T h e responding discussed in the preceding p arag rap h is the principal
exam ple of such m ediating behavior.
The reinforcem ent as an occasion fo r nonreinforcem ent
T he stim uli associated w ith the presentation of a reinforcer and w ith the a p p ro
priate consum m atory behavior (eating, cleaning, etc.) enter into the fixed-interval
FIXED INTERVAL
135
contingencies in an im portant way. Because they constitute an occasion upon which

a response is never reinforced, a low rate quickly develops im m ediately after rein
forcem ent. T h e d u ratio n of this control is in p a rt a function of the tem poral proper
ties of the stim uli. R esidual stim uli from food in the m outh, swallowing, etc.
m ay extend past the m om ent of reinforcem ent. O th er behavior m ay be set in motion
(e.g., w ashing for the rat) w hich m ay also control a low rate of responding because
of its relation to nonreinforcement. V ery roughly speaking, the effect of reinforcement
as a stim ulus of this sort appears to last about 30 seconds for the pigeon. T he effect
is to start the new interval with a period of zero or a very low rate of responding.
TRANSITION FROM CONTINUOUS REINFORCEMENT TO A FIXED-INTERVAL
SCHEDULE
Introduction
W hen a bird w hich has previously been reinforced on continuous reinforcem ent
(erf) is placed directly on a fixed-interval (FI) schedule, a characteristic perform ance
is usually observed. Figure 117 illustrates the im portant features.
Fig. 11 7.
Stylized curve o f the transition from erf to FI
1. T he preceding erf produces a negatively accelerated extinction curve, suggested

by the dashed curve a t a. These curves are quite variable with respect to the n u m
ber of preceding reinforcem ents, as unpublished experim ents by W . H. M orse have
shown. W hen the preceding erf has a substantial effect, however, the first p a rt of the
fixed-interval perform ance is negatively accelerated. T he rate usually reaches a low
level of responding, as at c, which is considerably below the rate which will eventually
be m aintained by the fixed-interval schedule of reinforcement.
2. Reinforcem ents are now being received on schedule at each vertical line. (R e
inforcem ents are usually delayed at this stage because the bird is not responding when
a reinforcem ent becomes available, as in the interval at c. T he next interval, at e,
is tim ed from the reinforcem ent d.)
136
E ach such reinforcem ent is generally followed by an increase in rate, an d the in

terval is usually m arked by a small negatively accelerated segment (b, b . .). T he
larger negative acceleration a ttrib u te d to extinction is com bined with these sm aller
curves. T h e cu rv a tu re is not often very uniform because of the rough grain usually
prevailing at this stage.
3. A fairly uniform rate of responding emerges during an interval and from interval
to interval, as at e. This constant rate seems to develop regardless of the size of the
interval, a n d is presum ably due to the special probability of reinforcem ent a t low rates
arising from the contingencies up to this point.
4. Because of this uniform ity in rate, the n u m b er of responses at reinforcem ent be
comes fairly constant. This condition appears to produce the fourth characteristic
occasional b rief runs at higher rates, as at f
These are not the com pensatory high rates
which follow pauses (g). T he rates are often as high as in early ratio performances.
(See C h a p te r Four.) No instances of rates this high have been observed up to this
point, nor, of course, have such rates ever been reinforced. T he brief runs appear,
therefore, to be due to the autom atic reinforcem ent resulting from progress tow ard the
num ber of responses characteristically prevailing at reinforcement. Since such a run
destroys the constancy of this num ber, the situation is unstable.
W hen a pigeon is placed directly on FI after erf, the perform ance will depend in
p a rt upon the am ount of extinction resulting from the erf, the distribution of inter-re sponse times or the tendency for pecks to occur in bursts (the grain of the record),
a n d the accidental contingencies resulting from the interaction of this perform ance
w ith the schedule of reinforcem ent. Seventeen pigeons, after varying am ounts of erf,
were placed directly on FI schedules ranging from 1 to 45 m inutes. Eleven of these
transitions will be described to show particular features.
Transitions
Figure 118 shows a perform ance u n d e r FI 1 after erf w hich begins

with only a small extinction curve (a). (T he 4 lines of the figure were recorded con
tinuously.) T h e lowest rate of responding occurs at b. T hen the rate becomes fairly
constant in the region c. A short b u rst of responding a t a higher rate occurs at d
a n d m uch later a t e (cf. Fig. 117/). T h e over-all rate during the session rem ains low,
and the grain of the record is rough. T here is already some tendency tow ard a lower
rate ju st after the reinforcem ent (e.g., a t f , g, h, i, a n d j ).
Figure 119A shows the 1st session of a transition, erf to FI 1, which begins with a
m uch higher initial rate of responding. T he first 3 intervals average about 100 re
sponses per interval. T he rate then falls. At a a n d b the reinforced responses occur
after pauses. U n fortunately, the curve is too short to show other features; b u t in
the following session the over-all rate continued to decline, and it reached a stage sim
ilar to the last segment of Fig. 118.
R ecord B in Fig. 119 shows a sim ilar transition to F I 2 after erf. T he over-all nega
tive cu rv a tu re at the beginning of the session shows the effect of the previous con-'
F I 1 and F I 2.
FIXED INTERVAL
137
5 M IN .
Fig. 1 18.
Fig. 119.
Transition from erf to FI 1
Transition from erf to FI 1 and FI 2
tinuous reinforcem ent. T he negative curvature w ithin each interval is the result of
th e in te rm itte n t reinforcem ent. T he linear phase of the typical perform ance ju st
begins to appear at c.
F I 4. Figure 120 shows an exceptionally irregular transition from erf to FI 4.
T his bird started slowly u n d e r erf. T h e present session begins w ith a pause of m ore
138
th a n 1 m inute at a. T he 1st interval contains only a single burst of responses, and

the 1st reinforcem ent (at b) occurs after a pause of more than 2 minutes. It is followed
by a small, negatively accelerated curve. Before the next reinforcem ent at c, how
ever, the bird begins to peck rapidly, a n d the next interval shows a large, negatively
accelerated curve. T h e reinforcem ent (at d) is received after a pause of a b o u t 2
m inutes, and 4 intervals thereafter a prolonged pause precedes the reinforced response
at e. Responding begins im m ediately after reinforcem ent at the highest sustained rate
yet seen, a n d a large, negatively accelerated curve extends through the interval to
the next reinforcem ent at f . T hereafter, a m ore uniform rate of responding emerges
w hich yields a stable over-all rate before the end of the session, betw een g a n d h.
R esponding d u ring this period is characterized by rough grain, which is due to short
bursts of responding separated by pauses.
Figure 121 shows a transition from erf to FI 5, w ith a rap id developm ent of a high
over-all rate of responding. T he negatively accelerated curve due to the preceding
erf extends through the first 2 segments. T h e first 10 segments show negative cu rv a
ture within each interval. As the session progresses, the rate imm ediately after rein
forcement falls, and beginning at a the over-all curve is nearly linear. T he m ean rate
has increased to about 0.6 response per second, com pared with 0.5 response per sec
ond a t the start of the session. As the over-all perform ance becomes m ore linear to
w ard the m iddle of the session, instances of brief periods of rapid responding occur a t
b, d, and i, an d m ore sustained high rates at c a n d e. M any short pauses are followed
by com pensatory increases in rate, w hich give the impression of a nick or bite in a
sm ooth over-all curve, as at f , g, h, i, a n d j . Nicks also ap p e ar in the other direction,
at b an d k, w here a b rief m om ent of ra p id responding is followed by a pause. T h e
grain of the record rem ains rough throughout this 1st session.
FIXED INTERVAL
139
F I 8. Figure 122 shows a transition from erf to FI 8. T he first p a rt of the curve,

to the reinforcem ent at a, is negatively accelerated as a whole; and each interval shows
a sim ilar acceleration. T he rate of responding im m ediately after the reinforcement
becomes lower an d sustained for fewer num bers of responses. T he rate is roughly
lin ear for the rem a in d e r of the session, w ith the over-all rate increasing gradually.
Bursts of responding at rates which have never been reinforced occur at b, c, and d.
By the end of the session, a pause of the order of 10 or 15 seconds appears after each
reinforcem ent, and the linear perform ance gives way to m arked oscillations, as at e.
F I 17. In the transition from erf to larger fixed intervals, extinction has a greater
140
Fig. 123.
o p p ortunity to occur. V ery low over-all rates of responding m ay prevail before the
final perform ance on the schedule is developed. Figure 123 shows a transition from
erf to FI 17. T he top line comprises a large extinction curve consisting of over 500
responses, m ainly due to the previous erf. T he bird is still responding at an in te r
m ediate rate at the 1st reinforcem ent, at a. In the 2nd interval the rate reaches zero,
and the 2nd reinforcem ent, at b, occurs after a 6-m inute pause. T he reinforcem ent
reinstates a high rate, which then declines th rough the next interval. T he same p er
form ance is repeated in the next 4 intervals, beginning at c, d, e, and f when the size
a n d rate of the initial run, as well as the total n u m b er of responses in the interval, d e
crease. T he rate is very low for the rest of the session except following the reinforce
m ent at g. Reinforcem ents at h, i, and j occur long after the designated interval has
elapsed. T ow ard the end of the session a m ore sustained rate of responding develops
(last line). T his low rate was m ain ta in e d th rough the 2nd session. T h e over-all
rate does not increase further until the end of the 2nd session, after 13 hours of exposure
to the schedule.
A nother transition from erf to FI 17 is shown in Fig. 124, where extinction from the
preceding erf generates a smaller curve. Except for the run at a, the first 6 intervals
show fairly sm ooth negative acceleration, a n d reinforcem ents occur after pauses. A
FIXED INTERVAL
141
substantial rate of responding is reached m ore quickly th an in Fig. 123. A lthough

the grain of the record is rough, with responses occurring in bursts separated by pauses,
the over-all effect, beginning in the region of b, is a fairly uniform rate of responding.
F I 45. Tw o birds were p u t on F I 45 im m ediately following erf. T h e extinction
of the preceding erf did not produce any substantial responding, and both birds showed
a very low ra te for m an y hours. O ne bird show ed no effect of the fixed-interval
schedule after 5 hours. D uring this period only 6 reinforcem ents an d 400 responses
occurred. M ost of the reinforced responses occurred long after the designated fixed
interval h a d elapsed. T he 7th reinforcem ent led to a period of fairly active respond
ing (at a in Fig. 125) followed by a negatively accelerated segment suggesting an ex
tinction curve. This segment is followed by a fairly uniform rate, although some
rough grain is present. Figure 143 shows subsequent sessions in which the over-all
ra te rises slowly. T h e effects of th e early reinforcem ent were not so m ark ed for the
Fig. 125.
142
second bird in the experim ent. This bird m aintained a considerably lower over-all
rate, and took a longer period of tim e to reach a constant rate of responding.
Summary
For very short fixed intervals, the 1st reinforcem ent occurs so soon th a t the rate
due to the preceding erf does not decline greatly. All reinforcem ents tend to occur
while the rate is high. For long fixed intervals, however, reinforcements usually oc
cur after the rate of responding has fallen nearly to zero.
W ith exposure to the fixed-interval schedule the negatively accelerated fixed-in
terval segm ents give w ay to a linear perform ance w ith rough grain. This condition
occurs because the high rate of responding early in the segm ent is correlated w ith
nonreinforcem ent, while the low rate tow ard the end constitutes a favorable stimulus.
T he negatively accelerated fixed-interval segments generate low rates, and the rela
tion betw een the low rate and the reinforcem ent is the most im p o rtan t difference be
tw een transitions from erf to fixed-ratio and fixed-interval schedules.
In the early exposure to a fixed-ratio schedule, reinforcem ents occur when the local
rate of responding is high. Very few instances of reinforcem ent are observed when the
local rate is low.
THE DEVELOPMENT OF FI AFTER erf
FI 1 and FI 2.5
T he transition from erf to FI 1 shown in Fig. 118 was followed by the performance
in Fig. 126, for 3 sessions on FI 1. T he 2nd session, R ecord A, begins with a constant
over-all rate of the order of 0.4 response per second, which gradually increased until
it reached 0.7 response per second a t the end of the session (R ecord B). T he rela
tively constant rate (yielding a relatively constant num ber of responses at reinforce
m ent) in the early intervals of R ecord A produces a b u rst of 20 responses a t m ore
th an 3 responses per second at a. T he whole curve shifts upw ard (that is, no com pen
satory drop in rate occurs). A slight tendency to pause after reinforcem ent appears
a t b an d c an d becomes quite m arked in R ecord B. A n increase in term inal rate ac
com panies the developm ent of this pause. T he term inal rates a t d and e are about
1.25 and 1.5 responses per second, respectively. T he over-all rate rem ains the same
because of the longer pauses after reinforcem ent. In R ecord C, the 4th session follow
ing erf, the interval segments are consistently scalloped; the term inal rate reaches 1.5 to
1.7 responses per second, and the period of pausing and acceleration extends over h alf
of the interval. T he over-all rate of responding rem ains about the same.
A transition to FI 2.5 from erf is shown in Fig. 127,J w hich presents 16 segments
a t the ends of the first 6 sessions. Any effect of the previous erf has already disap
peared in R ecord A. T he rate is roughly constant, and a slight pause begins to fol
low reinforcement. T he pauses increase in the 2nd and 3rd sessions (Records B and
1 W e are indebted to G eorge V ictor for help in this experim ent.
_</>
cr
in
5 MIN.
Fig. 1 2 6 .
D evelopm ent on Fl 1
Fig. 127.
D evelopm ent on Fl 2.5
144
C); but in the 4th session a linear perform ance returns tem porarily. Brief periods of
high rates of responding occur a t a an d b. R ecord E shows consistent pausing and
c u rv a tu re ; b u t a brief re tu rn to a constant rate occurs on the following day. E x
cept for R ecord E the over-all rate increases consistently throughout 6 sessions. A
second-order effect (see below) appears at c.
T h e further developm ent of FI 2, the transition to which was shown in Fig. 119B,
is indicated by the segments in Fig. 128, from the 2nd, 4th, and 6th sessions following
erf. R ecord A shows a fairly constant rate, w ith brief increases at a, b, an d c. P au s
ing or slower responding after reinforcem ent is clear by R ecord B; a brief period of an
exceptionally high rate appears at d. By R ecord C, pausing is m arked, with the usual
concom itant higher term inal rate.
Accidental contingency on FI 1
Figure 129 shows an exam ple of the effect of an accidental contingency in the early
stages of FI. T h e bird developed a sequence of responses in which it pecked the panel
at the side of the key before pecking the key itself. R ecord A is the 13th session on
F I 1 after erf. A lower rate is still resulting from the accidental chaining of the re
sponse to the panel. Such responses are most com m on n ear the end of the intervals,
FIXED INTERVAL
145
an d they produce small S-shaped curves (as at a, b, and c). Traces are still evident in
R ecord B for the 23rd session (cf. d, e, and J), a n d a slight roughness of grain and a
tendency to retard before reinforcem ent (as at g and h) can still be seen by the 56th ses
sion (Record G).
FI 5
Figure 130 shows further developm ent of FI 5 (after the transition in Fig. 121 ). R ec
ord A reports the 40th to 45th reinforcements, with a somewhat reduced rate beginning
to show at a a n d b. R ecord B, for the 75th to 80th reinforcem ents, and R ecord C,
for the 83rd to 88th, show a further developm ent of the pause, w ith a slight increase
in term inal rate. In Record D, the 161st to 166th reinforcements, the pause after rein
forcement is consistently of the order of 45 seconds. T he term inal rate of responding
is still not very high, and there is no extended acceleration. Record E, for the 193rd
to 198th reinforcem ents, shows the sam e order of m agnitude of pausing, with a higher
146
term inal rate and less grain. Occasional instances occur where the lower rate of re
sponding following the reinforcem ent is more extended (as at c). In the next session
a m uch m ore m arked pause an d acceleration begin to show (R ecord F). T he pause
following reinforcement is now as long as 90 seconds (as at d), and the acceleration from
the zero rate fo llow ing th e reinforcem ent to the term inal rate is occasionally quite ex
tended (as at e).
Tw o birds previously used in an experim ent on m atching-to-sam ple, where one class
of responses was reinforced (erf) a n d the other extinguished, were placed directly on
FI 5. They had had no previous exposure to a fixed interval. Nevertheless, smooth
accelerations throughout the interval developed. Figure 131 shows sample records of
both birds. R ecord A follows 109 reinforcements on F I 5; R ecord B follows 44 rein-
Fig. 131.
Tem porary smooth scallop on FI 5
FIXED INTERVAL
147
forcem ents on FI 5. T he curv atu re is m uch m ore m arked th a n in Fig. 130F after
m ore than 200 reinforcements. T he performances shown in Fig. 131 were transient,
however. T he smooth scallops appeared 4 reinforcem ents before the curves shown
in the graph, and they disappeared after a few other reinforcements. At this stage the
over-all perform ance was linear except for pauses after reinforcement.
FI 8
Figure 132 shows the developm ent of a linear perform ance on FI 8 after erf. T he
segm ents are from the first 3 sessions. R ecord A was taken 10 hours after the begin
ning of the 1st session on FI 8. T h e grain is very rough, the over-all rate low, and
the curvature unrelated to the schedule. By Record G, after 26 hours on the schedule,
the rate is constant except for slight pauses after reinforcem ent. T he following ses
sion, shown com plete in Fig. 133, is an excellent exam ple of the instability of the con
tingencies generated by the FI schedule. A lower rate after reinforcement begins to
ap p ear at a, b, c, an d d. T he curvature at e is m ore m arked, a n d acceleration extends
148
th roughout the following interval (/). T h e scalloping is unstable, however, and by

the end of the period the over-all perform ance is essentially linear (g).
FI 17
Further developm ent of the perform ance on FI 17 begun in Fig. 123 is shown in Fig.
134 (2nd session com plete) an d Fig. 135 (segments at 15, 22, 25, 35, 37, and 48 hours
after erf). Figure 134 begins with some recovery from the level of responding of Fig.
123 (probably because of special stim uli from the starting of the experim ent). Before
the end of the session a steady but low rate has been assumed. T he rates at a and b
are of the order of 0.1 response per second, and at c and d are of the order of 0.15 re
sponse per second.
Fig. 1 35.
Developm ent on FI 1 7 to
4 8 hr a fte r erf
Fig. 1 36.
Developm ent on FI 1 7 to 72 hr
a f ter crf
FIXED INTERVAL,
149
In Fig. 135 a brief break-through occurs at a (R ecord A , 15 hours), and a smaller one
at b (R ecord B, 22 hours). A very low rate then supervenes (R ecord C, 25 hours).
At R ecord D (35 hours) a higher rate prevails, and short bursts at about 3 responses
per second a p p e a r in R ecord E (d, e), 37 hours after erf. T h e over-all rate has now
risen, b u t it falls to the very low value seen at R ecord F (48 hours).
A still further developm ent of F I 17 for this bird is shown in Fig. 136, w here single
interval segments have been selected. R ecord A a t 52 hours shows sm ooth curva
ture, rough grain, a n d a slight negative acceleration tow ard the end of the interval.
R ecord B at 60 hours shows a sim ilar perform ance, in which the curvature is smoother
and the grain less m arked. In R ecord C a t 61 hours the period of lower rate at the
start of the interval is m ore extended, and a sharp decline in rate reduces even further
the n um ber of responses in the interval. R ecord D a t 65 hours shows a frequently
occurring perform ance, in w hich the interval begins w ith an extended period a t a low
Fig. 137.
FI 17: whole session 78 hr a fte r erf
rate (a) th a t leads abruptly to an im m ediate term inal rate (b) m aintained for the rest of
the interval. R ecord E a t 72 hours shows an instance in which the rate of responding
accelerates throughout most of the interval. T h e over-all curve is smooth in spite of
the 4 inflections at c, d, e, an d f . In the sessions following Fig. 136 the over-all rate a l
tern ated betw een high and low v alu es, w ith wave-like rate changes. T he two lines
in Fig. 137 form a continuous portion of the curve 78 hours after erf, and exemplify the
fairly smooth oscillations in rate.
A nother bird on FI 17 after erf (transition shown in Fig. 124) also m ain tain ed a
low over-all rate in which accelerations related to the schedule ap p ear an d disappear.
Figure 138 shows a complete session beginning 23 hours after erf. N ote th at curva
tu re appears in spite of the low term inal rate.
Figure 139 shows features of the early developm ent of F I 17 after erf in another bird.
In R ecord A (6 hours after erf) there is still some tendency to respond rapidly ju st after
Fig. 138.
Fl 17: 23 hr a fte r erf (second bird)
FIXED INTERVAL
151
reinforcem ent (a). A ratio-like break -th ro u g h a t a very high rate appears at b.
In R ecord B, after 11 hours, the rate is fairly constant except for a slight pause follow
ing the reinforcem ent (as at c). In R ecord C, at 36 hours, a scallop has developed, the
over-all and term inal rates have increased, and the grain is finer. After 72 hours on
the schedule, oscillations such as those shown in Fig. 140 are evident. Figure 141 shows
a complete 9-hour session for this bird after 87 hours of exposure to the schedule. T he
over-all rate declines progressively during the session, because of the lower rates of re
sponding appearing in the first parts of the intervals. Sm ooth curvature (as a t a, b,
an d d) is infrequent, and there are only a few instances of pauses longer than a few sec
onds following reinforcem ent. M any intervals show inflection points as a t e and f ;
and the grain m ay be extremely rough, w ith pauses and bursts of responding occurring
even after the term inal rate is reached, as a t c and h. T he performance at 111 hours
after erf (Fig. 142) continues the trend of Fig. 141. T he over-all rate falls off even
more sharply, a n d extended curvature develops earlier in the session.
152
Fig. 141.
FI 17: complete session a fte r 8 7 hr (third bird)
FI 45
Figure 143 shows segments at 11, 14, 19, 26, an d 34 hours on FI 45 following the
transition from crf shown in Fig. 125.
In R ecord A a characteristic break-through occurs at a and grain is rough. Record
B shows an unusually long pause at b and m inor break-throughs at c and d. Record C
shows a m oderately high rate following the reinforcem ent at e which falls progressively
d u rin g the rem ain d er of the interval. T he grain is m arked by short runs of 10 to 20
responses. Record D shows the emergence of an almost linear perfomance, although
occasional break-throughs occur, as a t f . R ecord E is the final perform ance recorded
for this bird before the schedule was changed to tan d F /F R . (See C hapter Eight.)
A second bird on FI 45 showed a low rate in the 1st session following crf. A pproxi
m ately 1000 responses occurred during a 15-hour session. Figure 144 shows the com
plete 2nd 15-hour session following crf. T here are m any instances of m oderately high
rates which tend to fall off smoothly as in extinction. Some of these occur following re
inforcem ent (as at a, b, and J), while others occur during the interval (as at c, d, and
e). N e a r the end of the record, a t g, the bird assumes a rate of about 0.3 response
RESPONSES
300
Fig. 143.
Early developm ent on FI 4 5 a fte r erf
154
per second im m ediately following reinforcement; it m aintains this rate for more than 30
m inutes, after which the rate falls off to nearly zero.
Transition from erf to FI 10 in the rat
In experim ents with rats which are started an d stopped autom atically, some stim
ulus is m ade the occasion for nonreinforcem ent so th a t it will produce a zero rate of re
sponding. (See C h ap ter T hree.) This prelim inary training gives the rat consider
able exposure to extinction, w hich doubtless has some effect on the first exposure to a
fixed-interval schedule. These curves also need to be qualified by the am ount of re
inforcem ent th a t was used. T he reinforcem ent was increased from 0.05 gram to 0.1
gram later as a result of experience in this experim ent.
T hree rats were first reinforced continuously in the presence of one stim ulus and
extinguished in the presence of a second, until responding in the presence of the second
FIXED INTERVAL
155
was essentially zero. They were then placed on FI 10 in the presence of the first stim
ulus, and the second stimulus was used to start and stop the experiment. Figure 145
shows the first 7 reinforcements for each rat on FI 10. T he over-all rate of responding
is low, with little curvature or local changes in rate. Except in the early intervals, very
few pauses longer th a n a few seconds occur, an d all of the reinforcem ents are received
w ithin a few seconds of the scheduled tim e. T he over-all rate increases slightly
through the whole first experim ental session.
Figure 146 shows the com plete 2nd session for the ra t in Fig. 145B. A pause an d
lower rate follow the 4th reinforcem ent in the session (a). A 2nd scallop occurs at
156
b, after an initial acceleration.
A nother exam ple of this appears at c. At d the rate

is slightly higher th an heretofore and falls off before the end of the segment.
Figure 147 shows a continuation of the perform ance by the ra t in Fig. 145C. R ec
ord A (top 2 curves, recorded continuously) shows the 9th to the 22nd reinforcements.
Pauses after reinforcem ent begin to appear a t a, b . . . and scalloping (with rath er
rough grain) appears a t c, d, and e, possibly because of a too-short interval p ro
gram m ed accidentally at c. M arked acceleration to a higher term inal rate is evident
a t f in Curve B, which shows the 56th to 61st reinforcements. At g a pause after rein
forcem ent extends through the interval.
Figure 148 shows segments from the session continuing Fig. 145A with features simi
lar to Fig. 146 and 147.
FIXED INTERVAL
157
THE EFFECT OF SUSTAINED FI AFTER erf
F11
Figure 149 shows the further developm ent of the perform ance on FI 1 shown in Fig.
126. T he over-all rate and degree of scalloping rem ain fairly stable over the 5 ses
sions represented by Segments A through E. A 15- to 30-second pause after reinforce
m ent is followed by a sm ooth acceleration to a term inal rate, w hich is m aintained
until the next reinforcem ent. Frequently, however, the term inal rate is assum ed im
m ediately following the reinforcem ent an d is m aintained until the next reinforcem ent
(at a through g). Two intervals together (3 in the case a t a) thus form a single posi
tively accelerated curve.1 R ecord F, the 16th session, shows a lower over-all rate
which is due m ainly to a fall in the term inal rate. Frequently, the pause and low rate
after reinforcem ent extend through the m ajor p a rt of the interval, as a t h, i, an d k.
Instances still occur, as at j, w here the rate of responding is high throughout the in
terval.
FI 2
Figure 150 shows further developm ent of a perform ance on F I 2 (following Fig. 119B
and 128). Record A shows the 22nd session; Record B, the 41st; and Record C, the
50th. In R ecord A a m oderate over-all rate results from a consistent period of pausing
and slow responding during the first half of the interval. Occasional instances of the
term inal rate being m aintained throughout the interval ap p ear at a and c. A sugges
tion of a large second-order effect occurs in the 4 intervals following b. In Record B
the term inal rate of responding has increased greatly, and on occasion the bird does not
1 See The Behavior o f Organisms, p. 123, on these second-order effects.
158
reach the term inal FI rate (as at d and g). These segments are characteristically fol
lowed by pauses th a t are shorter th an usual. Pauses after reinforcem ent are otherwise
of the same order of m agnitude as before, a n d the increase in the over-all rate of re
sponding is due to the higher term inal rates. N egative curvature shows slightly, as
a t e and f .
By the 50th session after crf (Record C) the pause after reinforcem ent is extended,
a n d it usually gives way to an a b ru p t shift to a higher rate. Frequently, as at h, i, and
j , the bird begins to respond at an unusually high rate, which declines smoothly to a
m ore m oderate value.
FI 4 or FI 5
After prolonged exposure to FI 4 or F I 5, a pause following the reinforcem ent is

rarely absent, and most segments show a smooth acceleration to a m oderately high ter-
Fig. 151.
F I4 : 38 hr a fte r crf
FIXED INTERVAL
159
m inai rate. T here is some grain and an occasional knee in the accelerated portion.
Successive intervals are related to each other in the m anner of a second-order effect:
a slow developm ent of the term inal rate in one interval, a n d hence a low num ber of
responses during the interval, is generally followed by an interval in which the pause is
shorter a n d the acceleration to the term inal rate m ore rapid.
Figure 151 shows a com plete daily session on F I 4 (38 hours after crl) for the bird
whose transition from erf was shown in Fig. 120. A pause of from 1 to 2 m inutes is fol
lowed by smooth acceleration to a term inal rate of approxim ately 2.5 responses per sec
ond. Occasional instances of a lower sustained rate occur, as iit g, where the rate is 1.25
responses per second. W hen very few responses occur during an interval, as at a, c,
and e, the next interval usually shows a shorter pause th an usual (b, d, an d J). Fig
ure 152 shows a later perform ance for the same bird, 66 hours after erf. H ere, the te r
m inal rate has fallen slightly, and the pause and curvature after reinforcem ent have
increased. R esponding usually begins earlier in intervals which follow intervals con
taining small num bers of responses (d). T he 3 following intervals comprise a good ex
am ple of the second-order effect. Occasionally, the bird shifts abruptly to a rate only
slightly less th a n the term inal rate during an FI segm ent, as at b. N ote the excep
tional case a t a, w here 4 or 5 responses occur im m ediately after reinforcem ent but
the rate then falls to zero and the usual F I perform ance emerges.
A perform ance on FI 5, 66 hours after erf, is shown in Fig. 153, w hich shows the
first 34 reinforcem ents of a session. T he perform ance is sim ilar to th at in Fig. 152 ex
cept th at the term inal rate is lower and the curvature more extended. Second-order
effects are frequent, as a t a, b, d, a n d i. O ccasional negative curv atu re appears n ear
160
th e end of the interval, as at c. Irreg u lar knees occur a t e , f g, and h. At j a few

responses im m ediately after the reinforcem ent are followed by a pause and a typical FI
performance.
FI 8
Late performances. Figure 154 shows a perform ance on FI 8 for a complete session b e
ginning 186 hours after erf. A high over-all rate is m aintained, but the perform ance
in the interval is by no m eans stable. T he term inal rate m ay be m aintained through
out the interval except for a b rief pause after reinforcem ent (as at f ) or after a longer
pause (as at a). Smooth curvature m ay lead to the term inal rate (as at c), or a m arked
knee m ay occur (as at e). Short runs at a higher rate m ay ap p ear (as a t d). T he
term inal rate m ay decline slightly before a reinforcem ent is received (as a t b).
A wide variety of types of curves ap p eared d u rin g the 186 hours of exposure to the
FI 8 schedule of the bird whose late perform ance is shown in Fig. 154. Figures 155
through 163 show examples of the various types of perform ances, which have been col
lected a n d grouped in terms of the shapes of the curves. These performances occurred
a t m any stages of exposure to the schedule.
FIXED INTERVAL
161
T he following is a rough account of the developm ent leading up to the perform ance
in Fig. 154. After the usual early features already described, the curve becomes essen
tially linear except for brief pauses presum ably controlled by the preceding reinforce
ments. Figure 155 shows some of these linear segments, taken from various points in
the whole experiment. By the 11th hour, m arked deviations from a straight line b e
gin to appear. T he first good exam ple of a smooth positive acceleration of the sort
exem plified in Fig. 156 appears in the 28th hour, an d several instances occur in the 2
or 3 hours following. These then becom e fairly com m on, alternating w ith straight
segments (Fig. 155) or with slighter curves. In a tem porary stage in the 38th hour, the
grain is rough a n d a high sustained rate is not reached. Straight segments then p re
dom inate, a n d a m ore positive curvature emerges. Second-order effects are clear by
the 52nd hour. T he over-all rate is now high, and the term inal rate increasing. M u l
tiple runs, exem plified by Curve 1 in Fig. 162, begin to appear. T he linear segments
of Fig. 155 retu rn from the 70th to the 74th hours, followed by a m ore m arked c u r
vature an d occasional intervals a t a low over-all rate or w ith only a slight accelera
tion. In the 78th and 79th hours, almost all intervals show fairly smooth curvature,
w ith only a slight suggestion of a second-order effect. By the 81st hour a m uch lower
over-all rate has appeared, m arked by rough grain and deviations of the sort presented
in Fig. 159 and 160. Second-order effects are frequent, although a series of repeated
FIXED INTERVAL
163
smooth scallopings m ay extend through as m any as 8 or 10 successive intervals. By

th e 100th hour, scallops are quite sm ooth, there are some second-order effects, and
th e grain is fair. Negative curvature appears occasionally, as exemplified in Fig. 157.
V ertical displacem ent sim ilar to C urves 6, 7, a n d 8 in Fig. 158 appears in the 110th
hour. Between 110 and 130 hours the scallops are fairly uniform , m arked by only m i
nor deviations or second-order effects. T he over-all rate rem ains m oderate (0.5 re
sponse per second). At the 142nd hour a very good series of successive scallops free of
second-order effects appears. T he later perform ance is w ell-represented by Fig. 154.
Types o f rate changes under sustained exposure to F I 8. Figure 155 shows linear segments
w hich are especially com m on early in the history of such a schedule. (Cf. Fig. 151
for sim ilar segm ents on a shorter interval.) Figure 156 exemplifies sm ooth accelera
tions extending throughout the segment. T he curvature ranges all the way from arcs
of circles, Curve 3, to a sharp acceleration im m ediately after reinforcement, as in Curve
4, where a further increase in rate is only very slight. W e take these curves to be typi
cal of basic processes in FI. T h e following figures note various kinds of deviations
which m ay be imposed upon them .
164
O ne type of deviation consists of an early acceleration to a high rate, which is not,

however, m aintained. Exam ples of such negative acceleration at different over-all
rates are shown in Fig. 157. In Curves 1 through 5, Fig. 158 shows various abrupt
changes in rate; and in Curves 6 through 8, displacem ents due to a brief run for which
no com pensatory decline occurs.
A com m on deviation is a convex b u m p or knee. In Fig. 159, C urve 1 shows
only a very slight exam ple; in Curves 2 and 3 the effect is m ore pronounced. M inor
variations an d evidence of more th an one inflection are ap parent in Curves 4 and 5.
In Curve 6 a fairly high rate falls off to zero; the rate then shifts abruptly to the final
ru n n in g rate. In Curve 7 an a b ru p t shift occurs early, and the rate falls off slightly
and again recovers in a smooth curve. In Curve 8 the m ajor changes are abrupt, as a t
a and b, b ut the negative curvature in the knee is fairly smooth. Curve 9 resembles
Curve 8 except for the less a b ru p t changes. Curve 10 presents a variation on Curve 9,
in w hich the knee occurs very late a n d little tim e rem ains for the m aintenance of a
term inal rate. Curve 11 shows a sim ilar effect at a very low over-all rate throughout
the interval.
A different type of deviation is a concave rath e r th an convex deflection of the curve.
Curve 1 in Fig. 160 seems to be a fairly smooth, positively accelerated curve, except
FIXED INTERVAL
165
th at a pause appears early in the curve; this pause is followed by a rapid com pensa
tory ru n which brings the curve roughly back to the extrapolation of the earlier p o r
tion. Curve 2 shows another exam ple. W hether C urve 3 belongs in Fig. 159 or Fig.
160 is difficult to say. Curve 4 shows a nick which is followed by a continuing decline
a n d the recovery of the term inal rate. T he possibility of repeated deviations of both
sorts is suggested in Curves 5, 6, 7, 8, 9, and 10.
T he kind of deviation exemplified in Fig. 160 frequently occurs ju st after reinforce
ment. (See Fig. 161.) It suggests th a t com peting behavior prevents the bird from
returning to the key, an d th at the delay is m ade up in a com pensatory run.
Some of the most irregular segments encountered in such an experim ent are illus
tra ted in Fig. 162. In general, the rough course of the record m ay be guessed at in
spite of the deviations.
Some instances of second-order effects from these experim ents are shown in Fig. 163.
In Curve A the resum ption of responding at d alm ost im m ediately after reinforcem ent
is related to the only m oderate pausing at c and the low rate during the whole inter
val at b. T h e over-all effect is a positive acceleration extending through 3 intervals.
In this particular case some negative acceleration sets in a t e. O ne interpretation of
this instance is th a t the reinforcem ent a t a, occurring after a fairly sustained ru n and
w ith a high count, accentuates the nonoptim al ch aracter of the pause after reinforce
m ent. T he bird responds only late an d irregularly during the interval. But the re-
FIXED INTERVAL
167
inforcem ent at c occurs after an irregular low rate and at a low count. It has an im m e
diate effect in th at it makes the situation just after reinforcement more favorable.
H ence, responding begins fairly soon. T h e rate is still below m axim al, so th a t the
effect carries through the next reinforcem ent (at d). Both the count an d rate become
excessive, however, and negative curvature then develops at e.
A pplying this in terpretation to R ecord B, we could say th a t the very high count a t
168
SC H ED U LES O F R E IN F O R C E M E N T
reinforcem ent f produces the low rate a t g; b u t the reinforcem ent at a low rate a n d
low count at h reverses the effect. Therefore, responding begins early in the following
interval a n d develops a high count an d rate before reinforcem ent a t i. This devel
opm ent in tu rn reduces the rate at j . Here, the second-order curve consists of only 2
intervals. In Record G the fact th at a second-order effect during only 2 intervals shows
some negative curvature a t the very end suggests th a t the count a t k has become ex
cessive. R ecord D shows an exam ple of an exceptionally long pause after reinforce
ment. T he postponed reinforcem ent is received w ith only 1 response. T he effect is
to reinstate responding almost im m ediately, because the pause following reinforcement
resem bles the pause after w hich reinforcem ent has ju st been received. T h e bird
quickly assumes the term inal rate at I and continues throughout the subsequent in te r
val. T h e curves a t Records E, F, G, an d H show other exam ples of second-order ef
fects at various over-all rates.
A second bird on FI 8 showed fewer deviations from a smoothly scalloped perform
ance. Figure 164 shows the perform ance a t 50 hours (R ecord A) and a t 100 hours
(R ecord B). M arked pauses a n d extended curvature tend to alternate w ith intervals
in which the rate is constant throughout.
Fig. 1 65.
FI 8 a fte r 51 hr (third bird)
F IX E D IN T E R V A L
169
10 M I N U T E S
Fig. 166.
FI 8 a fte r 162 hr (third bird)
Figure 165 shows an interm ediate perform ance on FI 8, 51 hours after the transition
from erf shown in Fig. 122. T he over-all rate is lower th an in the preceding cases.
T he distinguishing feature is a species of runn in g -th ro u g h . T he term inal rate con
tinues for from 5 to 20 responses after the reinforcem ent an d then declines, often
smoothly, to either a norm al scallop (as at c, d, an d e) or to a lower rate of responding,
which m ay or m ay not be sustained for the rem ainder of the interval (as at a and b).
Figure 166, 105 hours later, still shows instances of responding ju st after reinforce
m ent at the term inal-interval rate. W hen a pause follows reinforcement, it still does
not exceed m ore th a n a few seconds, as a t a a n d d. R ough second-order effects m ay
be seen at b, c, and e. T he final rate is considerably higher th an before.
A nother b ird w ith the sam e history gave the p erform ance shown in Fig. 167, 100
hours after erf. A high term inal rate develops early in the experim ent, and ru n
ning-through does not occur. M ost of the interval segm ents are sm ooth curves w ith
varying speeds of acceleration. At c, however, a pause of only a few seconds is fol
lowed by an a b ru p t shift to the term in al rate. At a a n d d, acceleration continues
170
SCH ED U LES O F R E IN F O R C E M E N T
throughout the whole interval. W here the bird begins to respond early in the interval,
th e uniform length of pause, as at b, c, e, an d g, suggests a separate control of a low
rate by the reinforcem ent. Second-order effects are evident a t b , f and h.
FI 16 o r FI 1 7
Figures 168 and 169 show 2 successive daily sessions after 272 and 280 hours, re
spectively, on FI 16. These records show the principal features of a fixed-interval
perform ance at this stage. In Fig. 168 the relatively low over-all rate is due to a very
pronounced scalloping, w here a low rate often extends through m ore th an h a lf the
interval. Following the reinforcement, there is a consistent brief pause from a p
proxim ately 30 seconds at b to over 1 m inute a t the start of the interval at a. I n
stances still occur where the term inal rate is reached soon after the reinforcem ent, but
responding never begins im m ediately after reinforcem ent as in a run-through. T he
perform ance in the following session, in Fig. 169, shows a m uch higher over-all rate,
which is due to an increase in the term inal rate from about 1 per second in the previous
session to about 1.5 per second in this session. The curvature becomes more m arked
during the period and shows considerable uniformity. (Com pare the last 6 intervals
of the session.)
T he schedule was m aintained for a total of 338 hours. T he last 2 sessions are
shown in Fig. 170. Here, instances occur where responding begins im m ediately fol-
171
lowing reinforcem ent, as a t a, b, and c; and the pause following reinforcem ent varies
from only a few seconds, as at e, to alm ost 2 m inutes a t d. Slight negative curvature
m ay appear, as at f . T he rate changes w ithin the interval are now less reproducible
from interval to interval th an those in the earlier performance.
Figure 171 shows a late perform ance on FI 17 (100 hours after erf). (Early stages
for this b ird are represented by Fig. 136 a n d 137.) T h e curves a t a an d d show con
tinuous acceleration through the interval. O th e r intervals show a m ore rap id ac
celeration to a term inal rate (as at b a n d g). T he rate often declines tow ard the end
of the interval (as at c a n d e). A short (reinforcem ent-controlled) pause occurs at
the beginning of each interval. A t h the term inal rate is not reached, an d only a few
responses occur during the interval. T he bird characteristically showed a higher
over-all rate of responding early in the session. T h e F I 17 reinforcem ent schedule
was continued for a total of 334 hours.
A still further developm ent is shown in Fig. 172, which contains the first 16 intervals
of a session beginning after 277 hours of exposure to the schedule. T he segments
occurred in the order shown. At the end of each interval the recorder reset to the bot
tom of the graph; reinforcem ents are indicated by the reset line. T he perform ance
at this stage of developm ent is radically changed. No instances occur here of a smooth
RESPONSES
300
I
RESPONSES
300
Fig. 173.
Sustained Fl 10 in the rat
173
acceleration d u ring the interval. T h e characteristic p a tte rn is a rap id assum ption

of the term inal rate after a pause of the order of 20 to 30 seconds after reinforcem ent.
Extrem e negative curvature appears (as a t a a n d b). T he num ber of responses per
interval at reinforcem ent, as well as rate, varies widely. Such a perform ance interact
ing with the scheduling device produces no consistent set of contingencies.
The effect o f a sustained FI 10 in rats
Figure 173 shows later perform ances for the rats responsible for Fig. 147 and 148.
T he low over-all rates are due m ainly to the am o u n t of reinforcem ent (0.05-gram pel
let), which was later determ ined to be too small. T he first rat, R ecord A (50 hours
after crf), has a consistent pause following reinforcem ent, and curvature extends
throughout most of the interval despite a rough grain. In R ecord B, for the second
rat (62 hours after crf), the term inal rate is higher and the grain smoother, but m any
intervals show a very slow acceleration to a low rate. In R ecord C, for the th ird ra t
(140 hours after c rf), the over-all rate is extrem ely low. R einforcem ents are fre
quently postponed an d are achieved w ith only 1 or 2 responses per interval. W hen
responding begins, however, a fairly typical fixed-interval perform ance m ay be pro
duced, as in the last interval shown.
These rats were th en used on a m ult F I avoidance schedule. (See C h a p te r T en.)
As a control condition in this experim ent, the schedule of reinforcem ent u n d er both
stim uli was FI 10. At this stage of the experim ent, we discovered th a t the reinforce
m ent h a d been in ad eq u ate; a n d we increased the am o u n t to 0.1 gram by loading
th e m agazine w ith 2 pellets instead of the 1 w hich h a d been delivered previously.
U n d e r these conditions all 3 rats showed sim ilar perform ances. Figure 174 shows a
Fig. 174.
FI 10 in the ra t w ith larger am ount o f reinforcem ent
174
daily session after 94 hours on FI 10 alone after the m ult FI 10 avoidance schedule.
T he perform ance from interval to interval is fairly reproducible. Occasionally, only
a few responses m ay occur during an interval, as at a; but the usual performance is a
period of 5 to 8 m inutes when the rate is either zero or near zero followed by a quick
acceleration to a term inal rate of from 1.5 to 2 responses per second.
TRANSITION FROM ONE VALUE OF FI TO ANOTHER
In the transitions from one value of FI to another the actual contingencies on the
second value are the product of the new schedule an d the perform ance generated by
the old. Eventually, a fairly stable set of contingencies will arise under the new sched
ule. In an early experim ent on this point the value of FI was advanced rapidly from
5 to 10, 15 to 25, an d finally to 35. T he perform ance was not allowed to stabilize
a t each value. T h e actu al contingencies of reinforcem ent therefore varied widely.
T he result was an irregular perform ance, m an y parts of w hich suggest a variable interval schedule (C h a p te r Six) or th a t special case of variable interval called a two-
Fig. 1 75.
Third session on FI 10 a fte r FI 5
175
valued schedule. A term inal rate which is often m aintained for m any hundreds of
responses develops quickly, though it m ay be preceded by quite irregular fluctuations in
rate. T he term inal rate often falls off fairly smoothly; this condition suggests extinc
tion after a variable-interval schedule. Tw o samples for 1 bird are shown in Fig. 175
(the 3rd session on FI 10 after FI 5) a n d in Fig. 176 (two sessions on FI 35 after
F I 25). N ote the stable term inal rate, the occasional interval curvature, the knees
a n d o th er irregularities early in each interval, a n d th e ru n n in g -th ro u g h after m any
reinforcements.
FI 5 to FI 30
Figure 177 shows one tran sitio n from F I 5 to F I 10. T h e record begins w ith a
fairly stable perform ance reached after 55 hours on FI 5. R unning-through is espe
cially m arked. After a short period of slow responding, a high term inal rate is m ain
tain e d th ro u g h o u t most of the interval. At the arrow the interval is increased to 10
m inutes. T h e break in the first 10-m inute interval at a is possibly due to a slight
Fig. 177.
Transition from FI 5 to FI 10
176
disturbance while the controlling circuit was changed. In the second 10-minute in
terval the term inal rate is held. D uring the rem ain d er of the session the term inal
rate declines som ew hat, an d the period of slow responding early in the interval is ex
tended. At e , f a n d g a low rate is m ain ta in e d for h a lf or m ore of the interval. At
th e end of the session, 3 intervals ending a t h are ru n off w ith alm ost no pause fol
low ing the reinforcem ent. M arked negative acceleration then occurs in the next in
terval. Small bursts of responses im m ediately after reinforcem ent continue to appear,
varying from approxim ately 5 responses in interval e to approxim ately 30 responses
a t d. Exam ples of knees are evident at b and c.
Figure 178 gives the transition to FI 20 after 122 hours on FI 10. T he 1st interval
shows a scallop com m on under F I 10, and the term inal rate is m aintained throughout
the new 20-m inute interval. This produces an exceptionally prolonged lower rate
after reinforcem ent a t a, but the num ber of responses em itted in the 2nd interval is still
above 1000. Scalloping becomes progressively m ore extended as the session proceeds.
At d the initial period of lower rate of responding extends over the m ajor p a rt of the
interval. This is true for most of the rest of the session. Pauses sometimes follow rein
forcem ents (as at a, c, a n d h), although they last only a few seconds a n d are probably
based entirely on the effects of the recent reinforcement. W ell-m arked knees appear
(as at h), and m ultiple knees (as at b a n d / ) . M any cases occur of fairly smooth ac
celeratio n to the term in al rate (as a t d a n d e), though the cu rv a tu re is m arred by
rough grain. As in the previous transition, the first effect is an increase in the num ber
of responses em itted during the interval as the high term inal rate from the preceding
schedule is m aintained throughout the larger interval. A more characteristic FI p er
form ance emerges as the term inal rate of responding falls and the extent of the scal
lop becomes greater.
Figure 179 shows the final perform ance on FI 20 after 300 hours of exposure to the
schedule. By this tim e the session begins w ith a pause a n d a fairly smooth an d rapid
acceleration to a term inal rate (as at a). T he rest of the session is characterized by
177
a rough kind of alternation between performances (as at b, d, and e) where slow respond
ing extends over more than half the interval and is followed by a high term inal rate,
an d a perform ance (as at c) w here the rate is assum ed im m ediately after reinforce
ment. A slight pause follows reinforcement, but generally some running-through oc
curs even w hen the term inal rate is not im m ediately assumed.
T h e transition from FI 20 to F I 30 was lost th rough a recorder failure, b u t the p e r
form ance after 64 hours of exposure to F I 30 is shown in Fig. 180. T he reinforce
m ent at a is followed by a few responses a t the term inal rate. After a short pause, the
term inal rate continues w ithout m uch change until the next reinforcement, a t b, giv
ing m ore th a n 2000 responses in this interval. T he next interval shows runningthrough, a m arked period at a low rate, a n d a shift to the term inal rate which con
tinues w ith some grain (at c) until the next reinforcem ent. T he following interval
shows an early start at the term inal rate at d, a decline m arked by rough grain at e, but
a quick resum ption of the term inal rate. T he rate falls slightly just before reinforce
m ent at f . T he interval starting at f is low throughout. T he following interval be-
Fig. 1 80.
FI 3 0 a fte r 6 4 hr
178
gins a t th e term in al rate b u t leads to m ark ed negative acceleration (at g) before the
next reinforcem ent. T he next interval begins w ith a low rate at h, followed by an
acceleration with rough grain to the term inal rate.
Figure 181 shows an early perform ance on F I 45, recorded 29 hours after the
change from FI 30. Except for the 1st interval, scalloping is not w ell-m arked; and
the grain is rough, even in long stretches at the term inal rate. Later, however, strong
Fia. 1 83.
FI 4 5 a fte r 100 hr
179
scalloping develops (Fig. 182). M any intervals now contain very few responses. This
decline in over-all rate at FI 45 continues. A session a t 100 hours after the change
from FI 30 is shown in Fig. 183.
A second bird showed a m uch slower transition from FI 10 to F I 20. Successive
stages are shown in Fig. 184, 185, 186, a n d 187. T he transition (not shown) is sim ilar
to those described: the term inal rate holds through the 1st extended interval, a n d the
2nd interval begins w ith a slight pause a n d a fairly a b ru p t shift to the term in al rate,
Fig. 185.
FI 2 0 a fte r 61 hr (second bird)
180
SCH ED U LES OF R E IN F O R C E M E N T
which falls off ju st before the end of the interval. Subsequent intervals showed sub
stantial pauses, occasional knees, a n d rough grain. Figure 184 shows a portion of the
session after 55 hours on F I 20. T here is consistent ru n n ing-through of the term inal
rate. At a the term inal rate is then held throughout the next interval. In general,
however, as at b, c, a n d d, the rate declines a n d then accelerates to the term inal value
again.
Figure 185, at 61 hours after erf, shows a change in the interval curvature and a long
second-order acceleration. Figure 186, at 110 hours after erf, shows a return to the
earlier type of perform ance. By 220 hours after erf (Fig. 187), m any intervals begin
w ith deep scalloping (as at a, d, and e), others w ith m arked knees and a briefer post-
181
ponem ent of the term inal rate (as at b), while some show the term inal rate throughout
(as at c). N o pausing occurs after the reinforcem ent. (T he fine grain of this and the
following record is an artifact due to excessive pen pressure above a fluted p aperfeed cylinder.)
Figure 188 shows the transition to F I 30. T he term inal rate, once assumed, usually
continues until reinforcem ent; b u t the strains im posed by the longer interval reduce
ru n n in g -th ro u g h to a few responses. L ater, however, w ith further reinforcem ent on
the longer interval, running-through increases, an d whole intervals are sustained a t
th e term inal rate, as in Fig. 189, taken 15 hours after the beginning of F I 30. Figure
190 shows the perform ance after 27 hours on F I 30.
182
FI 30 to FI 45
T h e transition from FI 30 to F I 45 is shown in Fig. 191. T he term inal rate as

sum ed at a is held during the new interval except for the 2 pauses at b and c. T he
pause at b seems to follow a period of higher-than-norm al rate. T he rest of the session
shows a few good scallops appropriate to the new interval, with some runningthrough. T he rate is low at d a n d the grain rough, possibly because of the very large
n u m b e r of responses em itted per interval. A late r p a tte rn is exem plified by Fig.
192. T he 1st interval shows a rough b u t deep scallop; but the 2nd is executed
throughout at the term inal rate (from a to b). As a result the following interval shows
a low rate an d rough grain. A fairly sm oothly accelerated curve begins at c, with
reinforcem ent at the term inal rate at d. T he next interval shows a low rate, negative
curvature, and rough grain. Figure 193 shows a later perform ance 57 hours after the
transition to FI 45. T he fixed-interval segments from 1 session have been assembled
in ord er of the n u m b er of responses. T he num bers indicate the order in the session.
At this stage in the developm ent of FI 45 the perform ances from segment to segment
are quite diverse. O nly 3 segments, 2, 6, and 8, show a lower rate of responding early
183
in the interval a n d a later acceleration to a higher rate. Tw o segments, 3 a n d 9,

show sustained responding im m ediately after the reinforcem ent, b ut the rate is not
held.
T he last recorded perform ance on FI 45 is shown in Fig. 194 in order of the n u m
ber of responses. By this tim e th e effect of the fixed-interval reinforcem ent is m ore
clearly evident. M ost of the curves show responding im m ediately after the re in
forcement, although the segments a t a, b, a n d c begin w ith substantial pauses. T he
m axim al rate of responding has dropped to about 1.25 responses per second, com pared
with slightly less th an 2 per second in Fig. 192.
A t one point in th e experim ent on F I 45 ju st described th e bird was briefly extin-
184
Fig. 193.
guished.
FI 4 5 a fte r 5 7 hr
Figure 195 shows the result.
Fig. 194.
FI 4 5 a fte r 3 3 0 hr
T he session begins with a scallop (a) appro
priate to the prevailing intervals of reinforcem ent; an d the term inal rate is held with
only a slight decline in rate until b, 97 m inutes after the start of the experim ent. At
this point the rate drops to zero. T he resulting pause initiates another extended scal
lop, reaching a som ew hat lower term inal rate m arked by greater irregularity of grain,
which is sustained until c, 67 m inutes after point b, where the rate falls to zero again.
A 3rd slow acceleration shows a small knee a t d. A som ew hat lower sustained rate
then declines gradually as the grain grows rougher, at e.
Figure 196 shows a fairly stable perform ance on FI 39 after 240 hours, arranged by
the n u m b e r of responses, for a th ird pigeon w ith a history of m ore th a n 700 hours
on various variable-interval schedules. T he over-all rate of responding is relatively
low. M any intervals frequently do not show the term inal rate. Some segments are
Fig. 1 95.
185
Extinction a fte r FI 45
roughly scalloped, but a few scattered responses usually occur shortly after reinforce
ment.
TIME OUT
The effect o f time out on FI 45
Figure 194 showed a fairly stable perform ance u n d er

In the following session a 20-m inute T O occurred after each reinforcem ent.
Time out after reinforcement.
FI 45.
186
T h e com plete 1st session is shown in Fig. 197, w here the segments are arran g ed in
term s of the num ber of responses. T he order of occurrence is indicated.
T he principal effect of the T O is to prevent running-through. All segments begin
with pauses. T he over-all rate of responding also shows a decline. T he 2nd session
w ith T O after reinforcem ent (Fig. 198) shows a better adjustm ent to fixed-interval
reinforcem ent. Each segment begins w ith a pause (ranging from approxim ately 1
m inute at a to 6 m inutes at c and d)\ and all except the last show an acceleration to
a higher rate. In the segment m arked b the term inal rate is reached before the m id
dle of the interval, and negative curvature appears before the end.
Time out after alternate reinforcements. W e m ade a further effort to determ ine the ef
fect of a T O by arranging a 20-m inute T O after every other reinforcem ent, the in te r
vening cases serving for comparison. Curves for 2 sessions were divided into segments
a n d separated according to the location of the T O . Segments preceded by T O are
assem bled in Fig. 199; those not preceded by T O , in Fig. 200. They are arranged in
order of the num ber of responses em itted in the interval. T he preceding T O produces
a substantial pause at the start of all segments. T he curves are for the most p a rt u n
evenly scalloped, however, a n d the acceleration is m arked by rough grain, with re
sponses occurring in bursts of 15 to 30. T he alternate segments which were not
preceded by T O (Fig. 200) show a m uch higher over-all rate of responding. Some re
sponding occurs im m ediately after reinforcem ent, although a t a an d b this is followed
by a zero rate before the acceleration to a term inal rate. T he term inal rate is some-
10 M I N U T E S
Fl 4 5 TO 2 0 , second session
300
RESPONSES
Fig. 198.
Fig. 2 0 0 .
FI 4 5 w ith TO 2 0 a fte r a
reinforcem ents, no TO records
Fig. 19 9 .
FI 4 5 w ith TO 2 0 a fte r a lte rn a te

reinforcem ents, TO records
188
tim es reached im m ediately (as in the 1st segm ent, which later shows a slight negative
acceleration).
A complete daily session 50 hours after the alternating T O procedure was begun is
show n in Fig. 201, in w hich the segm ents a p p e a r in their actual order. T he TO s of
20 m inutes are m arked by arrows. T he bird starts slowly at a, the beginning of the
session, and the 1st reinforcem ent is received a t b, after only scattered responding. No
tim e out follows this reinforcem ent, an d the following interval also begins at a low
rate, at c. T his interval is com pleted w ith an extended a n d fairly sm ooth scallop to
the 2nd reinforcem ent, at d, w hich is followed by a 20-m inute T O . A lthough the
ra te of responding ju st before the reinforcem ent a t c was high, the following interval
begins with a pause and ends with a fairly typical acceleration. T he 4th interval, at
e, is not preceded by a T O , an d the high rate prevailing ju st before the reinforcem ent
continues throughout the whole interval until the reinforcem ent, a t f
A 20-minute
T O follows at g, and the following interval contains only a few responses. A reinforce
m ent becomes available a t h, but is not received until approxim ately 5 minutes later.
Fig. 201.
FI 4 5 w ith TO 2 0 a fte r alternate reinforcements a fte r 5 0 hr
189
A lthough the next interval is not preceded by a T O , it begins with a pause and the rate
accelerates to a high value. T he next interval, following a T O , begins w ith a long
pause a n d shows a typical F I acceleration. A rate som ew hat higher th a n usual is
reached a t i; an d this rate is followed by a drop to the norm al term inal rate a tj. N o
tim e out follows reinforcem ent at j, and the subsequent interval begins at a substan
tial rate of responding, at k.
In sum m ary, this figure confirms the general effect of the time out seen in a com pari
son of Fig. 199 an d 200. T he 2 instances w hen an interval beginning w ithout T O
shows a pause and a norm al acceleration to a term inal rate are preceded by intervals
in which only a few responses occurred.
T h e 1st intervals of the 2 sessions are om itted from Fig. 199 an d 200 because they
have no relevance to the problem of the effect of the preceding interval. Actually,
the 1st segm ent could be thought of as following a 15-hour period of T O . T he 1st in
tervals from 3 sessions each for 2 birds have been collected in Fig. 202 and arranged by
the num ber of responses. T hey all resem ble intervals after T O , with either a consist
ent pause a n d acceleration following the start of the interval or a very low rate con
tinuing throughout the interval.
In these experim ents a T O after reinforcem ent results in a pause at the beginning
of the next interval which m ay or m ay not be followed by an acceleration to a term inal
rate. W hen T O is om itted after reinforcem ent, some responding usually occurs im
m ediately (running-through). T he over-all rate is highest w hen T O is omitted.
T he difference is presum ably attrib u tab le to the loss of the effect of the preceding
term inal rate during the T O . A strict alternation of T O and no T O encourages a dif
ference. T he higher rate following no T O also follows an interval with relatively few
responses, because this interval in tu rn was preceded by a T O .
Effective duration o f time out. In order to determ ine the m inim al effective d uration
of T O in an interval perform ance, 0-, 10-, 20-, and 30-m inute T O s were program m ed
to alternate after reinforcements. This procedure was in force for 9 sessions. A nal
ysis of the results showed no differential effect betw een T O s of 10, 20, or 30 m inutes,
either upon the m agnitude of the pause following reinforcem ent or on the am ount of
tim e elapsing before the term inal rate is reached. Figures 203 through 207 reproduce
all intervals except for segments following no T O and 8 anom alous segments. They
have been arranged roughly in order of the num ber of responses em itted in the interval
a n d collected in term s of sim ilarity of curvature. T hey provide an exam ple of the
perform ance under a large fixed interval w ith a T O after reinforcem ent, and illustrate
the kinds of cu rv a tu re w hich are p ro d u ced by this type of schedule. Figure 207
shows the anom alous segments. T he principal feature is the negative curvature which
sets in before the end of the interval.
Interspersed am ong the perform ances shown in Fig. 203 through 207 are segments
preceded by no T O after reinforcem ent. These show a large num ber of responses
per interval, as in the previous experim ent.
T h e m inim al effective T O was then sought am ong sm aller values. A full session
RESPONSES
300
10 M I N U T E S
Fig. 2 0 2 .
First intervals o f the session
Fig. 2 0 3 .
190
FI 45 TO ( 10-30): collected
segments
RESPONSES
300
Fig. 2 0 4 .
Fl 4 5 TO (1 0 -3 0 ): collected
segments
Fig. 2 0 5 .
191
Fl 4 5 TO (1 0 -3 0 ): collected
segments
192
Fig. 2 0 6 .
FI 45 TO (1 0 -3 0 ): collected
segments
Fig. 2 0 7 .
FI 4 5 TO (1 0 -3 0 ): collected
segments
each was devoted to T O s of 0, 1, 3, 7, a n d 10 m inutes. T h e effect was largely on the

perform ance im m ediately following the reinforcem ent; but this condition, in turn, af
fected th e cu rv a tu re a n d general orderliness of the F I 45 perform ance. In order to
com pare the effects of these different lengths of T O , portions of the curves in the re
gion im m ediately preceding an d following reinforcem ent have been collected for 2
com plete sessions on each value in Fig. 208. C olum n A shows all instances of rein
forcem ent which were not followed by T O . T he top 8 instances show a rapid assum p
tion of the term inal fixed-interval rate, w ith some responding im m ediately after rein
forcem ent in 5 out of the 8 cases. In the next 5 instances, substantial pauses either
follow the reinforcem ent or a small am ount of ru nning-through occurs. T he bottom
3 show a low rate both before an d after reinforcem ent. C olum n B for a 1-m inute
T O resem bles C olum n A fairly closely. T he curves are irregular; and, in general,
some responding after reinforcem ent occurs which m ay continue at the term inal rate
or lead to pauses with a return to the term inal rate later. T he effect of a 3-m inute T O
in C olum n C is m arked. Responding occurs im m ediately after T O in only 3 instances.
193
10 M I N U T E S
Fig. 20 8 .
FI 4 5 TO (0-10)
Fig. 2 0 9 .
FI 4 5 TO 2 0 a fte r 2 3 hr (second
bird)
T he T O s of 7 and 10 m inutes (Colum ns D and E) always yield a complete cessa

tion of responding im m ediately following reinforcement.
T he effective duration of T O for preventing carryover from the preceding interval is
in the region of 3 to 7 minutes. Longer T O s should be used to m inimize the in te r
action betw een successive intervals, whenever the T O is not too costly in experim ental
time.
Second bird. For the second bird the im m ediate effect of T O on FI 45 was lost b e
cause of apparatus difficulties; b u t a perform ance after 23 hours of reinforcem ent w ith
T O after reinforcement is shown in Fig. 209, where the segments have been arranged
in terms of the num ber of responses in the interval. T he over-all rate of responding fell
194
Fig. 2 1 0 .
'
'
+ 1 MIN.
FI 4 5 w ith TO 2 0 a fte r a lte rn a te

reinforcem ents, TO record
below th at which prevailed previously w ithout T O after reinforcement, as, for exam
ple, in Fig. 183. But w here a substantial level of responding does occur, it follows the
p attern of a period of acceleration with rough grain to a term inal rate, which is gen
erally held until the reinforcem ent. A break followed by a 2nd acceleration is a p p a r
ent at a. T he over-all rate of responding is low because of the very frequent ap p ear
ance of intervals in w hich the term inal rate is not reached even though responding
begins w ithin the first 15 m inutes of the interval.
W hen T O s are arranged after alternate reinforcements, this bird confirms the re
sult already noted. Segments from a single long session 55 hours after exposure to
alternating TO s have been separated into 2 figures. Figure 210 shows segments pre
ceded by a 20-m inute T O . Long pauses and delayed acceleration are usual. Figure
211 shows segments not preceded by TOs. Segments 1, 4, and 5 show some FI scal
loping w ith fairly consistent positive acceleration. Segm ents 2, 6, and 7, however,
show m arked negative curvature. T he m ean over-all rate is, of course, m uch higher
th a n in Fig. 210.
In Fig. 212, 213, 214, and 215, segments from 3 com plete daily sessions on FI 45 T O
have been arranged in term s of the type of curvature. W ithin each figure the seg
m ents are arran g ed roughly in order of the n u m b er of responses. T h e first 3 seg
ments from these sessions are reproduced separately in Fig. 216.
Extinction was carried out after 490 hours on F I 45 T O . T he previous history was
620 hours on FI 5 to FI 45 after crf. T he curve is slightly displaced in the coordi
nate fram e of Fig. 217. T he perform ance begins with the characteristic scallop
(at a), an d the term inal rate is m aintained w ith only a slight decline for about 7000 re
sponses. A sharp break occurs at b to essentially a zero rate, persisting for almost 2
hours. A 2nd interval scallop then follows a t c, a n d the term inal rate is sustained for
alm ost 2000 responses before it falls off w ith rough grain to am cher period of low rate
at d. This period is again followed by an interval scallop and eventually by a more
a b ru p t break to a zero rate of responding. A few b rief bursts near the term inal rate
Fig. 2 1 1 .
Fl 4 5 w ith TO 2 0 a fte r a lte rn a te
reinforcem ents, no TO record
RESPONSES
300
Fig. 2 1 5 .
Fl 45 TO: collected segments
197
occur at f b u t the curve ends at essentially a zero rate a t g. T he most obvious in

terpretation is th a t the intervalperform ance follows whenever the bird has been paus
ing for any length of time. Cessation of responding at b, d, and f suggests some kind
of exhaustion, which is probably the essential property of the process of extinction, over
and above changes in rate due to self-generated stimulus changes.
The effect o f time out on other values o f FI
F I 1. Figure 129C describes a fairly stable perform ance on FI 1, 56 sessions fol
lowing erf. Reinforcem ents are usually followed by a pause of 10 to 30 seconds, w ith
later acceleration to a term inal rate. Occasionally, this bird began responding im
m ediately after reinforcem ent, continuing a t a constant rate through the interval. In
the session im m ediately following Fig. 129C a 5-m inute T O followed each reinforce
m ent. T he first effect is shown in Fig. 218A. T he vertical lines following reinforce
m ents show some responding during the 5-m inute T O . R esponding during the
1-m inute interval is roughly constant. T he 3rd pen excursion of the session is shown
THOUSANDS
IN
RESPONSES
HOURS
Fig. 21 7.
Fig. 21 8.
Extinction a fte r FI 4 5 TO
The effect o f TO 5 a fte r reinforcem ent: FI 1
199
in R ecord B; responding during T O is now only slight, and pauses are beginning to a p
pear after reinforcem ent (at a, b, c, an d d ). A lower rate during the earlier p art of
the interval develops slowly and irregularly w ith continued exposure to the schedule.
R ecord G is a segm ent from the 9th session on FI 1 T O 5; it shows m ore pausing a t
the start of the interval, though still less th an in Fig. 129. A pause or lower rate at the
start of the interval is frequently lacking. T he term inal rate is higher th an w ithout
T O , and negative curvature frequently appears after the highest rate is reached (as
at e , f an d g ). T he T O s did not occur in the 11th session (R ecord D). T h e im m edi
ate effect is the com plete elim ination of the pause following reinforcem ent; b u t the
pause returns after the 5th reinforcem ent (h ), a n d the rest of R ecord D shows a fairly
norm al FI perform ance. Four sessions later (R ecord E) the negative curvature ob
served in Fig. 129 appears. This negative curvature probably represents a retu rn to
the superstitious behavior th a t was seen earlier in the experim ent. A 5-m inute T O
followed reinforcem ent in the next session, an d the result is shown in Fig. 219A. T he
T O produces a m arked increase in oyer-all rate and elim inates the pause after rein
forcement, except at a and b. After 4 sessions of further exposure to FI 1 T O 5 (R ec
ord B), the pause after the reinforcem ent has increased slightly, and occasional instances
of negative curv atu re (as at c an d d) occur. T h e over-all rate rem ains high. W hen
the T O was rem oved from the program at the start of the following session, R ecord
G was roughly sim ilar to Fig. 218D. T h e over-all rate declines through the first few
segments. T he grain of the record is rough, a n d the earlier term inal rate is seldom
200
reached or sustained. T he last recorded perform ance on F I 1 alone, 4 sessions later, is

shown in Record D, where a pause and lower rate during the early p art of the inter
val are m ore m arked an d the over-all rate lower (cf. Fig. 218E). T he perform ance
is irregular, however; an d negative curvature often occurs after the term inal FI rate is
reached (as at e , f an d g ).
In sum m ary, a T O after reinforcem ent on F I 1 first reduces the pause and lower
rate at the start of the interval and increases the term inal and over-all rates of respond
ing. W ith further exposure to the schedule, some pausing at the start of the interval
returns, although it is not so m arked as w ithout T O . W hen a T O is rem oved, the
pause after reinforcem ent is tem porarily elim inated. In this bird, rough grain and a
failure to hold a term inal rate characterized the final perform ance under FI 1 w ith
out T O ; but this condition is probably due to the factors which caused the negative
acceleration in Fig. 129A.
F I 2. Figure 220A shows a fairly stable perform ance on FI 2 after 1300 reinforce
m ents. (For earlier stages, see Fig. 119 and 150.) T h e pause after reinforcem ent
varies considerably; but no instance occurs where the reinforcem ent is not followed by
some pause. Occasionally, a high rate develops in the m iddle of the interval and is
followed by slight negative acceleration to a lower rate. Figure 220B shows the session
following R ecord A, in which a 5-m inute T O followed each reinforcem ent for the first
tim e. Some responding during T O is evident at a. Pausing is reduced m arkedly
a n d the term inal rate increased. W here a pause occurs, the acceleration to the higher
rate is m ore ab rupt than in R ecord A, where a few responses occur in most segments
before the term inal rate is struck. T he 5th session on FI 2 T O 5 (R ecord C) shows
201
m arked negative curvature (as a t c, d, a n d e). Pauses after reinforcem ent are either
absent or m uch reduced. (R esponding during the T O occurs at b.) In R ecord D,
after 10 further sessions with T O , strong negative curvature appears (as a t f and g).
No pause takes place at i, and responding occurs during the T O at h. Pauses and
scalloping are considerably less than before the introduction of T O .
In the session following Fig. 220D the T O was removed. T he perform ance (Fig.
221 A) shows very rough grain and irregular rate changes. Slow responding appears
at a, b, a n d c, a n d m arked negative curvature is still evident. R ecord B shows the
beginning of the next session, also w ithout T O . Some further progress is m ade tow ard
recovering the original FI 2 perform ance of Fig. 220A. This session continued for a
Fig. 2 2 1 .
Removal o f TO 5 from FI 2
total of 528 reinforcements, generating a very long satiation curve. T he rem ainder
of the session will be presented in Fig. 380 and 381 later, under the section on satiation.
T he bird took several days to return to the designated running weight. Figure 222A
shows the 1st perform ance following the satiation curve. A substantial period of slow
responding follows most reinforcem ents. T ow ard the end of the session, however,
m arked negative curvature appears later in the interval, along w ith an over-all lower
rate. W hen the T O was added, at the start of the following session (Record B), the
Fig. 2 2 2 .
Return to Fl 2 TO 5 a fte r Fl 2
F IX E D IN T ER V A L
203
pause after the reinforcement im m ediately disappeared and the term inal rate increased.
As the session progresses, a short pause following the reinforcem ent reappears. W hen
a pause occurs, the shift to the term inal rate is m ore a b ru p t than in Record A.
T h e FI 2 T O 5 was continued for 145 reinforcements, an d the final perform ance is
shown in Fig. 223A. A pause or period of slow responding at the start of the interval
develops slowly as the session progresses. T ow ard the end of the session the segment
is usually S-shaped. T h e grain is rough, an d there are m arked local fluctuations in
rate. R esponding during tim e out occurs occasionally (as a t a). In the following
session (R ecord B) the T O after reinforcem ent was rem oved for the second tim e. A
pause following the 1st reinforcem ent a t b is absent, b u t the 2nd reinforcem ent is fol
lowed by a substantial pause, and the rem ainder of the period shows m arked scallop
ing. T he term inal rate is not held, however, and most intervals show negative curva
ture. This is a m ore rapid transition th an the 1st retu rn to no T O . Record C shows
a later perform ance, 190 reinforcem ents after R ecord B. H ere, the over-all rate of
responding has declined m ainly because of a greater tendency for lower rates to occur
during the 1st p art of the interval. T he negative curvature during the latter p art of
the interval is m arked and the grain rough.
Summary. T he effect of the tim e out on FI 2 was sim ilar to the results reported for
FI 1. T he T O after reinforcem ent increases the over-all rate of responding by increas
ing the term inal rate of responding during the interval, and elim inating or shortening
the pause after reinforcement. U nlike the perform ance on FI 1, the final perform ance
on FI 2 T O show ed m arked negative cu rvature, w hich th en persisted even w hen the
T O was rem oved. At this size FI the introduction of a T O does not produce a good,
scalloped interval perform ance. W hen the T O is removed, the FI 2 performance con
tinues to show irregularities beyond those which occur in FI 2 after erf.
F I4 . A 5-m inute T O after reinforcem ent was added to the fairly stable perform
ance on FI 4 shown in Fig. 152, the last recorded segments of which are shown in Fig.
224A. In the following session, R ecord B, the T O elim inates the pause a n d scal
lop at a and b; b u t slower responding appears after the reinforcem ent in the 4th in ter
val (c), with a smooth acceleration to a term inal rate. T h e curvature increases p ro
gressively, a n d a pause appears after the reinforcem ent a t d. By the 10th reinforce
m ent (at e) the pause and the scallop resemble those in R ecord A. Figure 225A shows
the last perform ance after 10 hours of FI 4 w ith T O after reinforcem ent. T h e p e r
form ance here is som ew hat m ore irregular th a n in Fig. 152 a n d 224, possibly because
of some aversive effect of the T O . T h e pause after reinforcem ent and the term inal
rate (cf. a a n d b) vary considerably. T h e bird m ay begin to respond soon after rein
forcement and strike the term inal rate early (as in c and d). W hen T O after reinforce
m ent was rem oved on the following session, the performance (Record B) showed con
siderable disruption. T h e bird responds soon after reinforcem ent, although a few
pauses still occur (as a t e and h). T he grain is rough, a n d m arked rate changes occur
after the term inal rate is reached (as a t g). Slow responding m ay extend th rough
the interval i f ) , and second-order effects ap p ear (as at ).
Fig. 2 2 6 .
205
Return to FI 4 TO 5 a fte r FI 4
Tim e out was again added to the reinforcement. T he first performances were lost
th rough trouble w ith the apparatus. Figure 226A, however, shows the 3rd session of
FI 4 w ithout T O . T he session begins w ith a high rate sustained throughout the in
terval. Pauses and acceleration in the early p art of the interval develop progressively.
T he last p art of the session shows the earlier standard perform ance. W hen T O was
ad d ed to the reinforcem ent on the following day, the responding was im m ediately re
sum ed after the 1st reinforcement. T he experim ent was stopped; but it was contin
ued the next day (R ecord C), giving a perform ance sim ilar to R ecord A. Figure 227
shows a later perform ance on FI 4 T O 5. T h e over-all rate of responding is som e
w hat higher than norm al; and several instances occur in which the bird starts soon after
206
the T O after reinforcement, with subsequent m arked curvature (b, d, and e). Negative
curvature is occurring in m any of the intervals (as at a, c, and f ) .
A com plete experim ental session 7 hours after a second retu rn to F I 4 only is shown
in Fig. 228A. T he term inal rate is lower th an in Fig. 227, w ith a resulting lower over
all rate. M arked negative curvature occurs (as at a, b, a n d c), and the grain is m uch
rougher th a n w ith T O . This perform ance gives way to th a t shown in R ecord B, 2
hours late r, w hich resembles the final FI 4 perform ance shown earlier in Fig. 152. A
rem n an t of the previous negative acceleration is ap p aren t in the short runs a t higher
rates occurring a t d and e.
F I 8. Figure 229A shows the first exposure to FI 8 T O 5. T he im m ediately pre-
207
ceding session on FI 8 (186 hours after erf) presented earlier (in Fig. 154) shows a
very irreg u lar perform ance, although every reinforcem ent was followed by at least
some pausing. T he first tim e out at a is followed by sustained responding th rough
out the FI segm ent; the rem ainder of the record shows a higher over-all rate, a re
duction in the am o u n t of scalloping, an d instances of responding at the start of the
interval (a, b, c, and d).
T he pause an d lower rate at the start of the interval become slightly more p ro
nounced at the start of the 2nd session (R ecord B); and in Fig. 230A, at the start of
the 4th session w ith T O , the pause an d scallop at the start of the interval are of the
same order as w ithout T O . T he ab ru p t shift from the pause to the term inal rate, how
ever, has given way to fairly smooth curvature. By the start of the 6th session (R ec
ord B) the pause again becomes shorter and the shift to the term inal rate more abrupt.
R esponding occurs at the start of the 2nd interval in spite of the T O . Segments at a,
b, an d c show the highest rate of responding in the m iddle of the segment. T h e T O
was then rem oved for 70 hours, and the return to FI 8 T O 5 (in Fig. 231A) resembles
the first tim e the T O was used, although the transition is somewhat faster. A constant
rate throughout the interval gives way to a pause and scallop th a t become very m arked
by the 7th segment. N early all of the segments show a pause w ith smooth and ex
tended curvature 5 sessions later (Record B).
A 2nd return to FI 8 T O 5 after 165 hours w ithout T O produces an even quicker d e
velopm ent of a pause and smooth scallop (Fig. 232). By the end of the session, most
of the intervals are showing long pauses and smoother and m ore extended curvature
th an heretofore.
Figure 164 showed a perform ance on FI 8 w ithout T O in which a pause after re
inforcem ent and a smoothly curved scallop were common. A nother bird in the same
experim ent failed to develop sim ilar segments. Figure 233A shows a perform ance
after 170 hours on FI 8 after erf. T he over-all rate is low, as it h a d been th roughout
208
th e experim ent, an d the grain is rough. Slight pauses m ay or m ay not occur after
reinforcem ent. Breaks occur elsewhere, and little or no scalloping appropriate to the
schedule appears. A 5-m inute T O after reinforcem ent was introduced at the start of
the following session (R ecord B). T he over-all rate of responding increases to 3100
responses per hour, com pared w ith a m ean rate of only 2300 responses per hour for a
10-hour sample before the introduction of T O . T he scallop at a at the beginning of
the session is characteristic of this bird and occurred w ithout T O in earlier sessions.
Fig. 2 3 3 .
209
The effe ct o f TO 5 on FI 8
(second bird)
U p to this point the only effective T O was overnight in the hom e cage. T h e 1st T O
(at b) reproduces a lower rate a n d smooth acceleration, which are repeated at c and
d. T hereafter, bad grain develops, an d the over-all p a tte rn is not very different from
R ecord A except for the over-all rate. Figure 234 shows the rem ainder of the 1st
session with T O after reinforcem ent following im m ediately after Fig. 233. For some
time the perform ance remains roughly linear, with occasional pauses or slower respond
ing after reinforcem ent (as at a a n d b). Slight curv atu re begins to ap p ear consist
ently at c and d, and the curvature becomes well-m arked, at e , f and g, before the end of
the session. In a single experim ental session of ab o u t 90 m inutes, a 5-m inute T O
produces w ell-m arked scallops, although these had not developed during 170 hours u n
der FI 8 alone. T he bird was apparently unable to develop a scallop or to respond
appropriately to the fixed-interval schedule because of the carryover of stimuli from
the preceding interval.
T he subsequent history of this bird on FI 8 T O 5 is sam pled in Fig. 235 through 238.
These show an instability ranging all the way from a sm ooth perform ance conform
ing to the fixed-interval pattern of reinforcem ent to a perform ance showing high over
all rates with only a slight decline after reinforcement.
Figure 235 shows the entire 4th session after 12 hours of F I 8 T O 5. P a rt of this
performance is not very different from th at prevailing in the absence of T O (Fig. 233A).
210
Occasional instances of substantial pauses or substantial reductions in rate after re

inforcem ent appear, however. T he m any instances of negative curvature th a t occur
(as a t b, c, d, e, a n d f ) suggest th a t the higher rate a n d earlier start due to the T O
produce more responses th an can be sustained on FI 8. Extinction therefore sets in
during almost every interval; but most segments begin with some positive curvature a p
p ro p riate to the fixed interval. This cu rv a tu re is most conspicuous at a, g, an d h.
T he grain is rough, and an occasional interval is run off a t a very low rate.
A higher term inal rate later develops as in Fig. 236 after 32 hours of exposure to
F I 8 T O . A longer pause after reinforcem ent prevents an excessive n u m b er of re
sponses per interval, b u t the term inal rate is frequently broken by a decline before the
Rs
150
Fig. 2 3 6 .
Fl 8 a fte r 32 hr TO 5 (second bird)
212
interval is com pleted (as a t b a n d c). O ccasional instances occur of fairly sm ooth
curvature to a term inal rate which is sustained almost to reinforcement (as at a). At e,
after a long pause, the rate begins higher than usual and declines before the term inal
rate is again reached and m aintained.
Figure 237A shows a somewhat different perform ance after 39 hours of exposure to
FI 8 T O 5. Pauses after reinforcem ent are sharply reduced; instead, intervals tend to
begin w ith periods of slower responding, with rough grain. T he term inal rate rem ains
high until the reinforcement. This is a passing phase, however, and pauses again re
tu rn by the end of the session shown at Record B. Occasionally, as best exemplified at
g, there is a smooth interval scallop similar to the m ore regular types of perform ance
on this schedule of reinforcem ent. A t a, b, c, a n d d a series of intervals show a pro
gressive deepening of the scallop. A long pause after reinforcem ent leading abruptly
to a high rate followed by negative curvature is evident at h. M arked examples of
knees occur at e and f .
T he change from one to the other of these different perform ances in a single ses
sion must be due to very subtle conditions. Figure 238 shows a portion of a session
after 4 hours; the over-all rate is considerably lower, because of m arked pauses and
scallops and failure to m aintain the term inal rate of responding (a, b, c, and d).
Figure 239 shows a com plete session tow ard the end of the experim ent, beginning
after 72 hours of exposure to FI 8 T O . H ere, a variety of changes in rate occur in sin
gle segments: (1) negative curvature in a, b, c, d, etc.; (2) a series of intervals showing a
g rad u al increase in the am o u n t of scalloping, as a t e , f g, h, i, a n d j ; an d (3) fairly
sm ooth acceleration to a term inal rate in spite of a rough starting grain, a t k and I.
T he over-all rate shifts from a low value during the 1st hour of the experim ent to a
higher value in the m iddle p a rt of the session.
213
Figure 240A shows the final perform ance for another bird on FI 8 (already repre
sented in Fig. 164) for the 140th h o u r after erf. It is m ark ed by reinforcem ents fol
low ed by sm ooth acceleration to a term in a l ra te (a a n d c), b u t also by occasional
intervals in w hich the term inal rate is assum ed im m ediately after reinforcem ent and
m aintained until reinforcem ent (at d). Some ru n n in g -th ro u g h appears a t b. O n
the following day, Record B, a 5-m inute T O followed each reinforcement. In general,
scalloping is m ore m arked. T he 1st interval of the session a t e is typical of the start
of the session. T he T O is effective a t/e v e n though the term inal rate is reached fairly
quickly. However, no instances occur here, or in any subsequent session, of imme-
Fig. 2 4 0 .
The e ffe ct o f TO 5 on FI 8
(third bird)
214
diate responding a t the start of the interval. A n early effect is to produce occasional
intervals w ith alm ost no responding (as a t g and h).
W ith further exposure the T O produces a m ore uniform perform ance from interval
to interval. Figure 241 shows the com plete record for the 2nd session with T O after
reinforcem ent. T h e pauses following reinforcem ent are m ostly brief, except for the
segments at b an d i. M any instances occur of fairly smooth curvature (as a t b, d, e,
an d j ) , a n d also instances of fairly a b ru p t shifts to the term inal rate (as a t a a n d c).
Some intervals for e x a m p le ,/a n d g never show a term inal rate. A slight negative
acceleration occurs a t h. In general, however, the record shows a fairly narrow range
of the n um ber of responses em itted in intervals and only an occasional succession of
intervals w hich could be regarded as a second-order effect. A nother characteristic of
FI 8 T O 5 exem plified by this record is an occasional set of intervals showing a p ro
gressively slower b u t smooth acceleration to the term inal rate.
In order to show sets of intervals in w hich the n u m b er of responses progressively
declines, all intervals from a daily session have been arranged on a common baseline in
Fig. 242 in the order of their occurrence. This perform ance occurred 16 hours after
the first introduction of TO . T he 2nd interval, at a, showed the largest num ber of re
sponses per interval during the day a n d is the first of a set of 7 intervals showing a
progressive decline. At the end of this series, reinforcem ents occur after a small n u m
ber of responses; this occurrence appears to set up a new contingency, which leads to
215
a higher term inal rate, at b. T he set of segments beginning at b shows a rough decline,
as does a n o th e r set of 5 intervals beginning a t c. T o w ard the end of the session,
m ark e d negative cu rv a tu re has begun to a p p e a r (at d an d e) in intervals in w hich a
high rate is reached early in the segment. T he high rate presum ably occurs because
increasing the num ber of responses is now reinforcing.
These orderly changes show th a t even w ith a 5-m inute T O after reinforcem ent,
th e perform ance on FI 8 is not stable. C ontingencies arise under one level of perfor
m ance which produce a change. Specifically, in this case, if the num ber of responses
from interval to interval becomes stable and small, num ber becomes a discriminative
stim ulus which reinforces early responding in the interval. This in tu rn increases the
rate a n d disrupts the stable FI perform ance. This instability m ay well be a p ro p
erty of the m iddle range of FI values in which num ber is of the m agnitude required
to show a sensitive control.
O ccasionally, a fairly stable condition m ay be m ain tain ed for some tim e. Figure
243 begins after 34 hours of exposure to F I 8 w ith T O after reinforcem ent. T he
n um ber of responses em itted per segment does not change greatly during the intervals
en d in g a t a, b, c, d, a n d e. H ow ever, some progressive reduction occurs, a n d the
small count at f sets off an oscillating disturbance. In order to show the kinds of curva
tu re w hich occur u n d e r F I 8 T O 5, a com plete daily session after 50 hours on this
schedule is given in Fig. 244, w here the segm ents have been arra n g e d in separate
groups roughly according to curvature. W ithin each group the curves are arranged
in order of the n u m b er of responses in the interval. T he n u m b er of responses per
interval here ranges betw een about 40 a n d 400. T h e set of segments in G roup A
contains 17 intervals, which seem appropriately classified as examples of simple positive
acceleration. At the low er rates of responding there is some sticking of the trig
ger. O ne curve (at a) shows a continuous acceleration throughout the whole
segment. T he rest start at zero and then accelerate to a constant term inal rate be
fore the end of the interval. T he term inal rate is low in the lower h alf of these curves.
216
Fig. 2 4 4 .
Types o f curvature under FI 8 TO 5
G roup B shows 4 segments with negative acceleration. T he decline is only ju st set

ting in at b, but it is more clearly m arked in the other cases. A group of 5 intervals
at C show a considerable delay in starting, and then a shift to a rate which appears
to be a com pensation for the delay. A nother group of 4 intervals showing delay in
starting, followed by a shift to a high rate briefly, an d then negative curvature is at
D. T he n um ber of responses in these segments is very small, however; and it is not
clear w hether these records could possibly form p a rt of the series in G roup A. O ne
interval (at the start of the session) showed early responding followed by a very long
pause extending 9 m inutes beyond the end of the figure before a response occurred
a n d was reinforced. (See R ecord E.)
F I 16. T h e effect of a 5-m inute T O on FI 16 was tested on the perform ance already
described in Fig. 170. T he T O was introduced in the session following R ecord B in
th a t figure. It reduced or elim inated pauses at the beginning of the interval, an d the
quicker acceleration to the term in a l ra te increased th e over-all rate. Figure 245
shows later samples of the perform ance. In R ecord A, the start of the 4th session, the
217
ra te accelerates fairly sm oothly to a term in al ra te w hich is m ain ta in e d until rein

forcem ent. Pauses after reinforcem ent are absent a t a an d b, however, a n d the p re
vious term inal rate is m aintained for a few responses. R ecord B, the end of the 6th
session, shows a sim ilar perform ance, w here there is a slight run-through at c. At this
stage the rate following the reinforcem ent tends to be som ew hat lower th an in Fig.
170. In Fig. 246, the 7th session after the introduction of T O , the intervals arranged
on a com m on baseline in the order in which they occurred in the experim ent show
a progressive increase in the am ount of curvature over the first 9 intervals. By the 9th
interval, however, the curvature is so extended th a t a term inal rate is not reached.
After reinforcem ent at this low rate, the segments begin to oscillate betw een high and
low num bers of responses.
W hen a T O no longer followed reinforcement, the performance was badly disrupted.
Figure 247 shows the 1st session. T he term inal rate is assumed quickly after the 1st
reinforcem ent; and although the local rate varies considerably, a large n um ber of re
sponses is em itted in the interval. N evertheless, the term inal rate is im m ediately
assumed. M arked negative curvature then appears a t a. Thereafter, the rate oscil
lates slowly an d has little relation to the schedule of reinforcem ent.
W hen the FI 16 schedule was continued for a total of 35 hours, the perform ance sta-
Fig. 2 4 6 .
FI 16 a fte r 6 sessions o f TO 5
300
RESPONSES
218
bilized a n d a n orm al FI perform ance (sim ilar to th a t in Fig. 170) em erged. T h e

T O was th en ad d ed to the reinforcem ent for the 2nd tim e; the result is shown in Fig.
248. R ecord A shows the start of the 1st session, and R ecord B shows the end of a ses
sion 25 hours later. Note the progressive increase in the extent of the scallops at a,
b, a n d c. A low er rate of responding a n d pauses after th e reinforcem ent develop
m ore rap id ly th a n during the 1st introduction of the T O . T he final perform ance
(R ecord B) shows a m ore extended curvature and a m ore consistent interval-to-interval perform ance than under FI 16 w ithout TO .
W hen T O was again rem oved from the program , considerably less disruption re
sulted th an w hen the change was first m ade (Fig. 249). T he perform ance is never
theless irregular, with running-through at a and b, a n d m any intervals showing a sus
tained rate of responding throughout (c and d).
After a fu rth er 71 hours of FI 16 w ithout T O (Fig. 250), a fairly good fixed-interval
perform ance develops. T he over-all effect resembles the perform ance with T O after
reinforcem ent, except th a t T O after reinforcem ent has a greater depth of scalloping.
Note th at in Fig. 250 the scalloping is more m arked an d the performance more con-
219
sistent th an in Fig. 170, which was the final perform ance after m ore th an 300 hours
of exposure to FI 16. T he intervening exposure to FI 16 T O has resulted in a more
m arked and stable interval curvature w ithout the aid of TO .
T h e most regular fixed-interval perform ance shown by this bird w ith T O after
reinforcement is reproduced in Fig. 251, in which the intervals are represented in the
order of their occurrence but on a com m on baseline. Except for the last 2 intervals,
the num ber of responses from interval to interval varies w ithin a narrow range. T he
grain is rough, and the curvature varies considerably (cf. b and c). T he pause after
reinforcem ent varies from zero at a to several m inutes a t d. T h e term inal rate is
fairly uniform and the num ber of responses per interval fairly constant.
Time-out probes in FI 1
A regularly scheduled T O , as in the experim ents just described, can enter into the
reinforcing contingencies as an im portant stimulus. An occasional T O is relatively
free of this com plication. It can be used as a probe, as described in C hapter Three.
In a fairly stable perform ance on FI 1 (sim ilar to th a t in Fig. 149) a 30-second T O
Fig. 2 5 0 .
FI 16 showing scallops w ithout TO
Fig. 2 51.
M ost re gular Fl 1 6 TO 5 perform ance
10 MINUTES
Fig. 2 5 2 .
TO 30sec probes in the m iddle o f every 10th FI 1 segment
10 MINUTES
Fig. 2 5 3 .
TO 30sec probes a fte r every 10th reinforcem ent
221
probe was program m ed after every 10th reinforcem ent. T he 9 intervening segments
provided a baseline forju d g in g the effect of the T O and for reducing, if not prevent
ing, the developm ent of a stim ulus function.
T he FI 1 segment was first probed m idw ay in the interval. T he term inal rate of re
sponding had usually been reached at this point. Figure 252 shows segments from
3 daily perform ances over a 5-day period. Tw o reinforcem ents following the T O and
one preceding it are presented. Arrows m ark the probes. No reinforcem ents oc
c u rre d at these points, of course. R eco rd A shows th e 1st session in w hich th e T O
probe occurred. In no session does it have any effect. (T h e last T O at a occurs
before the term inal rate is reached, and a som ew hat lower rate follows.)
Fifteen sessions later, the T O probe was program m ed im m ediately after every 10th
reinforcement. It had the effect of elim inating the pause. Figure 253 shows 3 exam
ples from a single-session probe typical of the perform ances of 2 pigeons.
In a later session with the same birds the probe occurred 45 seconds after the start of
the interval. Figure 254 shows a representative perform ance approxim ately 10 ses
sions after Fig. 253. H ere, there appears to be some effect on the following interval seg
ment. T he probe reduces the am ount of pausing after the next reinforcem ent. Note
the negative acceleration during the intervals after the probe. This suggests th at the
probe has the effect of an aversive stim ulus, w hich has been dem onstrated elsewhere.
M easurem ents were m ade over the 10-day period in which the T O probes occurred 45
seconds after every 10th reinforcem ent. T he num bers of responses in the interval
segments preceding an d following the probed interval were counted. In one bird the
segm ent following the probed segment shows m ore responding in 21 cases out of 27,
a n d the m ean num bers of responses are in the ratio of 1.5 to 1. In a second bird the
figures are 19 an d 29 in a ratio of 1.3 to 1. A th ird bird, how ever, failed to con
firm this result.
T he effect of a T O probe in elim inating the pause at the start of the fixed interval
seems to indicate th a t the norm al pause a n d period of slow responding u n d er FI 1 are
222
due to a discrim ination based on recent eating. T he T O probe covers the tim e when
eating is an effective stimulus. Since a 30-second T O eliminates the pause, we m ay
conclude th at the effective duration of the reinforcem ent as a controlling stimulus is 30
seconds or less.
Any effect on segments subsequent to the one in which a T O probe occurs, as in Fig.
254, w ould presum ably be felt from a reduction in the stim ulus carryover from re
sponding in earlier segments from either the num ber of responses or the rate of respond
ing. T h e order of m agnitude of the effect of the T O after reinforcem ent, however,
indicates th at the m ain controlling factor in FI 1 is the preceding reinforcement. At a
larger value, num ber and rate m ay be m ore effective.
Sum mary o f time out
T h e effect of a T O depends on the value of the FI. At shorter FIs (up to 4 m in

utes) the T O after reinforcem ent increases the over-all rate by elim inating the pause
and curvature. C ontinued reinforcem ent w ith the T O re-forms the scallop, although
not so well as w ithout the T O .
T h e final perform ances on FI 4 a n d FI 8 w ith a n d w ithout T O were sim ilar except
th a t the curvature is som ew hat sm oother w ith T O .
O n larger FIs (e.g., 16 m inutes and over) the T O produces a stable performance with
a pause and extended curvature at the start of the interval. In contrast, curves for
perform ances w ithout T O are very irregular, a n d pausing an d sm ooth scalloping are
either rare or absent. Instances still occur, how ever, w here there is responding a t
th e start of the interval in spite of the T O . Those becom e rare as the FI becomes
larger (FI 45).
T h e transition from T O to no T O and vice versa tem porarily elim inated the scal
lop an d stable perform ance except for F I 45 to F I 45 T O . Each transition produced a
tem porary increase in the over-all rate w hen the T O was either added or removed.
Time-out effects upon FI schedules in the rat
T h e tim e out we have m ost frequently used w ith pigeons is a b lac k o u t, p ro

g ram m ed by tu rn in g out all the lights in the a p p a ra tu s. A n u n tra in e d pigeon will
ordinarily not respond in the dark. Because a ra t is nocturnal, its behavior cannot
be controlled in the sam e way, i.e., via illum ination of the experim ental cham ber. As
noted in C hapter T hree, a well-developed discrim ination m ay be used instead.
In the following experim ents the ra t was first subm itted to a procedure in which it
was always reinforced in the presence of a light a n d never reinforced in its absence.
T echnically, this schedule is m ult erf ext. In a given case lever presses occurring
in the absence of the light are not reinforced for a 5-m inute period. T he light is then
turned on, the response reinforced, etc. Eventually, responses only rarely occur in the
absence of the light, which can then be used as a T O in the same m anner as a blackout
with pigeons.
Time out after reinforcement. Records A through I of Fig. 255 contain a representa-
Fig. 2 5 5 .
223
Early developm ent o f FI 5 TO 5

in the ra t
tive segment from each of the first 9 sessions on FI 5 T O 5 after a history of m ult erf ext.
T h e 1st session (R ecord A) shows a slow developm ent of a fairly constant rate. R e
sponding d u ring the T O was near zero; it was not significantly higher th a n during
the previous m ult erf ext. By the 4th session, in R ecord D, a fairly sustained over-all
rate develops. Some scalloping is clear in Record E, the 4th session; and by the 6th
session, in R ecord F, a higher term inal rate is evident, although reinforcem ent is post
poned in some intervals because the ra t is not responding. T he final perform ance
in the 9th session following erf (R ecord I) shows a higher over-all rate of responding,
rougher grain, and m uch less pausing after the reinforcem ent, even though the seg
ments are consistently scalloped.
Figure 256 contains the entire daily session for this ra t after 32 hours of exposure to
FI 5 T O 5. A w ell-m arked pause follows each reinforcem ent, and the rate acceler
ates fairly sm oothly to a term inal value except at a a n d b, w here pauses cover most of
the interval. Two other rats showed sim ilar performances.
Time-out probes. T he im p o rtan ce of T O in th e F I 5 perform ance shown in Fig.
Fig. 2 5 6 .
FI 5 TO 5 a fte r 32 hr (rat)
224
256 was probed by the omission of the T O after single reinforcem ents or small groups
of reinforcem ents. All cases are presented in Fig. 257, w hich gives portions of 4
daily sessions. At the arrow , T O s after reinforcem ent were om itted. In R ecord A,
after 31 hours of FI 5 T O 5 after reinforcem ent, the omission of T O produces some
responding im m ediately after reinforcem ent, alth o u g h this is followed by a zero rate
and norm al acceleration. In R ecord B, for a second rat, after 18 hours of FI 5 T O 5
after reinforcem ent, the over-all rate of responding is very low and the omission of the
T O has no effect. However, the sam e rat showed a higher over-all rate after 6 hours
of further exposure to the schedule (R ecord C), an d the omission of the T O (at the
arrow ) produces a continuation of responding a t the term inal rate in the preceding in
terval. In R ecord D, for the ra t in R ecord A after 44 hours of FI 5 T O 5 reinforce
m ent, the T O was om itted for 4 consecutive reinforcements, indicated by the arrows,
at a, b, c, a n d d. Since the ra te of responding was low preceding the first probe, the
omission of the T O did not result in any increase in rate. A slightly higher rate of
225
responding occurs in the interval at a; a n d the 2nd reinforcem ent at b, not followed
by a T O , shows some responding im m ediately following the reinforcem ent. T he 3rd
reinforcem ent w ith om itted T O (at c) is followed by a substantial rate, which, after
some decline, leads quickly to a term inal rate. T he 4th omission of the T O at d also
produces a few responses im m ediately following the reinforcem ent, as well as a knee.
T h e rem a in d e r of th e session from e, w ith T O after reinforcem ent, shows consistent
pausing a n d m arked scalloping. (T he experim ent illustrated by Fig. 201 showed
th a t the omission of T O produced a high rate after reinforcem ent only when the bird
had reached the term inal rate of responding in the preceding interval.)
All TO s were then w ithdraw n while FI 5 continued in force. Figure 258 shows seg
ments from the 1st, 2nd, and 4th sessions, all w ithout T O , for 1 rat. In the 1st ses
sion (Record A) the over-all rate is low and no term inal rates com parable with those
w ith T O after reinforcem ent (Fig. 256) are reached. M ost intervals show pauses
after reinforcem ent, although some ru n ning-through occurs a t a a n d b. T he over-all
rate of responding increases in the 2nd session (R ecord B), a n d a t c, d, e, a n d f no
pausing occurs after reinforcem ent. In the segm ent beginning at d the term inal rate
is m aintained throughout. In the 3rd session (Record C) the performance is similar;
most intervals show a pause after reinforcem ent w ith a n o rm al acceleration to a te r
m inal rate, although at g the responding begins im m ediately after reinforcement.
After 12 hours of exposure to F I 5 w ithout T O , T O was again added. R ecord D
shows some responding during the T O , particularly at h and i. T he first 3 segments
226
Fig. 2 5 9 .
Extinction a fte r FI 5 TO 5 (rat)
show some negative acceleration and considerable disruption. Otherwise, however, a

more orderly perform ance emerges.
Extinction after FI. Figure 259 shows 2 extinction curves after F I 5 T O 5 in the pre
ceding experim ent. T he curve at A (com pleted by the segment above it) is for ex
tinction after 29 hours of FI 5 T O 5. It shows an orderly decline in rate with few
oscillations, suggesting extinction after V I. (See later chapters.) T he curve at B,
for an o th er rat after 36 hours on FI 5 T O 5, shows an FI interval effect m uch better
developed. W hen reinforcement is missing at a, 5 m inutes after the start of the record,
responding continues at the term inal rate and then declines by b. This pattern sets
off a typical interval perform ance; a low rate prevails for the rest of the session.
Summary
T he T O probes of the rats FI 5 perform ance confirm the findings of the preceding
pigeon experim ents. O m itting an occasional T O produced a continuation of the te r
m inal rate from the preceding interval.
T he perform ance with T O after reinforcem ent showed consistent pauses and accel
eration, contrasted w ith FI w ithout T O w here occasional instances occurred of re
sponding at the start of the interval. Each tim e the T O after reinforcement was added
or removed, however, the FI perform ance was disrupted.
T h e FI segments in the rat contain a relatively longer period of pausing a n d curva
tu re th a n com parable perform ances w ith pigeons. This probably is a species
difference.
T he over-all and term inal rates of responding in this experim ent reflect the inade
quate am ount of reinforcem ent (0.05 gram ) used in the experiment.
TRANSITION FROM CRF TO FI TO 5
P erform ances u n d er FI 1, 2, 4, 8, an d 16 were recorded following erf w ith T O 5

after every reinforcem ent. O ne bird was exposed to each schedule for approxim ately
20 reinforcem ents per day every other day. T h e first 4 sessions will be described for
each.
227
FI 1
T h e transition to FI 1 T O 5 shown in Fig. 260A is m ore like the transition to F R

th a n FI, w ith a small extinction curve a t a an d a low rate of responding at b, followed
by an a b ru p t shift to a higher over-all rate. Some responding during the T O oc
c u rred at c a n d d\ an d at e a n d f there are substantial periods of lower rates a t the
beginning of intervals. These are followed by a fairly linear perform ance a t g. In
the 2nd session, R ecord B, there is an over-all acceleration during the first 7 reinforce
ments. W ell-m arked scallops soon appear at h, i,j, and k. T he over-all perform ance
is irregular, however, and intervals occur, as in R ecord A, w here little acceleration
develops during the segment. T he 3rd session following crf (Record C) shows a
m arked pause and a period of lower rate a t the start of most intervals; this period of
lower rate is even m ore pronounced during the 4th session (R ecord D).
This transition should be com pared w ith Fig. 118 for F I 1 after crf w ithout T O .
T he m ain difference is the rapid developm ent of the pause after reinforcem ent an d a
high term inal rate of responding in Fig. 260. A com parable perform ance em erged
after 160 reinforcem ents w ithout T O . In the present experim ent a w ell-m arked
scallop appears after only 40 reinforcem ents. Despite the slower developm ent, how
ever, Fig. 118 shows a greater uniform ity from segment to segment.
228
SC H ED U LES OF R E IN F O R C E M E N T
FI 2
Figure 261 shows the first 4 sessions of the transition erf to FI 2 T O 5. T he over
all form of the 1st session (R ecord A) is sim ilar to the transition to FI 2 w ithout T O ,
w ith each segm ent showing negative acceleration w hich becomes progressively less
m arked until a roughly linear phase is reached (a to c). U nlike the norm al transition,
how ever, the linear phase very quickly gives way to a rough F I scallop. O ne in
stance a t b breaks into the lin ear phase. T he 2nd session (R ecord B) begins w ith a
roughly lin ear phase; b u t the 4th interval (d) shows a rough positive acceleration
w hich becomes m ore m arked at e, g, an d h. R esponding occurs d u ring the T O a t/ ;
and for the rem ainder of the session, the start of each interval shows a m arked pause
229
and scallop. T h e same perform ance is sustained in R ecord C beginning 40 reinforce

m ents after erf. By the 4th session (R ecord D) the scallops become even m ore
m arked, w ith the initial pause often extending through most of the interval, as at j ,
k, and /.
FI 4
Figure 262 contains the first 4 sessions on FI 4 T O 5 after reinforcem ent following
erf. O nly the first 6 segments of the 1st session are presented (R ecord A) because of a
recorder failure during the rem ainder of the session. These segments are all roughly
negatively accelerated, an d the available portion of the over-all curve shows over-all
negative curvature. T he com plete 2nd session is shown in R ecord B. T h e first 3
segments are negatively accelerated. But a very brief linear phase follows in the seg
m ent at b, after which there is a progressive developm ent of positive acceleration a t
c, d, and e, which extends throughout most of the segment, although with very rough
grain. Some responding during T O occurs at a a n d b. D uring the rem ainder of the
session, the pause following the reinforcem ent becomes longer, the curvature becomes
m ore a b ru p t, a n d the term inal rate increases and is sustained for a longer period of
tim e (as at f g, and h ). T he 3rd session following e rf (R ecord C) shows over-all nega
tive acceleration because of the progressive developm ent of pausing and scalloping
during the session. T h e 1st segm ent of the session shows a sustained high rate. T he
following 3 segments, at i,j, and k, show fairly linear perform ances with rough grain.
These are followed at / and m by extended curvature thro u g h o u t most of the interval,
alth o u g h w ith very little pausing following the reinforcem ent. After Segm ent m,
th e session shows a m ark ed decline in th e rate. T h e over-all rate falls to a p p ro x i
m ately 0.2 response per second. T he perform ance continues to be m arkedly scal
loped, w ith most of the responding occurring tow ard the end of the interval. As the
num ber of responses in the interval becomes very low, the pause following the rein
forcem ent becom es m ore m arked, ex ten d in g th ro u g h m ost of the interval (as a t n).
T he 4th session following erf show n in R ecord D begins a t a m uch lower over-all
230
rate th an the 3 previous sessions and shows a continuous decline, until a t the end of the
session only a few responses per interval segm ent occur. In spite of the extrem ely
low rate of responding, however, the reinforcem ent is not being postponed beyond the
designated fixed interval. Responding occurs during T O at o, p, and q. This tra n
sition should be com pared with Fig. 120.
FI 8
T he record of the 1st session was lost because of a recorder failure; however, Fig. 263
shows the com plete 2nd session. T he 1st interval shows slight negative acceleration
a n d rough grain. T he bird responds d u rin g the first T O a t a, and the 2nd interval is
roughly linear at a lower rate of responding. A few responses occur during the 2nd
T O , at b. W ell-m arked pauses then begin to ap p ear a t the start of the intervals (as
at c a n d d). T h e c u rv atu re becomes m ore uniform a n d extended. In the 3rd seg
m ent, at e, and in the 4th segment, at g, the bird pauses beyond the designated interval,
Fig. 2 6 4 .
231
Further developm ent on FI 8 TO 5

a fte r erf
an d reinforcem ent is late. In both subsequent intervals ( f a n d h) the c u rv atu re is

slight, responding beginning im m ediately at nearly the term inal rate.
Figure 264 shows the 3rd and 4th sessions after erf. T h e 40th to 60th reinforce
ments are shown in Record A; the 60th to 80th, in R ecord B. T he scallop is lost over
night. R ecord A begins w ith a roughly linear perform ance and shows only a slow
increase in positive curvature, w hich becomes quite sm ooth by the segments at a, b, c,
a n d d. T he over-all rate of responding shows the continuous decline th ro u g h the
session seen in performances under FI 1, 2, and 4. R ecord B, the 4th session following
erf, shows a n even lower over-all rate of responding, because of increased pausing
and failure to reach as high a term inal rate. Portions of R ecord B which were poorly
recorded are indicated with dotted lines, which are not to be construed as the probable
course of the curve. T his tran sitio n should be co m p ared w ith Fig. 132 a n d 133,
showing the transition from erf to FI 8 w ithout T O , w here the perform ance even after
m any hours of exposure to the schedule did not show any good adjustm ent to the
fixed-interval schedule.
FI 16
T he effect of the first 20 reinforcem ents in the transition from erf was lost because
of a recorder failure. Figure 265 shows the 2nd session, containing the 21st to 40th
232
reinforcem ents. T he 1st segm ent begins w ith a linear segm ent w ith rough grain.
A m ark ed pause a n d acceleration develop d u rin g the first 3 intervals, at a, b, a n d c.
T he am ount of pausing after reinforcem ent increases progressively, and instances occur
at d and e w here the reinforcem ent is postponed beyond the designated interval. T he
term inal rate is not held well; a n d although the over-all rate is not m arkedly different
during the early and latter parts of the interval, the natu re of the responding during
the latter p a rt of the interval shows considerable variation. R esponding occurs d u r
ing T O at f an d g.
233
Figure 266 shows the 3rd session after erf (the 41st to 60th reinforcements). T h e
1st interval, at a, shows an anom alous perform ance in which there is an extended pause
through most of the interval. T he 2nd interval, at b, is essentially linear. A pause
a n d acceleration develop at c. A progressive increase in the pause a t the start of the
interval follows as in the previous session in Fig. 265. A t d the reinforced response is
the only response in the interval, an d it is followed by a continuation of the near-zero
rate. T he reinforcem ent at e follows a long ru n at a high rate. In spite of the 5m inute T O the bird continues a t a high rate for a few responses, a n d slows down only
slightly before finishing the next segm ent a t the term inal rate. T h e rapidity of the
transition from erf to F I 16 w ith T O is evident from a com parison of Fig. 265 a n d 266
with Fig. 123 and 124, which show the transition from erf to F I 17 w ithout TO .
Summary o f erf to FI TO 5
All 5 birds studied here on FI 1 to FI 16 T O 5 after erf show a m uch m ore rapid
production of a m arkedly scalloped FI perform ance th an those w ithout T O . In every
case a stable perform ance with a m arked pause following the reinforcem ent and an
acceleration to a term in al rate occurred well w ithin 80 reinforcem ents following erf.
(P art of the rapidity w ith which the F I scallop develops here m ay possibly be due to
the fact th a t the reinforcements are spaced out over several sessions.)
T he difference between transitions from erf to FI with and w ithout T O after rein
forcem ent is greatest at longer fixed intervals. At FI 16, for exam ple, a m arked pause
following the reinforcem ent and acceleration to a stable term inal rate occur between
the 40th and 60th reinforcements after erf. This is an extrem ely rapid development of
a close conform ation to the fixed-interval p attern of reinforcement. The rapid tra n
sition is consistent w ith the view th a t the m ajor effect of T O is to prevent the carryover
of behavior as a stim ulus from one interval to the next. This carryover is especially
im p o rta n t at longer intervals of reinforcem ent, w here schedules w ithout T O result in
unstable FI perform ances conform ing only poorly to the tem poral properties of the
fixed-interval schedule.
TRANSITION FROM ONE FI TO ANOTHER WITH TO 5
AFTER REINFORCEMENT
Five schedules of reinforcem ent, F I 1, 2, 4, 8, a n d 16, plus T O 5, were ro ta te d

am ong 5 birds, so th a t each bird changed once from FI 2 to F I 1, F I 4 to FI 2, FI 8 to
F I 4, FI 16 to FI 8, a n d FI 1 to F I 16, all w ith T O . T h e transitions were from a
larger value of FI to a smaller, except for the change from FI 1 to FI 16. (Also, 1 bird
was changed from FI 2 to FI 16.) E arlier perform ances for these birds at values of
FI assum ed after erf are shown in Fig. 129, 150, 152, a n d 170. Fairly stable perfor
mances with T O after reinforcem ent have already been described in Fig. 219, 223,
227, 232, an d 251. In the present experim ents a given value of reinforcem ent was
m aintained until a fairly stable condition developed. T he transition to another value
of FI was then m ade. T he results provide inform ation about perform ances on var-
235
ious FI schedules of reinforcem ent w ith T O , and show the extent to which we are now
prepared to describe in detail the transitional states.
T h e 5 birds approached a given case of transition from different histories in this ex
perim ent. In the following reports each history is briefly indicated. Earlier experi
m ents on FI 1 with and w ithout T O are not indicated. All schedules included TO s.
FI 1 to FI 16
History: FI 2; FI 1 (240 reinforcements). Figure 267 shows the 1st ses

sion on FI 16 after FI 1. T he previous FI reinforcem ent produces a high over-all rate
of responding, w hich declines slowly th ro u g h o u t the session, as in extinction after
Bird 49T.
variable-interval reinforcem ent. A high rate is m aintained throughout the 1st FI 16

segment, which ends at the reinforcem ent at a. This high rate is the equivalent of
th e first p art of an extinction curve after FI 1. T hereafter, lower rates develop some
tim e after reinforcem ent. Oscillations involving either no responding (as at b) or a
lower rate of responding (as at c and d) are characteristic of the early transition.
T he rate during the first p art of the fixed-interval segment continues to fall during
the 2nd session on F I 16; and by the end of th a t session the fixed-interval curvature is
w ell-m arked, as in Fig. 268A. T he end of the 3rd session u n d e r FI 16 is shown in
Record B. T he period of slow responding during the first p a rt of the interval has b e
come m ore extended, as at b, c, and d. T he term inal rate of responding has increased
correspondingly. T he high rate im m ediately following the reinforcem ent a t a, break
ing into a more appropriate scallop, is probably a result of the previous FI reinforce
ment.
Figure 269 shows segments from the 4th, 6th, an d 9th sessions on F I 16. In Record
A, the end of the 4th session, the over-all rate of responding is low. By the end of
th e 6th session, in R ecord B, the pause after reinforcem ent is longer an d the term inal
10 M I N U T E S
Fig. 2 6 8 .
Second and third sessions on FIT 6 TO 5 a fte r FI 1 TO 5 (bird 49Y)
237
rate higher. At a the term inal rate from the preceding interval continues beyond
the reinforcem ent for approxim ately 10 responses, despite the 5-m inute T O period
sep aratin g the 2 segments.
B ird 5 2 G. History: FI 4; FI 2; FI 1 (350 reinforcem ents). Figure 270 represents
the transition. It is sim ilar to Fig. 267 in th at it shows a high over-all rate a t the start
of the session (a) which declines as the effect of the previous FI 1 reinforcem ent is extin
guished u n d e r the new schedule. H ow ever, the b ird showed a m uch m ore rap id
developm ent of a pause and a period of slower responding in the early p art of the in ter
val. T he high rate of responding resulting from the previous FI 1 perform ance at a
continued to appear throughout the rem ainder of the session, although the rate usu
ally fell off before reinforcem ent, as at g ,j, a n d /. In spite of these frequent reinforce
ments at lower rates of responding, the over-all rate an d num ber of responses per in ter
val rem ained high. R unning-through occurs at c, d, i, and m, possibly because of the
previous FI 1 reinforcem ent. T h e early appearance of a scallop in this bird m ight be
due to the earlier history on F I 4. T h e oscillations appearing at b, e, and f an d the
type of cu rv a tu re at h a n d k suggest previous perform ances on intervals of 4 m in
utes.
Figure 271 shows the subsequent perform ance on F I 16 for this bird. R ecord A,
from the end of the 2nd session, shows a well-m arked pause following the reinforcement
and consistent curvature to a well-sustained term inal rate of responding. Record B,
from the end of the 3rd session after 15 hours of exposure to FI 16, shows a m uch m ore
extended pause an d period of lower rate. At a, for exam ple, the term inal rate is not
reached until 13 m inutes after the start of the interval. T he term inal rate is now 1.5
responses per second, com pared w ith approxim ately 1 in R ecord A. R ecord C d e
scribes a segm ent from the perform ance a t the end of the 5th session after 25 hours of
exposure to FI 16. T h e first 2 segments of the record show a second-order effect a t
b, occurring in spite of the T O after reinforcem ent. A gain, the very low count a t the
reinforcem ent at b seems to be responsible for the early start of responding in the next
segment. Fifteen hundred responses are em itted before the reinforcement a t c. T h e
238
Fig. 2 7 2 .
First session on FI 1 6 TO 5 a fte r FI 1 TO 5 (bird 89R)
next segment is followed by a pause of norm al size a n d a very smooth acceleration to a

term inal rate of about 2 responses per second.
B ird89R. History: FI 8; FI 4; F I 2; FI 1 (120 reinforcements). T he 1st session (Fig.
272) shows a decline in the over-all rate which is m uch m ore m arked than in the fig
ures already described. T h e rapid transition m ay be due to the fact th a t this bird had
already undergone transitions to FI 8, FI 4, FI 2, a n d F I 1. T he high rate from the
preceding FI 1 reinforcem ent has fallen off by approxim ately one-third at the 1st rein
forcem ent, at a. T he 2nd, 3rd, an d 4th fixed-interval segments, ending at c, d, and e,
show m arked negative curvature. Tw o bursts a t high rates of responding a t b and f
m ay be rem iniscent of earlier shorter intervals. A m arked, irregular curvature c h a r
acterizes the rem ainder of the session. T he grain is extrem ely rough.
Figure 273 shows the further developm ent of the F I 16 perform ance. R ecord A
shows a segment 6 hours after the beginning of the 2nd session. T he performance is
still irregular, w ith rough grain a n d negative acceleration, but the over-all rate has in
creased considerably. R esponding at interm ediate rates is fairly continuous, unlike
the perform ance in the previous session, w here pauses occurred frequently a t all points
d u rin g the fixed-interval segments. A segm ent of the 3rd session after 15 hours of
exposure to FI 16 (Record B) shows a very regular scallop in which the acceleration ex
tends th roughout most of the fixed interval. N ote the slight ru n n ing-through (a and
b) in spite of the 5-m inute T O . In R ecord C, from the 4th session after 20 hours of ex-
239
posure to F I 16, the term inal rate is higher an d is reached earlier in the interval. T he
final perform ance in Record D after 25 hours of FI 16 reinforcem ent shows a somewhat
w ider v ariation in the length of the pause an d a period of lower rate of responding a t
the start of the fixed-interval segment.
Figure 274 shows a still later perform ance after 30 hours on F I 16.
B ird 98R. H istory: F I 16; F I 8; FI 2; FI 1 (120 reinforcem ents). T he 1st session
(Fig. 275) shows the same decline in the over-all rate as the other transitions from F I 1
to FI 16, but the transition is most rap id here. A pause follows the 1st reinforcem ent,
at a, an d the set of intervals beginning a t a through i shows a sm oothly developing
curvature. L ater intervals show a further increase in the extent of the scallop. Note
th at the bird h ad an earlier history of F I 16.
T he start of the 2nd session on F I 16 in Fig. 276 shows some effect of shorter fixed in
tervals at the start of the session, b u t orderly scallops w ith a substantial pause at the
start of the interval and a stable term inal rate develop soon thereafter.
B ird46Y. History: FI 2 (31 reinforcements). In Fig. 277 the transition from FI 2 to
FI 16 (m ade by accident) is sim ilar to Fig. 267, except th a t the 1st segment on FI 16
does not show a sustained rate, a n d the decline to a lower rate throughout the session
is less orderly. T he record is very rough and m arred by short rapid runs. But there
is no consistent rate change correlated w ith any feature of the schedule of reinforce-
/
Second session on FI 1 6 TO 5 a fte r FI 1 TO 5 (bird 98R)
300
RESPONSES
Fig. 2 7 6 .
Fig. 2 78.
Second to fifth sessions on FIT 6 TO 5 a fte r FI 2 TO 5 (bird 46Y)
241
m ent. In the 2nd session, after 7 hours on FI 16 (Fig. 278A), a rough scallop has de
veloped; but the perform ance is quite irregular, w ith negative curvature (as at a and
b) and m arked changes in perform ance from segment to segment. R ecord B, a seg
m ent from the 3rd session, after 15 hours of exposure to F I 16, shows less negative accel
eration. In R ecord C, a segm ent from the 4th session, after 20 hours on FI 16, the
pause following the reinforcement has become m uch m ore extended (as at c, w here only
a few responses occur during the whole 16-minute interval). T he segment at d does
not begin earlier, because of the reinforcem ent a t a low rate in the previous segment.
Record D is a segment from the 5th session after 23 hours of FI 16 reinforcement. Fig
ure 279A shows the start of the 6th session after 49 hours of FI 16 reinforcement. A
high term inal rate has developed and is m aintained, with a concom itant increase in the
pause following reinforcem ent. T he perform ance is not stable, however; a n d the end
of the session (R ecord B), after 53 hours of FI 16 reinforcem ent, shows negative curva
ture near the ends of intervals and wide changes in local rates uncorrelated with any
features of the schedule.
FI 1 6 to FI 8
Bird 52G. History: F I 4; F I 2; FI 1; FI 16 (25 hours). Figure 280A, the 1st ses
sion on FI 8, begins at the low rate characteristic of the start of the FI 16 interval. T he
Fig. 2 7 9 .
Fig. 2 8 0 .
Sixth session on FI 1 6 TO 5 a fte r FI 2 TO 5 (bird 46Y)
First and last sessions on FI 8 TO 5 a fte r FI 1 6 TO 5 (bird 5 2 G )
242
1st reinforcem ent, at a, occurs before the term in al rate is reached. Because of this
early reinforcem ent the 2nd segm ent shows a m ore rap id acceleration to the term inal
rate. T he 2nd reinforcem ent, at b, is followed by an even m ore rapid acceleration to
the term inal rate; and the 3rd reinforcem ent, at c, shows some running-through. T h e
rem ain in g segments of R ecord A then show a progressive deepening of the scallop,
because of the developm ent of a substantial pause at the beginning of the interval and
a very sm ooth acceleration to a high term inal rate. R ecord B shows the final p e r
form ance after 14 hours of exposure to FI 8. A short pause a t the start of the interval
is followed by a m oderate period of acceleration to a high term inal rate, which is m ain
tained for about 500 responses.
History: FI 2; FI 1 ; FI 16 (49 hours). T he 1st session (Fig. 281 A) shows

roughly the same pattern as Fig. 280A, except th a t the smooth curvature is replaced
here by a pause and an a b ru p t shift to the term inal rate. Record B shows a final p er
form ance after 17 hours. T he pause after reinforcem ent varies from slightly over 1
m inute at b to over 5 m inutes a t a. Responding begins abruptly at the term inal rate of
responding after the pause, and most of the segments show slight negative curva
ture. At the start of the interval at c the bird begins responding a t the term inal rate
in the preceding interval and continues a t a slightly lower rate for the rest of the seg
m ent. This perform ance is rarely observed w ith a 5-m inute TO .
Bird 98R. History: FI 16. T he 1st session (Fig. 282A) shows a rapid adjustm ent
to the curv atu re ap p ro p riate to the FI 8 schedule. In R ecord B, after 30 hours of
exposure to FI 8, each segment begins with a substantial pause and accelerates, often
ra th e r abruptly, to a term inal rate. An exceptional case of a knee occurs at a.
B ird49Y.
243
History: F I 8; FI 4; F I 2; FI 1; FI 16. This bird characteristically be

gan each session with a high over-all rate w ith no fixed-interval scallop. L ater in the
session, a very stable an d regular fixed-interval perform ance w ith m arked scallops
emerged. (Figure 273 is for this bird.) T he 1st reinforcem ent on FI 8, at a, in Fig.
283A is received at a high rate of responding, as is the 2nd, at b. T hereafter, a lower
rate develops progressively during the early p a rt of the interval. Figure 283B shows
th e perform ance after a total of 8 hours of exposure to the F I 8 reinforcem ent schedule.
A substantial pause follows each reinforcem ent, and the acceleration to a higher rate
is irregular. T he term inal rate varies from segment to segment, and instances of nega
Bird. 89R.
tive acceleration occur after the term inal rate has set in.
B ird 46T. History: F I 2; F I 16 (55 hours). T he previous perform ance on FI 16
244
10 M I N U T E S
Fig. 284.
First and last sessions on FI 8 TO 5 a fte r FI 1 6 TO 5 (bird 46Y)
shown in Fig. 279 was not a stable perform ance on th a t schedule. T he 1st reinforce
m ents on FI 8 in Fig. 284A reflect the previous F I 16 perform ance in which the te r
m inal rate was high. R esponding is sustained alm ost continuously through the first 4
segments. T he over-all rate then falls m arkedly during the next 3 segments, with a
substantial pause following the reinforcement. T he term inal rate is not reached at a
an d b. After 8 hours of reinforcem ent on F I 8, the curvature becomes appropriate to
the new schedule of reinforcem ent. After 23 hours on FI 8, the perform ance is still
irregular, as seen in Fig. 284B, which shows negative acceleration a t c, failure to accel
erate to the term inal rate of responding a t d, an d an early acceleration to the term i
nal rate at e. Figure 285 shows the following session complete. T he high sustained
rate of responding from the perform ance u n d e r the previous FI 16 reinforcem ent re-
10 M I N U T E S
Fig. 2 8 5 .
C om plete second session on FI 8 TO 5 a fte r FI 1 6 TO 5 (bird 46Y )
245
turns at the beginning of the session b ut gives way to a lower over-all rate. Pausing
appears after the 3rd reinforcem ent at a an d develops progressively throughout the
rem ainder of the session. T he acceleration to the term inal rate of responding is gener
ally prem ature, and in almost every case there is a m arked negative curvature before
the end of the interval.
FI 8 to FI 4
Bird 98R.
History: FI 16; FI 8 (32 hours).
fixed-interval curve on FI 8.
(See Fig. 282.)
T his bird h ad developed a stable

In the 1st session on FI 4 (Fig. 286) the
1st reinforcem ent at a occurs before the term inal rate from FI 8 reinforcem ent has been
reached. T h e early reinforcem ent produces im m ediate responding in the 2nd seg
m ent. A pproxim ate scalloping begins at b. After 13 hours of reinforcem ent on FI 4,
a perform ance approaching the final effect of the schedule is shown (Record B).
H ere, most intervals begin w ith a pause, and the curvature extends through most of
the interval (cf. f , g ). T here is some responding at the start of the interval at d and
responding during the time out at c and e.
Bird 52G. Figure 287 shows the transition (R ecord A) an d later perform ance (R ec
ord B) on F I 8 T O 5 following the perform ance on F I 16 T O 5 shown in Fig. 280B.
T he early reinforcem ent at a is followed by a m arked scallop before b. Strong neg
ative curvature appears later (c, d, e , f g).
246
B ird49Y. History: FI 2; FI 1; FI 16; F I 8 (17 hours). Figure 280B showed a stable

a n d smooth perform ance on FI 8. In the transition to FI 4 (Fig. 288A) the 1st rein
forcem ent (at a) occurs after sustained responding d u ring the first 4 m inutes o f the
session. T h e first 5 segm ents show sustained responding a t the term in al rate. A
pause occurs at the start of the interval (b), however, with an a b ru p t shift to the te r
m inal rate. T he next segment (c) shows curvature appropriate to the FI 4 schedule.
In the final perform ance in R ecord B, after 9 hours of exposure to F I 4, most segments
begin w ith a substantial pause an d show extended acceleration to a term inal rate.
However, instances occur (as at d an d e) w here the pause at the beginning of the fixedinterval segment is brief and the acceleration to the term inal rate relatively abrupt.
B ird 46Y. History: F I 2; FI 16; FI 8 (14 hours). A lthough this bird did not show a
very stable perform ance on FI 8 (Fig. 284B), an interval scallop appropriate to F I 4
reinforcem ent developed rapidly during the 1st session (Fig. 289A). Record B shows
the final perform ance after 6 hours of reinforcem ent on FI 4.
Bird 89R. H istory: F I 8; FI 4; F I 2; F I 1; F I 16; FI 8. T he 1st reinforcem ent on
F I 4 (Fig. 290A) produces sustained responding throughout the next 4-m inute interval
in spite of the m arkedly scalloped perform ance which h ad previously been occurring
on F I 8 (Fig. 283B). T h e linear perform ance then gives way to some positive curva-
First and last sessions on FI 4 TO 5 a fte r FI 8 TO 5 (b ird 49Y )
Fig. 2 8 9 .
First and last sessions on FI 4 TO 5 a fte r FI 8 TO 5 (b ird 46Y )
300 RESPONSES
Fig. 2 8 8 .
Fig. 2 9 1 .
First to tw e lfth sessions on FI 2 TO 5 a fte r FI 4 TO 5 (bird 5 2 G )
249
ture. T he final performance, Record B, after 13 hours of reinforcement on FI 4, shows

an over-all negatively accelerated curve as a result of: (1) the increase in the length
of the pause during the session; (2) a decline in the term inal rate; and (3) an increase in
the period during which acceleration occurs.
FI 4 to FI 2
B ird52G . History: FI 4. T he perform ance in Fig. 291A shows the first change in
schedule for this bird after the original transition from crf to FI 4. T he previous p e r
form ance under FI 4 T O h ad been showing S-shaped curves. T he 1st reinforcement
on FI 2 (at a) occurs after the term inal rate of responding has been reached. U nlike
the transitions from larger fixed intervals previously described, a sim ilar pause an d
lower rate at the beginning of the interval ap p ear in the second segment. Some re
sponding occurs d u ring the T O a t b an d c, a n d S-shaped curves soon appear. They
disappear, however, by the end of the 2nd session (R ecord B). R ecord C shows a
segment from the end of the 6th session of exposure to the FI 2 reinforcement, where the
pause a n d period of lower rate following the reinforcem ent have becom e m ore ex
tended and the term inal rate of responding has increased. T he increase in the term i
nal rate more th an compensates for the increase in the period of lower rate at the start
of the interval. By the end of the 8th session (R ecord D), the term inal rate has fallen
to a value of about the same order as in R ecord B, and the pause has increased, with a
resulting decline in the over-all rate. After 11 sessions of reinforcem ent on FI 2, very
m arked negative cu rvature sets in. T he entire 12th session is shown in R ecord E.
H ere, most of the intervals begin with pauses, followed by fairly ab ru p t accelerations to
a high rate. Before the end of the interval, however, this rate falls off by a factor of
3, and the reinforcement occurs at a m uch lower rate. T he S-shaped curves in Record
E proved to be a special property of this interval of reinforcem ent at this stage of de
velopm ent of the performance; later performances for this bird on FI 16 and FI 8 in Fig.
271 and 280 showed norm al, positively accelerated curves.
Bird 98R. History: FI 16; FI 8; FI 4. Figure 292 shows the first exposure to FI 2
after the final perform ance under FI 4 shown in Fig. 282. T he m iddle p a rt of the ses
sion was lost because of a recorder failure; hence, only the first 5 an d last 3 segments of
the session are shown. Some interval curvature develops. R ecord B shows the b e
ginning of the 2nd session, during which a pause after reinforcem ent develops, together
with an extended scallop throughout the interval. T he over-all rate falls as the pause
and scallop develop. Responding occurs during the T O at a. Record C shows the
final perform ance after 5 sessions of FI 2 reinforcem ent. R esponding occurs during
the T O at a, c, d, and e. Both the over-all a n d term inal rates have fallen considerably
below those of Record B.
Bird 89R. History: FI 8; FI 4. Figure 293A shows the com plete 1st session on FI 2
after the final performance on FI 4 already shown in Fig. 290. T he over-all rate de
clines slightly; but no pause develops after reinforcem ent or curvature within the inter
val, even though these were present on FI 4. By the 3rd session, a segment of whicl)
Fig. 2 9 3 .
First to ninth sessions on FI 2 TO 5 a fte r FI 4 TO 5 (bird 89R)
251
is shown in R ecord B, slight positive curvature begins to appear (as at a). This curva
ture becomes more m arked by the 7th session, represented by the segment in Record
C. T here is still no pause after reinforcem ent, a n d most of the intervals begin with a
short burst of responding at the term inal rate of the preceding segment. T h e interval
a t b shows the greatest extent of the curvature at this stage. Record D shows the entire
9th and last session on FI 2. Considerable responding occurs during the T O at d, e,
and f . T he performance is generally irregular, with frequent break-throughs a t high
rates of responding (as at g a n d h). A lthough most of the intervals are roughly scal
loped, no pause follows the reinforcement.
FI 2 to FI 1
Figure 294 shows the transitions for all 5 birds.

Previous perform ances can be identified through earlier figures as follows: A, Fig.
292; B, Fig. 289; C, Fig. 293; D, Fig. 291; E, Fig. 288. T ransitional () and late (2)
performances are shown. As usual, FI 1 T O is very irregular. In B2 and D 2, high te r
m inal rates give a fairly sm ooth over-all curve. In A2 a n d C 2, some slight stability is
evident; E 2 is highly variable, with responding during T O and m uch negative accel
eration.
Summary o f fixed-interval transitions with time out a fter reinforcement
From small to largerfixed intervals. In the first exposure to a large fixed interval after
reinforcem ent on a sm aller one, the term inal rate from the shorter interval is m ain
tained through most of the new larger interval. T he large num ber of responses per
interval thus produced cannot be m aintained, and the term inal rate begins to fall, as
pauses w ith later acceleration develop at the start of the interval. T he over-all curve
during the first several sessions shows negative acceleration. T he interm ediate rates
and acceleration to the term inal rate become irregular. T he transition is frequently
252
prolonged, as m any as 10 sessions being needed to develop the final performance on the
larger interval. D uring the transition the curvature m ay extend through most of the
interval. However, further exposure to the schedule of reinforcement produces a
longer pause after reinforcement, a more restricted interval over which the curvature
occurs, and a more sustained period of responding a t the term inal rate.
From large to smallerfixed intervals. T he transition from a larger interval of reinforce
m ent to a smaller one is m uch faster. It is sometimes complete within 2 sessions. T he
scallop w hich had developed u n d er longer fixed intervals gives way to a linear p er
form ance, often after a single reinforcem ent on the shorter interval. This perform ance
is followed by the developm ent of a scallop appropriate to the new fixed interval. An
interm ediate stage frequently occurs where the segment is uniform ly accelerated
throughout in a form sim ilar to the arc of a circle.
In this experim ent it was not practical to m ain tain the schedule of reinforcem ent at
each interval until a final perform ance had developed. In some transitions to a fixed
interval to which the bird h ad already been exposed, the previous perform ance was
not fully recovered. O ther experim ents strongly suggest, however, th at each schedule
would produce a similar performance.
LARGE FIXED INTERVALS
Tw o birds from the experim ent rep o rted in Fig. 267 th ro u g h 294 were exposed to
a series of progressively increasing fixed intervals up to 134 m inutes w ith tim e out.
T he d a ta show the transitions on some fairly stable perform ances at large fixed in ter
vals. Since the procedures were slightly different, the results will be described sepa
rately for the 2 birds.
Bird 89R
H istory: FI (1-16) 1000 hours. In one experim ent a pigeon which

h a d h a d a history of FI reinforcem ent a t values betw een 1 an d 16 m inutes ending
w ith FI 8 T O 5 was exposed for the first tim e to F I 64 T O 5. T he com plete session is
represented in Fig. 295. Each segment in the record is one 64-m inute interval. In
Segm ent 1 the bird accelerates to the form er term inal rate, which continues far b e
yond the preceding interval, dips smoothly to a low over-all rate m arked by occasional
oscillations, and is reinforced at an interm ediate value. Segment 2 begins at a lower
rate, accelerates m ore slowly to a term inal rate, and breaks somewhat m ore sharply to
a low rate at a. Reinforcem ent is received after some further acceleration (at b). Sub
sequent intervals show oscillations betw een a fairly high term inal rate a n d very low
values, most changes being m ade fairly smoothly. T he over-all rate declines. No
curvature appropriate to the 64-m inute interval appears. This bird failed to develop
a good 64-m inute perform ance, even after 107 hours on the schedule. Figure 296
shows the last session.
F I 29. A lthough the bird could not m aintain a substantial perform ance on FI 64
F I 8 to F I 64.
Fig. 2 9 5 .
First session on Fl 6 4 TO 5 a fte r Fl 8 TO 5 (bird 89R)
254
a t this stage, a change to F I 29 produced the fairly stable perform ances of Fig. 297 in
the 3rd session. T he continuous decline in the over-all rate has been characteristic
of this bird. T he session is characterized by extended curvature throughout the inter
val. Even tow ard the end of the session, in the segment at a, where the over-all rate
is of the order of 0.3 response per second, the curve is fairly uniformly accelerated
throughout.
A final perform ance during 2 successive sessions, beginning after 40 hours of rein
forcem ent on FI 29, is shown in Fig. 298, w here the fixed-interval segments have been
arranged according to the num ber of responses occurring during the segments.
F I 77. W hen the interval was extended to 77 m inutes, strong over-all negative cu r
vature appeared during the session; b u t each new session began with a high over-all
rate. T he 5th session is shown in Fig. 299, w here the order of occurrence of the seg
ments is shown by the num bers near the beginnings of the segments. T he 1st in ter
val is norm al, w ith about 4000 responses; Intervals 2 and 3 are almost empty; Interval
4 shows a norm al curvature to about 800 responses; Interval 5 has a norm al curvature
to ab o u t 500 responses, In terv al 6 to nearly 2000 responses, a n d In terv al 7 to about
RESPONSES
300
Fig. 2 9 9 .
Fl 7 7 TO 5
256
2000 responses. By this tim e the interval p a tte rn is well-established, although some
intervals are practically empty.
Bird 98R
Figure 300 shows the perform ance for the second bird on FI 29 T O 5, for 2
sessions beginning 30 hours after FI 16 T O 5, already described in Fig. 275. (Ten re
inforcements occurred on FI 2 accidentally in the 21st hour.) T he interval segments
F I 29.
Fig. 3 0 0 .
Early FI 2 9 TO 5
257
have been assembled according to the num bers of responses. T he perform ance is sim
ilar to th a t in Fig. 298. M ost segments show a pause following the reinforcem ent
a n d a m oderately smooth an d extended acceleration to a term inal rate which is m ain
tained until the reinforcement. Occasional responding occurs at the start of the in
terval (as in a, d, an d f ) and negative curvature tow ard the end of the segment (as at b,
c, and e). T he following session is shown in Fig. 301 in the original order of segments.
T his perform ance after 39 hours of exposure to FI 29 T O 5 is sim ilar to th a t in the
previous figure except th at no cases of negative curvature occur.
T he curve resulting from the 10 accidental reinforcements on FI 2, in Fig. 302, shows
an over-all shape almost identical with the curvature th at had been occurring under
FI 29 (cf. Fig. 300). T he slow developm ent of a higher rate of responding dem on
strates the degree of control by the stimuli responsible for the low rate of responding
during the early p art of the FI 29 performance.
F I 64. After 54 hours of FI 29, the schedule was changed to FI 64 T O 5. All seg
m ents from the 1st session are shown in Fig. 303 a n d 304, w here segments showing
positive curvature and negative curvature have been separated. T he positively accel
erated curves in Fig. 303 show rough grain throughout; b u t the acceleration is u su
ally continuous except for segments containing only a few responses. Note the absence
of a pause following the reinforcement at a in spite of the 5-m inute TO . T he 2 m arked
negative accelerations occurring in Segments 1 and 4 in Fig. 304 are presum ably the
258
result of strain produced by the term inal rate generated by the preceding FI 29 p er
formance.
Figure 305 shows the perform ance after 100 hours on FI 64 T O 5. T he upper ends
of the first 3 segments have been displaced to the left of the graph.
F I 77. T he effect of increasing the interval of reinforcem ent from 64 to 77 m in
utes is some tem porary negative curv atu re w ith occasional intervals a t low rates. A
fairly stable perform ance develops quickly. Figure 306 shows the performance during
2 sessions extending up to the 40th hour on F I 77. T he segments have been re a r
ranged in term s of the num ber of responses. In the first segments (A1 and B 1) of the
daily performance, responding begins early. All segments show some positive accel
eration, although the segments at the bottom of the figure do not reach the sam e te r
m inal rate. T he relation between the n u m b er of responses in the segment and the
tim e before the acceleration to the term inal rate begins is roughly linear.
F I 92. Figure 307 shows the transition from F I 77 T O 5 to FI 92 T O 5. All seg
m ents from the first 3 sessions on F I 92 have been arranged in order of the num ber
of responses except for the 1st segments, which have been om itted. These 1st segments
simply show a steady performance at a m axim al rate throughout the intervals, which
contain about 4500 responses each. T he bird sustains a fairly high over-all rate of re
sponding in spite of the infrequent reinforcem ent. T he segments under FI 92 rein
forcement are sim ilar to those under FI 77, except th at a larger num ber of responses
occurs per fixed-interval segment and the term inal rate is somewhat delayed.
T he other bird was also reinforced on FI 92 T O 5. T he performance resembled th a t
on transition from FI 64 to FI 77 (Fig. 299).
F I 111. T he m ain effect of a transition from FI 92 T O 5 to FI 111 T O 5 is a g rad
ual decline in the over-all rate, w ith further postponem ent of the acceleration to a ter-
300
RES PO NSE S
260
00
RES PONSES
261
300
RES PO NSE S
Fig. 3 0 6 .
Fl 7 7 TO 5
262
Fig. 3 0 7 .
First session on Fl 92 TO 5 a fte r Fl 7 7 TO 5

263
266
m inal rate of responding. Figure 308 shows the perform ance after 61 hours on FI 111.
T h e figure contains all but the 1st segments from 2 successive daily sessions reassem
bled in terms of the num ber of responses em itted. T he acceleration to the term inal
rate is roughly appropriate to the large interval of reinforcement. T he term inal rate is
never reached very rapidly, although the first 2 curves show fairly quick starts, with
alm ost 4000 responses per interval.
In the transition from F I 92 T O 5 to FI 111 T O 5 the other bird showed a low over
all rate with very irregular grain. A term inal rate rem iniscent of shorter fixed inter
vals was occasionally held for approxim ately 1000 responses, b u t negative accelera
tion usually set in before reinforcement. D uring 72 hours of the schedule the perform
ance consisted mostly of slow responding, with an occasional break-through to a higher
rate.
F I 134. Figure 309 shows an early perform ance on FI 134 T O 5. Fixed-interval
segm ents from 3 sessions beginning after 21 hours of exposure to the schedule have
been arranged in order of the num ber of responses. T he 1st segments (om itted to fa
cilitate presentation of these records) showed: (1) a low rate of responding during the
whole interval; (2) a fairly quick shift to an unsteady term inal rate, w ith the segment
containing more th an 4500 responses; and (3) a fairly norm al interval curve contain
ing about 1800 responses. In spite of the very infrequent reinforcem ent, the bird
m aintains a fairly substantial over-all level of responding. Some sign of strain, how
ever, appears in the frequent negative curvature.
After a total exposure of 84 hours on this schedule, the second bird continued to
show a very low over-all rate, w ith only occasional increases in rate in the form of the
S-shaped curves of Fig. 296. T he bird could not sustain a perform ance under FI 134
T O 5 at this level of deprivation.
FIXED INTERVAL PLUS ADDED CLOCK
To the extent th at the bird s behavior is an event varying in time and correlated con
sistently with the FI schedule, it can be thought of as a clock by which the bird m ay
modify its behavior w ith respect to reinforcement. T he term inal rate at the end of the
fixed interval represents the most optim al setting of the clock, and the curvature rep
resents interm ediate clock-settings. W ith a perfect clock an interval should contain
only a single response. T he bird s own behavior, however, is a relatively poor clock.
T h e inherent instability of an FI perform ance, w ith its oscillations in rate, destroys
the correlation between the num ber of responses and reinforcement, or the rate and re
inforcement.
W e have attem pted to get some notion of the control exercised by the b ird s own
behavior by adding an external stimulus th a t varies uniformly in some dimension d u r
ing the fixed interval. This stimulus is arranged by the experim enter, and can be con
sistently correlated with the schedule of reinforcement. In the following experiment,
various kinds of fixed-interval perform ances were studied, with an added clock similar
the added counter of C hapter Four. A spot of light is projected on a translucent
267
key th rough a slit, the length of w hich is determ ined by a circular wedge driven by a
synchronous motor. As the m otor turns the wedge, the slit grows from a small spot
approxim ately 1/16 inch square to a slit 1/16 inch wide by 3 /4 inch long, centered
horizontally on the key. T he reinforcem ent schedule was locked to the slit by the use
of the same m otor to program the reinforcement. By changing the speed and direc
tion of the motor, and the design of the wedge, the added stimuli could be m anipulated.
A short T O (of a few seconds) followed each reinforcem ent as the clock was reset to
zero. (T he recorders used for these experim ents were powered by a new type of syn
chronous m otor; we discovered later th a t these motors could run a t 2 speeds. Any
tim e the recorder m otor was stopped an d started [as a t reinforcem ent], it m ight shift
speed. This variation in recorded time did not affect the bird, however, since the m o
tor program m ing the slit and reinforcement schedule operated correctly. W hen in ter
preting the following records in fine detail, the length of the interval m ust be exam
ined to discover at w hat speed the recorder was running. Since we will not be
concerned with absolute rates but m ainly with changes in rate during the intervals, this
defect is not im portant for the conclusions which will be draw n from the experim ent.)
Development o f FI performance with clock
F I 10 + clock.
Figure 310 shows the first effect of introducing a clock. T he 1st ex

cursion of the record is the final perform ance for this bird after 38 hours of exposure
to FI 10, following 14 hours of FI 5 after erf. All 6 intervals show a brief pause follow
ing the reinforcem ent and a brief acceleration to a m oderate term inal rate. D uring
the 38 hours of exposure to FI 10 the key was illum inated through a 1/16-inch-w ide
slit, 3 /4 inch long. At the beginning of the interval, indicated by the arrow, the slit on
the key was at its smallest size; it grew linearly throughout the 10-minute interval to
the m axim um size at reinforcement. T he small size was a novel stimulus and has the
usual effect of suppressing responding for some tim e a t a. D uring this tim e the slit
is growing, however; and before the end of the interval the bird begins responding at a
m uch higher rate th an it h a d ever shown in its history. T he rem ainder of the in ter
val at b shows continuous negative acceleration; but the reinforcement at c occurs while
268
the rate is considerably higher th an norm al. T he next few intervals are run off at a
high over-all rate. Each successive interval shows m ore m arked curvature, however.
T he rem ainder of the session shows the developm ent of a longer pause following the re
inforcem ent, a rapid acceleration to the term inal rate, a n d a decline in the over-all
rate.
Figure 311 shows segments from the 2nd and 3rd sessions w ith the clock. R ec
ords A an d B show the 4th, 5th, 6th, 23rd, 24th, a n d 25th reinforcem ents of the 2nd
session. These show a further developm ent of the perform ance in Fig. 310. T h e
pause extends over more than half the interval. T he term inal rate, however, is now
Fig. 31 1.
D evelopm ent o f FI 10 + clock
269
higher th a n the interm ediate rate occurring d u rin g the previous session. Records C
a n d D show the 27th, 28th, 29th, 30th, 39th, a n d 40th intervals of the same session.
T he extent of the pause is about the same as before, but the acceleration to the term inal
ra te is m ore rap id a n d the term in al ra te higher. T h e last segm ent in R ecord D,
w here the tim ing is precise, shows a rate of 10 responses per second during the last few
seconds of the interval. R ecord E shows the 20th, 21st, an d 22nd segm ents from
th e 3rd session following the introduction of the clock. T he perform ance is similar to
R ecord D, a n d represents a perform ance very close to the final perform ance under
F I 10 + clock. Figure 312 shows a large segment from the final perform ance taken
after 25 hours of exposure to the clock. T he perform ance is uniform from segment to
Jl
Fig. 3 1 2 .
Final FI 1 0 + clock
segm ent, w ith a pause of from 7 to 8.5 m inutes, followed by a short period d u ring
which the rate accelerates to a term inal rate well above 10 responses per second. T he
variation in the n u m b er of responses from interval to interval is relatively small in
view of the very high rate at which these responses are em itted. V ariations of this o r
der are caused by differences of a very few seconds betw een the tim es a t w hich the
term inal rate is reached. T he term inal rate shows a 30- to 35-fold increase over the
rate of 0.3 response per second prevailing before the clock was introduced. A sec
ond bird, 23Y, confirm ed the general features of this transition.
T h e clock continues to control the rate even d u ring extinction, w hen the slit on the
key m aintains its usual cycle. An instance is shown in Fig. 313. T he usual proce
dure occurs at a, b, an d c. E xtinction begins at the arrow . T he cycling spot of light
produces scallops characteristic of the clock perform ance. T he over-all rate even-
270
tually declines as the n um ber of responses em itted at the larger slit-values decreases.
T he stim ulus control, however, does not become disturbed as a result of the extinc
tion.
F I 30 + clock. After an intervening set of experim ents w ith a clock (to be described
late r in the c h ap ter), the b ird responsible for the perform ance ju st described was
placed on FI 30, w ith the slit on the key growing from 1/16 to 3 /4 inch long during the
interval. T h e perform ance after 5 sessions is shown in Fig. 314, w here the last 7
intervals of the session have been arranged in the order of their occurrence from a com
m on m argin. At this value of FI the clock shows even better control th a n in Fig.
312 under FI 10. In the 1st segment of the figure, for exam ple, the term inal rate is not
reached until the slit is approxim ately 19/20 of its m axim um size. T he same order
of m agnitude of the slit at the start of the term inal rate appears in the rem ainder of the
figure. Sm all knees at a a n d b p ro b ab ly reflect the previous exposure to an o th er
kind of clock (to be described). T his schedule was co n tin u ed for 5 sessions w ith no
further change.
Development o f F I 10 + clock after extended history o f reinforcement on mix FIFI. Figure
769 in C h ap ter Eleven shows segm ents taken d uring the first 34 hours of exposure to
FI 10 + clock after an extended history of FI, V I, an d m ix F IF I, an d an im m edi
ately preceding history of m ix FI 10 FI 2 w ith a reduced counter. Records A through
D contain the last 2 segments of sessions after 8, 15, 24, a n d 29 hours of exposure to
the clock, respectively; and R ecord E is a segment from the m iddle of a session after 34
hours of exposure to FI 10 + clock. Segm ents a a n d b resem ble the term inal p e r
form ance on F I 10 + clock in Fig. 312, b u t the term in al rate is m ore m oderate. A
perform ance for a bird having a sim ilar history is shown in Fig. 769, R ecord A, in
C hapter Eleven. This perform ance is after 100 hours of exposure to FI 10 + clock.
271
Both figures undoubtedly show an incom plete developm ent u n d er this schedule.
F I 1 to F I 6. A clock was added to the F I perform ances of 3 birds on the 2nd ses
sion following erf. Figure 315 shows the com plete 1st session w ith clock for 1 bird.
A typical FI perform ance develops very rapidly, with a consistent pause after rein
forcement and a high term inal rate of responding. A lthough pauses after reinforce
m ent develop w ithout the aid of a clock, they develop m ore slowly (cf. Fig. 126 and
149), and the term inal rates do not reach the sam e order of m agnitude as w ith the
a d d e d clock. Figure 316 shows late r perform ances after 6 sessions u n d e r FI 1 +
clock for all 3 birds. Figure 316A shows little change beyond the perform ance at
the end of the 1st session with clock as shown in Fig. 31-5. R ecord G shows a m uch
lower over-all rate, although the order of consistency from interval to interval is a p
proxim ately the same. T he bird h a d been ru n by m istake on FI 10 briefly follow-
Fig. 3 16.
Final Fl 1 + clock
273
ing erf; a n d the lower rate of responding m ay be due to exposure to this schedule.
All 3 of these birds were th en placed on FI 3 w ith the same clock, and final p e r
form ances are shown in Fig. 317. T he birds are assigned the same letters in Fig. 316
a n d 317. R ecord A was taken after 5 sessions; Records B an d C , after 8 sessions.
T he clock grew to full size in 3 m inutes, instead of in 1 m inute as in the previous sched
ule. In the resulting perform ance the pause after reinforcem ent characteristically
extends for m ore th a n h alf of the 3-m inute interval. M ost segments show a fairly
a b ru p t shift to the term inal rate, alth o u g h varying degrees o f cu rv a tu re are present.
T he term inal rates are approxim ately 3 per second in R ecord C, slightly less than 4
per second in R ecord A, and slightly m ore th a n 5 per second in R ecord B. W hile
these do not approach the rate of m ore th an 10 per second recorded under FI 10 4clock, they are higher th an are usually recorded for this size of FI (cf. Fig. 150G for FI 2,
50 sessions after erf).
T h e session following Fig. 317 began w ith reinforcem ent on FI 3; b u t the interval
was soon increased to 6 m inutes, the clock growing from small to large in each inter
val. O ne perform ance is shown in Fig. 318. T h e effect (beginning at the arrow ) is
to enlarge the FI 3 curve roughly by a factor of 2. T he interval a t c is very close to a
Fig. 3 17.
Fig. 31 8.
Final FI 3 + clock
Transition from FI 3 + clock to FI 6 + clock
274
sim ple m agnification of the curve at b, while d suggests m agnification of a. W ith

fu rth er exposure to FI 6 w ith clock, however, the n u m b er of responses in the interval
becom es less as the bird pauses for a greater portion of the interval. T hus, in the
curves at e , f g, and h the num bers of responses in the interval are approxim ately only
one and one-half times those under FI 3.
A perform ance under FI 6 + clock after 4 sessions is shown in R ecord A of Fig. 319,
a n d for another bird after 5 sessions in Fig. 319B. (T he third bird h ad been dropped
from the experim ent after only a brief exposure to FI 6 + clock; how ever, its early
transition to this interval confirmed the performances of the other two.) T he term inal
rates in Fig. 319 are about the same as under FI 3, but the proportion of the interval
spent during the pause rem ains greater under FI 6. This characteristic is apparent in
the fact th at the num ber of responses per interval does not reach twice the value oc
curring under FI 3.
M ultiple clock
In ord er to com pare the kinds of perform ances established by various speeds of
clocks, a program was set up in which the bird was reinforced on any one of 4 intervals
of reinforcem ent: 3, 5, 7.5, or 10 m inutes. T h e slit on the key always began at the
Fig. 3 2 0 .
M ultiple clocks: FI 3, FI 5, FI 7 .5 , and FI 10
275
sm allest size ju st after reinforcem ent an d grew to m axim um size ju st before reinforce
m ent; thus, it grew most slowly on FI 10 a n d most rapidly on FI 3. Figure 320 shows
perform ances for one bird u n d e r 4 fixed intervals a lte rn a tin g a t ran d o m after 18
hours of reinforcement. T he segments have been arranged in groups corresponding to
the intervals of reinforcement. Segments of FI 3 are shown in Record A. Tw o in
stances occur of knees, which have been absent from the perform ance of this bird in the
past; b u t the clock control is otherw ise sim ilar to the final perform ance on FI 3 +
clock shown in Fig. 318. R ecord B shows the F I 5 segments, Record C the F I 7.5 seg
ments, R ecord D the FI 10 segments. T he acceleration just before the term inal rate
is slightly m ore prolonged th a n at a single clock value, possibly because other factors
contributing to the control vary from interval to interval in this experim ent. T he
ratio of the num ber of responses in R ecord D to th at in R ecord A is of the order of 3 to
8; therefore, the clock control m ust be greater at the longer intervals.
After 20 hours of reinforcement on 4 intervals (Fig. 320), the intervals were increased
to FI 7, 13, 19, and 25. These occurred in random order, with the clock com pleting
a full excursion during each interval, growing m ore slowly during the larger intervals.
Figure 321 shows the first 45 intervals of a session 87 hours later. T he records have
been rearran g ed according to the interval of reinforcem ent. As in Fig. 320 the clock
control is relatively better in R ecord A u n d er FI 25 th a n in R ecord D u n d e r FI 7.
T he shift to the term inal rate of responding is usually rapid, although instances of knees
sim ilar to those in Fig. 320A occur frequently.
T h e procedure of alternating 4 different intervals under the appropriate clock con
trol m inim izes the im portance of other stim uli related to the passage of tim e since
the last reinforcement. T he extent of such control was determ ined when the clock was
rem oved by fixing the slit at large. T he schedule then becomes a 4-ply m ixed FI
schedule. (See C hapter Eleven.) T he first effect of the new procedure was a constant
high rate of responding associated w ith the large slit. This rate falls, however, and
Fig. 322 shows the perform ance after 32 hours of exposure to the 4-ply schedule for the
bird giving the best curvature on this schedule. T he record begins with the start of
an experim ental session; and the 1st interval is ru n off with m arked acceleration at a to
a high rate of responding com parable w ith those which were occurring w ith the clock.
T he next interval shows a more m oderate term inal rate, m aintained with some grain
th ro u g h o u t the interval to the 2nd reinforcem ent (at b). T hereafter, each reinforce
m ent is followed by a period of positive acceleration, w hich continues u ntil term i
nated by the reinforcem ent. At c, several h u n d red responses suddenly ap p ear at a p
proxim ately 9 responses per second, ap p ro p riate to the fixed setting of the form er
clock. This rate then becomes m ore m oderate; but before the appearance of the next
reinforcem ent, the high rate again returns (at d) for a sm aller num ber of responses.
T he other bird also showed these sudden shifts to the term inal rate characteristic of the
clock perform ance. It did not show curv atu re related to reinforcem ent, however;
and the over-all curve showed wide oscillations in rate.
Fig. 322.
M ix Fl 7 FM 3 Fl 1 9 Fl 25 a fte r m ultiple clocks
277
Reversing the clock
T h e extent of the control by the slit of light on the key is w ell-illustrated w hen the
direction of m ovem ent of the clock is reversed. Figure 323A shows a final perform
ance on FI 3 + clock in an experim ent w hich has already been discussed. R ecord
B follows R ecord A im m ediately, b u t the direction of growth of the slit was reversed
at the arrow . T he slit now rem ained large after reinforcem ent and shrank as the in
terval progressed until it reached the sm allest size ju st before reinforcem ent. T he
large slit correlated with the end of the fixed interval now occurs im m ediately after the
reinforcem ent, and the small size correlated w ith nonreinforcem ent now appears just
before reinforcem ent. T he first 2 segments are inverted images of the standard clock
perform ance. T he rate of responding is com pletely determ ined by the value of the
slit on the key. Any stim uli from behavior or other stim uli also varying in tim e seem
to exercise no control. T h e new contingencies of reinforcem ent, however, quickly
alter this pattern. T he bird reaches a stage in which the rate of responding is roughly
constant (at b). Note th at the new control arises first a t the smallest size. J u s t be
fore the slit reaches this size, a t a, the b ird begins to respond. T h e next 2 segm ents
show a sim ilar effect. T he n a tu re of the clock is arb itrary , of course; an d although
this ex p erim en t was not c o n tin u e d w ith th e reversed direction of change, a n orm al
clock perform ance would have developed.
T h e direction of grow th of the clock was reversed twice for a n o th e r bird on F I 30,
278
whose perform ance has already been shown in Fig. 314. Figure 324 shows the first
segm ent in which the clock was reversed. At the beginning of the interval, when the
slit rem ains large instead of resetting to the smallest size, the bird begins responding
appropriately, and shows a declining rate as the slit shrinks to the small size which had
been present at the start of the interval and correlated with nonreinforcement. T he
curve is very close to the inversion of the sta n d a rd F I 30 curve + clock shown in Fig.
314. T he bird was not responding w hen the slit reached the small size, an d it re
m ain ed in the a p p a ra tu s for nearly 2 hours w ithout responding. T h e bird was re
m oved w ithout being reinforced on the small slit.
T h e bird was th en placed on a norm al F I 30 w ith ad d ed clock for 2 days. An ex
tinction curve was then taken during which the clock cycled in the reverse direction
+ IH R . 4 8 M IN .
continuously. T here was no history of reinforcem ent w ith a reversed clock. T he

record (Fig. 325) begins with the clock at sm all, at a; the clock shifts abruptly to the
large-size slit, at b. This shift institutes the high rate norm ally occurring when the slit
is large; but as the slit shrinks during the next 30 m inutes, the rate drops to zero, a t
c. This curve is similar to Fig. 324, where the direction of growth of the slit was also
reversed for a single segment. T he smallest value of the slit is reached at d, 30 m inutes
after b; and the clock abruptly changes to large, so th a t the high rate is again reinstated.
E xtinction is carried out for a total of 11 cycles, w ith continued control by the clock
throughout. T he duration of the responding a t the high rate when the slit is large be
comes less as extinction proceeds. T h e differential control rem ains in force. T he
result here closely resembles the perform ance of an o th er bird in Fig. 313, except th a t
the curves in the present figure are inverted because the clock has been reversed.
RESPONSES
00
Fig. 3 2 5 .
Clock reversal during extinction
280
Transitions from clock to stationary slit

Transition from F I 10 + clock to F I 10 + clock at the optimal size.
In the session fol

lowing th at in which the final perform ance on FI 10 + clock shown in Fig. 313 was re
corded, the clock rem ained stationary at the optim al setting after 3 reinforcem ents
on the form er schedule. In Fig. 326 the first 3 intervals w ith clock are shown at a, b,
a n d c, the slit rem aining large as m arked. T he optim al size of slit controls the te r
m inal rate th ro u g h o u t the first interval except for some irreg u larity a n d decline ju st
before reinforcem ent, a t d. T he next reinforcem ent begins a t a lower rate, which
continues only for a few responses before a shift to a higher rate at e; this higher rate is
m aintained for several hundred responses before falling off to a lower value at f .
Subsequent intervals show a g radually declining a n d slightly oscillating rate, w ith
little or no curvature appropriate to the FI 10 schedule in spite of the continuing rein
forcements.
Subsequently, FI 10 w ith clock was rein sta ted , a n d the transition from F I 10 +
clock to FI 10 w ith the slit a t the optim al size was repeated 12 times. Each daily ses
sion began with the program which was in force a t the end of the preceding session;
an d during the session the procedure was changed from FI 10 + clock to FI 10 with the
slit at the optim al size or vice versa. T he bird begins to shift somewhat more read-
Fig. 3 2 6 .
Transition fro m FI 10 + clock to FI 1 0 w ith slit a t o p tim a l setting
281
ily and m ore ab ruptly to the perform ance characteristic of the schedule. Figure 327,
for exam ple, shows the 4th transition to F I 10, w ith the slit a t the large value after
previous exposure to FI 10 + clock. T h e preceding FI 10 perform ance with clock is
not shown, but was standard. T he term inal FI clock rate is m aintained for approxi
m ately 600 responses to a; here, it drops to a lower rate for a short time before returning
to a higher rate briefly a n d continuing to the 1st reinforcem ent at a slightly lower
rate. This responding is of the same order of m agnitude as in the 1st transition to the
large slit in Fig. 326, but responding at the full, term inal FI clock rate is consider
ably reduced. T he rem ainder of the session shows an irregular and declining over-all
rate. Before the end of the session some tendency is evident tow ard a lower rate of
responding during the first p art of the interval.
Figure 328A shows the 6th tran sitio n , w here the segm ents have been a rra n g e d
w ithout breaks to show the over-all shape of the curve. W hen the slit rem ains large
after reinforcem ent at the start of the record, the usual term inal rate is m aintained
for only 500 responses (to a), w here it falls to a lower value. T h e term inal rate recov
ers briefly at b, but the rate falls again a n d the reinforcem ent a t c occurs at a lower
rate. This reinforcem ent reinstates the term inal rate, w hich is again m aintained for
almost 500 responses before it shifts ab ru p tly to an interm ediate rate at d. T he a c
tual am ount of tim e on the FI clock rate is about the same as for the 4th transition
RESPO NSES
300
Fig. 3 2 8 .
Sixth and tw e lfth transition to the o p tim a l clock setting
283
in Fig. 327. T he other features of this record, however, show th at the transition to the
FI perform ance w ithout benefit of clock occurs m ore rapidly.
Figure 328B shows the 12th transition from FI 10 + clock to F I 10 w ith the slit
stationary at the optim al size. T here is only an initial burst (at e) at the rate a p p ro
priate to the clock perform ance before the rate falls sharply (at f ) to an interm ediate
value which is m aintained thereafter. This rate is considerably higher th an is usually
m aintained by an FI 10 perform ance w ithout benefit of added stimuli. (The second
bird in the experim ent showed the same general pattern, but w ith greater irregularities
in the absence of a clock. See Fig. 330.)
Transition from F I 10 to F I 10 + clock. T he converse cases in this experiment, where
th e procedure was changed from FI 10 w ith the slit a t large to FI 10 + clock, showed
transitions sim ilar to those in Fig. 329. (T he uniform pause following the reinforce
m ent in the intervals a, b, and c is an artifact; the paper feed was allowed to run during
the short T O after each reinforcem ent, so th a t the clock could reset.) U n d er these
conditions, little scalloping occurs w ithout the help of the clock, as the segments at a, b,
a n d c show. Following the reinforcem ent at the arrow, the slit on the key was reset
to its smallest value a n d began growing through the interval in the m anner of a clock.
T he small slit controls a zero rate of responding im m ediately; and the bird begins to
respond late in the interval and reaches the term inal rate only w hen the slit becomes
alm ost m axim ally large. N ote th a t the larger value of the slit now controls a rate
of over 10 responses per second, com pared w ith an over-all rate of only 0.35 response
per second when the slit was at the same size but stationary. T he bird can make a
rapid transition between the 2 performances appropriate to the large-size slit under the
control of the other stim uli associated with m oving or stationary slits. T he num ber
of responses in the 1st interval w ith the clock, at d, is small; but beginning with the fol-
284
lowing interval, at e, the num ber rem ains appropriately large. T he bird consistently
showed a small n u m b er of responses d u ring the 1st or the first few intervals following
the transition to the clock.
As already noted, the perform ance of the other bird w ithout a clock was irregular.
T h e beginning of Fig. 330 shows a sample. Scalloping, frequent pauses after rein
forcement, intervals in which the term inal rate continues after reinforcem ent, and sec
o nd-order effects occur. W hen the slit resets to small (at the arrow ) and begins to
grow in the m a n n e r of an a d d ed clock, th e 1st in terv al is te rm in a te d by a single r e
sponse, even th o u g h the slit on the key has grow n to large before the response was
m ade. T he 2nd segment shows a fair clock perform ance, however, with a term inal
ra te of the order of 2.5 responses per second. T h e rem a in d e r of the session shows a
low over-all rate, w hich is the result of both a late start a n d a lower term inal rate
th a n is usual for this bird on F I 10 w ith clock. N ote th a t the term inal rates w ith and
Fig. 3 3 1 .
Transition from a clock to the slit a t the least o ptim al size
285
w ithout clock are here of the same order of m agnitude, whereas in Fig. 329 they dif
fered by a factor of 30. In other transitions from F I 10 w ith the slit stationary to FI 10
+ clock, this bird showed a sim ilar perform ance, w ith the rate recovering as the ses
sion proceeds.
Transition from F I 10 + clock to F I 10 with the clock stationary at the setting farthest re
movedfrom reinforcement. A final clock perform ance such as th at shown in Fig. 312 dem
onstrates a precise control by the size of the slit. T h e bird consistently pauses for the
m ajor p a rt of th e in terv al a n d shifts to the term in a l rate only w hen th e size of slit is
nearly m axim um . T he extrem e control exercised by the small slit in suppressing
behavior was shown when the slit rem ained a t its sm allest size for the first tim e (after
the series of reversals described in Fig. 326, 327, and 328, in which the slit was some
tim es statio n ary at large from the beginning of an experim ental session). T h e bird
rem ained in the ap paratus for 5 hours w ithout m aking a single response. T he 1st re-
Fig. 33 2 .
Transition from a clock to the slit a t the least optim al size (second bird)
sponse which occurred was reinforced, as shown in Fig. 331, at a. T he 1st reinforce
m ent in the presence of the small slit was followed by im m ediate responding at a low
m o d erate rate, w ith ra th e r rough grain. T h e 2nd reinforcem ent (at b) produced a
2nd increase in rate to a value which is roughly m aintained for the rem ainder of the
session. T ow ard the end of the session, at c and d, the rate tends to be somewhat lower
im m ediately after reinforcement.
T he same procedure was carried out w ith the second bird in the experim ent. T he
bird rem ained in the ap paratus w ithout responding during a com plete 12-hour ses
sion w ith the slit small. Placed in the a p p a ra tu s for a 2nd session, it w aited 2 hours
a n d 17 m inutes before responding a n d was th en reinforced, a t a in Fig. 332. T he
single reinforcem ent in the presence of the small-size slit produces a substantial rate of
responding through the 1st interval. T he 2nd reinforcem ent (at b) is followed by a
small am ount of negatively accelerated responding; an d the 3rd reinforcem ent occurs
only after a long pause. Subsequently, furth er increases occur in the over-all rate
of responding, w ith the rap id developm ent of a lower rate im m ediately following the
286
reinforcem ent. T h e procedure was th en changed to F I 10, w ith th e clock growing

in the usual m anner; an d 2 weeks later the experim ent was repeated. T h e 1st re
sponse occurs after only 12 m inutes of the small slit (at c in R ecord B) and is followed
by an im m ediate increase in rate. T he rem ainder of the session shows a fairly high
term inal rate and substantial pausing after reinforcem ent, which in some instances, as
at d, e, a n d f approaches the order of m agnitude occurring with the added clock.
T he irregularity shown by this bird in Fig. 330 is again evident.
W e fu rth er investigated the effect of changing from FI 10 + clock grow ing from
small to large to FI 10 w ith the slit at small by m aking the change in the m iddle of
the session on successive days. E ach session began w ith the same procedure as the
end of the preceding; a n d in the m iddle of the session the slit was changed to sta
tionary at small if it had been growing, or growing in the m anner of a clock if it had
been statio n ary at sm all. Figure 333 shows a characteristic transition for 1 bird.
T he clock performances at a and b before the transition show considerable curvature

as a result of the m any m anipulations of the clock. W hen the slit rem ains small at the
arrow , the rate is zero m uch longer th a n in the usual interval. W hen a response
finally occurs an d is reinforced at c, however, there is an im m ediate retu rn to the p a t
te rn of responding characteristic of this bird on F I 10 w ithout clock. T h e rest of the
figure shows the progressive developm ent of sm ooth an d extended interval curvature,
w ith occasional pauses alm ost 5 m inutes long (d, e). T he small slit probably favors a
lower rate of responding after reinforcem ent because of its general tendency to control
a lower rate in the norm al clock perform ance. Conversely, transitions to FI 10 w ith
th e slit at the optimal value probably interfere w ith the developm ent of a lower rate or
pause after reinforcem ent because of the past correlation.
T h e second bird showed a sim ilar result of the sam e transition in this experim ent.
Figure 334A shows the 1st transition. T h e last perform ances w ith clock occur at a
a n d b; a n d beginning at the arrow the slit rem ains small for the rem ainder of the ses
sion. A first response with the slit at small is reinforced at c, and the general FI
perform ance being displayed at this stage of the experim ent is reinstated. T he scallops
Fig. 3 3 4 .
287
First and third transitions from a clock to the slit a t the least optim al size (second bird)
here are sm oother and the pauses m ore extended and m ore appropriate to the in ter
val than in the 1st transition for this bird in Fig. 332A. Record B shows the 3rd transi
tion. T he segm ents a t a a n d b are th e p revailing clock perform ances before the
transition, w ith the interval at b showing an unusually sm all num ber of responses.
W hen the slit rem ains stationary at the small size (beginning at the arrow ), a welldeveloped FI scallop is im m ediately assumed, the pause after reinforcem ent being of
the order of m agnitude w hen a n ad d ed clock is used. T h e term inal rate is m ore ir
regular, however.
Transition from F I 10 with the slit stationary at small to F I 10 + clock. Interspersed
with the transitions described in the preceding paragraphs were changes m ade during
the session from the slit statio n ary a t the sm allest size to the clock grow ing in the
norm al m anner. Figure 335 shows the 1st transition for I bird. T h e final perform
ances w ith the slit stationary a t small occur at a, b, a n d c. T h e various m an ip u la
tions of the clock prevented any substantial scalloping. W hen the added clock is rein-
Fig. 3 3 5 .
Transition from slit a t small to clock
288
stated (at the arrow ), the first effect is to continue the rate prevailing under the
small-size slit. As the slit becomes larger, however, the ra te increases (d) a n d as
sumes the usual high term inal clock rate, at e. T he small spot continues to control a
substantial rate in the following segment (at f ) ; but the term inal rate is reached sooner,
an d a large num ber of responses are em itted at the term inal rate (at g ). T he smallsize slit again controls substantial responding at h an d i, but the control grows progres
sively weaker during the rem ainder of the session.
T h e following session continued w ith the usual clock grow th, an d in the m iddle of
the session the slit was left stationary a t small. A 2nd transition from FI 10 with the
slit at small to FI 10 + clock occurred in the m iddle of the session on the following day.
In Fig. 336 the perform ance with the slit stationary at small shows a linear perform
ance (as at a and b) and an occasional pause an d curvature (as a t c). T he return to
FI w ith ad d ed clock a t th e arrow is sim ilar to the 1st transition in Fig. 335. T he
residual control of the small spot is still to be observed. H igh term inal rates return
im m ediately, as usual.
Figure 337 shows the beginning of a session w ith added clock, following a 3rd transi
tio n from reinforcem ent a t sm all to the usual clock. T h e previous session ended
w ith a good clock perform ance, in w hich responding after reinforcem ent stopped a l
most com pletely and the shift to a high term inal rate was abrupt. O n the day shown
in Fig. 337, however, some control exercised by the small size appears as substantial
responding im m ediately after reinforcem ent, at a, b, and c. A smooth negative curva
tu re brings the rate to zero as the clock grows to the interm ediate values, a t which
reinforcem ent has never occurred. T he latter p a rt of the interval shows a norm al shift
to a very high term inal rate.
As we have seen, these variations in the contingencies of reinforcement involving sizes
of slit produce an irregular perform ance in a second bird. Figure 338 shows the 1st
tran sitio n from FI 10 w ith the slit statio n ary to F I 10 w ith the usual clock growth.
RESPONSES
300
Fig. 3 3 8 .
Transition from slit a t small to clock (second bird)
290
R ecord A begins with the slit stationary at small. Except a t a, each reinforcem ent
is followed by a pause, which in tu rn is followed by a substantial am ount of scallop
ing, although the grain is rough an d considerable variation exists from interval to in
terval. B eginning at the arrow , the clock grew in the norm al fashion. T h e only
im m ed iate change in perform ance is a n increase in the term in a l rate of responding
when the slit reaches the larger sizes. T here is considerable irregularity. T he m id
dle p art of the session has been om itted from the figure. By the end of the session
(R ecord B) a nearly norm al clock perform ance has developed except for small knees
and a term inal rate disturbed by occasional pauses.
Figure 339 shows the 2nd to 6th transitions for this bird. T he parts of the records
before the arrow represent the perform ances w ith the slit stationary at sm all; else
w here, the clock grow th is usual. A m ore a n d m ore effective transition to the clock
291
perform ance occurs w ith each successive reversal. T h e transition in R ecord A, for
exam ple, is sim ilar to th a t in Fig. 338, the first for this bird. T he 3rd transition (R ec
ord B) shows a complete absence of clock control in the first segment after the arrow.
By R ecord C the deviations from a final clock perform ance are less m arked, and by
Records D and E a fairly good clock perform ance develops as soon as the slit is m ade
to move. T he perform ance before the clock was added is om itted in R ecord D.
A discontinuous clock
T he physical dimensions of a clock are arbitrary. T he clock used in the preceding

experim ents is only one of m any possibilities. A series of discrete color changes, a
succession of com plicated shapes changing in tim e, or a pointer m oving over a dial in
th e m anner of a timepiece m ight have been used. In the following experiments the
small slit was used as in the previous experim ents, but reinforcem ent was correlated
w ith the m iddle-size slit. T he sequence of events is as follows. J u s t after reinforce
m ent the slit is at the m iddle size. It reaches its m axim um w idth halfw ay th rough
th e fixed interval, an d at th a t point changes a b ru p tly to the sm aller size. R e in
forcem ent occurs when the m iddle size has again been reached. T he cycle is then re
peated d u rin g the next interval. In this clock the stim ulus on the key ju st before
the reinforcem ent is almost identical with the stim ulus at the start of the next interval.
T he greatest change in the stim ulus on the key occurs abruptly in the m iddle of the
fixed interval.
T he subjects were the 2 birds from the preceding experiments. A final perform
ance on FI 10 + clock, consisting of a slit which is small after reinforcement and
grows to its m axim um size at the end of the fixed interval, h a d been developed after
the transitions described in the preceding section. T he clock was m aintained w ith
out change for 3 full sessions. Figure 340A shows 2 segments from the final p e r
formance. Reinforcement was then begun with a discontinuous clock. The 1st in
terval of the session in Fig. 340B begins (at a) w ith the slit at the m edium size. A
/
R 's /S E C
Fig. 3 4 0 .
First exposure to a discontinuous clock
292
Fig. 3 41.
Second session on the discontinuous clock
m oderate rate prevails. A smooth acceleration then leads to the usual high term inal
clock-rate at b as the slit grows to large. T h e slit reaches its m axim um size a t c,
halfw ay th ro u g h the interval, a n d changes a b ru p tly to the sm allest size. T h e ra te
drops to a lower b u t still substantial value w hich continues unchanged until re in
forcem ent. T h e b ird s perform ance u n d e r the previous continuous clock obviously
was not controlled solely by the absolute value of the slit. L ater segments in Fig.
341B show a sim ilar control. T he interm ed iate rate d u rin g the second h alf of each
interval, w here the slit grows from sm all to m iddle size, continues practically u n
changed for m any intervals; but the scallop deepens progressively a t the start of the in
terval, w here the slit grows from the m iddle size tow ard large. T h e high term inal
ra te at the large size holds this high value u n til the 11th interval. In the last 3 seg
m ents of the figure some suggestion of an acceleration continuing throughout the in
terval appears. H ow ever, this factor does not necessarily im ply any control by the
discontinuous clock, since perform ances of this order of m agnitude can occur on FI 10
R 's / S E C
h
/ '/4
1
10 M IN U T E S
Fig. 3 4 2 .
End o f the sixth session on the discontinuous clock
Fig. 3 4 3 .
293
N inth session on the discontinuous clock
alone. This exposure to a discontinuous clock has, however, at least elim inated the
control over the rate established w ith the continuous clock. Figure 341 shows the
2nd day of exposure to the discontinuous clock. This perform ance exemplifies an
overnight loss in stim ulus control which is com m only observed. T he evidence of a
developing control by the discontinuous clock is clear. M ost of the curves are Sshaped. T he stim ulus on the key ju st following the reinforcem ent is the same as th at
present at reinforcem ent. But both large and small slits present during the m iddle
p a rt of the interval are now occasions on w hich reinforcem ent never occurs; hence,
they control a lower rate of responding. At the end of the session a norm al scallop
occurs. Figure 342 shows a further developm ent under the discontinuous clock for
the last p a rt of the 6th session. Definite control by the clock has now been estab
lished. R esponding ceases at reinforcem ent, a n d the pause is very m arked; accel
eration to the term inal rate occurs late in the interval, in the m anner of the contin
uous clock except for a low term inal rate. O th e r stim uli are clearly operating, in
add itio n to the size of slit, to prevent responding to the m iddle-sized slit ju st after
reinforcem ent. By the 9th session (Fig. 343) the perform ance is not very different
from th a t w ith a continuous clock.
T he second bird showed a sim ilar transition to the discontinuous clock, although
the rate of developm ent was very different. Figure 344A shows the start of the 1st ses-
294
sion w ith the discontinuous clock. T h e segm ents are sim ilar to those in Fig. 340.
T he large slit controls a high rate because of the previous exposure to the continuous
clock; a n d the small slit produces an a b ru p t shift to a lower but still substantial rate
instead of the zero rate previously appropriate to th at size. In Record B, in the m iddle
of the sam e session, the high rate of responding in the presence of the large slit has
largely disappeared, and the term inal rate has increased. By the 7th hour of the ses
sions shown in R ecord C the rate of responding tow ard the end of the interval (as
the slit approaches its m iddle value) is very high (at a, c, e, g, h, and j ) ; but the high
rate is recovered in the presence of the large-size slit, app ro p riate to the continuous
clock of the previous session at b, d , f a n d i. By the end of the session, after 12 hours
of exposure to the discontinuous clock (R ecord D), there is less of a tendency to re
spond w hen the slit is large in the m iddle of the interval, as for exam ple, at k and
/; but at m the interm ediate-size slit controls a high rate. T he 3rd session of expo
sure to the discontinuous clock shown in Fig. 345 shows a further developm ent. At a
the high rate controlled by the large-size slit returns because of the history on the
previous clock; but a fairly norm al fixed-interval curve appears at b. T he appearance
of this curve m ay or m ay not reflect control by the discontinuous clock. This p e r
form ance is followed by an interval in which a high rate is m aintained substantially
throughout. T hree intervals then follow which show an extended pause after rein
forcem ent a n d acceleration to a high term in al rate before the end of the interval.
These curves indicate control by the discontinuous clock. Following the reinforce
m ent at c, however, a high sustained rate appears thro u g h o u t the interval, m arked
by a low er rate in the region of the large a n d very sm all slit sizes. Tw o fu rth e r in
stances, at d a n d e, also show increased control by the interm ediate-size slit. T h e
5th session of exposure to the discontinuous clock is represented in Fig. 346. H ere, the
segments are either scalloped, as at b, e, and f , or show a high rate of response after
reinforcem ent which falls off as the slit reaches the largest size, as at a, d, and g. Figure
Fig. 3 4 5 .
Third session on the discontinuous clock (second bird)
295
347 shows the final perform ance for the second bird at the end of 100 hours. T he
rate is zero for considerably more th an half the interval; the term inal rate is fairly high
but irregular. Nevertheless, a high rate still occasionally appears near the start of the
interval (as a t a) an d the term in al rate at the end of the interval is sometimes not
reached (as at b and c).
Return to the continuous clock on FI 30
These 2 birds were returned to a continuous clock, and the interval was increased
to 30 minutes. The transition still gives us some inform ation about the previous clock
control. Figure 348 shows the 1st exposure to FI 30 with a continuous clock, where
the slit begins at the sm allest value following the reinforcem ent an d increases con
tinuously to the large size after 30 m inutes. O nly 4 intervals occur in the 1st ses
sion; they are shown in the order in which they occurred. A response was reinforced
a t the end of each segm ent. T h e general form of all 4 curves is S-shaped. T he
lowest rates of responding were at the beginning a n d end of the interval, when the
296
Fig. 3 48.
Return to continuous clock after

discontinuous clock
slit was very small and very large, respectively; and the high rates of responding were
in the m iddle range of the interval, w here the slit was m iddle-sized. T he form of
these curves, of course, corresponds to the likelihood of reinforcement on the previous
discontinuous clock. Note, however, th a t the very large an d very small slit values do
not control a zero rate of responding, as they w ould if the absolute size of the slit
were the only controlling variable. An increase in rate appropriate to the new sched
ule appears at a.
R E S P ONS ES
300
Fig. 3 4 9 .
End o f the second session a fte r
the return to the continuous clock
Fig. 3 5 0 .
Selected segments during FI 30
+ clock a fte r reinforcem ent on a discontinuous
clock
298
T h e transition to F I 30 w ith a continuous clock is almost com pleted by the end of the
2nd session after the discontinuous clock. T h e first 4 intervals of the session showed
high over-all rates of responding, largely because of the rapid responding in the m iddle
range a n d the effect of the large slit a t th e end of the interval. Figure 349 p re
sents the last 7 intervals of the session in the order in w hich they occurred. Some in
stances still exist of fairly p ro m in en t knees, p articu la rly a t a a n d b, w hich are
probably a result of the previous reinforcem ent at the m iddle slit-size. (A final p e r
form ance on F I 30 w ith continuous clock has already been described in Fig. 314.)
T he second bird showed a m ore rap id transition to the continuous clock, b u t also
a m ore prolonged effect of the previous reinforcem ent a t m iddle slit-sizes. Figure 350
contains 3 sets of intervals w hich have been collected in term s of the kinds of rate
changes occurring in the m iddle range of the interval. In order to perm it the greater
reduction a n d better com parison of the rate changes in the m iddle range, horizontal
portions of the curves ju st after reinforcem ent have been deleted. T he group of curves
in R ecord A contains knees consisting prim arily of an a b ru p t shift to a very high rate
of responding, followed by a decrease to zero an d th en a second sudden increase to
the term inal rate. T he knee a t a is exceptionally large, a n d a t b is probably insignifi
cant. Cases of 2 or m ore knees have been collected in R ecord B; included am ong
these is a single curve in w hich the a m o u n t of sustained responding before th e te r
m inal ra te is reached is so sm all th a t th e effect is m ore properly called a rough grain.
In R ecord C the knee consists of a high rate followed by negative acceleration to an
interm ediate rate, which then accelerates positively and eventually blends into the te r
m inal rate.
THE EFFECT OF TO DURING PART OF A FIXED INTERVAL WITH CLOCK
W e attem pted to exam ine some of the properties of a well-developed clock in F I 30

by im posing a 25-m inute T O . This T O determ ined, in particu lar, w hether the p ro
gressive change in stim ulus was im p o rta n t in the clock control relative to the absolute
control of each slit size. T h e first 25 m inutes of the interval was replaced by a T O
w ith the clock running. T he schedule actually was F I 5 T O 25, w ith the clock first
seen already advanced to the stage a t w hich it controlled a high rate on F I 30 (e.g.,
transition from continuous to discontinuous). Figure 351A shows segments for a full
session on F I 30 w ith clock a t a given stage of developm ent. These occur in the order
in which they were recorded, an d horizontal portions of the 30-m inute interval have
been cut off to facilitate reproduction. K nees are typical for this interval w ith clock;
the term in al rates are w ell-sustained. R ecord B represents the 1st day in w hich the
first 25 m inutes of the interval were replaced by a T O , all lights in the apparatus being
off a n d th e bird not responding. T h e recorder ra n d u ring the T O . T h e point a t
w hich the T O ended is shown by the small vertical dashes. In general, the curvature
in the 5-m inute period following T O tends to be som ew hat m ore m arked th a n th a t in
/
299
R's/SEC
10 M I N U T E S
Fig. 3 5 1 .
FI 5 TO 25 + the last portion o f FI 3 0 clock
R ecord A. Fewer exam ples of sharp knees exist, and the term inal rates are lower.
T he total effect is a sm aller num ber of responses per interval. This effect could be in
p a rt due to the early disturbances from a prolonged blackout; b ut it could also be due
to the fact th at the bird is beginning to develop a 5-m inute FI performance with the aid
of a clock which is now advancing over only one-sixth of its range during the inter
val. Record C shows the performance on the 9th and last day of this part of the exper
im ent. T he perform ance is sim ilar to th a t of FI 5 w ithout the help of the clock, a l
though some intervals, particularly the first 2 (at a and b), show an excellent clock
perform ance. H igher term inal rates have now been recovered, although the cu rv a
ture in approaching these rates is still m ore m arked th an when the whole interval was
ru n off without a TO .
W e further attem p ted to explore the effect of T O d u ring p a rt of the operation of a
clock by alternately replacing 2-m inute and 25-m inute portions of FI 30 clock by T O s
in a single session. T he schedule was essentially m ult FI 5 F I 28 w ith residual po r
tions of a clock. Figure 352 shows the results for 1 day. Those cases in which a 25m inute T O occupies the greater p art of the interval (up to the dashes) are collected
in Record A. T he bird can respond only at the end of the interval, and during this in
terval the clock covers only one-sixth of the total range. R ecord B shows the segments
in which 2-m inute T O s (up to the first dashes) leave most of the interval free for re
sponding. R esponding begins late because of the operation of the clock, which now
covers most of its range. Twenty-five m inutes is autom atically m arked for reference
purposes; but no change in procedure was m ade at these points. No great difference
exists betw een the 2 sets of curves, an d the absolute value of the clock appears to
control the performance.
300
Fig. 3 5 2 .
M ult FI 5 FI 28 w ith residuals o f an FI 30 clock
M agnitude o f the clock
W e have not m ade any extensive study of the m agnitude of stimuli used for clocks;
but 1 bird was exposed to FI 2 with a clock which consisted of only one-fifth the norm al
grow th of the spot of light. This bird had h ad a long history of FI with added counter
a n d m ult F IF I + clock, so th a t the developm ent of the perform ance is not signifi
cant. After 10 hours of FI 2 w ith the sm all-scale clock, however, the perform ance
was no better th an th a t in Fig. 353, which is a com plete session. Several instances oc
c u r here of no pause following reinforcem ent, particularly at a and b, but no case of
a pause com parable with those norm ally observed w ith the help of the clock. Instead,
there tend to be 2 term inal running rates: one is reached shortly after responding is
resum ed; and the other emerges m ore or less abruptly, for exam ple, at c and d, to con
tinue for the balance of the interval.
A nother bird w ith a long and com plex history of added counter an d clock failed to
301
show a good clock perform ance even w ith the norm al range of stimuli. Figure 354
shows a perform ance after 62 hours on FI 10 + clock. A lthough this performance is
unquestionably more strongly scalloped th a n th a t on FI 10 without the help of a clock,
a n d although m any intervals show good clock control (a an d b, for exam ple), the start
is generally earlier, and m ore curvature develops during the interval th an is charac
teristic of the added clock performance. A com panion bird with the same com plicated
history developed a very good clock performance.
302
FIXED INTERVAL WITH ADDED COUNTER
O ne explanation of the oscillation in perform ance in the later stages of an FI per

form ance is th at a tem porarily stable condition holds the num ber of responses per inter
val constant; this fact, as in F R , makes the production of num ber autom atically re
inforcing. Therefore, earlier an d m ore rap id responding sets in, an d the stable
condition is destroyed. Some exploratory experim ents on FI with added counter are
relevant to this interpretation.
Tw o birds used in this experim ent h ad had a history of FI 10. A th ird h ad had
an extended history of V I and fairly large FIs. T he perform ance of these 3 birds were
sam pled, one each on FI 5, FI 10, a n d FI 20, w ith the spot of light on the key at its
smallest size. T he spot was arranged to grow w ith the num ber of responses em itted
as an a d d ed counter by the bird. T h e slit reached its full length at ab o u t 600 re
sponses. In an interval containing a small num ber of responses, reinforcement occurred
a t a fairly small size of the slit.
FI 5 with added counter
O ne bird, which had shown a rath er rough grain on FI 5, developed some interval
curvature w hen the counter was added. T he term inal rate was high, and as m any as
700 or 800 responses m ight occur in each 5-m inute interval. T he bird was therefore
303
frequently reinforced with the spot at its largest size. T he bird did not develop further
curvature during 15 hours (extending over 7 sessions) on F I 5 w ith counter. T he size
of the slit had nevertheless developed a strong control. U pon changing to the slit th a t
was stationary w hen at large, a high rate of responding was m aintained for the b a l
ance of the session. M ore th an 10,000 responses occur w ithout any reduction in this
rate. This perform ance is shown in Fig. 355. T he first 3 excursions show a fair sam
ple of the perform ance with the aid of a counter on FI 5. T he m ain effect of w ith
draw ing the counter is to elim inate the pause after reinforcem ent and the interm ediate
stage of less th an the highest term inal rate. However, the other 2 birds at longer FIs
showed a very rapid developm ent of the effect of the counter.
As was usual with this bird at this value, m arked scalloping occurred b u t no pro
nounced pausing after reinforcem ent. T he small size of the counter controls a low
rate; but as soon as the spot has begun to grow, the rate steadily increases. At the
arrow in Fig. 356 the counter was stopped at small, th at is, at the size farthest from the
reinforced size. Some scalloping develops in the interval which follows, but the rate
rem ains very m uch below the term inal rate prevailing w ith the counter. In other
words, some evidence exists here th a t other factors besides the spot increase the rate
during the interval (a), but it is slight. T he 2nd interval (b) also shows curvature,
304
b ut again the term inal rate is not high. T he over-all rate then advances w ithout very
m uch interval curvature for the next 4 intervals (c, d, e, a n d / ) .
Figure 357 shows a later perform ance on FI 5 + counter. Although the rate follow
ing reinforcem ent is m arkedly depressed, the prolonged pauses produced by an added
clock are lacking. Pauses are also com m on in short intervals w ithout T O (cf. Fig.
152 for FI 4 after com parable exposure to the schedule). T h e n um ber of responses
em itted per interval is considerably larger w ith the help of the counter. O ne interval
a t a shows a low count p artly as the result of rough grain; b u t this perform ance is not
followed by m ore rapid starting in the subsequent intervals. T he production of count
acting as a reinforcem ent appears to get the bird off to an earlier start a t a higher rate
a n d results in a larger num ber of responses per interval.
T he bird chosen for the study of F I 10 with counter gave the performance on FI 10
alone shown in Fig. 358A. A short pause occurs after reinforcem ent, but the term inal
rate is rapidly reached. T he counter was added at the beginning of the session shown
at R ecord B. T he first few segments show a progressive increase of slow responding
after reinforcem ent (a, b, and c). By the 6th interval (at d) a w ell-m arked scallop has
set in.
T h e beginning of Fig. 359 shows a later perform ance on F I 10 + counter. Sub
stantial scalloping m ay occur. T he setting of the counter ju st after reinforcem ent is
very different from th at at most reinforcem ents; this negative control evidently cancels
the reinforcing effect of the production of count. N ote th at an occasional reinforce-
150 R's
Fig. 3 5 8 .
First e ffect o f the added counter on FI 10
306
m ent at a low count (a) has the im m ediate effect of m aking responding at a low counter
reading m ore probable, a n d th a t substantial pausing develops again only during a set
of segments showing progressive scalloping (a, b, and c).
At the arrow the counter did not reset. T he high reading controls a high rate for
most of the balance of the interval (cf. Fig. 355).
A perform ance after 95 hours of FI 10 w ith ad d ed counter is shown in Fig. 360,
where the segments have been arranged according to num ber and curvature. A later
perform ance after 115 hours is shown in the order of recording in Fig. 361. In both
figures the low reading of the counter after reinforcem ent delays responding in opposi
tion to any reinforcing effect of the production of count. T he curves are, however,
by no m eans as square as w ith an added clock. Figure 164 shows the same bird 2
m onths earlier on FI 8. Figure 166 shows another bird on FI 10. Both of these are
w ithout T O and w ithout added clock or count. Insofar as these provide a basis for
com parison, the effect of the added counter appears to be to deepen the scallop. This
307
effect resembles th at of a T O . Both provide an anchor point at the beginning of the

interval; ap a rt from any change in the size of the slit during the interval, one conspicu
ous result of a counter is th a t the interval begins u n d er a stim ulus (the smallest size of
the spot) which is absent at reinforcem ent. This should have the effect of elim inat
ing second-order effects in addition to favoring a pause an d scalloping after reinforce
ment. Note, however, th a t some instances of second-order effects continue to occur
under FI with added counter.
T he bird in the experim ent ju st described was accidentally reinforced on FI 20 with
added counter instead of FI 10 during 1 session (2 sessions before Fig. 359). As the
effect of the larger FI, almost 2000 responses were em itted in the 1st interval. T h e
counter, of course, reached its m axim al growth-size at about 600 responses. T he in ter
val segments in Fig. 362 appear in the order in which they occurred. T he segment
at a shows the beginning curvature previously developed u n d er FI 10 + counter, a l
though the spot is growing only half as fast as before. W hen the spot reaches m axim al
size, the term inal rate continues, as in experim ents on the transition to a count sta
tionary at m axim al size. W hen the count resets to zero a t the 1st reinforcement, a sim
ilar perform ance follows (Segment 2); b ut the 3rd segment shows some falling off (at
b) after the spot reaches a substantial size. This perform ance is somewhat more clearly
m arked in the 5th interval, at c. Sim ilar negative accelerations appear later, partic
ularly at d and e, and the over-all rate falls off to a value somewhat more appropriate to
FI 20.
T he bird assigned to FI 20 + counter h ad h ad the longer history m entioned above,

including a prolonged exposure to V I. O n FI 20 alone, it showed a low sustained
rate suggesting a V I perform ance. Figure 363 shows the 1st day with added counter.
T he tendency for the spot to control the rate is alm ost im m ediate, beginning a t the
2nd interval, at a; it becomes quite m arked before the session has advanced very far.
A later perform ance (after 28 hours) is presented in Fig. 364, where segments have
Fig. 3 62.
First session on Fl 2 0 + counter a fte r Fl 10 + counter
309
R 's / S E C
" /&
1
10 M I N U T E S
Fig. 3 63.
The firs t e ffe ct o f the added counter on FI 20
been arran g ed in order of the n u m b er of responses. O nly 1 instance occurs in which

the term inal rate is reached during the first 5 m inutes, a n d only 1 instance in which no
m arked acceleration exists during the interval. In m any of these intervals, however,
the term inal rate is never reached, and most of them show a fairly sustained slow accel
eration. A conspicuous exam ple of a knee exists a t a; b u t examples such as this are
rare com pared with performances w ith added clock. T he prolonged pausing after re
inforcem ent produced by a clock is almost entirely lacking. (Com pare Fig. 350 and
352 for a com parable FI with a clock.)
W e also checked the control exerted by the m axim al setting of the added counter
at this value of FI by allowing the counter to rem ain m axim al throughout the latter
part of a session. This was done at the arrow in Fig. 365; the result resembled those
of Fig. 355 a n d 359 except for the greater decline in rate produced by the m uch less fre
quent reinforcem ent at FI 20.
A later perform ance at FI 20 + counter is shown in Fig. 366, which should be com
pared with Fig. 364. T he acceleration is sharpened in Fig. 366. A straight line
draw n from a to b would pass close to the term inations of the periods of low respond
ing after reinforcement.
A second added counter
In one experim ent on F I w ith ad d e d counter the co u n ter com pleted the change
from small to large during a fixed num ber of responses which was usually less th an the
n um ber generated by the schedule. U n d e r these circum stances the first p a rt of the
interval was supplem ented by a counter, while the last p a rt occurred with the counter
stationary a t its largest size.
In the early stages of the experim ent, all 3 birds tend to respond a t a higher rate
while the counter is operating, a n d to drop to a lower rate shortly before or when it has
com pleted its cycle. Figure 367 shows p a rt of 1 session. T he schedule was FI 10,
and the counter completed the cycle in 240 responses. E ach reinforcement is followed
by a substantial pause or period of slow responding, w hich is in tu rn followed by a
311
relatively high rate, for from 100 to 250 responses. T hereupon, the rate drops to a
fairly stable interm ediate value for the rest of the interval. Thus, at this stage the in
crease in the counter reading is probably reinforcing an d produces a high rate; b u t
the counter near or at its largest size probably controls only an interm ediate rate, essen
tially as in an FI 10 alone.
T his stage comes an d goes an d eventually disappears. Figure 368 shows a later
perform ance for another bird after 105 hours on FI 10 w ith this p artial counter. T he
counter reaches its m axim al count in 240 responses. M ost of the intervals shown con
tain m ore th a n 240 responses, but no consistent tendency exists for a drop to a lower
rate after the counter has ceased to tally. H ere, the term inal rate is m uch higher than
th at in Fig. 367, and the over-all curvature suggests an FI + clock.
At a n earlier stage in the experim ent the bird responsible for Fig. 367 was acci
dentally reinforced on a fixed ratio of 120 for 8 reinforcem ents. T he counter, which
com pleted its count in 120 responses, was present. Figure 369A shows the result.
T h e rep eated reinforcem ent on F R 120 produced a quite uniform acceleration, b e
cause of an increase in the ru nning rate which reaches a high value at a, the reduction
of the pause after reinforcem ent, a n d the speeding up of the acceleration to the te r
m inal rate. T he apparatus was then corrected, and R ecord B was m ade a few m in
utes later in FI 10 + p a rtial counter. R ecord B begins w ith the perform ance ju st
developed on F R 120 w ith added counter, an d the early high term inal rate drops only
slowly from the m axim al value at b to the substantially lower value at c, ju st before
reinforcem ent. T he perform ance in the 2nd interval starts early and sustains a some
w hat lower term inal rate until the 2nd-interval reinforcem ent a t d. Thereafter, the
perform ance is essentially similar to earlier performances a t F I 10 with a partial counter
(cf. Fig. 368).
Fading counter
In speculating about the function of the b ird s own behavior as a counter, we as

sume th a t the counter resets to zero at reinforcement. This is unlikely unless a sub
stantial T O follows reinforcement. Also, some continuous resetting probably goes on, in
the sense th a t a given bit of behavior which m ay serve as a counter becomes m ore and
Early perform ance on F! 1 0 + counter reaching maximum size a fte r 2 4 0 responses
300
RE S P ONS E S
Fig. 3 67.
Fig. 3 68.
Fig. 3 6 9 .
Final FI 10 + counter (second bird)
Eight reinforcem ents on FR 1 2 0 + counter during FI 1 0 + counter

313
314
m ore rem ote in time. In a rough attem p t to duplicate this characteristic, we have
used a fad in g counter. A differential gear controls the slit in such a way th a t the
b ird s own responses increase the slit size while a clock slowly reduces it. Values were
chosen so th a t at a rate of 1 response per second the 2 changes were equal (and op
posite). R ates above this value would cause a grow th of the spot but a slower growth
th an with a nonfading counter, while rates below this value would perm it the spot to
shrink slowly tow ard the smallest size. T he counter would reset to zero if the pause
were long enough. A lthough we did not m ake any exhaustive study of the various
properties of such a counter, some evidence exists of its effectiveness.
F I 20. T he fading counter was introduced at the beginning of an experim ental ses
sion after 104 hours on FI 20 w ith the usual nonfading counter. Figure 370 shows
the first result. In the 1st interval the bird quickly reaches a rate (at a) at w hich the
counter grows, although m ore slowly th an heretofore. T he slit has probably reached
its full size by the tim e of the 1st reinforcem ent (at b). T he counter was always re-
Fig. 370.
First exposure to FI 2 0 + fad in g counter a fte r FI 2 0 + counter
set to zero at reinforcem ent. In the 2nd interval a rate which is ju st at the critical
value is assumed relatively late in the interval (beginning at c) and m aintained until re
inforcem ent (at d). T he spot probably did not grow very m uch during this interval.
T he following interval shows an even greater delay in reaching a substantial rate, and
again the rate is roughly the value at which the spot rem ained stationary. W hether
the counter had any substantial reading at the following reinforcem ent at e is doubtful.
Thereafter, the rate was clearly below the critical value, and the performance is sim i
lar to th a t at which the counter is replaced by a setting most unlike th a t at previous re
inforcem ents. In other words, from point e on, the perform ance w ould have been
the same if instead of using the fading counter, we had simply set the counter stationary
at the smallest-size spot. Good interval scallops are shown later in this experim ental
session, but the critical rate is not reached, except possibly for the end of 1 interval (at
/ ) . This perform ance shows the gradual developm ent of a scallop w ithout the help of
the counter.
T h e critical rate was not exceeded in the next 2 sessions of 12 hours each. T he rate
315
rose above the critical value in the 2nd interval of the 3rd experim ental session, how
ever, and frequently reached th at value thereafter. T h e rate tends to be either sub
stantially above the critical value at which the counter is tallying, or below it, in which
case the bird is essentially operating on FI with counter a t its m inim al setting. T o
w ard the end of this 3rd session the lower value generally prevails, and a great deal of
ru nning-through results, followed by m arked periods of low rate. In the following
session there is again an alternation betw een rates significantly above the critical rate
and very uniform ju st below it. A later perform ance for this bird under FI 20 w ith
fading counter is shown in Fig. 371, for the first p art of a session beginning 60 hours
after the introduction of the counter, and Fig. 372, for the end of the same session. A t
the beginning of the session (Fig. 371), several instances occur of rates which exceed the
critical value as d u ring the greater p a rt of the 1st interval, at a. T he slope a t b is
alm ost precisely the critical value w hich m aintains the spot, although some grow
ing m ay have occurred. A clear growth of the spot is indicated at c. In the follow
ing interval the critical rate is reached a t approxim ately point d, an d the last interval
shown in the figure was run off at a rate considerably above the critical rate throughout
the interval. T ow ard the end of the session, however, as Fig. 372 shows, no instances
occur of sustained running above the critical rate, and only occasional instances (as
at a and d) at which the rate th a t just m aintains the size of the spot is approxim ated.
At b and c, sustained runs occur at lower th an the critical value. These intervals m ust
therefore be regarded as FI without benefit of counter. They show well-m arked
running-through after reinforcement, but long periods of no responding or very slow
responding before the assum ption of the interm ediate term inal rate.
316
Figure 373 shows the result of rem oving the fading elem ent 62 hours after it was in
troduced. T h e first 2 intervals show the perform ance at this stage w ith the fading
counter. In the 3rd interval the fading elem ent was rem oved at the arrow; the im m e
diate effect was to increase the rate to a m uch higher term inal value (as a t a). T he
term inal rate was then m aintained throughout the period. T he b ird s perform ance
with a nonfading counter now resembles Fig. 366.
F I 10. Figure 374 shows another exam ple of the first effect of a fading counter on
FI 10. This bird h ad been giving a w ell-m arked scallop on FI 10 + counter. (See
Fig. 361 for a late stage of this perform ance, and Fig. 360 for an earlier stage. A n
other sam ple for this bird occurs a t the beginning of Fig. 359.) In Fig. 374 the effect
of the fading counter is clearly sim ilar to th a t w hen the counter rem ains small (cf.
Fig. 356 for another bird). W hen the 1st responses fail to increase the size of the spot
because they are below the critical rate of 1 per second, the small spot continues to
control a low rate, and the earlier acceleration characteristic of this bird (as at a) fails
to m aterialize. O nly d u ring the 3rd interval, ap p ro x im ately a t point b, is a rate
reached which increases the size of the spot. This rate leads to nearly the c h a ra c te r
istic term inal rate at c. T he next interval shows a very rapid adjustm ent to this new
Fig. 3 7 3 .
Transition from FI 2 0 + fa d in g counter to FI 2 0 + counter
Fig. 3 74.
317
Transition from FI 1 0 + counter to FI 10 + fa d in g counter
condition. At d a high rate of responding is quickly assum ed which rapidly increases

the size of the slit. This in tu rn continues to control a high rate of responding
throughout the interval to the reinforcem ent at e. T he following 3 intervals, how
ever, show rates which are only very briefly above the critical rate of 1 per second, and
the effect is essentially th a t of FI 10 w ithout counter. Some curvature exists in the
3rd of these intervals (at f ) , an d brief runs at this point exceed the critical value. But
the value just able to cause the spot to grow is reached again only at g. It is barely
m ain tain ed until reinforcem ent a t h. An alm ost im m ediate advance occurs a t i to a
rate well above the critical value.
A nonfading counter was then used for 2 sessions, and the bird was then exposed to
th e fading counter again. U nfortunately, the record for the session was badly re-
Fig. 375.
Second session o f return to FI 10 + fa d in g counter a fte r nonfading counter
318
corded; but it shows a quick return to a perform ance which was m aintained during the
com plete 2nd session shown in Fig. 375. T h e term inal rate tends to exceed the criti
cal value. R esponding m ay continue a t a rate below the critical value, however, an d
hence essentially w ithout benefit of counter. C om pared with the set of segments in
Fig. 360, the segm ents in Fig. 375 show 2 values of term in al ra te , one above a n d one
below the rate sufficient to increase the counter. T he term inal rate is assumed m ore
abru p tly after the pause following reinforcem ent. T h e beginning of Fig. 376 shows a
late perform ance w ith a fading counter. At the arrow the change to a nonfading
counter produces generally higher term inal rates, which are partly reflected in the large
num bers of responses per interval. Before the end of the session the term inal rate
tends to fall somewhat.
F I 5. T h e beginning of Fig. 377A shows a perform ance after 8 sessions of 2 hours
each on FI 5 w ith fading counter. At this size of interval the effect is quite variable,
Fig. 376.
Final perform ance on the fa d in g Counter and the transition to the nonfading
counter: FI 1 0
319
th e grain is rough, pauses after reinforcem ent m ay or m ay not occur, an d the te r

m inal rate is of a m oderate-to-high value. At the arrow the fading com ponent was
rem oved. Low rates after reinforcem ent begin to a p p e ar; scalloping has becom e
consistent by the end of the session (as a t a, b, c, d, e, a n d f ) . A n even better interval
cu rv a tu re develops on the following d ay in the absence of the fading com ponent.
R ecord B represents the end of this 2nd session. T he over-all rate is som ew hat less
here th a n in Fig. 357 for the sam e bird, w hich shows an earlier perform ance after
58 hours with the usual counter.
THE EFFECT OF A NOVEL STIMULUS O N FI 20
A perform ance after extended exposure to F I 39 has been reproduced in Fig. 196.
T he bird was showing u f drly stable perform ance, w ith w ell-m arked scallops and a
fairly regular acceleration to a term inal rate. In p reparation for an experim ent w ith
added counter, an opaque mask was placed over the key, so th a t only a small slit of
light was exposed. At the same tim e, the schedule was changed to FI 20. Figure 378
shows the effect of the novel stimulus. T he fact th a t alm ost every interval has
m arked negative curvature suggests the early stages of erf to FI. Occasional break
throughs occur, as at a and b, at higher rates. T he grain is unusually rough. D uring
the rem ainder of this session, the perform ance was essentially linear, with some slight
continuing negative curvature leading to a low rate ju st before reinforcement. This
rate suggests the 2nd phase in the transition from the erf. T he 2nd session (Fig. 379)
m aintains the parallel with the transition to FI after erf, with an over-all linear p er
form ance an d a slight tendency to respond faster im m ediately after reinforcem ent.
This constant rate of responding occurs despite a long history in which the bird showed
a m arked scallop.
!
320
Fig. 3 79.
The effect o f a novel stimulus: FI 2 0 (second session)
SATIATION UNDER FI 2
Figure 380 is a plot of responses em itted d u ring an 18-hour session, w hen a bird
received over 500 reinforcements on FI 2. T he extent of this degree of satiation is sug
gested by the fact th a t 60 reinforcem ents m ain tain ed the bird at approxim ately its
experim ental weight. T he over-all curve is negatively accelerated over the first 9
hours, with very few deviations from an orderly decline in rate. For the last 9 hours
of the session, it is roughly constant at about 0.35 response per second. Figures 220
and 221 showed this b ird s transition to FI 2 T O 5, following a history of 1300 rein
forcem ents on FI. T he satiation curve was taken w ithout T O after reinforcem ent
during the session following Fig. 221, after a history of 450 reinforcements with T O .
Details of Fig. 380 are presented in Fig. 381. A segm ent has been taken from the
larger curve every 63 m inutes. T he top segm ent, after 40 reinforcem ents, shows a
perform ance similar to the previous session, in Fig. 221. As satiation proceeds, how
ever, the effect is an increase in the pause following the reinforcem ent an d the ex
tent of the curvature. By the 4th segment, strong negative acceleration occurs.
F rom Records E to H the over-all rate of responding falls sharply, largely because
of a decline in the term inal rates. By R ecord H , intervals occur in which the grain
is very rough a n d a term in al rate is not reached. T ow ard the end of the session,
how ever, the few responses per interval occur to w ard the end of the interval, an d
most reinforcements are received almost as soon as scheduled.
RESPONSES
OF
THOUSANDS
HOURS
Fig. 3 8 0 .
S atiation curve under FI 2 reinforcem ent
322
Fig. 3 8 1 .
Segments excerpted from a satia
tion curve under FI 2 reinforcem ent
THE EFFECT OF A LOSS OF BRAIN TISSUE O N FI
W e studied FI perform ances in 2 birds after various parts of the b rain were re
moved. (See p. 85.) T he records of these birds are presented m erely as examples of
th e kinds of disruptions th a t can occur in FI perform ances because of brain lesions
b u t not p a rticu la r lesions. T h e final perform ance after a long history of m ultiple
schedules, the last of which was m ult FI 15 tan d FI 15 F R 10, was sim ilar to th at of a n
323
other bird, shown in Fig. 667 in C hapter Ten. T he bird was then subm itted to an
operation, the postm ortem result of w hich was described as a cyst in forw ard p art of
brain 9 m m in depth by 3 m m diam eter extending to the right cerebral hem isphere
. . . m arked loss of tissue with evidences of tra u m a to the other half.
O ne m onth later it was run on FI 15. T he key was greenthe color present d u r
ing F I 15 on the m ultiple schedule. Figure 382 shows the 1st session. T h e stan d
ard FI perform ance is severely disrupted; the over-all rate is low; the grain is irregular;
and m any intervals show negative curvature. Figure 383 shows a session beginning
20 hours later. Segments are arran g ed in order, beginning a t the top. A high rate
of responding occurs for about 1 hour in the m iddle of the session, but the over-all
rate is otherwise low, w ith rough grain an d the occasional brief em ergence of the ter
m inal rate at any time during the interval (as at a and b). T he next sessions showed
only sporadic responding, w ith m any intervals containing only reinforced responses.
D uring the next 4 sessions, the bird was tested on m ult FI 15 FI 60; b ut the over-all
level of responding was so near zero th a t the m ultiple schedule was discontinued by a
change to FI 1 at the end of the 4th session. Figure 384 shows the result. T he rec
ord begins at the low rate from the previous m ult F I 15 FI 60 schedule, and the 1st re
sponse on the new schedule is reinforced at a. After 7 reinforcem ents on FI 1, a sub
stantial rate develops (at b). Some oscillation occurs in the over-all rate (as at c), and
a long segm ent (at d) shows an in te rm e d ia te rate. Before the end of the session,
pauses and lower rates follow reinforcement, and most intervals show some positive ac
celeration.
In the 3rd an d 6th sessions on FI 1, Fig. 385A a n d B, respectively, each reinforce
m ent is followed by a substantial pause and slow acceleration. T he over-all rate is
considerably lower th a n norm al. In spite of this very low over-all rate, reinforce
m ents are not postponed; and a substantial term inal rate usually gives the effect of
an extended scallop.
In the following session (Fig. 385C) the interval of reinforcem ent was increased to
5 m inutes. T h e beginning of each interval resem bles the perform ance in Records A
324
Fig. 3 8 3 .
P erform ance a fte r 2 0 hr o f FI 1 5

a fte r brain lesion
a n d B; an d the low term inal rate, once assum ed, is m ain tain ed for most of the in
terval. Negative curvature sets in at a and b, however, a n d the over-all curve shows a
decline in rate which nearly reaches zero before the end of the session (not shown in
the figure). T he bird could sustain a perform ance on FI 1, b u t it was unable to sus
tain a perform ance on FI 5. This performance is similar to that when smaller am ounts
of reinforcements are given, and is probably related to the lack of m otivation th at
occurs as a result of brain dam age by tum ors. A second bird in the experim ent con
firmed this picture very closely.
Fig. 3 8 4 .
Fig. 38 5 .
Increase in the over-all rate on FI by reducing the interval to 1 min
Later perform ance on FI 1 and transition to FI 5 a fte r brain lesion
Chapter Six
VARIABLE INTERVAL
INTRODUCTION
A V a r i a b l e -I n t e r v a l (VI) schedule is one in which the intervals betw een reinforce

ments vary in a random or nearly random order. T he V I schedule is designed to p ro
duce a constant rate by not perm itting any feature of the b ird s behavior to acquire
discrim inative properties. In contrast, in fixed-interval and fixed-ratio schedules the
fixed p attern establishes a correlation betw een behavior and reinforcement.
In some V I schedules, however, the intervals in the series are not a rran g ed to pro
duce a constant rate. As a lim iting case, only 2 intervals of reinforcem ent occur in
ra n d o m altern atio n . (This schedule produces th e effects discussed m ore generally
u n d e r mixed F IF I.)
Since the V I reinforcem ents are determ ined by the passage of tim e, such a schedule
involves the same differential reinforcem ent of low rates as FI. (See Introduction to
C hapter Five.) Once a V I has generated a constant rate, the fact th at reinforce
m ents occur at one rate could establish this rate as a discrim inative stim ulus. T here
is no explicit differential reinforcem ent in respect to this rate, however.
Specification o f the variable-interval schedule o f reinforcem ent
T he most im portant specification is the m ean interval of reinforcement. Different

distribution of intervals m ay produce the same m ean interval, however, b u t still have
diverse effects. T he most frequently used distributions are arithm etic series, adjusted
to give a slightly higher frequency of reinforcem ent at shorter intervals.
An arithm etic schedule is not specified com pletely until we state the largest and
smallest intervals. M ost V I schedules will have zero or near zero as the shortest in ter
val of reinforcem ent, in order to prevent the developm ent of a pause after reinforce
m ent. Once the m ean interval of reinforcem ent and the largest and smallest intervals
have been specified, the size of the steps still has latitude. For exam ple, one a rith
m etic V I schedule begins 0, 20, 40, 60, 80. . . .
A schedule with similar m ean and
extrem es could begin 0, 10, 20, 30. . . . T h e n u m b er of steps in a series and the
num ber of series scram bled in a program are usually practical m atters, lim ited by the
326
V A R IA BLE IN T E R V A L
327
V I program m er an d the length of the daily experim ental session. In a geom etric
variable-interval schedule the schedule is specified by the m ean interval of reinforce
m ent, the shortest interval, and the progression for generating the intervals in the series.
T h e final step in constructing an actual variable-interval schedule is the random i
zation of several sets of the intervals. Strict random ization is not attem pted because it
always entails the probability of m any short or long intervals in a row. O nly w ith
infinite experim ental tim e could true random ization be used.
Development o f a stable perform ance under

variable-interval reinforcement
VI 1
In one experim ent the schedule of reinforcem ent consisted of an arithm etic series of
intervals ranging from 0 to 120 seconds in steps of 10 seconds. These were rearranged
in ran d o m order. T he effect of this schedule after crf is shown in Fig. 386, which
contains the first 1000 responses of the 1st, 2nd, 4th, 6th, 9th, 12th, 14th, a n d 16th ses
sions after crf. Record A (the entire session following crf) shows slight negative cu r
vature and an over-all rate of responding of approxim ately 0.4 response per second.
T he over-all rate falls to 0.3 response per second a t the start of the 2nd session in R ec
ord B. T hrough Records C , D, E, and F, the over-all rate increases to 1.5 responses
per second, which is roughly m aintained during the rem ainder of the sessions shown in
the figure. Except for slight oscillations, the final perform ance after 16 sessions is an
approxim ately constant rate of responding w ith no rate change correlated w ith any
feature of the schedule.
Figure 387 is a plot of the average rates of responding during the first 10 sessions
on a 1-m inute, variable-interval schedule after continuous reinforcem ent for 2 other
birds on the same schedule as in Fig. 386.
328
_J______ L _
I
__ L ___ I
5
SESSIONS
Fig. 3 87.
M ean rate during first 10 sessions o f VI 1
A bird which had received erf in a separate a p p a ra tu s gave the curve shown in Fig.
388 when subjected for the first tim e to an arithm etic V I 1. T he transition from erf
to V I is disturbed by novel stimuli.
No responding occurs for the first 9 m inutes. T h e first response, at a, is reinforced.
Successive reinforcem ents occur at b, c, a n d elsewhere as m arked in the usual way.
A continuous acceleration in rate leads to an over-all rate of approxim ately 0.8 re
sponse per second, at d, which is m aintained for the rem ainder of the session. M ost
of the positive acceleration represents a reduction in the disturbance caused by novel
stimuli in the new apparatus.
329
300
RESPONSES
VARIABLE INTERVAL
Fig. 3 8 9 .
Transition from erf to VI 2
VI 2
Figure 389 shows a transition from erf to V I 2. T he schedule of reinforcem ent was
composed of an arbitrary selection of intervals in the following order: 0, 30, 120, 60,
240, 230, 120, 10, 240, 180, 0, an d 210 seconds. T he over-all curve for the 1st ses
sion consists of 4 negatively accelerated portions ending at a, b, c, and d. T he grain of
the record is rough. M ost of the responding occurs in bursts at rates m uch higher
than the prevailing rate. T he highest local rates of responding tend to occur after
reinforcem ent. T he over-all curve is also negatively accelerated, reaching a very low
over-all rate by the end of the session. By this tim e, there are no consistent rate
changes correlated with reinforcement.
Figure 390 shows the complete 2nd and 3rd sessions for this bird. T he 2nd session
(R ecord A) begins with a negatively accelerated portion (ending at a) sim ilar to those
in the 1st session. T he rate then increases, and a 2nd negatively accelerated portion
appears between a and b. T he over-all rate of responding rem ains approxim ately 0.45
330
response per second for the rem ainder of the session, with rough grain, m arked oscil
lation in local rates, and no consistent rate change correlated with reinforcement. T he
3rd session (Record B) begins at approxim ately 0.6 response per second and declines
gradually to slightly m ore th an 0.4 response per second tow ard the end of the session.
Beginning at e the over-all rate is fairly constant. A pause begins to appear after some
reinforcements, as at c, d, f , g, and h.
T he entire 4th session, shown in Fig. 391, begins w ith a very low rate of responding
for the first 6 m inutes which we c a n n o t n o w account for. Following the 2nd rein
forcement, however, a norm al performance similar to Record B of Fig. 390 is reinstated.
T he bird responds at a roughly stable over-all rate thoughout the session, beginning
at approxim ately 0.6 response per second a n d falling to approxim ately 0.4 by the end
of the session. T he tendency to respond slowly just after reinforcem ent becomes m ore
m arked, and beginning at a, every reinforcem ent is followed by a pause of the order
of 10 seconds. This perform ance is alm ost certainly due to the abbreviated form of
the variable-interval series, which does not provide for enough reinforcem ents after
short intervals.
T he developm ent of a second V I 2 perform ance w ith the sam e series is shown in
Fig. 392, where segments from various stages of the first 3 sessions have been chosen
to show the character of the local rate changes. R ecord A, which shows the perform
ance im m ediately following continuous reinforcem ent, should be com pared w ith Fig.
389. T he over-all curve is negatively accelerated, and reinforcem ents tend to be fol
lowed by higher rates of responding which fall off sharply to lower values. Reinforced
VARIABLE INTERVAL
331
responses tend to occur after pauses. R ecord B shows a segment from the 2nd hour of
the 1st session following erf. T he over-all rate has increased, the grain of the record
is rough, an d the rate has a slight tendency to be higher im m ediately after the rein
forcem ent. R ecord C contains the end of the 1st session, w here the over-all rate has
increased and the local rate is fairly constant except for the fine grain of the record.
R ecord D is the start of the 2nd session after erf. T he earlier negative acceleration
after each reinforcem ent returns; b ut in the 2nd hour of the session (R ecord E) the
over-all and local rates are considerably smoother. Bursts of a few responses at a high
rate occasionally occur, as at a. R ecord F, a segm ent from the start of the 3rd ses
sion, shows only a very slight return to negative cu rv a tu re , and there is no consistent
rate change correlated w ith reinforcem ent. By the end of the 3rd session, in R ec
ord G, after a total of 8 hours of exposure to V I 2, the rate is lower a n d shows a slight
tendency to fall after reinforcem ent. Figure 393 shows the entire 4th session on V I 2.
T he over-all rate is roughly constant. (Both birds show a slow oscillation in rate.)
Short bursts of responding at a high rate occur at a and b. Slow responding follows
m any reinforcements tow ard the end of the session, as a t c, d, e, and f .
An increase in the frequency of shorter intervals elim inates the pause after rein
forcement developed by those 2 birds. Figure 394 represents the session following Fig.
332
Fig. 3 9 4 .
Disappearance o f the pause a fte r reinforcem ent with higher frequency o f short
intervals
391. At a, b, c, d, and e an ap p aratu s failure produced successive reinforcem ents of

from 3 to 10 responses. These are all followed by brief rapid responding. T he pause
after reinforcem ent no longer occurs.
VI 3
T h e developm ent of a perform ance on V I 3 during the first 17 sessions after crf is
show n in Fig. 395, w here the final segm ent from each session is reproduced. T he
schedule was composed of intervals as follows: 300, 30, 280,120, 360, 300, 0, 240, 220,
180, 10, 280, 100, and 60 seconds. By the end of the 1st session in Record A a constant
rate of approxim ately 0.35 response per second h ad developed. T he over-all rate in
creased progressively during the next 5 sessions (Records B through F), reaching an over
all rate of approxim ately 1 response per second by R ecord F. Reinforcem ents are not
VARIABLE INTERVAL
333
m arked in Records B, C, and D. At this stage approxim ately 20 responses are em itted
rapidly ju st after reinforcement; they are followed by an ab ru p t shift to a lower rate b e
fore the prevailing variable-interval rate is reached. This is m ore pronounced by the
end of the next session (R ecord G). T h e rate ju st after reinforcem ent is now about
3.8 responses per second, com pared w ith a prevailing rate of approxim ately 1.4 re
sponses per second. For the rem aining 10 sessions (Records H through P) the over-all
rate rem ains at approxim ately 1.25 responses per second. But the high rate im m e
diately after the reinforcem ent a n d the a b ru p t shift to a lower rate becom e m ore p ro
nounced.
Figure 396 shows the perform ance for a second bird after 45 hours on the same
schedule of reinforcement. T he bird m aintains a constant over-all rate throughout the
session, with m arked local changes, and shows the same tendency to respond at a higher
rate im m ediately after reinforcement.
A nother bird on a variable-interval schedule sim ilar to the V I 3 just described
showed the perform ance recorded in Fig. 397. This figure contains segments from
altern ate sessions during the first 26 sessions on the schedule. T he actual series of in
tervals was: 5, 50, 190, 190, 170, 10, 300, 80, 50, 280, 20, 290, 120, 355, and 310 sec
onds. T h e average interval is 164 seconds, although for brevity the schedule is here
called V I 3. T he first segment of the figure occurs after 12 hours on V I 3 after erf,
and already shows a m arked tendency for the rate of responding to be highest im m edi
ately following the reinforcement. O nly the reinforcement at a is not followed by an
increase in the rate. The over-all rate is approxim ately 0.75 response per second, very
nearly as high as the final rate generated under this schedule in the last segment of the
figure. At b the over-all rate has reached a value com parable w ith the high rate typi
cally following reinforcement. This continues after reinforcem ent and shows no break
to a lower rate. Elsewhere, however, very square changes in rate occur after re
inforcem ent. T ow ard the end of the session, instances of a b ru p t shifts to a higher rate
occur at other points in the schedule. Figure 398 shows a larger sample of the p e r
form ance on this variable-interval schedule from a session following the last in Fig. 397.
334
T he bird has h ad 72 hours of exposure to V I 3. A lthough the local rate frequently

changes considerably, particularly after reinforcem ent, the over-all perform ance is sta
ble, an d the effects are uniform from segment to segment. A high rate im m ediately
after reinforcem ent m ay continue for from 20 to 40 responses a t a rate of about 2.7 re
sponses per second. This rate is followed either by an a b ru p t shift to a rate of 1.1
responses per second, which prevails throughout most of the b ird s perform ance, or to a
pause of a few seconds followed again by the higher rate. W here a second reinforce
m ent occurs soon after the first, while the rate of responding is still high, a pause m ay
VARIABLE INTERVAL
335
follow the second reinforcem ent instead of the usual high rate (for exam ple, a t a, b,
and c). W hen the bird begins responding at the end of the pause, however, a brief
period of high rate com parable w ith th at after reinforcem ent follows.
Figure 399 shows the perform ance of a second bird after 80 hours on the same sched
ule. T he bird shows several kinds of rate changes after reinforcement, but the small
square curve is readily observed.
After the perform ance on V I 1 shown in Fig. 386, the bird was reinforced for 29
hours on V I 2 and V I 3 schedules sim ilar to those already described. T he variable
interval series was then changed to one of the form: 0, 1, 1, 2, 3, 5, 8 . . . , in which
successive terms are determ ined by adding the preceding two (the so-called Fibonnacci
num bers). T he largest interval was 987 seconds, and the m ean, 152 seconds. Fig
ure 400 shows the performance after 16 hours on this schedule. T he sustained over-all
Fig. 4 0 0 .
VI 152 sec w ith Fibonnacci series
337
rate of responding is m ore th an 2.5 responses per second. Each reinforcement is fol
lowed by an unusually high rate; but instead of the a b ru p t decline to the lower rate, the
rate falls slowly to an only somewhat lower value.
T h e lack of local effects ju st after reinforcem ent is presum ably due to the preponder
ance of short intervals and the gradualness w ith which longer intervals appear in the
series.
In a study of the effects of deprivation level on perform ance when a V I 3 schedule
rem ained unchanged over a long period, the birds showed several types or stages of p er
form ance. Figure 401A reproduces a segm ent for each of the 3 birds, showing an a l
most linear perform ance. A lthough slight local changes in rate occur, they are not
associated w ith the reinforcem ent or any other feature of the schedule. These curves
represent tem porary states interspersed between the other performances shown in the
figure. R ecord B contains one segment for each bird showing a comm on deviation in
Fig. 4 0 1 .
Varieties of local rate changes occurring during long exposure to VI 3
w hich the rate increases briefly after reinforcem ent a n d then declines rapidly to zero
with a later retu rn to an interm ediate value. Occasionally, the higher rate after rein
forcem ent is not followed by a pause, so th a t the whole curve is displaced upw ard.
Thus, in the segments for the 3 birds in R ecord C the high rate im m ediately after rein
forcement is followed by an a b ru p t shift to an interm ediate rate, which is m aintained
until reinforcem ent. In R ecord D a segm ent for each bird illustrates sim ilar changes
executed m ore slowly and w ith fairly smooth curvature. In R ecord E segments for
2 of these birds show an unusual decrease in rate after reinforcem ent. O th e r types of
change follow adjacent reinforcements.
In spite of the local changes in rate produced by a p a rticu la r set of intervals, V I
schedules almost invariably produce sustained and relatively constant over-all rates.
Figure 402 shows a very rare exception which we cannot explain. T he bird was show
ing a fairly linear perform ance on V I 3, as the 1st segment shows (followed imm edi-
338
Fig. 4 0 2 .
Unusual deviation
ately by the second). At a, however, the rate falls over a period of a m inute or two to
a very low value, w hich is m ain tain ed in spite of reinforcem ents at b, c, an d d. A
sm ooth acceleration then leads to a rate which is slightly higher th an norm al at e, but
drops to a norm al value thereafter.
V ariable-interval reinforcem ent sustains a fairly uniform rate for long periods.
Figure 403 shows a com plete 14-hour session on V I. T he over-all rate varies slightly,
an d in three instances a low rate is in force for from 5 to 10 m inutes. M ost of the
tim e, however, the bird is fully under control of the schedule. It emits approxim ately
87,000 responses d u ring the session. A perform ance of this m agnitude m ay be re
peated on a daily schedule. A nother bird in a 9-hour session on the same schedule
gave a perform ance represented by Fig. 404.
Fig. 4 0 4 .
339
Sustained performance on VI during a 9-hr session
The behavior o f the rat on VI 3.5
An arithm etic V I 3.5 schedule of reinforcem ent was used in a study on pre-aversive
stimuli in the rat. Figure 405 shows 2 complete daily sessions. T he deviations from
a roughly constant rate show a sort of long-term oscillation, b u t show no relation to the
schedule.
Geometric VI 7
W e m odified the series of intervals in the experim ent shown in Fig. 400 by om itting
the zero interval a n d one 1-second in te rv a l, an d by extending the series u n til the
Fig. 4 0 5 .
The behavior of a rat on VI
341
longest interval was approxim ately 45 m inutes. T h e m ean interval was 7 m inutes.
Figure 406 presents the tran sitio n from the series in Fig. 400. T h e rate is at first
app ro p riate to the previous V I 3; b u t the 1st interval in the schedule is unusually long
an d the rate breaks before reinforcem ent a t a. T h e next few reinforcem ents occur
after intervals of the same order as u n d er the preceding schedule. T he longest in te r
val in the series follows the reinforcem ent a t b. A high over-all rate is sustained for
approxim ately 10 m inutes, breaking sharply at c and showing a long pause before rein
forcem ent 45 m inutes later at d. This reinforcem ent reinstates the prevailing varia
b le-interval rate; a n d for the rem a in d e r of the session the rate begins high following
each reinforcem ent and declines smoothly until the next reinforcement. D uring the
longer intervals the reinforcement occurs after pauses, as at e and f . D uring the shorter
intervals, the reinforcem ent occurs at a high rate, as at g.
Figure 407 shows a later perform ance on this V I 7 after 162 hours of exposure to the
schedule. T he figure contains the entire session except for the first 2 \ hours. Each
reinforcem ent is followed by a m axim al rate, which falls off smoothly to a lower value.
Occasionally, the higher rate appears spontaneously d u ring a longer interval, as a t

T he interm ediate rate, showing some negative acceleration, is usually
sustained throughout the interval. At d, however, a pause of m ore than 5 m inutes oc
curs. T h e next reinforcem ent is then followed by over 200 responses at 3 responses
per second, a considerably more sustained run than typically occurs.
Figure 408 presents a later perform ance for a second bird on the same schedule
and after a sim ilar history, after 240 hours of reinforcem ent on V I 7. Record A shows
the start of the session; R ecord B shows the 3rd hour; R ecord C, the 7th hour; and
Record D, the end of the session at 10 hours. T he over-all rate declines from approxi
m ately 1.5 responses per second in R ecord A to 0.35 response per second at the end
of R ecord D. T h e decline is executed m ainly as an acceleration in the change to a
lower rate after reinforcement.
A th ird bird w ith a sim ilar history was u n ab le to sustain com parable rates on this
variable-interval schedule. Figure 409 is a sample of its perform ance after 83 hours
of V I 7. T h e session starts (R ecord A) w ith a roughly linear perform ance; a n d the
a, b, c, an d e.
342
over-all rate of responding declines slowly, reaching 0.6 by the 3rd segm ent, a d e
cline of approxim ately 30%. By the 5th an d 6th hours of the session (R ecord B), the
over-ll rate has fallen to 0.3 w ith the highest rates im m ediately following the rein
forcement. T h e 7th and 8th hours of the session (R ecord C) show a further decline in
the over-all rate to 0.2 response per second, with pauses occurring during the longer
intervals, 19 m inutes at a a n d 21 m inutes a t b. In spite of the very low over-all rate
a n d the long pauses, each reinforcem ent is followed by rates of the sam e order as
those in R ecord A. R ecord D shows the 14th and last hour of the session. T he over
all rate has increased b u t has not reached the value a t R ecord A. R esponding is
sustained throughout the longer intervals, although at very low rates.
After 138 hours of reinforcem ent on this schedule, an alm ost linear perform ance
developed; the over-all rate was 0.6 response per second, w ith only a slight drop during
the session. Figure 410 shows an entire session. W hen the over-all rate is suffi
ciently high, as at a and b, the reinforcem ent is not followed by an increase in rate. At
lower prevailing rates most reinforcem ents are followed by a slight increase (as at c).
U nlike the earlier perform ance for this bird, the interm ediate rate of responding is
m aintained, w ith only a slight a n d orderly decline through the longest interval in the
series, ending a t c.
After 252 hours of reinforcem ent on this V I 7 schedule the over-all rate shows a fu r
ther increase of the order of m agnitude shown in Fig. 411, which contains the first 7
hours of a daily session. T he over-all rate declines only slightly from 1.5 responses per
VA RIA BLE IN T E R V A L
343
second at the start of the session to 1.25 responses per second at the end. M arked
rate changes after reinforcem ent are still prom inent, however, a n d the prevailing p e r
form ance is a more or less rapid decline to an interm ediate rate which is m aintained
through the interval. Occasionally, the highest rate of responding reappears after the
rate has fallen to an interm ediate value.
T h e experim ents ju st described were followed by others on the effect of a T O on V I
to be described later. In the experim ents on T O a new V I 7 schedule was in force,
in w hich, as in the preceding schedules, the intervals com prised a roughly geom etric
series. T he longest interval was 1200 seconds, and the m ean, 7 m inutes. (The ac
tu al intervals a n d the order were as follows: 360, 960,10, 180, 20, 720, 1, 40, 960, 720,
240, 360, 360, 20, 480, 240, 960, 720, 40, 1200, 1, 80, 120,480, 40, 360, 1200, 5,
480, 40, 120, 10, 1200, 200, 80, 1, 120, 5, 80, 240, 5, 80, 10, 20, 240, 960, 120,
20, a n d 720 seconds.) This schedule produced a m uch m ore linear perform ance.
Figure 412 shows the first perform ance after the rem oval of the T O . Except for very
brief local rate changes, the over-all rate is quite constant through the entire 8-hour
session. Figure 413 shows the perform ance of a second bird in the first session w ithout
T O after a sim ilar history. T he longest pauses occurring during the 40,000 responses
in the figure are of the order of 30 seconds. T h e over-all rate shows some decline
d u rin g the session, a n d this bird continues to show a slightly higher rate im m ediately
after the reinforcement.
344
Tw o m ore pigeons developed stable perform ances on the same roughly geom etric
V I schedule. Figure 414 shows an interm ediate stage of developm ent after 105 hours
of exposure to the schedule. T he figure contains the 5th and 6th hours of the ses
sion. T he bird responds at either one of two rates. Im m ediately after reinforcem ent,
a high rate is m aintained for 60 to 100 responses, after which there is a rap id shift
to an interm ediate rate of approxim ately 0.3 response per second m aintained until re
inforcem ent. Occasionally, the reinforcem ent is not followed by an increase in rate
of responding, as at a; a high rate sometimes appears elsewhere, as at b.
V A RIA BLE IN T E R V A L
Fig. 4 1 4 .
345
Intermediate development of geometric VI 7
T he second bird showed a m uch higher over-all rate of responding after a com par
able exposure to V I 7. Figure 415 shows a com plete session after 132 hours on th e
schedule. R ate changes are lacking in the vicinity of reinforcem ent so long as the
over-all rate is high. A m arked decline tow ard the end of the session results from the
appearance of a second and lower rate, an d reinforcem ents are now followed by in
creases.
346
In another experim ent 4 birds passed directly from crf to a roughly geom etric V I
schedule. Two birds placed on the geometric schedule discussed above gave perform
ances sim ilar to th at in Fig. 416 for a session beginning 215 hours after crf. Two or
m ore stable rates shift fairly abruptly from one to the other. T he shifts are only occa
sionally correlated w ith reinforcem ent. T he perform ance suggests the special case
of V I called a tw o-valued interval discussed in the sections on m ixed schedules. Tw o
of these birds were on another schedule, composed of intervals in the following order :
1343, 4, 10, 69, 10, 712, 38, 32, 224, 47, 56, 136, 47, 1800, 0, 537, 69, 104, 224, 1800,
20, 978, 172, 4, 392, 20, 32, 978, 82, 136, 0, 712, 394, 1, 82, 2700, 15, 15, 104, 56, 38,
72, 25, 2700, 25, 537, and 299 seconds. Figure 417 is a sam ple of the perform ance
after 105 hours. Except for a higher over-all rate, it resembles Fig. 414.
EXTINCTION AFTER VARIABLE-INTERVAL REINFORCEMENT
T h e extinction curves to be reported were m ainly the results of ap paratus failures

a n d do not represent any systematic study of the extinction curve after variable-interval
reinforcem ent. T he general type of curve, however, can be com pared w ith those
u n d e r other schedules. Figure 418 shows a com plete extinction curve after 30 hours
of reinforcem ent on the Fibonnacci schedule for the bird whose later performance on
this schedule was shown in Fig. 408. T he extinction curve contains slightly more th an
1000 responses; an d except for the a b ru p t shift to the long pause at a an d a com pen
sating rate at b, the curve shows a rough negative acceleration. Figure 419 shows an
extinction curve for the same bird 50 hours later on the same schedule. A portion
of the perform ance on the V I 7 schedule is shown before the arrow . T h e prevailing
ra te is now higher th a n th a t before the extinction curve in Fig. 418. T h e curve in
347
Fig. 419 contains approxim ately 4000 responses distrib u ted th rough 2 \ hours, w hen
the rate of responding fell continuously and smoothly to a very low value.
Figure 420 contains the extinction curve of a second bird after 225 hours on Fibonnacci V I 7. Figure 411 showed the irregular perform ance for this bird at this stage
of the schedule. A high rate follows reinforcem ent, an d the rate oscillates consider
ably d u rin g the longer intervals. Figure 420 follows im m ediately upon Fig. 411,
both figures com prising one session. D uring the first p a rt of extinction the rate oscil
lates in a m an n er sim ilar to th at under the previous variable-interval reinforcement.
348
As the ra te begins to fall d u rin g extinction, how ever, the decline in the rate becomes
m ore continuous. Pauses become longer and are separated by responding at approxi
m ately 0.8 response per second, though w ith rough grain. Even at the end of the
curve, if the bird responds at all, it is a t approxim ately this rate.
Figure 421 illustrates a later extinction curve for the same bird after approxi
m ately 25 hours on the geometric schedule which produced the perform ance shown
in Fig. 412. This bird had had 325 hours of reinforcem ent on the V I 7 schedules. T he
extinction curve has been broken into two parts. Between R ecord A and R ecord B a
low over-all rate of approxim ately 200 to 250 responses per hour was sustained for 8
hours. T h e extinction curve begins at approxim ately 1.5 responses per second and
shows a decline to approxim ately 1 response per second by the segment of the record at
a. T hereafter, the rate of responding falls sharply, reaching approxim ately 0.2 re
sponse per second at b. T he rate returns briefly to 1 response per second for approx 8 H R . A N D 1700 R' s
OMITTED
im ately 250 responses at c, after w hich it again falls off sharply. A reduced g raph of
this extinction curve, showing the whole curve in a continuous graph, would show an
initial leg consisting of about 16,000 responses, declining from 1.5 responses per second
to 1 response per second, followed by an ab rupt shift to a very low but sustained rate for
over 8 hours. T he return to the initial high rate of responding similar to that in the
start of the session, shown in Record B, would appear as a slight deviation from the long
period of sustained slow responding. T h e initial leg is evidently the effect of the V I
schedule in building up a sustained high rate during a long interval.
Figure 422 shows a later extinction curve after 168 hours of reinforcement on the geo
m etric V I. This bird had been showing a fairly linear perform ance except tow ard
the end of the session, when the over-all rate was reduced by low rates during the longer
intervals. Extinction begins at 1.25 to 1.5 responses per second, which is m aintained
for the first 8000 responses to a. T hereafter, the rate of responding falls fairly con
tinuously, reaching a very low value at the end of the figure. Figure 423 shows the
Fig. 4 2 2 .
Extinction a fte r 168 hr geom etric VI 7
Fig. 4 2 3 .
Continues Fig. 4 2 2
350
rem aining 7 hours of the extinction period. T h e low rate a t the end of Fig. 422 gives
w ay to periods of sustained responding (as a t a, b, c, an d d) separated by periods of
long pauses or very low rates. T he sustained rate, however, does not reach the original
variable-interval value or the earlier ra te in extinction in Fig. 422. As extinction
progresses, the over-all rate again declines, because of the increasing length of the
pauses separating the periods of sustained responding as well as a lower actual rate.
Figures 424 a n d 425 contain extinction curves for 2 birds reinforced on the geo
m etric V I. T he final perform ance of the bird in Fig. 424 has already been shown in
Fig. 417. Both birds were exposed to V I 7 im m ediately after crf. T he extinction
curve in Fig. 424 was taken after 127 hours after crf. A portion of the performance on
V I 7 is shown before the arrow . This was recorded tow ard the end of the session,
w hen the over-all rate was characteristically low for this bird. In extinction the rate
continued to fall fairly continuously; an d rates com parable with those which occurred
d u rin g the first segm ent of the figure do not ap p ear a t any point for the rem ainder
Fig. 4 2 5 .
Extinction a fte r 102 hr geom etric VI 7
of the curve.
351
Thus, changes in extinction m ay be related to changes occurring d u r
ing the variable-interval session.

Figure 425 shows only the first 4000 responses of the second extinction curve. T he
rate falls off sharply after the first excursion, as it did during the longer intervals in
the geom etric series. T hereafter, high rates of the order of those which form erly fol
lowed reinforcem ent alternate with near-zero rates. Extinction was continued for a
total of 14 hours, when the over-all rate of responding fell continuously. T he curve
continued to show periods of sustained responding a t a high rate, separated by long
periods of very slow responding.
Return to variable-interval reinforcement after extinction
Figure 426 shows the return to variable-interval reinforcement immediately after the
extinction curves described in Fig. 422 and 419. T he figure begins with the final por
tions of the extinction curves. T he m agazine was re-connected, so th at reinforcem ent
occurred at the arrows. A single reinforcem ent reinstates the highest rate th at was
norm ally observed under this schedule, and in both records the ensuing performance is
not distinguishable from the previous variable-interval performance.
THE EFFECT OF TIME OUT O N A VARIABLE-INTERVAL PERFORMANCE
If a variable-interval schedule is successful in generating a nearly constant rate of

responding, a tim e out after reinforcem ent should have no effect on the perform ance.
The time out makes a difference only to the extent th at the bird s own behavior controls
local rates because of differential reinforcem ent. T his possibility perm its us to ex
plore the failure to generate a linear perform ance in every case. In one experim ent to
352
be described in this section, we studied the effect of T O on V I in 2 birds by a tta c h

ing a T O to each reinforcem ent. In another experim ent on 3 birds, T O occurred in
dependently of the reinforcement.
Time out a fte r reinforcement
A 7-m inute T O was introduced after every reinforcem ent in a perform ance under a
geometric schedule similar to those already described in Fig. 415 and 416. T he first
p art of Fig. 427 shows a perform ance after 220 hours of exposure to a geometric V I 7 T O
(following a previous history of 227 hours of a geom etric V I w ithout T O ). T h e rate
oscillates radically betw een a high value im m ediately following T O to interm ediate
rates, w ith rough grain sustained during most of the longer intervals. Frequently, re
inforcem ent occurs w hen the bird is responding a t a high rate, as at a a n d b, even
though these reinforcem ents occur after relatively long intervals. W hen the T O s no
longer follow reinforcement (beginning at the arrow), the rate remains relatively con
stant, at approxim ately 1 response per second, or slightly below the m axim al rate under
V IT O . A single break-through at a high rate occurs at c, where approxim ately 150
responses are em itted at 1.7 responses per second. T he effect of the removal of the T O
in producing a more orderly and linear perform ance is tem porary, however; and in
subsequent sessions the perform ance becam e as irreg u lar as th at in the first p a rt of
Fig. 427.
T he 2nd bird in the experim ent showed a m ore linear performance under V I 7 T O 7,
as Fig. 428 shows, after 200 hours of reinforcem ent on V IT O . T h e T O after rein
forcement was discontinued at the start of the following session, and the performance
is shown in R ecord B. T he over-all rate an d the c h aracter of the changes in rate re
m ain about the same, b u t the records give the impression of a narrow er range of rates
and a generally more orderly performance.
A lthough adding or rem oving a T O after every reinforcem ent usually has an im m e
diate effect, the perform ance generally settles down to a common pattern. T he sched-
353
ule producing the wide rate changes of Fig. 407 and 408 continued to give the same
patterns when a 2-m inute T O was added to every reinforcement. Figure 429B should
be com pared w ith Fig. 408, and Fig. 429C w ith Fig. 407. R ecord A is for a 3rd bird
with the same history.
Time out not correlated with reinforcement
T he effect of a T O not correlated w ith reinforcem ent was tested on the 3 birds in
Fig. 429. Instead of the Fibonnacci series in the earlier studies, the schedule was now
a geometric series with a m ean of nearly 10 m inutes and a largest interval of 45 m in-
354
utes. T he schedule generated a m uch m ore linear perform ance. Figure 430 shows
th e first effect of the 2-m inute T O occurring independently of reinforcem ent. T h e
birds had been exposed to the new geometric variable-interval series for approxim ately
100 hours. T he T O s are indicated by the dashes above the curves. T he recorder
and program m ing equipm ent did not, of course, operate during the T O . Reinforce
m ents are recorded in the usual m anner. T he T O s have little effect. Records B and
C m ay be slightly m ore irregular th a n the prevailing perform ance w ithout T O , a l
though the effect, if any, is slight.
T he T O procedure was continued for a total of 44 hours, during which no effect
ap p eared except possibly w hen the over-all rate h ad fallen tow ard the end of the ses
sion, so th a t each reinforcement was followed by a negatively accelerated curve. A t
this phase the occurrence of a T O where the rate is changing from one value to another
provides a check on w hether the prim ary variable responsible for the change is tim e
or some other condition of the experim ent. Figure 431 represents a continuous record
w hich has been broken up into segments beginning w ith reinforcem ent. T h e T O s
occurring every 7 m inutes are indicated by the dashes above the curves. T he rate
changes are not sufficiently predictable to enable us to ju d g e w hether any p a rticu la r
change is due to TO s. However, T O s (at a, b, c, d, a n d j) occur at substantial rates
a n d always show a slight decline in rate. At g, h, an d i, w here the rate of respond
ing is very low, the T O has no observable effect. T he most extrem e breaks in the
curve occur at c and e, where T O is followed by a zero rate for m any minutes. T he
T O s during these pauses do not reinstate responding. At f there is little, if any, effect.
Fig. 4 3 0 .
TO not correlated with reinforcem ent
355
VARIABLE-INTERVAL REINFORCEMENT WITH LIMITED HOLD
In the interval schedules already discussed, reinforcem ent is determ ined by a p ro

gram m ing clock which closes the circuit from the key to the m agazine w henever a re
inforcem ent is to be m ade available. T he circuit rem ains closed until a response
occurs a n d is reinforced. T h e reinforcem ent need not rem ain available indefinitely,
however. W hen the program m ing clock sets u p a reinforcement, the circuit from
th e key to the m agazine m ay close only for a designated interval of tim e. If the bird
responds w ithin this interval, it is reinforced; if it does not, th at reinforcem ent is lost
and responses will continue to go unreinforced until the program m ing clock once again
sets up a reinforcement. Such a schedule m ay be qualified as lim ited hold.
In the following experim ents on V I lim ited hold the intervals during which rein
forcem ent is available are of the order of fractions of a second. L ater experim ents by
W. H. Morse an d R. J . H errnstein have shown th a t lim ited holds of the order of 10
seconds produce m ajor changes in fixed-interval performances.
Development o f VI 1 limited hold
T he transition from V I to V I lim ited hold was m ade from an early perform ance on
an arithm etic V I 2. A session under this schedule has already been described (Fig.
391). T he over-all rate is roughly linear betw een 0.4 a n d 0.5 response per second,
w ith a slight pause following reinforcem ent. In the next session the schedule was
changed to V I lim ited hold; each reinforcem ent was set up for only 0.75 second. A
decline in th e over-all frequency of reinforcem ent was an tic ip a ted because the bird
m ight not be responding during the hold. T h e m ean interval of reinforcem ent was
therefore reduced to 1 m inute. W e determ ined this figure by estim ating the num ber
of reinforcements th at would be received on lim ited hold if the bird continued respond
ing as in Fig. 391. W e estim ated th at the bird would receive about half the reinforce-
356
m ents designated by the schedule; this proved to be the case. D uring the first session
under V I limited hold, 34 out of the 64 possible reinforcements were received.
A n early perform ance is shown in Fig. 432, w here representative segments from the
first 5 sessions with lim ited hold are given in Records A through E. D uring the 1st
session (R ecord A) the over-all rate is approxim ately the same as th a t under variable
interval reinforcem ent. T he pause following reinforcem ent, however, has now given
way to a brief burst of responding at a higher rate. Local changes in rate are con
siderably sm oother th an those in the previous session, and the long-term oscillations in
th e over-all rate are missing. By the 2nd session (R ecord B), instances ap p ear where
th e local rate increases very m arkedly. For exam ple, the rate at b reaches 1.5 re
sponses per second. At the same tim e, however, long pauses and periods of slower
responding also appear, as at a, as well as interm ediate rates, as a t c. T he bird con
tinues to respond faster im m ediately after the reinforcement, and bursts are in tu rn fol
lowed by a pause an d scallop or an a b ru p t shift to the variable-interval rate. T he
3rd and 4th sessions (Records C and D) show similar performances. T he 5th session
(Record E) shows a severe decline in the over-all rate with a long pause at e, after an
a b ru p t shift from the term inal variable-interval rate at d.
W hile the term inal variable-interval rates of responding are of the same order seen
after com parable exposure to a variable-interval schedule alone, it is clear th at the
lim ited-hold contingency has an effect beyond th a t of a change in the num ber of rein
forcements. Pauses as long as those occurring at a and e would not ordinarily appear.
T h e following session (Fig. 433) begins w ith a pause of ab o u t 15 m inutes after
w hich the rate accelerates briefly to approxim ately 1.25, ab o u t the sam e value as the
highest rates in the previous session. Responding continues at about this level until
V A R IA B LE IN T E R V A L
357
ju st after the reinforcem ent at a; here, following the short period of higher rate after
the reinforcem ent and a short pause, the bird begins to respond at a rate of ap p ro x i
m ately 4 per second. T he highest local rate was previously about 3 per second. T he
exceptionally high rate continues for alm ost 1000 responses, until, at c, it a b ru p tly
shifts to a lower value; for the rem ainder of the session the rates oscillate between 4 per
second an d from 1.25 to 1.5 per second. T he shift from one rate to another some
tim es occurs at reinforcement.
T h e arrow s in the figure show the 2 shortest intervals in the series. R einforce
m ents were received here because the rate of responding im m ediately after one re in
forcem ent is high enough to guarantee a response w ithin the 0.75-second hold. T he
high frequency of these successive reinforcements relative to the other intervals of rein
forcem ent, where the lim ited hold is m ore costly, presum ably explains the continua
tion of a high rate im m ediately after reinforcement.
Figure 434 gives the 8th session on V I lim ited hold, showing an interm ediate d e
velopm ent of the performance. T he m ean rate of the first 3 \ segments is 1.52 responses
per second. A t a the rate shifts a b ru p tly to 3 responses per second. F or the re
m ainder of the session the rate is high after the reinforcem ent, but drops fairly rapidly
though smoothly to a lower value before the next reinforcem ent. Following the re
inforcem ent at b, the next reinforcem ent is not received for alm ost 20 m inutes. T he
lower rate of responding is sustained for a full excursion of the pen. T he higher rate
then reappears.
Figure 435 shows the perform ance after 34 sessions (102 hours) on V I 1 w ith lim ited
hold. R esponding is m aintained a t a fairly constant rate of from 2 to 2.5 responses
per second. T he session begins w ith a slight w arm -up at a, b u t after a brief accelera
tion a m axim um rate is m ain ta in e d except ju st after reinforcem ent. T h e arrow s
indicate the 0-second intervals of reinforcem ent, where 2 successive responses are rein-
150 R's
\
Eighth session VI 1 limited hold
150
R's
Fig. 4 3 4 .
Fig. 4 3 5 .
VI 1 limited hold after 34 sessions
359
forced. T he double reinforcem ent now controls a lower rate of responding, since no
instances exist w here 3 successive responses are reinforced. By this tim e the value of
the lim ited hold h ad been reduced to 0.24 second. W e were unable to detect any
effect of the change from 0.75 second. A lthough the schedule set up 60 reinforce
m ents, only about 15 were received, because of the lim ited hold. T he initial w arm
u p ap parent in Figs. 433, 434, an d 435 is characteristic of the perform ance under V I
Fig. 4 3 6 .
Development of warm-up on VI 1
limited hold
lim ited hold. O n V I w ithout lim ited hold the rate is usually highest at the start of
the session. Figure 436 shows the developm ent of the w arm -up. T he first 1000 re
sponses of each of the first 5 sessions are shown in Records A through E. These are
the sam e sessions from w hich the segments in Fig. 432 were taken. In R ecord C the
rate has some tendency to be lower at the beginning of the session; this lower rate
becomes very pronounced in the 4th a n d 5th sessions, where the prevailing rate of re
360
sponding is not reached until alm ost 15 m inutes after the start of the session. T he
scattered responding early in R ecord E indicates th a t the bird is facing the key a n d
occasionally pecking it; nevertheless the rate rem ains low. T he developm ent of a
slow start coincided w ith the developm ent of pauses and a b ru p t shifts in the local rate
d u rin g the session. D uring the rest of the experim ent a w arm -up has not always
been observed. Figures 434 an d 435 show slight examples. T he lim ited-hold con
tingency would tend to perpetuate any tendency to begin the session with a low rate
of responding, since once a low over-all rate is established, the frequency of reinforce
m ent decreases correspondingly.
W hen the schedule of reinforcem ent was changed to V I 3 w ithout the lim ited-hold
contingency, the effects of the previous lim ited-hold reinforcem ent were found to be
only partially reversible during 28 sessions.
Figure 437 shows the first 920 responses
of each of the last 12 of these sessions on V I 3. A w arm -up at the start of the session
persists, alth o u g h reinforcem ents now occur w ith eq u al likelihood d u rin g the lower
rates of responding.
T he perform ance w hich has ju st been described was confirm ed by a second bird in
the experim ent, except th at the rate increased a little less abruptly when the limitedhold contingency was added. T he adjustm ent was quicker when the lim ited-hold
contingency was removed; also, a decline in rate of the order of 50% resulted, as well
as a loss of w arm -up. Effects of the previous lim ited-hold reinforcem ent continued to
show, however, in the continuing oscillation between high and interm ediate rates.
361
Development o f VI 1.5 limited hold
Figure 438 shows a late perform ance on V I 1.5 lim ited hold for the birds whose
final perform ance on V I 3 was shown in Fig. 398 an d 399, respectively. T he lim ited
hold was 0.5 second. W e carried out the transition to the lim ited hold by doubling
the frequency of reinforcement from V I 3 to V I 1.5 so th at the actual over-all frequency
of reinforcem ent would not change radically under lim ited hold. T he complete daily
sessions shown in Records A an d B of Fig. 438 were recorded after approxim ately
100 hours of reinforcem ent on V I 1.5 lim ited hold. T he over-all rate of responding
has increased from about 1.25 responses per second under the previous V I 3 to about
2.5 responses per second. This order of increase m ight have occurred norm ally as the
result of prolonged exposure to the original V I 3 reinforcem ent. T he effect of the
lim ited-hold contingency is evident, however, in the general lack of interm ediate rates
characteristic of later V I lim ited hold. Pauses generally occur a few responses after
the reinforcem ent, and the rate changes before and after are generally abrupt. If for
Fig. 4 38 .
Late performance on VI 1.5 for two pigeons
any reason the over-all rate of responding drops to a value so low th a t the likelihood
of a response occurring during the lim ited-hold interval is very small, the bird m ay stop
responding entirely.
At one stage during the developm ent o fV I lim ited hold, the birds were extinguished.
(See Fig. 441 and 442 for extinction after V I lim ited hold.) Late in the session the
m agazine was re-connected, but the rate had reached so low a value th a t no responses
coincided with a lim ited-hold interval. In one case the following session began with
a 3-hour period during which no response occurred. T he beginning of Fig. 439 shows
the end of this 3-hour period. Before the arrow a single reinforcem ent h a d been set
up in the norm al fashion so th a t the first response occurring at any tim e thereafter
would be reinforced. The first reinforcem ent (at the arrow) is followed by a high
rate, which falls off sharply to an interm ediate value w ith rough grain. A second re
sponse coincided with a lim ited hold at a. This reinforcem ent reinstated a substantial
rate, under which the frequency of reinforcem ent was sufficient to m aintain the behav
ior. This figure illustrates one of the m ajor differences betw een a lim ited-hold con-
362
tingency and a norm al variable interval. T h e norm al variable-interval schedule is

self-correcting ; th a t is, any tendency to slow dow n causes relatively frequent rein
forcem ent. This condition has the effect of increasing the rate. T h e figure also
shows a n exam ple of a V I lim ited-hold perform ance at an earlier stage th a n th a t in
Fig. 438. Because of a consistent pause after reinforcem ent, the shorter intervals of
reinforcem ent in the variable-interval series are missed. This fact in turn m aintains
th e pause following the reinforcem ent. W h eth er the perform ance shows runs after
reinforcem ent, as in Fig. 433, or pauses, as in Fig. 439, depends upon the early history
w hich brings about one or the other condition. Both are self-perpetuating.
F igure 440 shows th e re tu rn to V I lim ited hold after extinction for the second
Fig. 4 4 0 .
Return to VI lim ited hold a fte r extinction (second bird)
363
bird in the experim ent. D uring the session preceding the figure, extinction h ad pro
duced a low rate at w hich few reinforcem ents w ould occur. O nly 350 responses
were em itted during 5 hours and only 1 reinforcem ent occurred tow ard the end of the
session. T he following session (Fig. 440) begins with a positively accelerated segment
to a m oderately high rate at a. T he schedule specified a reinforcem ent during the 10m inute period represented by this first segment, but the bird was pausing during the
hold. T he rate then falls to a low value followed by a rough acceleration to an in
term ediate rate at b, where a reinforcem ent is received despite the low over-all rate a n d
rough grain. This first reinforcem ent at b leads to the characteristic perform ance
under V I lim ited hold after a short period at an interm ediate rate.
Extinction a fter variable-interval reinforcem ent with limited hold
T he response was extinguished after 25 sessions (60 hours) on V I 1 lim ited hold,
with the hold varying from 0.75 to 0.24 second. Figure 441 shows the resulting curve
for one bird. T he tendency to respond at a single rate is strong. D uring 3 hours
16,000 responses occur; several h u n d red of these, in the range from a to b, occur at
o th er th a n the prevailing rate. T h e ra te changes a t a a n d c are very sharp. (T he
breaks near the bottoms of the records are due to a defective recorder.)
A later extinction curve, after 7 further sessions (21 hours) on V I 2 and V I 3 without
lim ited hold still shows the effect of the lim ited-hold reinforcement. Figure 442 be
gins on a V I 3 schedule of reinforcem ent. Extinction begins at the arrow . T he re
sulting curve is similar in over-all form to Fig. 441. T he bird responds either at the
prevailing variable-interval rate or not a t all, except for brief accelerations at a an d b
and a brief period of slow responding at c. These curves resem ble extinction after
fixed-ratio reinforcem ent in the general absence of interm ediate rates of responding.
Both F R an d V I lim ited hold reflect the fact th a t the probability of reinforcem ent
does not increase with the passage of tim e during a pause.
Figure 443 describes the retu rn to a variable-interval perform ance following the ex
tinction shown in Fig. 442. This figure simply continues the curve in Fig. 442.
364
Fig. 4 4 3 .
Return to VI 3 after extinction
T he m agazine had been re-connected, and the first response at a was reinforced. By
the th ird reinforcem ent thereafter a variable-interval perform ance is reinstated es
sentially unaffected by the prolonged extinction.
THE EFFECT OF DEPRIVATION O N PERFORMANCE
UNDER VARIABLE-INTERVAL REINFORCEMENT
W e conducted two experim ents in w hich the rate of responding u n d er v ariab le

interval schedules of reinforcem ent was m easured as a function of the body-w eight of
the bird. A single schedule of reinforcem ent was m ain tain ed th ro u g h o u t the ex
perim en t, a n d the body-w eight of the bird was changed from day to day d u ring the
experim ent. As already noted, body-weight is affected by recent drinking, eating of
365
grit, a n d defecation. W hen th e b ird is fed a t the sam e tim e a n d the sam e am o u n t
each day, these factors become fairly stable; but w ith the radical shift in metabolism
th a t occurs w hen gross changes are m ade in the body-w eight of the bird, they are
likely to have a disturbing effect. In in te rp re tin g the following results it is also im
p o rta n t to note th a t a given body-w eight represents a different condition of d ep ri
vation if it follows higher weights th an if it follows lower ones.
W e altered body-w eights by daily feeding or deprivation until a given level was
o btained. T h e successive sessions of the experim ent therefore do not alw ays im ply
successive days.
Rate o f responding as a function o f body-w eight
Experiment I. Final perform ances for two birds on a geom etric variable-interval
schedule w ith a 7-m inute m ean have alread y been described in Fig. 408 a n d 411.
These birds were then run on this schedule while their weights were varied over a fairly
wide range. Figure 444 shows the results. T h e num bers beside the points show
the order of the sessions. T he bird represented by the circles showed the higher rate
of responding. Its body-weight ranged from 420 to 569 grams. T he ad lib weight
Fig. 4 4 4 .
BODY WEIGHT
GRAMS
Plot of mean daily rate of responding as a function of body-weight
366
of the bird determ ined 2 years earlier was 522 gram s. H ow ever, the highest weight
recorded, 568 gram s, is probably very close to the weight w hich this bird would
have reached a t the tim e of the experim ent on a free-feeding schedule. T h e g rap h
shows a rough relation betw een the w eight of this bird a n d the rate during the first
5 hours of the session. Betw een 490 a n d 530 gram s the rates scatter widely. T h e
ra te is low betw een 530 a n d 568 gram s, b u t increases sharply as the weight falls
from 530 to 490 gram s. T h e rate increases m ore slowly as the w eight is fu rth er re
duced. Samples of variable-interval performances at the lowest, middle, and highest
body-weights are shown in Fig. 445 from the sessions m arked a', b', and c' on the graph.
In R ecord A deviations from a constant rate are m inor. At an interm ediate bodyweight, Record B, the m axim um rate has fallen slightly, largely because of the ap p e ar
ance of grain and slight pauses; and the over-all rate has fallen considerably, largely
because of the m arked negative curvature w hich appears during the longer intervals.
Each reinforcement characteristically reinstates the m axim um rate. In the perform
ance at the highest body-weight (Record C) high rates still appear after reinforcement,
b u t these are sustained for only a few responses.
T he second bird, represented by the triangles in Fig. 444, shows roughly the same
relative cu rv atu re, a lth o u g h th ere are fewer points a n d th e range of body-w eight is
sm aller. T he b ird s ad lib weight, determ ined 2 years earlier, was 500 gram s; but
th e range in the present experim ent is from 408 to 480 grams. Figure 446 shows seg
ments from the lowest, m iddle, and highest body-weights, at a, b, and c, respectively,
in Fig. 444. T he perform ance at the lowest body-weight, R ecord A, is sim ilar to th at
of the other bird except for the lower over-all rate. However, a t the m iddle bodyweight, R ecord B, both the m axim um rate and the average rate fall. A t the highest
body-weight recorded, Record C, the perform ance resembles th at in Fig. 445.
Experiment II. W e conducted a second experim ent on the effect of deprivation on
the perform ance under variable-interval reinforcem ent with the birds which had
reach ed a final perform ance u n d e r V I 3 after the history of V I lim ited hold already
described. T he experim ent com prised approxim ately 120 sessions each, during
w hich the body-weights were varied from 370 to 505 gram s in one bird and from 382
to 540 gram s in another. Each dot in Fig. 447 a n d 448 represents a daily session
at the beginning of which the body-weight was as indicated on the abscissa and during
which the average rate of responding was as indicated on the ordinate. W eight an d
corresponding rates have been averaged for successive blocks of 10 sessions a n d p re
sented in the solid line in the figures. T h e num bers to the right of the points indi
cate the order of occurrence of the blocks of 10 sessions.
D uring the first 50 sessions in Fig. 447 the relationship betw een the average rate
of responding a n d the w eight of the bird is roughly linear. N ear-zero rates of re
sponding were recorded when the weight approached 500 grams. M axim al rates
occurred a t 420 to 425 grams. Between the 50th an d 90th sessions, however, when
the weight of the bird was reduced further to 380 gram s, the rates of responding gener
ally fell to lower values; and between the 90th and 120th sessions when the weight of
368
1---------- r
169 YP
9
8
7
6
5
4
3
2
I
0
380
390
400
410
420
M EA N
Fig. 4 4 7 .
430
440
WE I GH T
450
460
470
480
490
500
AT START OF SESSION (GRAMS)
Plot of mean daily rate of responding as a function of body-weight (first bird)
510
520
530
370
the bird was now increased again to 420 gram s, the rate of responding continued to
fall.
T he sequence in both birds shows th a t a b ru p t and gross changes in body-weight are
likely to produce greater changes in rate th an a slow gradual change. Some of the
variability in the two figures is undo u b ted ly due to the relatively rough m a n ip u la
tions of weight.
T he decline in rate at the very low body-weight is possibly due to inanition. T he
continued decline in rate as the weights were increased tow ard the end of the experi
m ent suggests either a progressive effect of deprivation or a change in the v ariab le
interval baseline, because of prolonged exposure u n d er conditions of extrem e depriva
tion.
The m axim um change in rate produced by the change in deprivation is by a factor of
4 for the bird in Fig. 447 and of
for the bird in Fig. 448. T he range of rates oc-
o
37 0
Fig. 4 4 8 .
J
380
i
390
l
400
_______I_______I_______L______ I_______1______ I_______I_______ I_______ I_______I_______I_______I_______

410
4 20
430
440
450
460
470
480
490
500
510
520
530
540
MEAN WEIGHT AT START OF SESSION (GRAMS)
Plot of mean daily rate of responding as a function of body-weight (second bird)
curring in individual sessions, however, is considerably greater. Figure 449 shows

segments from 4 sessions at 4 extrem e rate ranges. R ecord A at 423 gram s shows one
of the highest rates recorded, in the vicinity of Point 6 in Fig. 448. Record B at 463
gram s in the vicinity of Point 4 shows an in-betw een rate. R ecord D at 499 gram s
shows one of the lowest rates recorded in the experim ent, to the right of Point 1 in the
figure. Record C shows an interm ediate rate which occurred at a weight of 408 gram s
(lower than the body-weight which produced the high rate of responding in Record A).
R ecord C occurred in the vicinity of P oint 11, w here the curve shows a decline in rate
as the w eight is increased.
371
Rate o f responding as a function o f pre-feeding
Various am ounts of food, ranging from 10 to 80 grams, were fed to a norm ally d e
prived bird just before the start of an experim ental session in an attem pt to vary the
deprivation-level in another way. Tw o birds showing stable perform ances on a geo
m etric V I 7 schedule of reinforcem ent were used. (These are the same birds whose
variable-interval performances were recorded as a function of body-weight in Fig. 444.)
Figure 450 shows the rates during a 5-hour session. Except a t the largest am ount of
pre-feeding used, the procedure h ad no effect on the variable-interval perform ance.
T he only substantial decline in rate was produced by feeding 80 grams to one bird ().
T he other bird showed an over-all rate of responding well w ithin the norm al range,
although it was fed 80 gram s or 20% of its body-weight. This result differs from an
earlier report of the effect of pre-feeding on FI perform ances in rats (Skinner, 1938).
We have not repeated this experim ent with FI reinforcement, b ut the difference m ay
372
GRAMS PREFED
Fig. 4 5 0 .
Plot of mean daily rate of responding as a function of pre-feeding
be due to the use of a V I schedule, w hich appears, in general, to be less sensitive to

variables such as level of deprivation and drugs.
Feeding a pigeon im m ediately before an experim ent and changing its weight over
a m atte r of days w ith a given feeding regim en are clearly very different operations.
But the negative result of pre-feeding is surprising, even so. M ost long experim ental
sessions show a decline in rate, which appears to be due in p a rt to the food ingested
during reinforcement. T he tem poral relations between the ingestion and the effect on
rate are similar to those in the pre-feeding experiments. T he gradual intake of food
m ight be m ore effective th an the ingestion of a single large am ount.
O ne factor to be considered in all discussions of the effect of deprivation on V I is the
rate contingency generated by the schedule. T he preceding experim ents involved
long-standing perform ances on V I. R einforcem ents usually occur at a single rate, or
at a rate varying w ithin narrow limits. H ence, once this rate is assumed, it tends to
p erp etu ate itself. Extinction curves after brief V I show m ore gradual acceleration.
E xtinction curves after prolonged V I show a stepwise structure in which a rough high
rate predom inates.
373
T he severe decline in over-all rate during a long session on a well-developed V I a p

pears to show a process of satiation in conflict with a tendency to m aintain a single rate.
An extrem e exam ple is given in Fig. 451 an d 452, w hich were recorded continu
ously in this order. T he fairly stable V I perform ance a t the beginning of Fig. 451
gives way to perform ances suggesting the records taken a t different body-weights (Fig.
445 and 446). H ere, im m ediate feeding (in a series of reinforcements) produces the
same result as a long-term modification of body-weight.
V ariable-interval performance under w ater deprivation
W e studied the perform ance on an arithm etic V I 3 in 3 birds in an apparatus w hich

contained m agazines for both food and w ater reinforcem ent. In the early stages of
the experim ent the effective conditions of deprivation were not known, and factors such
as the a m o u n t of w ater per reinforcem ent, the n u m b er of hours of w ater deprivation,
an d the am o u n t of w ater fed in the hom e cage were varied freely until values were
discovered which produced stable responding u n d er the variable-interval reinforce
ment. T he birds were deprived of both w ater and food, b ut these operations are not
independent. Extrem e w ater deprivation reduces food consumption, and both of
these reduce weight. H ow ever, stable conditions for w ater reinforcem ent can be set
up.
Figure 453A shows a com plete daily session after 12 sessions of w ater reinforcem ent
on V I 3. T he session begins at a high rate, which falls off during the first 2000 re-
Fig. 4 5 1 .
17 -hr session on VI 7
375
sponses to a value which is m aintained for the rest of the session. T he rate has a slight
tendency to be low after reinforcem ent w ith the schedule used. After 6 further ses
sions of w ater reinforcem ent, the w ater m agazine was disconnected and the food m ag
azine substituted. T he perform ance in R ecord B was recorded after 3 sessions of food
reinforcem ent. T he rate of responding is higher th an th a t u n d er the preceding w ater
reinforcem ent, and the rate im m ediately after the reinforcem ent is now relatively high.
T he perform ance in R ecord A represents the highest rate w hich we could generate
u n d er w ater deprivation. Food deprivation and reinforcem ent ap p ear to be some
w hat m ore effective in controlling a high level of activity.
Figure 454 shows the transition from food to w ater reinforcem ent under conditions of
both food an d w ater deprivation. At the arrow, food reinforcements are discontinued
376
an d w ater reinforcements substituted. T he first w ater reinforcem ent a t e is followed

by a decline in rate, which becomes most m arked a t f . This perform ance should be
contrasted w ith th a t a t the start of the record w here the rates following the reinforce
m ent with food at a, b, c, and d. increase slightly over the prevailing rate.
Figure 455 com pares food an d w ater perform ances on V I reinforcem ent for another
bird. This bird had a higher over-all rate on both food and water. This figure also
shows a higher rate of responding u n d er food reinforcem ent (R ecord B) th an under
w ater reinforcem ent (Record A). However, the rate im m ediately after reinforcement
is roughly the same in both cases.
THE EFFECT OF A PRE-AVERSIVE STIMULUS
O N A VARIABLE-INTERVAL PERFORMANCE
A pre-aversive stim ulus is a stim ulus which characteristically precedes an aversive

stimulus. An exam ple is a buzzer which is characteristically followed by a shock, as
in the procedure used by Estes a n d Skinner (1941). A h ungry ra t is reinforced on a
fixed-interval schedule. D uring the session a buzzer is presented for 3 m inutes; the
ra t is then shocked through a grid floor. As a result, the rate of responding declines
possibly to zero w hen the buzzer, the pre-aversive stimulus, is present. T he effect
is sim ilar to the kind of disruption in h u m an behavior called anxiety. T h e effect of
a pre-aversive stim ulus of food-reinforced behavior has been used extensively by H u nt,
Brady, Lindsley, and others to study the effects of electroconvulsive shock, drugs, an d
radiation.
T he effect of the pre-aversive stimulus on the performance of the pigeon on V I was
studied in the following way. A stable perform ance on V I 3 was established. T h en ,
the general illum ination in the ap p aratu s was dim m ed for 30 seconds. This proce
377
dure was repeated every 5 m inutes, until the novel stim ulus no longer h ad an effect on
the V I rate. Each 30-second period during w hich the light was dim was then fol
lowed by an 8-second electric shock of approxim ately 400 volts, 60 cycles a c, delivered
through a floor grid (independent of a response). T he pigeons feet were coated with a
light film of graphite paste, and the polarities of the grid were constantly shifted at a
high rate in order to prevent serious shorting of th e circuit. W e determ in ed the
level of the shock by adjusting the voltage to elicit vigorous leg-lifting m ovem ents.
T he intensity was kept below values which produced violent reactions.
Development o f a fin al perform ance with the pre-aversive stimulus
Figure 456 (the 2nd session in w hich the pre-aversive stim ulus was followed by
shock) shows the developm ent of a lower rate of responding d u rin g the pre-aversive
stim ulus. (T h e vertical lines below the record refer to the start of the pre-aversive
stim ulus a n d the occurrence of th e shock at the end of th e pre-aversive stim ulus.)
C om plete suppression of the variable-interval perform ance is evident a t c a n d d, a n d
partial suppression at a an d g, w here some responding occurs during the 30-second
interval. At b the rate rem ains low beyond the term ination of the pre-aversive
stim ulus a n d shock; b u t, in general, th e n o rm al V I ra te is assum ed im m ediately
after the shock, at the end of the pre-aversive stimulus. Note th at the schedule of rein
forcem ent is m aintained through the pre-aversive stim ulus, and th a t reinforcem ents
378
occasionally occur a t th a t tim e, as a t g. At the arrow the duration of the pre-aversive

stim ulus was increased to 60 seconds. A zero rate is first m aintained for about the
previous duration of 30 seconds, and is followed by some responding during the second
half of the pre-aversive stimulus, as at h, i, a n d j. For the rem ainder of the session,
however, either the rate during the pre-aversive stim ulus is lower but sustained,
as at / an d m; or (tow ard the end of the session) responding completely stops, as at n
and o. W hen the pre-aversive stim ulus was not present, the perform ance rem ained
appropriate to the schedule of reinforcement.
Figure 457 shows the perform ance on V I w ith a 60-second pre-aversive stim ulus
every 5 m inutes after a total of 62 sessions (240 hours). T he rate d u ring the preaversive stim ulus is now very close to zero, although some responding m ay occur, as a t
e , f and i, during the first few seconds. T he rate under V I is often suppressed after
the shock, as at b, e, h, a n d i, although an im m ediate shift to the highest observed rate

m ay take place, as, for exam ple, a t c, d, an d g. T h e session also begins w ith some
w arm -up, and the prevailing V I perform ance is not reached until possibly 15 m inutes
after the start of the experim ent. T he grain is m uch rougher th an th a t in the e a r
lier V I perform ance in Fig. 450, and the over-all rate has fallen, largely because of the
lower rate im m ediately following the shock.
Occasionally, as at d in Fig. 457, the higher rate after the shock suggests com pen
sation for the pause d u rin g the pre-aversive stim ulus. Figure 458 contains several
sim ilar instances from the im m ediately preceding an d following sessions for the same
bird. T he dotted lines in Records A a n d B are intended to suggest the ex trap o la
tion of earlier parts of the curves. T h e rate after the pre-aversive stim ulus an d shock
exceeds the prevailing V I rate until the extrapolation is roughly reached.
V ARIA BLE IN T E R V A L
379
T h e p resentation of the pre-aversive a n d aversive stim uli on an F I schedule (e.g.,

every 5 m inutes) eventually sets up a tem p o ral p attern . T h e buzzer is no longer
the only pre-aversive stim ulus. T h e b ird s own behavior is a stim ulus which can e n
ter into the contingencies of the schedule of shocks a n d acquire the pow er to sup
press the rate. Such stim uli show tem poral gradients.
Figure 459 shows a late r perform ance for a n o th e r bird on V I 3, in w hich a 60second pre-aversive stim ulus occurred every 10 m inutes. This is the 67th session of
the pre-aversive procedure. T he intervals betw een aversive stim uli h ad been 5
m inutes until the 11th session preceding the figure. T he particu lar V I schedule pro-,
duces a brief run after reinforcem ent. T he rate during the pre-aversive stimulus is
380
sometimes zero, although responding is usually slow but substantial. T he session c h a r

acteristically begins w ith a very low ra te w hich increases grad u ally as the session
progresses. T h e prevailing V I perform ance is not reached until at least 45 m inutes
after the start of the session. T he rate during the 10 m inutes between the presenta
tions of the pre-aversive stim ulus freq u en tly shows a m ark ed decline, as a t a, b, c, d,
an d e. At the end of the pre-aversive stim ulus the rate quickly accelerates to a m axi
m al value, a n d th e n declines continuously u n til the next p resen tatio n of the preaversive stim ulus. This effect is m ost m arked a t the beginning of the session. A
sim ilar perform ance 5 sessions later, shown in Fig. 460, illustrates the reproducibility
of the m ain features.
The effect o f the pre-aversive stimulus during extinction
Figure 461 shows the effect of pre-aversive and aversive stimuli on extinction after
V I 3. After 74 sessions of V I reinforcem ent with a periodic pre-aversive stimulus (Fig.
381
459 a n d 460 are samples), reinforcem ent was discontinued. T he w arm -up p a rt of
the session has been om itted in Fig. 461. This figure shows compensatory fast respond
ing after the term ination of the pre-aversive an d aversive stimuli, as at a. (Cf. Fig.
458 for the other bird.) T h e m agazine was disconnected at the arrow , b u t the preaversive and aversive stim uli continued to ap p ear every 10 m inutes as before. T he
resulting extinction curve contains 2 segm ents of approxim ately 1000 responses each
at the prevailing V I rate, separated and followed by segments where the rate is con
stant but near zero. T he second pre-aversive stim ulus after extinction has begun (at
b) shows only a slight effect. H ow ever, the first an d fifth presentations, occurring
against a background of a high rate, show the suppression effect. T he fourth, occur
ring during slow responding, m ay be followed by a reb o u n d . In general, this is
very sharp curv atu re after so long a history of V I 3. It suggests th a t the aversive
stim uli are hastening the extinction process.
Removal o f pre-aversive stimulus
Following the final perform ances shown in Fig. 457 and 459, the pre-aversive stim
ulus was om itted. T he same V I 3 was in force throughout the session, and a shock
occurred every 5 m inutes, although it was no longer preceded by a pre-aversive stim
ulus. This procedure was designed to show the disruptive effect of the preceding
shock on the V I perform ance a p a rt from any effect of the pre-aversive stimulus. Fig
ure 462 shows a perform ance after 9 sessions during w hich the shock appeared u n
a n n o u n c e d every 5 m inutes. T h e m ajor effect has been severe suppression of the
over-all rate d u ring the early p a rt of the session as an intensification of the earlier
w arm -up. T he session represented shows a continuous acceleration of the over-all
rate during the first 4 or 5 excursions of the recording pen which cover about 2 hours
of the session. T h e periodic shocks were not m arked, b u t no rate changes having a i
382
period of 5 m inutes are apparent. T here is no evidence th a t the periodic presenta

tion of the shock has any consistent effect on the rate com parable with the curvature of
Fig. 459 and 460. This procedure shows th a t the pre-aversive stimulus tends to re
strict the effect of the shock to the pre-aversive period.
Removal o f the aversive stimulus
T he shock was removed from the program in the following session. T he schedule
of reinforcem ent was now sim ply V I 3. Figure 463 shows the sam e low ra te a t the
sta rt of the session a n d the acceleration to the n o rm al V I perform ance, alth o u g h
these are considerably less m ark e d th a n those in Fig. 462. A n o rm al V I p e rfo rm
ance is reach ed w ithin an hour; a n d the over-all rate is higher (1.5 responses per
second com pared with m axim um over-all rates of 1.25 in the previous session).
383
Extinction w ithout aversive or pre-aversive stimuli
Figure 464 shows extinction in the absence of both pre-aversive and aversive stimuli.
This occurs in the session im m ediately following Fig. 457. T he session begins with
the slight w arm -up which this bird has been characteristically showing. In the a b
sence of aversive stim uli the rate reaches a higher th an usual value, however. T he
highest rate (at a) is approxim ately 1.1 responses per second. D uring the first 6000
responses the rate declines fairly smoothly; it falls nearly to zero for 10 hours (om it
ted from the figure), d u rin g w hich only 350 responses occurred. E xtinction con
tinues with the low over-all rate in the last 3 segments of the figure. O scillations in
rate at b through h are more m arked th an usual in extinction after V I reinforcement,
an d m ay be the result of the previous aversive an d pre-aversive stim ulation. T he
periodicity could reflect the period of aversive stim ulation; or it could m ean th at the
10 HR. AND 3 5 0 R' s
OM ITTED
presence of a rate at which shocks have frequently occurred suppresses the rate, b ut
th at the new rate autom atically elim inates the suppressing stimulus.
Pre-aversive stimulus in the rat
For comparison, Fig. 465 shows the effect of a single pre-aversive stimulus on p er
form ance on V I in the rat. T h e pre-aversive stim ulus was 5 m inutes long a n d oc
curred in the m iddle of each daily session. T he aversive stimulus was a shock. T he
figure shows 15 successive daily sessions in w hich the pre-aversive and aversive stimuli
occurred. T he lever pressing stopped alm ost completely a t the end of the 5-m inute
period, but nearly all segments show responding for the first few m om ents of the preaversive stim ulus. T his perform ance represents a tem poral discrim ination. T h e
V I rate is considerably depressed im m ediately following the shock, a n d the over-all
rate rem ains lower during the latter p a rt of the session. T h e norm al decline in the
V I schedule of reinforcem ent makes it impossible to attrib u te this effect entirely to the
pre-aversive program .
Fig. 4 6 5 .
Pre-aversive stimulus in the rat
V A R IA B LE IN T E R V A L
385
CO NTINUO US ADJUSTMENT OF DRUG LEVEL IN TERMS OF BEHAVIOR
We used variable-interval perform ances in studying the effects of certain drugs.

In one experim ent we explored the possibility of controlling the level of sodium pento
barbital during a session by autom atically adm inistering the drug in relation to the
rate of responding on a variable-interval schedule. This possibility was suggested as
a convenient alternative to other criteria for the repeated adm inistrations of a drug,
such as the am o u n t of th e d ru g in blood sam ples, or collateral effects such as ra te of
breathing. T he plan was to establish a stable perform ance on V I and to adm inister
sodium pentobarbital slowly and continuously until the rate fell below a certain value.
T he drug would be discontinued w henever the rate of responding was below the given
value a n d re-adm inistered w henever the rate of responding was above it. (W ith a
drug which h ad an excitatory effect the procedure w ould be the opposite; the drug
w ould be adm inistered w henever the rate fell below a certain value an d w ithheld
w hen it increased above it.)
In the present experim ent we adm inistered the d rug orally by infusing the grain
w ith an aqueous solution of sodium pentobarbital. A given am ount of grain was in
fused w ith a specified solution a n d allow ed to dry quickly. Some of the d ru g was
undoubtedly lost through oxidation u n d er these conditions. W e were not interested
at this stage in specifying the am o u n t of sodium p en to b arb ital actually given from
day to day but rather in am ounts relative to the concentrations of the infusing solution.
In order to evaluate the effect of a specific drug, dose levels would, of course, have to
be controlled m ore accurately. This control could be achieved with better m ethods
of preparing the drug for oral adm inistration, by adjustm ent of the concentration of a
gas in an am bient atm osphere, or by a continuous m ethod of intravenous injection.
T h e exp erim en tal ch a m b e r co n tain ed 2 food m agazines, one of w hich co n tain ed
clean grain a n d the other grain infused w ith sodium p entobarbital. A n a rb itra ry
rate of responding was specified to determ ine w hether reinforcem ent would be drugged
or clean. W e tried several values of criterion rates of responding. In the experi
m ent to be reported here, reinforcem ents were drugged w henever the rate was above
0.3 response per second for 3 m inutes. Conversely, reinforcem ents were m ade w ith
undrugged grain w henever the rate was below 0.3 response per second for 3 m inutes.
Figure 466 contains graphs of com plete daily sessions u n d er the various procedures
carried out in the experiment. T he curves are reduced in order to emphasize the over
all rate changes occurring d u rin g the sessions. R ecord A shows a control V I p e r
form ance w ithout sodium pentobarbital. This was taken after approxim ately 34 ses
sions on V I during most of which the drug h ad been autom atically adjusted. T he
over-all rate is roughly constant, w ith the usual slight negative curvature. A second
bird in the experim ent showed a sim ilar curve u n d e r these conditions. R ecord B
shows the effect of a small am o u n t of sodium p e n to b arb ital delivered in all reinforce
ments. T he curve in R ecord B is negatively accelerated throughout a n d shifts
abruptly to a low rate before the end of the session. R ecord H shows a perform-
19
18
17
16
15
14
13
12
I!
10
7I
6|
5>
4 r
3;
2|
H O U R S
466,
D aily sessions under various conditions o f drug adm inistration
387
ance in which all reinforcements were drugged w ith a larger am ount of sodium pento
b a rb ital. This dose level produced the m ost m arkedly negative acceleration an d
the lowest num ber of responses for the session. (In Curve J a slow-down contingency
has been added to the variable-interval schedule. See C hapter Nine.) In Curve H
the decline in rate is not continuous. T he over-all negative curvature is produced by
increasingly longer pauses and periods of low rate. In R ecord C the adjusting p en
to b a rb ita l doses produce a rap id oscillation in rate. H ere, the bird is either not re
sponding and receiving no drug, or it is responding at a rate higher th an the critical
value a n d receiving all drugged reinforcem ents. Figure 467 shows the sam e curve
enlarged. T he first 5000 responses of the session are ru n off at essentially a n o r
m al V I rate except for b rief runs a t an in term ed iate ra te after some reinforcem ents.
All reinforcem ents are drugged. T he first response following the pause at a is rein
forced w ith clean grain, an d a substantial rate quickly develops. D rugged grain is
then received until b, where the rate again falls off to zero for approxim ately 5 m in
utes. As the session progresses, the oscillation becomes m ore m arked and the pauses
longer. T h e pause at c, for exam ple, is 25 m inutes. A m ore prolonged accelera
tion brings the rate back to its highest value at d, and extended negative curvature over
the next 1300 responses leads to a zero rate again at e.
R ecord D in Fig. 466 shows a daily session in w hich the adjusting d ru g level p ro
duces an in term ed iate rate of responding w ith less rap id oscillations th a n those in
388
Fig. 4 6 8 .
Fig. 4 6 9 .
D aily session showing interm ediate rate under the adjusting drug level
Last p a rt o f a d a ily session showing oscillation and over-all negative curvature

under la rg e r am ounts o f adjusting drugs
Record C. T he over-all curve is negatively accelerated. Figure 468 shows the entire
record in detail. At a and b the rate falls to zero under the influence of the sodium
pentobarbital in most of the preceding reinforcements. T he section of the record from
c to d shows an interm ediate rate of responding of the order of 0.3 response per second.
This is the rate above which reinforcem ents are drugged and below which they oc
cur w ithout the drug. T h e procedure does not, however, m ain tain a steady state.
L ater, the rate accelerates continuously as the effect of the now only occasionally a d
m inistered drug wears off.
R ecord E in Fig. 466 shows a daily session w ith a larg e r dose per reinforcem ent,
w here the criterion rate was 0.5 response per second for 3 m inutes ra th e r th a n 0.3
response per second as in the rest of the sessions described. T he session shows a
more m arked over-all negative acceleration and m ore responding at interm ediate rates
below the critical value. Figure 469 shows p art of R ecord E, beginning at a in Fig.
466, in g rea ter detail. T h e change from a high rate a t a to a zero rate a t b is fairly
smooth. A quick return to an interm ediate rate at c is followed by a continuous
decline in th e over-all rate for m ore th a n an hour. H ere, a m ore graded effect has
been achieved, but by no m eans a stable state.
389
A closer approach is m ade in R ecord F, w hich shows a daily session w here scattered
d ru g g ed reinforcem ents produce an in te rm e d ia te ra te of responding th a t declines
continuously d u rin g the session. R ecord G shows a perform ance sim ilar to R ecord
C, w ith more extended pauses and a m ore gradual acceleration from one rate to a n
other.
O rd in arily , the rate of responding is decreased w hen reinforcem ents are discon
tinued. W hen reinforcem ents have contained a drug, however, a cessation of rein
forcem ent m ay produce the unusual effect of a tem porary increase in rate. Figure
470 shows the end of a daily session, in which oral adm inistration of sodium p en to b ar
bital at reinforcem ent h ad produced a very low rate of responding. After the rein
forcem ent at a, the m agazine was disconnected. T h e effect is a continuous increase
in the rate during the next 2500 responses as the effect of the drug wears off. At b
the rate begins to fall in extinction, b ut a reinforcem ent at c reinstates an interm ediate
variable-interval rate.
W e reduced the am ount of oscillation in this experim ent by adding a drl contin
gency to the variable-interval schedule. (See C h a p te r T en.) R ecord I in Fig. 466
shows a perform ance u n d e r a schedule in w hich reinforcem ents were available on
VI but a response was not reinforced unless 2 seconds had elapsed since the preced
ing response. T he over-all rate is approxim ately 0.6 response per second a n d the
curve is roughly linear, although the rate of responding has some tendency to be higher
at the very start of the session. T he rate ju st before the drl contingency was added
was 1.3 responses per second. (D etails of the perform ances before an d after drl was
ad d ed a p p e ar in Fig. 569 in C h a p te r N ine.) W h en the adjusting d rug procedure
was added to this schedule, it produced the stable interm ediate rate of responding seen
in R ecord J , Fig. 466. H ere, the over-all rate is 0.3 response per secondthe cri
terion value above w hich reinforcem ents are drugged an d below w hich they are not.
A slight am ount of hu n tin g is evident in the small grain of the curve. This curve
shows the successful m aintenance of a constant rate of responding by an adjustm ent of
the drug dosage in term s of the rate of responding.
T he infusion of the grain w ith sodium p en to b a rb ita l solution could possibly affect
th e taste of the grain for the bird. H u m a n subjects find sodium pentobarbital very
bitter. This factor m ay have been one cause of lower rates under drugged reinforce
ments. But a result such as th a t of Fig. 470, in w hich the effect of the d ru g wore
off during extinction, indicates th at the m ajor depressive effect of the drug is due largely
to action on the central nervous system.
O th er schedules m ight be m ore effective in providing a baseline for an adjusting
drug level, particularly because of the rate contingency which develops under V I and
which m ay have disturbed the experim ents on the effect of the level of deprivation.
A large fixed ratio showing some pause after the reinforcem ent, or a fixed-interval
schedule with an extended scallop, m ight be preferred. T he fixed-interval schedule
has the advantage th a t reinforcem ents will usually continue to be delivered on a fixed
schedule. This factor m ight elim inate some of the oscillations in rate th a t are due
390
Fig. 47 0 .
Increased rate o f responding during extinction when drug is discontinued
to the groupings of reinforcem ents on a variable-interval schedule. A nother possible

baseline for adjusting a d ru g level m ight be the ra te of responding d u rin g a preaversive stim ulus, provided a given dose of a given d ru g changes the a m o u n t of re
sp o nding th a t occurs d u rin g such a stim ulus, as is often the ease. A p ro g ram of
adjusting drug adm inistration could be used ultim ately in a m ultiple schedule in which
the schedule of reinforcement in the presence of one stimulus would be used to adjust
the dose level, while the effect of the given dose level w ould be observed 011 the second
schedule in the m ultiple program .
Chapter Seven
VARIABLE RATIO
INTRODUCTION
In a v a r i a b l e - r a t i o (V R ) schedule of reinforcem ent the reinforcem ent occurs after

a given num ber of responses, the num ber varying unpredictably from reinforcement to
reinforcem ent. T he schedule stands in the sam e relation to the F R as V I does to
FI.
Like F R it arranges for differential reinforcem ent of high rates. (See C h a p te r
Five, p. 133.) A cursory exam ination of the figures in this chapter will reveal very few
instances w here a reinforcem ent occurs after a pause.
V a ria b le-ratio schedules produce a v ariety of perform ances, depending upon the
distribution of the num bers of responses required for reinforcement. As in all sched
ules req u irin g a n u m b e r of responses, th e b ird will stop responding altogether if the
average num ber goes beyond a certain value. W e have attem pted to study variable
ratio schedules which produce stable perform ances w ith a constant over-all rate of re
sponding. As in the design of V I schedules, various systems m ay be used to generate
the numbers of responses in the series. A m ean num ber of responses per reinforcement
is usually fixed. T he sm allest n u m b er is usually 1; i.e., in some cases the first re
sponse after reinforcem ent is reinforced. T h e largest num ber and the progression
specifying the successive values in the series are other arb itrary points to be selected.
As w ith fixed-ratio reinforcem ent, we cannot observe the transition from erf to V R
w ith a large m ean, because extinction takes place. T h e larger m ean ratios are
reached after stable perform ances are established on lower means. T he m ean m ust
be carefully increased until the desired value is obtained.
DEVELOPMENT OF STABLE PERFORMANCES O N VARIABLE-RATIO SCHEDULES
A stable perform ance was generated on several variable-ratio schedules in 2 separate

experim ents. In the 1st the birds w ent directly to V R after erf; in the 2nd they were
exposed to 3 sessions of V I before VR.
Figure 471 shows the transition from erf to an arithm etic V R . In Record A the
m ean was a b o u t 40; in R ecord B it was 50. Som etim es, successive responses were
reinforced, an d the largest ratio was approxim ately 100. Both birds show a rapid
391
392
developm ent of a high over-all rate. T he high rate from the extinction of erf produces
a high frequency of reinforcement, which in tu rn sustains a very high rate of re
sponding. T he 2nd segm ent of R ecord A shows about 1.75 responses per second, the
last segment of Record B almost 3 responses per second. This is an extremely rapid
developm ent; b ut the frequency of reinforcem ent is, of course, high com pared w ith the
transitions from erf to VI.
D u rin g the next 16 sessions we increased the m ean ratio from the values shown in
Fig. 471 to 120, 240, a n d 360 responses by ad d in g successively larger n um bers of
responses in the series. E xcept for a relatively h ig h er density of shorter ratios, the
num bers formed an approxim ately arithm etic progression: 1, 10, 20, 30, 60, 100, 180,
240, 300, 360, 420, 480, 540, 600, 660, 690, 690, 720, and 720 responses. W e de-
V A R IA BLE R A T IO
393
signed the actual schedule by scram bling several sets of these num bers. Performances
after 6 sessions of exposure to V R 360 are shown in Fig. 472 an d Fig. 473. Both
birds show a higher rate im m ediately after reinforcem ent. From 25 to 75 responses
are em itted at rates varying from 2.8 to 3.5 responses per second in Fig. 472, an d
from 3 to 3.5 responses per second in Fig. 473. T he over-all rate, however, is m uch
lower for both cases approxim ately 1.5 responses per second in Fig. 472 a n d 2 to
2.5 in Fig. 473. T he short period of a high rate of responding im m ediately follow
ing the reinforcem ent is followed either by an a b ru p t shift to a lower rate w hich is
m ain tain ed until the next reinforcem ent, by a pause an d acceleration to th a t rate, or
by a pause an d an a b ru p t shift to the lower rate of responding. Figure 472 shows
a lower over-all rate and m ore curvature a t interm ediate rates.
394
Figure 474 shows examples of the lowest sustained rates occurring a t this time; one
exam ple is given for each bird. These occurred 4 sessions after the perform ance a l
rea d y given in Fig. 473. R ecord A now shows a m arked scallop following the p e
riod of high rate after reinforcement.
T h e ru n after reinforcem ent in these V R schedules is sim ilar to th a t in certain
v ariab le-in terv al schedules w here th e density of short intervals is relatively large.
T his is effectively a tw o-valued schedule of reinforcem ent. T h e perform ance im m e
diately after the reinforcem ent is ap p ro p riate to a short ratio. If no reinforcem ent
occurs within, say, 75 responses, a shift to a lower rate appropriate to the larger r a
tios occurs. T h e V R schedule rem ained in force for 60 sessions (except for a few ses
sions in which the schedule was changed to F R 3 6 0 ). Some variation existed from
session to session in the ability to sustain a high rate under V R . Figures 475 and 476
show the extrem es of perform ances on V R 3 6 0 for one bird. Figure 475 shows a
segm ent from a daily session in which the over-all rate is high. M ost rapid responding
still occurs im m ediately after reinforcement (of the order of 10 responses per second).
Sustained rates of responding, as in the first excursions of the figure, are of the order of
7 responses per second, w ith local rates reaching 10 responses per second during 20
to 30 responses.
VARIA BLE R A T IO
395
T h e perform ance shown in Fig. 476 is for the session following th a t in Fig. 475.
H ere, th e ra te in the few responses im m ediately following the reinforcem ent is of the
order of 8 responses per second, a n d the relatively a b ru p t shift to a high rate gives
way to long pauses. O ne of these (om itted from the figure) was m ore th a n 1 hour.
After the pause, several h u n d red responses m ay be em itted before the term inal rate
is reached. T he performances shown in Fig. 475 and 476 represent the limits of a sta
ble state, in the sense th a t the bird will m aintain a perform ance lying between these
two extremes indefinitely.
T he second bird never developed a sustained high rate on this m ean value of V R .
Figure 477 shows the perform ance after 60 sessions on V R 360, including several ses
sions on F R 3 6 0 . T h e entire perform ance is executed in bursts of from 10 to 75
Fig. 4 7 7 .
397
Late VR perform ance w ithout sustained responding
responses, at a very constant rate of 4 to 5 per second, separated by pauses varying

from 10 to 70 seconds. Reinforcem ents no longer generate a high rate as in the earlier
sessions of the experim ent. For exam ple, note the reinforcem ents at b, d, and e. T he
only sustained responding occurs at a, w here slightly over 200 responses are ru n off
w ith only a slight suggestion of pauses. In the region a t c the groups of responses are
consistently of the order of 10 to 20, while in the region of d they are of the order of
75 to 125 responses.
This 'performance is similar to th at on a variable-interval schedule with differential
reinforcem ent of high rates (see C hapter N ine), and m ay be considered as the effect of
the differential reinforcem ent of high rates resulting from ratio reinforcement. This
p a rt of the contingency is here acting to offset any o th er factors in the v ariable-ratio
schedule th at sustain responding for longer periods of tim e w ithout pauses.
Tw o birds were reinforced on V I 1 following erf for 3 sessions of 60 reinforcem ents
each. T hey were then exposed to a variable-ratio schedule. Figure 478 shows the
tran sitio n for one bird. R ecord A shows the last session on V I 1, w here the over-all
rate is app ro x im ately 0.35 response per second. T h e m ean n u m b e r of responses
em itted per reinforcem ent was 20. At the beginning of the following session (R ecord
B) the schedule was changed to V R 15. T he result is both an increase in over-all
rate to approxim ately 0.8 response per second and, necessarily, a m arked increase in
freq u en cy of reinforcem ent, resulting sim ply from th e m echanics of the schedule.
T h e second bird showed a sim ilar perform ance on th e transition to V R , except th a t
the higher rate developed m uch m ore rapidly.
After 3 sessions on V R 15 the schedule was changed to V R 60, and pauses after rein
forcem ent an d occasional low rates of responding betw een reinforcem ents began to
appear. For 35 sessions the m ean variable ratio was th en kept a t 82. T h e over-all
and local rates of responding increased gradually, and the pauses after reinforce-
399
m ent an d deviations from a constant rate eventually disappeared. Because of this

slow p u sh -u p , both birds show a stable high rate. T h e m ean ratio was th en in
creased to 110. Figure 479 shows the 12th session following. A stable over-all rate
of approxim ately 4 responses per second is m aintained except for occasional periods
of interm ediate rates, as at a and b, a n d the appearance of a pause after reinforcem ent
tow ard the end of the session, as a t c, d, a n d e. After 11 further sessions on V R 110,
th e p erform ance shows less responding a t in te rm e d ia te rates a n d fewer pauses after
reinforcement.
Figure 480A shows the 1st session of the transition from V R 110 to V R 173. T he
rate of responding falls considerably below the final value on V R 110, an d even b e
low the early developm ent on V R 110 in Fig. 479. After 12 sessions on V R 173, how
ever, R ecord B shows a three-fold increase in the rate of responding. By this time,
interm ediate rates of responding have again disappeared, and any deviations from the
sustained high rate of responding are a b ru p t shifts to a zero r a te , as at a and b.
THE CONTRIBUTION OF FREQUENCY OF REINFORCEMENT TO RATE
OF RESPONDING UNDER VARIABLE-RATIO REINFORCEMENT
W hen reinforcem ent is determ ined by the num ber of responses, as it is on any r a
tio schedule, the frequency of reinforcem ent increases with the rate of responding. W e
cannot be sure th a t the high rates generated by variable-ratio schedules are not due
400
to increased frequency of reinforcem ent ra th e r th a n to the differential reinforcem ent

of rates or groups of responses. T he following experim ent was designed to separate
frequency of reinforcem ent from other factors in the variable-ratio schedule.
T he problem was to design a control experim ent in which frequency of reinforce
m ent w ould be identical w ith a variable ratio, b u t in w hich none of the other factors
of a variable-ratio schedule would be present. T h e procedure was to yoke two ex
perim en tal boxes, as described in C h a p te r T hree. W hen a reinforcem ent occurs in
one box, it autom atically sets up a reinforcem ent in the second box. T he a p p a ra
tuses were in separate rooms a n d hence w ell-insulated from each other, except for the
electrical connection w hich set up reinforcem ents in the second box. T h e bird in
th e first box was reinforced on a variable-interval schedule. T he same schedule was
set up in the second box, since every tim e the first bird was reinforced, a reinforce
m ent was set up for the second bird. W e m atched rates of responding in the two situa
tions by varying levels of deprivation. W hen rates of responding were approxim ately
the same, the schedule of reinforcem ent in the first box was changed to variable-ratio.
Fig. 4 8 1 .
Final VI performances fo r matched pair
T h e values of responses per reinforcem ent were chosen to m atch the actual num bers
ap p earin g in the perform ance on the variable-interval schedule. T he frequency of
reinforcem ent in the lead bird was thus u n ch an g ed at the start of the transition from
v a ria b le -in te rv a l to v ariable-ratio. A ny in itial increase in ra te m ust result from
some other factor, such as the differential reinforcem ent of high rates. As soon as the
ra te has increased, how ever, the frequency of reinforcem ent is increased, a n d the
process m ay continue in an autocatalytic fashion. T he bird in the second yoked a p
paratus, however, still rem ains on a variable-interval schedule. T he actual intervals
are determ ined by the perform ance of the first bird on the variable-ratio schedule, b u t
have no relation to the num ber of responses em itted by the second bird. T he sched
ules of reinforcem ent of the 2 birds are v aria b le -in te rv a l a n d variab le-ratio , respec
tively, while the frequency of reinforcem ent is identical. T h e extent to w hich the
increased frequency of reinforcem ent in the v ariable-interval schedule is responsible
for the increased rate can be determ ined from the increase in rate under the variab le
interval schedule in the yoked apparatus.
V A RIA BLE R A T IO
401
W e conducted the experim ent w ith 2 pairs of birds. T hey were placed on a geo
m etric V I 5 im m ediately after erf. T h e intervals in th e schedule ran g ed from a few
seconds (w hen successive responses were reinforced) to 33 m inutes. In m atch in g
th e rates of the 2 birds of each p a ir by adjusting the level of deprivation, we specified
an a rb itra ry rate betw een the 2 rates first observed; an d we adjusted the birds
weights in term s of w hether the over-all ra te for a session was above or below this
value. O ne bird in one pair showed signs of illness during the experim ent, an d to save
tim e, a n o th e r was substituted. T h e new yoked p a ir h a d a relatively short history
on V I before the transition to V R .
Matched pair I
Figure 481 shows final perform ances on V I for 1 p air of birds. T he figure shows the
m iddle parts of the session ju st preceding the transition to a variable-ratio schedule.
R ecord A for the first bird shows a typical geom etric V I perform ance w ith a n over
all rate slightly lower th an th at in Record B. Corresponding points in the schedules
are indicated by the small letters. T he over-all rate during the session is roughly con
stant. M arked short-term oscillations occur, an d the rate tends to be slightly higher
after reinforcem ent. Figure 482 shows the transition for the first bird from V I to V R .
T he record shows the start of the session; a n d the p a rt of the record before the arrow
shows th e final V I perform ance. T h e n um bers of responses req u ired for reinforce
m ent ranged from 1 (successive responses reinforced) to 1980. Because the rate of
responding following th e reinforcem ent tends to be som ew hat higher th a n elsewhere,
the first effect of the variable-ratio reinforcem ent is a m arked increase in the n u m
ber of reinforcem ents occurring in groups. T his effect, in turn, produces wide oscilla
tions in the over-all rate a n d sustained responding at rates higher th a n those under
VI. At a, for exam ple, the rate is above 2 responses per second for m ore th an 200 re
sponses; the highest sustained rate on V I was approxim ately 1.5 responses per sec-
402
ond. W e cannot be sure th a t this increase in the local rate of responding was produced
by the increased frequency of reinforcem ent, since differential reinforcem ent of high
rates m ay have produced some effect.
Figure 483 shows the perform ance of the yoked bird. T he record shows the usual
high starting r a te , which quickly falls to the usual value. T he point a t which the first
b ird s schedule was changed to variable ratio is indicated by the arrow. Subse
quently, the over-all rate oscillates som ew hat, although not so m arkedly as in Fig. 482.
Figures 484 a n d 485 describe the 2nd session for the yoked pair. T h e over-all
ra te on V R in Fig. 484 shows a fu rth er increase, w hile the perform ance of the yoked
b ird shows roughly the same rate as w hen under the previous variable-interval rein
forcement. However, the higher density of reinforcements after short intervals, result
in g from th e increase in ra te of reinforcem ent in th e first bird, produces a higher
rate of responding im m ediately following reinforcem ent.
T h e difference in over-all rate betw een the yoked birds becomes even m ore m arked
by the 4 th session. Figure 486 shows the first parts of the session. T he v a ria b le
ratio perform ance in R ecord A now shows an over-all rate of approxim ately 2 to 3 re
sponses per second w ith sustained rates reach in g 3.5 responses per second. I n te r
m ed iate rates of responding are in freq u en t, a n d m ost of th e changes in ra te involve
Fig. 4 8 4 .
Second session on VR
403
10 M I N U T E S
Fig. 4 8 5 .
Yoked bird during second session o f VR
a b ru p t shifts betw een a high prevailing ra te a n d pauses. T h e yoked bird, whose

concurrent perform ance is shown in R ecord B, displays a n over-all rate of slightly less
th a n 1 response per second, even though the frequency of reinforcem ent is identical
w ith th at of Record A. (The small letters on the record indicate corresponding points
in the schedules.) T he over-all rate is roughly constant; the rate has a slight tendency
to be higher after the reinforcements. Pauses of m ore th an a few seconds duration are
rare.
T he V R perform ance after 11 sessions is shown in Fig. 487A taken from the m iddle
p a rt of the session. W hile the local rate of responding is only 3 responses per second,
ra th e r th a n 3.5 responses per second as in Fig. 486, responding is sustained a t this rate
for longer periods. T here are clearly only two rates: the prevailing rate of 3 responses
per second an d zero. T he over-all rate is ab o u t 2 responses per second. T he cor-
487.
Yoked p a ir during the eleventh session o f VR
Fig. 4 8 8 .
Yoked p a ir a fte r 29 sessions on VR
VARIA BLE R A T IO
405
responding record for the yoked bird (R ecord B) continues to show over-all rates of
less th a n 1 response per second. Also, the rate changes continue to reflect the V I,
w ith sm ooth transitions from the higher rate following the reinforcem ent to lower rates
elsewhere.
Figure 488 records the final perform ance in this phase of the experim ent, after 29 ses
sions of V R for the first bird a n d 29 sessions of V I for the second. T h e final V R p e r
form ance consists of sustained runs averaging about 75 responses each at 3 responses
per second, separated by pauses of from 1 to 2 m inutes. T he shift from pausing to
the prevailing rate is usually instantaneous. Occasionally, slight curvature appears,
as at a an d b. T he yoked bird on V I shows rates as high as approxim ately 0.75 re
sponse per second. T he rate of responding im m ediately following the reinforcem ent
rem ains approxim ately 2.5 responses per second a n d is sustained slightly longer th an
th a t in the earlier perform ance (for exam ple, R ecord B of Fig. 487).
Matched pair II
T he second pair of birds did not constitute a repetition of the preceding experim ent,
because the first bird did not sustain substantial rates of responding under the variable
ratio schedule of reinforcem ent. Figure 489A shows the transition from V I to V R .
R ecord B gives the corresponding perform ance for the yoked bird. W hen the sched-
406
ule was changed to V R at the arrow , th e special (an d a t this stage largely accidental)
contingencies failed to take effect. T h e over-all rate continuously declined. R e
sponding reached so low a value following the last reinforcem ent a t b th a t the 1980
responses required for the next reinforcem ent in the V R schedule were not em itted d u r
ing the rem aining 4 hours of the session. N ote th a t this sam e ratio was com pleted
earlier in the series a t a. T h e yoked bird (R ecord B) shows a m uch larger extinction
curve th an Bird 50G, w ith a sm ooth decline in over-all rate. This bird h ad 47 ses
sions of reinforcem ent on V I, com pared w ith only 8 sessions for the bird in R ecord A,
w hich, as noted above, was added to the experim ent a t a late stage. T he bird w ith
the shorter history was assigned to the V R schedule in anticipation of an increase in

rate. This increase could then not be attrib u ted to a relatively long history; b u t it
failed to occur.
In the following session the schedule was changed to V I in order to reinstate a su b
stantial over-all level of responding. This change was successful; a n d a second re
tu rn to the variable-ratio schedule produced a more successful transition, at least to the
extent th a t the bird m aintained a substantial over-all level. Figure 490A shows the
perform ance after 8 sessions on the variable-ratio schedule. R ecord B shows the p e r
form ance on V I in the yoked apparatus. W hile the variable-ratio reinforcement has
not yet had its final effect, the perform ance in Record A begins to show a change in th a t
VARIA BLE R A T IO
407
direction. T h e over-all rate is roughly of the same order of m agnitude in both curves:
the only difference is in the character of the changes. T he rate changes in Record A
are relatively a b ru p t com pared w ith those in R ecord B, a n d the curve shows consider
able grain. G roups of from 5 to 10 responses em itted at high rates a n d separated by
pauses appear. Sim ilar pauses are relatively few in R ecord B u n d er the yoked V I.
T he bird in Record A is still unable to sustain substantial rates of responding during the
larger values of ratios or tow ard the end of the session, and the yoked bird also shows
a m arked decline in rate during the session corresponding to the growing infrequency
of reinforcements.
TRANSITION FROM VARIABLE RATIO TO FIXED RATIO
Figures 475, 476, a n d 477 recorded final perform ances on V R 3 6 0 , reached after
erf a n d a series of V R s of lower m eans. T h e lowest over-all rates of responding were
Fig. 4 9 1 .
Transition from VR 3 6 0 to FR 360
still substantial com pared w ith rates on F R , in w hich long pauses frequently occur
after reinforcem ent on ratios of the order of 300 responses a n d the over-all rate of re
sponding m ay approach the point w here responding ceases altogether. W e studied
this difference in perform ance on V R an d F R by changing the V R 360 schedule to
F R 360 an d observing transitional effects. T heoretically, the schedules are sim ilar
except for the possibility th a t the b ird s own behavior m ay serve as a counter w ith FR .
Differential reinforcem ent of high rates should be the same in both schedules.
Figure 491 shows a transition from V R to FR . T h e first excursion of R ecord A is
th e fourth in the session. T h e first 2 excursions show the prevailing perform ance on
V R . At the arrow the schedule becam e F R 360. T h e lower rate an d acceleration a t
a and b are characteristic of the variable-ratio perform ance during longer ratios. R ec
408
ord B shows the 11th an d 12th excursions, w ith a fu rth er developm ent of the fixedratio perform ance. T he pause a n d acceleration n ear the start of the ratio segm ent is
becom ing m ore m arked and occurs uniformly. R einforcem ent is first followed by a
prim ing run, however. R ecord G shows the 18th through the 21st excursions of the
recording pen. A brief period of responding at a high rate follows the reinforcem ent
a t c, but 45 m inutes is required for the next 360 responses. A long pause also occurs
near the start o f the fixed-ratio segm ent at d. T hereafter, 3 ratios are ru n off at m axi
m um rate, an d a rem aining ratio shows the character of the first excursion in Record C.
T he end of the session (R ecord D) shows consistent scalloping following the prim ing
run, b u t no m ore pausing th an ordinarily occurs under V R 360.
Figure 492 shows the entire 3rd session on F R 360. I t begins w ith instances of re
sponding at a high rate im m ediately after reinforcem ent before pausing, as a t a, b, and
c. But these prim ing runs becom e less frequent as the session continues. M ost of
the fixed-ratio segments now show a pause im m ediately after reinforcement. T he
over-all rate of responding falls progressively during the session as the pause an d lower
ra te following reinforcem ent becom e extended. D uring the final 70 m inutes of the
session, only 250 responses occur, a n d the last ratio is not com pleted. M an y of the
ratio segments are identical with a standard fixed-ratio perform ance, as a t d, e, f and g.
T o w ard th e end of the session (low er curves), segm ents begin to show a prolonged
Fig. 4 9 2 .
FR 3 6 0 three sessions a fte r VR 3 6 0
VA RIA BLE R A T IO
409
acceleration w ith rough grain, as a t h a n d i. W hen F R 360 was m ain tain ed for 6
sessions, the tre n d in Fig. 492 continued. T he over-all rate rem ained low, w ith sub
stantial pauses following reinforcements.
C hanging the schedule of reinforcem ent back to variable-ratio elim inated the longer
pauses an d reinstated the high rates im m ediately after reinforcem ent. T he tran si
tion was not im m ediate, however. In Fig. 493 the schedule of reinforcem ent was
changed to V R a t the arrow . A long pause occurs because of the recent F R , a n d
the first reinforcem ent on the variable-ratio schedule occurs a t a. A second reinforce
m ent occurs im m ediately after a few responses. T he reinforcem ent a t b occurs after
7 50 responses a n d is followed by a pause of ab o u t 2 m inutes. A sim ilar pause follows
other reinforcements a t c and d. But by the reinforcem ent at e, the transition to the
variable-ratio perform ance is practically complete, w ith consistent responding im m edi
ately after reinforcem ent. T h e ra te is not yet as high as will be characteristic of
this bird under the final V R . T he rem ainder of the session (Fig. 494) shows the re a p
pearance of pauses and accelerations following prim ing runs after reinforcement. T h e
m arked curvature and extended periods of interm ediate rates appear to be due to the
preceding FR , since further exposure to V R reduced them greatly. T he second bird
h a d shown a considerably lower over-all rate under V R and m uch m ore pausing u n
der FR . In the tran sitio n from F R to V R the change in perform ance was not com
pleted during the first session.
Fig. 4 9 3 .
Transition from FR 3 6 0 to VR 3 6 0
410
A second transition from V R to F R was m ade after 18 sessions of reinforcement on

V R 360, after the transition from F R shown in Fig. 493. A t the arrow in Fig. 495 the
schedule of reinforcem ent is changed to F R 360. T h ro u g h the rem ainder of the ses
sion, the pause after reinforcem ent an d acceleration to a term inal rate develop pro
gressively. T h e bird continues to respond at the term inal rate for from 25 to 75
responses after reinforcem ent, although short ratios are no longer being reinforced.
By the end of the second session, shown in R ecord B, the pause a n d acceleration d u r
ing the ratio segm ent have become m arked, b u t responding a t the term inal rate im
m ediately following the reinforcem ent is disappearing.
VARIA BLE R A T IO
Extinction a fter variable ratio
T h e variable-ratio perform ances described in this ch ap ter differ considerably from

one another, depending upon the particular schedule of reinforcem ent, the length of
exposure to the schedule, a n d the previous history of the bird. E xtinction curves fol
lowing variable-ratio perform ances reflect the prior perform ance. Figure 496 is an
extinction curve taken a few sessions after Fig. 487A an d a few sessions before Fig.
488A, w here the bird responded typically in rap id bursts of from 50 to 75 responses at
a rate separated by pauses of from 60 to 90 seconds. T h e extinction curve in Fig.
496 is roughly negatively accelerated and reaches a low over-all rate of responding after
7000 responses. T h ro u g h o u t the extinction the character of the perform ance from
the preceding variable-ratio reinforcem ent is preserved, with most of the responding
occurring in sustained runs separated by pauses of from 1 to 2 m inutes. T ow ard the
end of the session, longer and longer pauses ap p ear between groups of these repeated
short segments. A nother bird was extinguished 4 sessions after the perform ance
shown in Fig. 474B. T h e curve (Fig. 497) shows a rough negative acceleration reach
ing a very low rate after about 8000 responses. T h e early p a rt of the extinction curve
also consists of sustained responding in short bursts, separated by short pauses. T he
decline in over-all rate as extinction proceeds follows from the increasing length of
pause separating these bursts of responding.
Figure 498 contains an extinction curve after 47 sessions of variable ratios from 15 to
110. T he curve was recorded several sessions before th at of Fig. 480B. A segment
of the perform ance on V R 110 appears before the arrow. T he high rate of responding
th a t this bird eventually developed in Fig. 480 u n d e r a larger m ean has not yet oc
curred. Extinction continues a t essentially the variable-ratio rate for the first 3000
responses. In the segment at a the rate shifts to a somewhat lower value for approx
im ately 600 responses; after this the rem aining 2 \ hours of the session show mostly
300
RESPONSES
VARIA BLE R A T IO
413
long pauses separated by brief periods of responding at approxim ately the original
variable-ratio rate.
Figure 499 shows an extinction curve after a stable perform ance had developed on
V R 173, after a history of 56 sessions w ith various m ean values after erf. T h e record
begins w ith a small portion of the V R 173 performance. This perform ance is followed
by about 5000 responses at the original variable-ratio rate. An abrupt pause occurs a t
a. T he rem ainder of the extinction curve consists of longer and longer pauses, sepa
ra te d by segm ents at the beginning of w hich the bird responds a t n e a r the original
variable-ratio rate. T he transition to lower rates in these segments shows fairly ex
tended curvature ( b). Changes to higher rates are usually fairly ab ru p t, as a t c and d.
W hile these curves differ considerably in detail, they all represent fairly rapid ex
tinction, most of the responses being em itted early w ith the rate falling sharply to
zero. This perform ance contrasts w ith extinction after V I, in which the rate declines
fairly continuously from the original variable-interval rate through interm ediate rates
to a very low value. In Fig. 422, for example, 18,000 responses are em itted at a rate
which is falling continuously. Short-term rate changes also tend to be m ore a b ru p t in
variable-ratio extinction.
THE EFFECT OF SODIUM PENTOBARBITAL O N
A VARIABLE-RATIO PERFORMANCE
Figure 500 shows the effect of an in tram u scu lar injection of 5 m illigram s of sodium
pentobarbital on the perform ance of a bird on V R 360. T h e record contains the com
plete session. T he prevailing perform ance under V R is represented by the first 7 ex
cursions of the record. Follow ing the reinforcem ent a t the arrow the a p p a ra tu s was
shut off, the bird was rem oved from the ap p aratu s, injected, an d replaced, a n d the
ap paratus was started again. T he bird was out of the ap paratus for approxim ately 60
Fig. 5 0 0 .
The e ffect o f p e ntobarbital on VR
414
seconds, during which the recorder did not run. T h e injection produces an im m edi
ate cessation of responding, p robably because of both the injection a n d d rug effect.
(T he effect of the injection per se is very slight, ju d g in g from the effect of a saline injec
tion in Fig. 88. T he bird was on F R 400 w ith added counter and the 0.5-cubic centi
m eter injection im m ediately after reinforcem ent is followed by a pause of
m inutes
before responding begins norm ally. Since pauses usually occurred following the rein
forcem ent, this perform ance can be taken as the m axim um disruptive effect of the
injection perse.) T he first response occurs at a, 38 m inutes after the injection. A nother
pause of 35 m inutes soon occurs; a period of acceleration follows, during which a large
ratio is com pleted. Reinforcem ent occurs a t b, almost 800 responses later. A fairly
norm al variable-ratio perform ance th en appears. T h e break a t c and the long pause
at d possibly indicate some surviving effect of the drug. T he segments a t e, f , g, and
h show average rates of responding of almost 8 responses per second, com pared with
from 4 to 5 responses per second a t the start of the session before the injection. This
perform ance represents an excitatory phase of the drug, which occurs after sub
depressive doses an d some tim e after the m ajor depressive action of larger doses of the
drug have disappeared.
Chapter Eight
TANDEM SCHEDULES
INTRODUCTION
A t a n d e m s c h e d u l e is one in which a single reinforcem ent is program m ed b y 2 sched

ules acting in succession w ithout correlated stimuli. For exam ple, in tand FI 45 FR 10
the 10th response counted after 45 m inutes elapses is reinforced.
T hree types of tandem schedules m ay be distinguished, depending upon the values of
the com ponent schedules. T he first com ponent m ay be substantial and the second
very brief (tan d F R 150 FI 5 sec). T he second com ponent m ay be substantial and the
first very brief (tand FI 10 sec F R 150). O r, both components m ay be substantial (tand
F I 10 FR 150).
T he com ponent schedules m ay be any of those already exam ined. M ost tandem
schedules where both com ponent schedules are of the same sort are trivial. A tandem
F R F R schedule is sim ply a larger fixed-ratio schedule. A tan d em V IV I is usually a
variable-interval schedule w ith at least a slightly larger shortest interval. A tan d em
V R V R is sim ply a variable-ratio schedule w ith a t least a slightly larger smallest n u m
ber. T andem F IF I, however, has interesting possibilities. (T he 2nd interval is as
sum ed to be m easured from the 1st response after the expiration of the 1st interval.)
I f the first effect is sim ilar to a single-interval schedule, the pause developing in the
appropriate perform ance m ay at some point exceed the first of the tandem intervals.
T he result is th en an increase in the next interval, a still longer pause, an d so on. In
this respect it has the effect of tan d F R 1 FI.
W hen a ratio schedule is followed by an interval schedule, we have three im p o r
ta n t cases:
1. T he initial ratio schedule is substantial an d the following interval schedule very
brief (e.g., tan d F R 350 FI 5 sec). This differs from F R 350 alone in th a t it elim inates
the differential reinforcem ent of high rate associated w ith FR . T h e F I com ponent is
so short th a t the n u m b e r of responses from reinforcem ent to reinforcem ent does not
vary enough to greatly modify the fixed n u m b er of responses a t reinforcem ent.
2. T h e initial ratio schedule has a small value and the 2nd-interval com ponent is
substantial (e.g., ta n d F R 10 F I 10). H ere, the clock which times the fixed-interval
schedule starts only after the responses in the small ratio have been emitted.
3. Both the initial fixed-ratio schedule an d the following interval schedule have sub415
416
stantial values (e.g., F R 100 F I 5). T his is roughly equivalent to a variable-interval

schedule with a larger smallest interval of reinforcement. T he substantial initial fixedratio com ponent, however, elim inates the regenerative feature present in interval
schedules, in w hich single responses begin to be reinforced w henever the rate becomes
very low.
W hen the 1st com ponent is interval and the 2nd com ponent is ratio, we have 3 m ajor
cases:
1. T h e beginning interval schedule is substantial, a n d th e following ratio schedule
has a small value (e.g., F I 10 F R 5). This schedule provides a technique for adding a
differential reinforcem ent of high rates to an F I schedule w ithout otherwise distu rb
ing the m ajor properties of the schedule.
2. T he initial interval schedule is brief, a n d the tandem fixed-ratio schedule is sub
stantial (e.g., FI 30 sec F R 200). T his is a schedule in w hich the actual contingencies
a t reinforcem ent are very sensitive to the behavior generated. Responses m ade d u r
ing the short interval increase the effective ratio u ntil w ith a very large ratio, a pause
develops after reinforcem ent. Few er responses will th en be required a n d the sched
ule should stabilize a t some value w here the pause after reinforcem ent is roughly the
sam e as the interval in the initial com ponent. But pausing in a strain ed ratio is a
very subtle feature of the performance.
3. Both the initial interval schedule a n d the following ratio schedule are substantial
(e.g., tan d F I 5 F R 200). This is substantially a V R w ith a large smallest ratio of re
sponses. T he total num ber of responses em itted per reinforcem ent is again sensitively
related to the pause after reinforcement.
T h e differential reinforcem ent of rates (C h ap ter N ine) supplies special cases which
m ight be considered tandem schedules. For exam ple, a schedule in which a response
will be reinforced at the end of an FI w henever the rate satisfies a d rh contingency
is essentially a case of tan d F/crfdrh.
TANDEM FIXED-INTERVAL FIXED-RATIO SCHEDULE
Tand FI45 FRI O
Development to a fin a l performance. Tw o birds were placed on tan d F I 45 F R 10 whose
transition from erf to F I 45 has already been described in C h a p te r Five. T he p e r
form ance of one bird will be described in detail, and th a t of the second used to confirm
th e m ain points of the experiment. Figure 501 shows the transition from FI 45 to tan d
FI 45 F R 10. T he perform ance of FI 45 was not well-developed. T he bird was show
ing a low over-all rate of approxim ately 0.2 response per second, w ith rough grain,
an d little or no consistent scalloping. In the 2nd session following (shown complete in
Fig. 502 except for the first hour) the tandem ratio has produced both local and over-all
changes. T he grain now has a stepwise character; the bird responds at about 3.5 re
sponses per second in short bursts of about 10 responses. T he over-all rate increases
from 0.2 to 0.6 response per second. Even a t the highest rates the performance consists
TA N D EM SCH EDULES
417
of short bursts of responses. No evidence exists yet of the development of an interval

scallop. Figure 503 shows 45-m inute segments, one each from the 3rd, 4th, 5th, and
6th days. R ecord A is from the first hour om itted in Fig. 502. Records B and D
exhibit a slight tendency tow ard interval scalloping. Records A and C, however, show
a roughly linear performance. Fifteen sessions later, after a total of 170 hours of re
inforcem ent on the tan d em F I 45 F R 10 schedule, th e over-all rate of responding has
been increased and the scallop has become prom inent. This stage is shown in Fig.
504, which contains one 45-m inute segment from each of 4 successive sessions. In R ec
ords A and B the over-all rate accelerates rapidly until the m iddle of the interval, where
it begins to decline to a lower term inal value. In R ecord C the acceleration is slower
b u t continuous, reaching a final over-all rate of approxim ately 0.6 response per sec
ond. In R ecord D the acceleration is rapid to a n over-all rate of responding of the o r
der of the high rate reached in the m iddle of the interval in Records A and B, which
is m aintained throughout the interval, except for slight local changes a t a a n d b. All
of these over-all interval performances are executed even though the local rate is dis
continuous. A later perform ance on tan d F I 4 5 F R 10 (after 290 hours of reinforce
m ent) is illustrated in Fig. 505 and 506, which comprise a complete session. In Fig.
505 a period of from 5 to 10 m inutes follows each reinforcem ent (except a t c) during
which the rate increases to a term inal value of approxim ately 2.5 responses per second.
10 MINUTES
TA N D EM SCH ED U LES
Fig. 5 0 5 .
419
Tand Fl 4 5 FR 10 a fte r 2 9 0 hr
T h e term inal rate is m aintained for the rem ainder of the 45-m inute interval. At a,
b, and f there is a fairly ab rupt shift to the term inal rate, while at d, e, and elsewhere,
the over-all curvature is smooth. T he curvature results from a series of increasingly
shorter pauses between bursts of responses. Even a t the term inal rate, the fine grain of
the record consists of small bursts separated by pauses, although by this tim e the pauses
are of the order of from 2 to 5 seconds. T h e running rate, therefore, is considerably
higher th a n the over-all rate of about 2.5 responses per second. A rate of 8 responses
per second m ay be reached for runs of 25 to 50 responses.
T ow ard the end of the session (Fig. 506) the over-all and term inal rates are lower.
T he running rate in each burst, however, is little changed.
T he last 4 recorded sessions on ta n d FI 45 F R 10 are represented in Fig. 507 by the
3rd interval segment from each session. T he curves a t Records A and B contain 3300
and 4200 responses, respectively, and have been broken for better reproduction. T he
term inal rate is high until the end of the interval in R ecord A, b u t falls somewhat in
Record B. Prolonged interval scallops are evident in Records C and D. T he smooth
over-all increase in rate during the first p art of the fixed-interval segment is accom-
420
Fig. 5 0 7 .
Segments from the last fo u r sessions on tand FI 4 5 FR 10
plished by the shortening of the pauses betw een bursts of responses. T he sustained
rap id responding during the latte r p a rt of each interval is composed of short bursts of
very fast responding, separated by pauses of from 1 to 2 seconds.
T he second bird showed a sim ilar first effect of the tan d em F R 10. T he transition
occurred after 30 hours of reinforcem ent on FI 45, in which the final perform ance was a
low over-all b ut steady rate of approxim ately 0.03 response per second. Figure 508
shows the entire 1st session on tan d F I 45 F R 10. Because of the very low over-all rate,
th e ta n d F R 10 postpones m any reinforcem ents far beyond the designated interval.
T he reinforcem ent a t a, for exam ple, occurs 87 m inutes after the previous reinforce
m ent, an d th a t at b occurs 59 m inutes after. T he perform ance developed m uch m ore
slowly th an w ith the first bird. After 162 hours (Fig. 509), the curves are composed
alm ost en tirely of runs of from 10 to 30 responses a t from 3 to 4 responses per second,
separated by pauses of approxim ately 1 m inute. H igh over-all rates occasionally
em erge in which the long pauses separating bursts of responses are absent. T he
T A N D E M SCHEDULES
421
TH
~1
Fig. 5 0 8 .
First session on tand Fl 4 5 FR 10 (second bird)
change is gradual, however, an d the over-all curvature is fairly smooth, as a t a, b, c,

a n d d. A slight tendency tow ard the developm ent of interval curvature is evident.
Figure 510 shows the final recorded perform ance after a total of 754 hours on this
schedule. T he bursts of responses a t from 4 to 5 responses per second are now shorter
a n d usually separated by shorter pauses. T he over-all rate has declined from the
earlier performances. Little, if any, consistent interval curvature exists.
Transition from tand F I 45 F R IO to F I 45. In the session following R ecord D in Fig.
507 the ta n d F R 10 was rem oved, the b ird being reinforced simply on FI 45 for a total
Fig. 5 1 0 .
Tand FI 4 5 FR 10 a fte r 7 5 4 hr (second bird)
422
of 10 sessions. Figure 511 shows the entire last session after 61 hours of reinforcem ent
on FI 45. T he removal of the tan d F R 10 has reduced the over-all rate. T he high
est term inal rate is now of the order of 1.25 responses per second, com pared w ith 2.5
responses per second in Fig. 505. T h e acceleration d u ring the early p a rt of the inter
val is occasionally very prolonged, as at a an d e. T h e whole record gives the a p p e ar
ance of m ore curvature extending over longer periods of time. Substantial responding
m ay occur im m ediately after reinforcem ent, as a t b. In th a t case the term inal rate
is reached very quickly, but m ay give way to lower rates with fairly smooth negative
cu rv atu re. A second acceleration before the reinforcem ent frequently appears.
W hile th e grain of the record d u rin g the period of low or interm ediate rates of re
sponding still consists of pauses sep arated by runs, this perform ance has becom e less
pronounced; a n d instances now occur, a t c an d d, w here responding is sustained w ith
out pauses.
W hen the tan d e m F R 10 is again ad d e d to the schedule, the previous perform
ance on this schedule is quickly recovered. Figure 512 shows the 11th session, 72
hours after the schedule was changed a second tim e to ta n d F I 45 F R 10. T h e over
all rate of responding a n d especially the term inal rate of responding in the interval
have increased to the sam e ord er of m ag n itu d e as before. T h e tendency to respond
im m ediately after the reinforcem ent is m arked, even though this responding is fol
lowed by a long pause a n d acceleration as in a norm al fixed interval. Occasionally,
the term inal rate of responding is sustained throughout the whole 45-m inute period,
as at a, alth o u g h most segments show a long pause followed by an eventual accelera
tion to a term inal rate.
Tand F I4 5 F R 10 with TO 15 after reinforcement. A 15-m inute T O after reinforcem ent
was ad d ed to the schedule in the session im m ediately following Fig. 512. T he re
sult (Fig. 513) is a m arked decline in the over-all rate. From 10 to 15 responses u su
ally occur ju st after the term ination of the T O 15, as a t a, b, c, and e; an d following
Fig. 51 3.
First effect o f TO 15 a fte r reinforcem ent on tand FI 4 5 FR 10
424
the last reinforcem ent in the figure, a t f responding is fairly sustained. O nly at d does
th e tim e out have the effect of elim inating responding a t the beginning of the in ter
val, as in the experim ents reported in C h ap ter Five. T h e tim e o ut does, however,
postpone the developm ent of the term inal rate.
These effects are tem porary, however. By the 2nd session on tand FI 45 F R 1T O 15
(Fig. 514), the over-all rate of responding has increased to the earlier order of m agni
tude, an d fairly consistent scallops occur in most of the fixed-interval segments. Some
responding begins soon after reinforcem ent; a n d most interval segments show a fairly
sm ooth acceleration to a term inal rate, w hich is m ain ta in e d u ntil the reinforcem ent.
O ne m arked deviation occurs a t a, w here the term in al rate is reached early. T he
ra te shifts ab ruptly to a very low value, a n d a new scallop begins. T he term inal rate
Fig. 5 1 4 .
Second session tand FI 4 5 FR 10 TO 15
is reached a second tim e, before reinforcem ent a t b. Seven sessions later, after a total of 69 hours of reinforcem ent on the tan d em schedule w ith tim e out after reinforcem ent, th e effect has becom e m ore consistent (Fig. 515). H ere, a reasonably long
pause occurs after reinforcem ent a n d a sm ooth over-all acceleration to a term inal
rate , w hich is m ain ta in e d steadily u n til the next reinforcem ent. Especially good
exam ples of extended and sm ooth curvature occur tow ard the end of the session, at
a, b, a n d c. T h e acceleration is caused by a change in the d u ratio n of the pause
ra th e r th an in the actual local rate.
T h e pause after reinforcem ent and the period of acceleration to the term inal rate
of responding become longer as the experim ent continues. Figure 516 shows the p er
form ance 4 sessions later. T h e over-all ra te of responding has fallen slightly, b e
cause of the longer pauses and m ore extended interm ediate rates.
426
T he final recorded perform ance

u n d er ta n d FI 45 F R 10 T O 15 was shown in Fig. 516. In the following session the
ta n d e m F R 10 h a d been rem oved. T h e schedule h a d now becom e F I 45 T O 15.
This schedule was m aintained for 10 sessions, during w hich no noticeable change oc
c u rre d in th e perform ance. T h e e n tire 10th session is show n in Fig. 517, w hich
closely resembles Fig. 516. In the transition from ta n d F I 45 F R 10 to F I 45 w ithout
tim e out, how ever, both the over-all ra te a n d the c h a ra c te r of the curves changed
in the direction of the usual FI 45 perform ance. Possibly, further exposure to the
FI 45 reinforcement in this experim ent would have produced a lower rate of respond
ing and rate changes m ore characteristic of a n F I 45 perform ance. A difference is to
be expected from the higher term inal rates in the present experim ent. These rates
m ake it less likely th a t reinforcem ent will occur after a pause. T h e m arked run-andpause c h aracter of the tan d em perform ance w ithout tim e out m akes it m ore likely
th a t reinforcem ent will be set up during a pause.
T h e final perform ance of the second bird on tan d F I 45 F R 10, already described
in Fig. 510, showed little evidence of interval curvature after 754 hours of exposure to
the tandem schedule. A m arked pause and scallop appeared when a 15-m inute T O
was added after reinforcem ent. Figure 518 shows the entire 2nd session w ith T O 15.
Transition from tand F 1 4 5 F R 10 TO 15 to F I 45 TO 15.
RESPONSES
300
Fig. 5 1 8 .
S econd session ta n d FI 4 5 FR 10 TO 15 (second bird)
428
T h e fixed-interval segments, w hich have been arra n g e d in order of the n u m b er of

responses in the interval, show th a t the over-all rate is roughly the same as before tim e
out; but the term inal or highest sustained rates of responding are considerably higher.
Pauses after reinforcem ent vary from 9 m inutes a t a to 19 m inutes a t b. Figure 519
shows the final recorded effect of the T O , 10 sessions later. T he highest term inal rate
has increased to alm ost 2 responses p er second, com pared w ith from 1.2 to 1.5 re
sponses per second in Fig. 518. Reinforcem ents are still uniform ly followed by pauses,
a n d the fine grain of the record consists of short bursts of responding a t from 3 to 4
responses per second.
Extinction after tand F I 45 F R 10. T he response was extinguished 30 hours before the
final perform ance of tan d FI 45 F R 10 shown in Fig. 512. Figure 520 contains the e n
tire extinction curve. It begins with the slow acceleration to the term inal rate of r e
sponding at a which norm ally characterizes the start of the session. T he term inal rate
is sustained for alm ost 8000 responses before it falls off, a t b. For the rem ainder of
Fig. 5 2 0 .
Extinction a fte r tand FI 45 FR 10
T A N D E M SCH ED U LES
429
the curve the bird oscillates betw een interm ediate rates of responding, consisting of
bursts of from 10 to 30 responses separated by pauses (as in the segments a t c and e),
a n d periods of m ore sustained responding a t rates a p p ro ach in g the form er term in al
ra te at d a n d f . T h e h ig h er ra te is achieved by th e shortening of th e pauses b e
tw een bursts.
A second extinction curve taken in the session following Fig. 517 confirms the fact
th a t the perform ance still retains the effect of the earlier tan d e m ratio. Figure 521
shows th e curve. It begins w ith the no rm al acceleration to a term in a l ra te at a,
w hich is sustained for 20,000 responses until it falls off at b. T he rem ainder of the
curve is characterized by long periods of zero or near-zero responding (m any of which
are o m itted from the figure), sep arated by periods of responding a t n e a r-te rm in a l
rates for a few thousand responses. T he fine grain of the record shows no interm ediate
ra te of responding. T h e responses w hich occur in bursts show a rate of th e order
of 5 responses per second. Except for the longer pauses, this curve resembles Fig. 520,
tak en after tan d F I 45 F R 10 w ithout T O .
T he extinction curve after tan d FI 45 F R 10 for the second bird appears greatly re
duced in Fig. 522. T he rate declines fairly steadily for the first 3 or 4 hours, and
thereafter the level of perform ance becomes relatively stable. T he grain th ro u g h
o ut shows th e effect of the ta n d e m ratio. A sam ple tow ard the end of the curve has
been reproduced on the usual scale.
Large FR fo llo w in g ta n d F I4 5 F R
Following the experim ents on tan d FI 45 F R 10 described above, the tandem ratio
was progressively extended u p to F R 400. A lthough this extension postponed rein
forcem ent, the term inal rate was so high an d so consistent th at little relative change
in the 4 5 -m inute interval was produced. T h e perform ance rem ain ed unch an g ed .
M any thousands of responses per reinforcem ent were being em itted. A ratio of this
m agnitude could not be held on a straight F R basis. In order to investigate the special
powers of this tandem schedule to sustain this level, we changed the schedule to F R
875, the T O rem aining in force. Now the bird could com plete the ratio required for
reinforcem ent in a short tim e. Figure 523 shows the perform ances for b oth birds.
R ecord A follows 20 reinforcem ents on F R 875 T O 15, a n d R ecord B, 17 reinforce
ments. T h e periods of acceleration before the term in al rate is reached are effects
of the earlier interval com ponent. T h e term in al ra te is of the same ord er as before,
an d relatively low com pared w ith F R perform ances reached directly from erf.
Sim ilar performances were m aintained for a total of 22 hours. T he T O 15 was then
rem oved. T he ratio continued to be h eld for the rem aining 77 hours of the ex
perim ent, though pauses increased an d the general picture closely resem bled a largerratio perform ance. T h e second bird sustained the ratio less successfully. (Cf. Fig.
113 a n d 114 for the later perform ances of the bird in Fig. 522, a n d Fig. 115 for th at of
th e other bird.)
T A N D EM SCH ED U LES
Fig. 5 2 2 .
Fig. 5 2 3 .
431
Extinction curve a fte r tand FI 4 5 FR 10 in reduced form
Early perform ance on FR 8 7 5 TO 15 a fte r tand FI 4 5 FR 10 TO 15
Tand F I 8 F R 3 and ta n d F I 1 0 F R 3
W e studied the effect of a sm all, tan d e m fixed ratio ad d ed to a shorter fixed in

terval w ith 2 birds w ith the following history: (1) erf to FI. This has been described
in Fig. 166 and 167. (2) F R w ith ratio determ ined by the m ean num ber shown in
th e interval. T he birds failed to develop ratio perform ance. Pauses were extended
indefinitely. (3) FI. Recovery of the earlier perform ance, one bird on FI 8, the other
on FI 10.
A tan d e m ratio of only 2 responses was th en added. A fter 6 sessions, this was
changed to 3 responses. For convenience, the schedule will be called F I F R 3 through
out.
Figure 524 shows representative parts of 4 sessions a t various stages in the devel
opm ent of the perform ance on ta n d FI 8 F R 3 a n d of a 5th session after the re tu rn to
FI 8 alone. T he interval segments are in the recorded order. Records A, B, G, and
D contain 8 successive interval segments at 25, 45, 80, and 150 hours of exposure to the
tandem schedule. T he term inal rate, the rapidity of the acceleration to the term inal
rate, a n d the pauses after reinforcem ent all increase progressively d u ring this period.
W hen th e tan d e m ratio was rem oved, how ever, th e tre n d continued. R ecord E
shows segments from the 16th session, or 115 hours after the tandem ratio was dropped.
T he term in al rate is ap p ro x im ately 2.5 responses per second, a n d pauses of 3 to 5
Fig. 5 2 4 .
D evelopm ent on tand Fl 8 FR 3 and Fl 8
TA N D EM SCH ED U LES
433
m inutes follow the reinforcem ents. A lthough we cannot be sure th a t the progressive
change in the c h aracter of the perform ance in Records A, B, C, a n d D was due to
th e ta n d e m ratio ra th e r th a n sim ply a result of th e continuous exposure to th e fixed
interval, the perform ance in R ecord D with its high term inal rate is unusual.
T he perform ances in Fig. 524 represent various stages of the experim ent fairly ac
curately. Deviations in the form of knees sometimes occurred, however. Some of
these are shown in Fig. 525, which gives successive interval segments from 4 sessions
after the return to FI 8.
Figure 526 presents an extinction curve after 95 hours of reinforcem ent on tan d FI 8
F R 3, 3 sessions after R ecord C in Fig. 524. A continuous decline in over-all rate is
m arked by a strong oscillation. No prolonged pauses occur in the curve, even when
the over-all rate of responding reaches a very low value. T he curve suggests th a t the
tandem com ponent has had a more m arked effect than appears in the tand FI 8 F R 3
perform ance. T he grain is ratio-like, an d the perform ance is sustained m uch beyond
extinction after FI.
Figure 527 shows a second extinction curve, tak e n shortly after R ecord D in Fig.
524, w hen the term in a l ra te a n d pause after reinforcem ent on F I 8 h a d increased
considerably. T h e sm ooth oscillation betw een low an d high rates is even m ore
m arked. T h e oscillation is betw een the n ear-zero a n d high term in a l rates of the
ta n d F IF R perform ance. N ote th a t 4 hours in which 840 responses occurred are
om itted from the record.
T he second bird, on tan d FI 10 F R 3, also showed a progressive increase in the pause
after reinforcem ent and in the term inal rate. T h e final tandem perform ance shows
a large pause, a higher term inal rate, a n d m ore responses per interval th a n a norm al
F I perform ance. T h e effect is again not reversible. Figure 528 presents the final
perform ance on FI 10 after 370 hours on F I 10, tan d F I 10 F R 3, and FI 10. Except for
occasional knees, at a, c, d, an d e, the bird pauses or responds a t a very low rate for the
greater p art of the interval a n d then changes a b ru p tly to the term inal rate. T he te r
m inal rate is between 3 and 4 responses per second, and the pauses following the re
inforcem ent m ay be as long as 8 m inutes (at b, for example). M any of the segments
are sim ilar to those under a fixed interval w ith clock. T he clock perform ance is less
variable. (N ote th a t the perform ance in Fig. 528 is w ithout benefit of T O .) T he ir
reversibility of the tandem effect is not surprising. W hen a term inal perform ance of
this sort is m aintained, w hether the last 3 responses are tim ed or counted is of little sig
nificance. W e could have em phasized the missing tandem ratio com ponent by dif
ferentially reinforcing low rates and thus slowing dow n the whole performance.
RESPONSES
300
Fig. 5 2 5 .
Deviations from a smooth interval perform ance

434
Fig. 5 2 6 .
Extinction a fte r tand FI 8 FR 3
436
T A N D E M FIXED-RATIO FIXED-INTERVAL SCHEDULE
In the tan d F R F I schedule the reinforcem ent is based on a fixed num ber of re
sponses, but the com pletion of this n u m b er does not produce reinforcem ent; instead, it
starts a clock which, after an interval, sets up a reinforcem ent. T he next response
is reinforced. It provides a technique for elim inating the differential reinforcem ent of
high rates associated w ith fixed-ratio reinforcem ent while m aintaining an ap p ro x i
m ately constant fixed num ber of responses per reinforcem ent. If the tandem fixed in
terval is only a few seconds long, the variation in the n u m b er of responses from fixedratio segm ent to fixed-ratio segm ent will be slight. B ut the relation betw een rein
forcem ent a n d im m ediately preceding responses will be th a t of an interval schedule,
where the probability of reinforcem ent increases w ith tim e since the last response.
T he likelihood of reinforcem ent after any given pause is slight, however, since only
responses are reinforced which occur after pauses beginning after the fixed ratio has
been counted out. A schedule of this type is im p o rtan t in an analysis of the high rate
in a fixed-ratio schedule.
E xperim ent I
1. Early development after erf In a first experim ent a brief interval of 10 seconds
was ad d ed to a fairly large ratio of 240. T h e schedule was p u t in force im m ediately
after erf. T h e b ird p ro b ab ly could not have m ad e th e tran sitio n d irectly to F R
240. T he tandem schedule, however, takes im m ediate effect.
Figures 529 an d 530 comprise the entire (13 hour) 1st session of the transition
from erf to ta n d F R 240 F I 7 sec. Figure 529 begins w ith a succession of negatively
accelerated curves, common in transitions to F I an d F R after erf. T he over-all rate
increases g rad u a lly d u ring th e first 8 reinforcem ents, to a, after w hich an y fu rth er
change is only m oderate. T he over-all rate reached approxim ately 0.75 response per
second by the last segment of the figure. D uring the rem aining 9 hours of the ses
sion (Fig. 530), both the over-all a n d local rates of responding increased progressively,
reaching 1.9 a n d 2.5 responses per second, respectively, by the last excursion of the
record. Pauses began to develop after reinforcem ent, as a t a, b, a n d c, th o u g h they
disappear tem porarily in the region of d. In the m iddle p a rt of the figure the
rate oscillates a n d some curv atu re occurs d u rin g m any of the ratio segments. Some
responding occurs im m ediately after reinforcem ent a t e, before the pause, an d accel
eration frequently leads to the term inal rate. C u rv atu re in a norm al fixed-ratio p e r
form ance usually foreshadows long pauses after reinforcement, and two of these occur
at f and g, 167 m inutes an d 34 m inutes long, respectively. An almost standard fixedratio perform ance emerges by the end of the session. T he last reinforcem ent of the
session was followed by a 40-m inute pause, after which the session was term inated.
W hy the tandem interval helps the bird to m aintain a substantial level of respond
ing u n d er this large fixed ratio is not clear. T he very late developm ent of a high te r
m inal rate m ay be the explanation. N orm ally, a high term inal rate is quickly
437
Transition from erf to tand FR 2 4 0 FI 7 sec
reached. This produces pausing, possibly because of the contrasting count just after
or ju st before reinforcem ent. If the sm all tan d em interval delays the developm ent of
the high term inal rate until the ratio schedule is m ore effective in establishing count
as reinforcem ent, the present result w ould follow.
T he second bird in the transition from erf to F R 240 FI 7 showed too low a rate to
p roduce even 1 reinforcem ent on the ta n d e m schedule d u rin g the first 2 hours of the
session. T h e size of the fixed-ratio com ponent of the tandem schedule was therefore
reduced to 80, and then later increased to 140. Both the local and over-all rates of re
sponding increased during the first 6 sessions, resulting in the perform ance shown in
Fig. 531 for th e 6th session after co ntinuous reinforcem ent on ta n d F R 140 F I 7 sec.
T he term inal rates of responding are characteristically betw een 2.5 a n d 3 responses
per second, although they vary considerably, w ith instances as low as 1.4 responses per
second, as at a. A pause after the reinforcem ent is sometimes absent, as at c. T h e
change from pause to term inal rate varies from the sm ooth c u rv atu re a t b and d to an
438
a b ru p t shift, as at e, or an a b ru p t shift to an interm ediate rate, which then acceler

ates to the term inal rate, as a t f . Figure 532 shows the perform ance 13 sessions later,
after the size of th e fixed ratio has been increased to 200. T his perform ance was
fairly stable a n d was m aintained for the next 30 sessions. Pauses after reinforcem ent
vary from a few seconds, as at d, to 12 m inutes, as a t a. T he term inal rate of respond
ing is stable a t a b o u t 2.8 responses per second. M ost of th e changes from pause to
term in a l ra te are now relatively a b ru p t. T h e perform ance differs from a sim ple
fixed-ratio schedule in the frequent appearance of interm ediate rates of responding, as
at b a n d e, in th e failure to m ain ta in th e term in a l rate once it is reached, as a t c, a n d
in the grain of the term inal rate, as a t e.
T h e pause after reinforcem ent grad u ally disappeared d u rin g 3 sessions 30 hours
later, w hen the fixed-ratio com ponent of the tandem schedule was reduced to 80. Fig
ure 533 shows the 3rd session on tan d F R 80 FI 7 sec. T he term inal rate of respond
ing rem ain s u n c h a n g ed , a n d sustained in te rm e d ia te rates, as a t a a n d b, still occur.
439
N ote th e S -shaped curve a t c. T h e b ird late r failed to m a in ta in a sustained p e r

form ance on the tandem schedule, although the fixed-ratio com ponent was reduced to
as low as 60. D u rin g th e last sessions of the exp erim en t, not enough responding
occurred to produce reinforcem ents. T h e schedule was th en changed to ta n d F R 10
F I 7 sec in o rd er to restore the response. Figure 534A gives the result. T h e first
fixed ratio was com pleted at a, w ithin the first half-hour of the session. A second rein
forcem ent occurs a t b, 85 m inutes later. T h e size of th e fixed ratio was th e n in
creased to 20 for the rem ainder of the session. T he rate of responding increased during
th e next 8 segments. D uring the next 9 sessions of the experim ent, the size of the
ratio com ponent of the tan d em schedule was g radually increased to 100. R ecords B
th rough J show the first 10 reinforcem ents of each session. A pause after reinforce
m ent develops as the size of the ratio is increased, a n d the term inal rate of responding
increases to from 2 to 3 responses per second. Even in R ecord J , however, the local
T A N D E M SCH EDULES
441
rate varies considerably, from a p p ro x im ately 2.2 responses p er second a t c to 1.7 re

sponses at d.
In this second bird the tandem 7-second interval seems to have given little if any help
in m aintaining a ratio perform ance.
Later development. T h e first bird in the experim ent was m aintained on tan d F R F I
for m ore th a n 60 sessions, during w hich the fixed ratio varied between 170 and 335.
T he perform ance shown at the end of the session represented in Fig. 530 continued es
sentially unchanged for 11 sessions except for a slight increase in the rate of respond
ing. T h e size of the fixed ratio was then increased to 335. D uring the 3rd session of
this ta n d F R 335 F I 7 sec, sustained responding a t 4 responses per second suddenly
occurred, w ith no pausing after reinforcem ent and few deviations from a constant local
rate. Figure 535 shows the session. In co m p arin g this figure w ith Fig. 529 a n d
530, note th a t the recorder scale has been changed. A given slope in these first 2 fig
ures represents a low er rate th a n th a t in th e figures for the rest of the experim ent.
T h e bird h ad been showing a perform ance sim ilar to the first few excursions of the rec
ord, w ith only an occasional emergence of sustained responding through several ratio
segm ents. B rief responding a t a n in te rm e d ia te ra te w ith ro u g h g rain continues to
occur occasionally a t a, b, a n d c. R a te changes of this ord er are rare on a sim ple
fixed-ratio schedule except im m ediately after reinforcement.
D uring subsequent sessions on ta n d F R 335 FI 7 sec, pausing after reinforcem ent
increased; a n d in th e 9th session a perfo rm an ce such as th a t in Fig. 536 developed.
R ecord A shows th e first 2 hours of th e session, R ecord B, th e 5th a n d 6th hours.
T h e pause after reinforcem ent increases progressively thro u g h o u t the session, leading
occasionally into sm ooth and extended curvature, as a t c, a n d in any case to extended
scallops. A term inal rate of responding of the order of m agnitude of 6 responses per
second is usually reached before reinforcem ent, b u t not very long before. A t d kn in
term ediate rate is sustained in a knee, a n d at a a n interm ediate rate of respond
ing is m aintained until reinforcem ent. Occasionally, an a b ru p t shift occurs from the
pause after reinforcem ent to an interm ediate rate, as a t b and e.
442
P au sin g a n d heavy scalloping co n tin u e to develop in th e following session, all 15

hours of which are shown in Fig. 537 to 540. T ow ard the end of Fig. 537, the seg
m ents at a, b, a n d c show longer pauses an d m ore extended curvature th an those in
the previous session (Fig. 536). Figure 538 shows the 29th through 52nd reinforce
ments, w here the over-all rate rem ains about the same, although the individual seg
m ents show a variety of rate changes, including 2 m arked knees a t a an d b. By the
53rd reinforcem ent in Fig. 539, the pause following the reinforcem ent becomes m uch
m ore extended th an heretofore, producing the lowest over-all rate yet seen. Except
Fig. 5 3 7 .
Progressive developm ent o f pause and curvature
TA N D E M SCH EDULES
Fig. 5 4 1 .
443
Later perform ance on tand FR 3 3 5 FI 7 sec showing developm ent o f pause

and curvature
for the segments a t a a n d b, where a term inal rate of responding is never reached, and c,
where the rate falls off just before the reinforcem ent, all segments accelerate to a rate
of m ore th a n 4 responses per second before the end of the ratio. T he low over-all rate
continues in the final p a rt of the session, which is shown in Fig. 540. T he segments
beginning a t a, b, a n d c show a very prolonged period of acceleration to the term in al
rate, w hich has fallen to the ord er of 3 responses per second. A fter the reinforce
m ent a t d, 40 m inutes elapse before the bird begins to respond. T he 75 reinforce
ments received during the entire 15-hour session probably did not produce any sub
stantial satiation to explain the decline in the over-all rate.
T he over-all rate increased in the following session, w ith a perform ance sim ilar to
th a t in Fig. 535, except for slightly longer pauses after reinforcem ent. Five sessions
later, however, pausing and m arked scallops reappear tow ard the end of the session.
In the next session, shown com plete in Fig. 541, the perform ance begins norm ally b ut
goes through a decline in over-all rate sim ilar to the changes represented in Fig. 537
th rough 540. By the end of the session the bird frequently fails to reach the term inal
rate, a n d long periods occur in which the rate is zero or n e a r zero.
T he character of the perform ance on the tan d FRF1 was explored a t lower values
444
of the ratio com ponent. Occasionally, the bird sustained a daily session of 60 rein
forcem ents w ith very little decline in over-all rate, b u t a m ore characteristic perform
ance was a progressive decline resulting from the elongation of the pause after
reinforcem ent, a n d to some extent by a m ore extended period of curvature. Figure
542 contains a representative sam ple of this perform ance, w hen the schedule of rein
forcem ent was ta n d F R 170 F I 7 sec, 30 sessions after th e fixed-ratio com ponent h a d
been reduced from the F R 335 of Fig. 541. T he schedule of reinforcement was then
changed to F R 170 w ithout a tan d em interval. Figure 543 shows an entire daily ses
sion, 250 reinforcem ents later. T h e over-all ra te still falls off d u rin g th e session.
T h e elim ination of the tan d em interval has increased the local rate of responding a n d
th e n um ber of ratio segments sustained w ithout pausing. C u rv atu re an d in term e
d iate rates of responding are reduced. T o w ard th e end of the session, for exam ple,
w here nearly all segments in the ta n d F R F I showed m arked curvature, instances of
a b ru p t shifts to the term in al rate occur (as a t f a n d g ). T h e perform ance still shows
effects of the tan d em schedule in the knees a t a, b, an d c, the interm ediate rate a t d,
a n d occasional m arked curvature, as a t e.
TA N D E M SCH ED U LES
445
E xp e rim e n t II
In a second experim ent on ta n d F R F l the size of the fixed-ratio com ponent was
m ain ta in e d a t values w hich did not produce m arked pausing or prolonged curvature.
Figure 544 illustrates a tran sitio n from e rf to ta n d F R 150 FI 6 sec. T his is sim ilar
to the 1st transition observed in the previous experim ent, except th a t the grain is some
w h at rougher. T he transition begins w ith negatively accelerated curves after rein
forcem ents (at a, b, an d c) leading into a linear phase (at d). H igh rates after
reinforcem ent retu rn a t e , f g, a n d h. T h e end of the session is roughly linear, except
for one break-through at i. T h e over-all rate reaches approxim ately 0.75 response
per second by the last excursion, a n d b rief pauses a p p e a r after the reinforcem ents at j ,
k, a n d I.
T h e 2nd session on th e ta n d e m schedule show n in Fig. 545 begins a t ro ughly the
sam e rate as th a t a t the end of the preceding session. A pause of from 5 to 10 sec
onds frequently follows reinforcem ent. By the 3rd excursion, pauses ap p ear consist
ently, a n d the acceleration to the term inal rate becomes m ore extended. M arked
scallops occur at a and b; an d in a final segment, a t c, a very low rate for approxim ately
15 m inutes is followed by a 55-m inute pause, after which the session was term inated.
T he over-all rate of responding passes through a m axim um in the m iddle of the session,
w ith the highest term inal rate of responding slightly over 1 response per second. By
the end of the session, both over-all an d term inal rates fall to about the sam e values as
those at the start of the session.
This bird was not able to m aintain a sustained perform ance on tan d F R 150 FI 6 sec.
T h e perform ance shown in Fig. 546 for th e end of the 3rd session shows a co n tin u
ing decline. N ote th a t in spite of the low over-all rate a n d the m arked curvature, the
term in al rates are still 0.75 response per second, as in the previous 2 sessions. T he
next session shows a progressive increase in curvature an d pausing and a lower over-all
446
rate. Tw o sessions later, responding essentially ceased. T he fixed-ratio com ponent

of the tandem schedule was th en reduced to 50. Figure 547A shows the entire 1st ses
sion, w hich begins w ith only a slight pause after reinforcem ent a n d a constant rate
du ring each ratio segment. T h e term inal rate increases from 0.9 response per second
at a to 0.95 response per second at b, 1.5 responses per second a t c, 1.7 responses per
second a t d, a n d the highest value in the session 2.2 responses per second a t f .
T hereafter, the term inal rate falls to the order of 1.25 responses per second. T he high
est over-all rate occurs in the excursion at e. T h e decline in over-all rate beginning
a t g is in p a rt probably a result of satiation from the 180 reinforcements received during
TANDEM SCHEDULES
447
the session. (Sixty reinforcem ents ordinarily keep the bird at a constant daily
weight.) T h e start of the next session (R ecord B) shows occasional instances of curva
ture to the term inal rate, as a t h a n d i, and S-shaped segments, as at j.
T h re e sessions later, the fixed-ratio com ponent of the tan d e m schedule was in
creased to 100, an d the pause following the reinforcem ent a n d the period of accelera
tion to the term inal rate of responding becam e longer. T he various types of
perform ances present at this stage ap p ear in groups (Fig. 548). Segments containing
very uniform curvature are evident at a, b, c, and d. T he segments from e through f
show sim ilar th o u g h sq u arer types of perform ance, w hile th e segm ents a t g, h, a n d i
show only slight pausing. B eginning a t i, m an y segm ents show m arked negative
5 M IN .
Fig. 5 4 9 .
550.
Tand FR 1 0 0 Fl sec showing

S-shaped segments
Tand FR 150 Fl 6 sec showing S-shaped segments
TANDEM SCHEDULES
449
cu rv atu re. These are p articu larly m ark ed at j a n d k, alth o u g h instances still occur
w here the term inal rate is m aintained until the reinforcement, w ith no decline, as a t
/ and m. T he tendency for the rate to fall during the latter p a rt of the session on F R F I
increases d u ring the next 2 sessions. Figure 549 shows the 1st hour of the session,
in which S-shaped curves are particularly m arked. T h e higher rate of responding in
the m iddle of the fixed-ratio segment is now being m aintained for a larger num ber
of responses, a n d th e drop to a low er ra te ju s t before the reinforcem ent has becom e
m ore abrupt.
T h e size of the ratio com ponent of the tan d e m schedule was then increased to 150
for 3 sessions. T he last of these is represented in Fig. 550. T h e 1st a n d 2nd, 8th
an d 9th, and 16th and 17th excursions are shown in Records A, B, an d C, respectively.
T h e pause an d acceleration following reinforcem ent become m ore extended. T he
S-shaped curves occur th ro u g h o u t the session, alth o u g h they becom e m ore frequent
an d prom inent tow ard the end in R ecord C. In R ecord A, w here S-shaped curves
are not present, two instances of pauses occur a t a a n d b, at about the same position in
the ratio segment as the point of m axim um curv atu re in the later negatively accel
erated segments. A t c a n d d the shift to the lower rate occurs m ore ab ruptly and e a r
lier in the fixed-ratio segment than heretofore.
In the session following Fig. 550 the schedule of reinforcem ent was changed to tan d
F R 125 F I 6 sec a n d m a in ta in e d for 34 hours (except for a period of 6 hours d u rin g
which the tandem interval was removed). T h e segments in Fig. 551 show the final
perform ance after this exposure to the tan d e m schedule. T he pause after reinforce
m en t varies from zero to approxim ately 1 m inute a n d is followed by an in sta n ta
neous shift to th e highest rate of responding, a b o u t 4.5 responses p er second. This
rate is m aintain ed for from 40 to 75 responses, after w hich a relatively a b ru p t shift
occurs to a rate of approxim ately 2.5 responses per second. T he higher rate following
the pause brings the end of each segment close to an extrapolation of the end of the
preceding segment. W here the reinforcem ent is not followed by a pause, as a t a, re
sponding is resum ed a t the lower rate, m aintaining the over-all linear envelope.
T h e short period of higher rate has the effect of com pensating for the pause, which is
seldom seen in simple F R performances.
T he other bird m aintained less substantial responding, and stopped responding alto
g eth er soon after the first tra n sitio n from e rf to ta n d F R 150 F I 6 sec. W h en the
ratio com ponent of the tandem schedule was reduced to 65 responses, a stable over-all
rate developed, w ith a perform ance very sim ilar to th a t alread y described for the
other bird in Fig. 547. W hen the size of the fixed ratio was later increased to 150, the
b ird was able to m ain ta in a sustained perform ance, as shown in Fig. 552, the 8th
session following the transition from erf. T he pause following the reinforcement varies
from zero to 11 m inutes, w ith only m oderate acceleration to a low term inal rate of
the order of 1 response per second. T he perform ance gives the over-all impression of
a fixed-ratio schedule, except for the low term inal rate.
D uring the 2nd an d 3rd sessions following Fig. 552, the term inal rate of respond-
Fig. 5 5 1 .
Final tand FR 125 Fl 6 sec shovying

S-shaped segments
TANDEM SCHEDULES
451
ing increased. A few ratio segm ents from th e 2nd session a re show n in Fig. 553A,
where the term inal rate at a reaches 2.5 responses per second. T he 3rd session (R ec
o rd B) shows a som ew hat lower term in a l ra te of 2 responses per second; b u t by this
tim e it is m aintained more uniformly w ith less extended curvature. T he experim ent
was term inated after the perform ance in Fig. 553.
TANDEM VARIABLE-RATIO FIXED-INTERVAL SCHEDULE
In tand F R V l we modify a fixed-ratio schedule by m inimizing the differential rein

forcement of high rates. T he relation between reinforcem ent and the few preceding
responses is th a t of an interval schedule, w hile all o th er properties are those of a
fixed-ratio schedule. In a tan d e m variable-ratio fixed-interval schedule a v ariab le
ratio schedule is specified; b ut a reinforcem ent becomes available only after a few
seconds have elapsed since the last response in a ratio is em itted. T he m ajor fea
tures of the V R are preserved; th at is, the reinforcem ent is prim arily determ ined by
a num ber of responses which varies from reinforcem ent to reinforcement. T he differ
ential reinforcem ent of high rates resulting from the relation betw een reinforcem ent
and the few preceding responses is, however, reduced or elim inated. T a n d Fi?FI dif
fers from V I in th at long pauses are usually not reinforced (unless they begin during
the relatively short FI 6 sec com ponent of the tandem schedule). In V I whenever the
over-all rate of responding is low for any reason, reinforcements will occur on sched
ule or close to schedule if any responding occurs at all. In tan d V R F \, however, w hen
ever the over-all rate becomes low, no further reinforcem ents will occur for possibly
a long time.
452
Experiment I
In a first experim ent tan d V R F \ was studied directly after erf. T he variable-ratio
p rogram was as follows: 50, 5, 130, 100, 20, 120, 140, 100, 5, 120, 80, 160, 10, 60,
20, 155, 30, 70, 40, 150, 40, 120, 5, 90, 80, 155, 1, 80, 100, 10, 60, 70, 110, 1, 90, 50,
60, 130, 40, 150, 90, 140, 110, 1, 150, 130, 10, 50, 140, 30, 160, 70, 110, 155, 20, 160,
a n d 30 responses. T h e m ean is 80. T h e ta n d e m fixed interval was 6 seconds.
T he perform ance during nearly 100 hours on tan d V R 80 F I 6 sec is shown in Fig. 554,
where 1000-response segments have been selected from various stages of the develop
ment. Each excursion is the 2nd segment of its session. Records A and B, a t slightly
over 1 response per second, show the perform ance 9 hours and 16 hours, respectively,
A B C
DE
F G H I J K
L M
after erf. T h e local rate is considerably irregular; a n d reinforcem ents are variously
followed by pauses, continuations of the preceding rate, or increases in the rate. In
R ecord C, 44 hours after continuous reinforcem ent, the ra te has increased to 1.7 re
sponses per second, w ith frequent pauses after the reinforcem ent. A t this stage of
ta n d VRF1 the ra te fell to very low values for periods up to 2 hours, a n d in one case
m ore th a n 10 hours. (This exceptional perform ance is shown in Fig. 555 for the
entire session, after 35 hours of reinforcem ent on the tandem schedulebetw een R ec
ords B and C of Fig. 554. T he session begins with a perform ance sim ilar to th at in
R ecord B. After reinforcem ent a t a, however, the rate falls off sharply b u t sm oothly
to an interm ediate value m aintained for m ore th an 2 hours, during which reinforce
m ents occur relatively infrequently. A t the end of the figure the rate of responding
TANDEM SCHEDULES
453
falls to zero. T he bird rem ained in the box for 10 hours w ithout responding to the
key.) In R ecord D of Fig. 554, after 54 hours on ta n d V R F I, the over-all rate falls
below th a t of R ecord C, b u t sustained local rates of responding now occur of the
o rder of 2.5 responses per second. In R ecord E, at 55 hours, the over-all rate reaches
2.8 responses per second; but this falls to a lower rate in the next session (R ecord F
a t 57 hours), as a result of a lower local rate. In Record G, 64 hours after erf, the over
all rate again reaches 2.8 responses per second, w ith pauses or periods of lower rate
occurring only after a few reinforcements.
An extinction curve shown in Fig. 556 was taken betw een Records F and G, after
62 hours of reinforcem ent on the tan d e m schedule. T he curve consists of sustained
responding for almost 2000 responses in th e m an n er of the current tan d V R F I p er
form ance followed by an instantaneous shift to a zero rate of responding a t a. T he
Fig. 5 5 6 .
Extinction a fte r 62 hr tand

VR 8 0 FI 6 sec
454
bird does not respond for over 2 hours. W hen responding resumes, the m agazine is
reconnected a t b, a n d a single reinforcem ent reinstates the typical variable-ratio p er
formance. By Record H in Fig. 554 the over-all rate reaches the m axim um value seen
in the experim ent, 4 responses per second, 74 hours after erf; an d it rem ains in this re
gion for the rem ainder of the tan d V R F I. T he F I com ponent was removed starting
at R ecord K, an d simple V R prevailed through Records L and M to a total of 94 hours
after erf. T he performance shows no appreciable change, except possibly a decrease
in over-all rate. A second extinction curve was taken after 87 hours of reinforcem ent
on tan d V R F I and a short exposure to V R alone, ju st after R ecord K. Figure 557
shows the entire curve, which is m uch larger th a n th a t in Fig. 556. A n initial period
of sustained responding at the prevailing rate is m aintained for m ore th an 9000 re
sponses before the rate falls at a. T he rem ainder of the curve consists of 3 segments, a t
b, c, a n d d, of 800, 600, a n d 150 responses, respectively, executed a t the prevailing
v ariab le-ratio rate, separated by long periods of zero or near-zero rates. W hile the
transitions from one rate of responding to an o th er are not so a b ru p t as in the p re

vious extinction curve, the transitions are considerably m ore a b ru p t th an in the usual
variab le-in terv al extinction curve. If p lotted in a single continuous segm ent, the
over-all curve w ould consist m ainly of a long leg a t the prevailing variable-ratio rate,
falling sharply to a zero rate, punctuated by relatively small deviations a t the higher
rate. C om pare Fig. 496 through 499.
T he second bird in the experim ent developed a sim ilar final perform ance on tan d
V R F I, although the developm ent was slower and followed a somewhat different course.
Figure 558 shows the change in the over-all rate of responding during the first 7
sessions (34 hours) of the experiment. T he over-all rate rem ains very low for the first
14 hours, after which it increases m ore rapidly to almost 0.5 response per second by the
end of the figure. By this tim e, however, sustained local rates of responding appear
which are considerably higher th an the prevailing over-all rate. Figure 559 gives the
entire session represented by point a in Fig. 558. T his is the 6th session, after 33
hours of reinforcem ent on the tandem schedule. It begins w ith a perform ance similar
TANDEM SCHEDULES
455
HOURS
Fig. 5 5 8 .
Change in over-all rate during the firs t seven sessions on tand VR 8 0 FI 6 sec
(second bird)
to the early stages of the other bird, w ith an over-all rate of approxim ately 0.8 re
sponse per second. Pauses after reinforcem ent an d subsequent acceleration to the
term inal rate are som ew hat m ore extended, however. As the session progresses, both
the pause a n d the period of acceleration become m uch m ore extended, leading to
m arked scallops, as at a an d b, a near-zero rate of responding, as at c, and a final over
all rate for the last segm ent of the figure of approxim ately 0.2 response per second.
This perform ance m ay represent a stage of developm ent of ta n d V R F I sim ilar to th a t
of Fig. 555, w here a later increase in over-all rate was preceded by a period w here
the rate, and consequently the frequency of reinforcem ent, fell to a low value.
Figure 560 represents the rem ainder of th e experim ent on tan d V R F I, from the 8th
to the 67th sessions. T he second pen excursion has been selected from the record for
every 4th session. Records A th rough N show a progressive increase to a value in
the vicinity of 3 responses per second. T he over-all rate varies from segment to seg
m ent, and pauses are variously present or absent after reinforcement. After the session
represented by R ecord N, the fixed-interval com ponent of the tandem schedule was
456
rem oved. Records O and P for the 4th and 8th sessions on the simple V R 80 show
no significant change.
Both birds in this experim ent developed rates a n d perform ances sim ilar to those on
V R . If the tandem FI 6 sec has any effect, it is probably in the greater variation in the
rate from segment to segment and in the phase where over-all rate falls far below the
rates prevailing a t the time.
Experiment II
T he developm ent of a final perform ance on V R 360 has already been described in
C hapter Seven. Figure 475 shows a well-sustained level; Fig. 476, on the following
day, shows considerable pauses; an d Fig. 477, for the second bird, shows a m arked step
wise perform ance. Both birds were placed on a tan d V R 360 FI 6 sec. T h e condi
tions at the m om ent of reinforcement have thus changed, a t least potentially, to those
of an interval schedule, while the other contingencies of the variable-ratio schedule of
reinforcem ent were m aintained. T he previous perform ance in Fig. 477 was optim al
for transition to the tandem schedule, since the brief sustained responding an d the fre
quent pauses m ust increase the likelihood th a t the variable-ratio requirem ent of the
tandem schedule will be m et ju st before a pause, so th a t the first response after a pause
will be frequently reinforced. Figures 561 and 562 show the 7th session after transi
tion. T he early effect of the FI 6 sec com ponent of the tandem schedule is a decline in
the over-all rate an d the appearance of smooth curvature from the prevailing rate to
the pause. T h e transition from pausing to responding still rem ains instantaneous.
T ow ard the end of the session, in Fig. 562, interm ediate rates of responding begin to a p
pear, as a t a, w here sustained runs of approxim ately 50 to 75 responses are separated
by short periods of interm ediate rates rath e r th an pauses. T he rate im m ediately after
reinforcem ent frequently takes an interm ediate value, as a t a and b in Fig. 561 an d
TANDEM SCHEDULES
457
b in Fig. 562, possibly because of the reinforcem ents after pauses. T h e reinforcem ent
a t b in Fig. 562 is an instance in which the variable ratio was counted out just before
the bird was about to pause, and the reinforcem ent occurred ju st after the pause. Fig
ure 563 contains the latter p a rt of a later experim ental session beginning 137 hours
after the change to the tandem schedule. Both the over-all a n d local rates decline
m arkedly. Reinforcem ents occur after pauses or at lower rates of responding, as a t
b, for example. Pauses now occur after reinforcem ents, as a t b, c, e, and f , although
these were rare in the previous V R perform ance. T h e pauses presum ably occur b e
cause successive responses are no longer likely to be reinforced, as under the previous
variable-ratio schedule, since the m inim um tim e elapsing betw een reinforcem ents is 6
seconds. T he variable-ratio character of the perform ance is preserved in the m any
fairly a b ru p t shifts to zero rates at tim es other th a n a t reinforcem ent, as a t a, d, and g.
In general, the addition of the FI 6 sec schedule has produced a more irregular perform
ance.
T he second bird, which had been showing the perform ance in Fig. 475 and 476 on
V R , took longer to show the effect of th e FI 6 sec com ponent of the tan d em schedule.
U nder sustained responding at high rates w ith but few shifts to lower rates, the p ro b
ability th at the variable ratio w ould be counted out ju st before a pause was very small.
458
Figure 564 shows a perform ance after 8 sessions on tan d V R 360 FI 6 sec. T he p rin
cipal change from Fig. 475 is the appearance of sustained interm ediate rates. Even
in the poorly sustained perform ance of Fig. 476, com parable portions of an interm edi
ate rate are lacking.
Chapter Nine
DIFFERENTIAL REINFORCEMENT
OF RATE
INTRODUCTION
In
o u r a n a l y s i s of the effects of schedules the rate at the m om ent of reinforcement is
a n im portant variable. For exam ple, in FI the probability is m uch greater th a t a re

inforced response will follow a pause th an in the otherwise not-so-different tan d F IF R .
But a given schedule does not guarantee such a relationship a t every reinforcement.
Indeed, u n d er exceptional perform ances the relationship m ay vanish or m ay even be
reversed.
T he im portance of this variable can be checked by direct control of rate conditions a t
reinforcem ent. T h e resulting perform ance will enable us to estim ate the extent to
w hich m om entary rates function as a stim ulus in any set of contingencies of reinforce
ment.
By rate at reinforcem ent we m ay m ean the reciprocal of the inter-response tim e
preceding a single reinforced response, or of the elapsed tim e required by the last n re
sponses. At this stage, practical considerations have been most im portant in dictat
ing procedures. T he differential reinforcem ent of low rates (drl) is easiest to arrange
in term s of a single inter-response tim e (IR T ); the differential reinforcem ent of high
rates (drh) is easiest when a given num ber of responses m ust be executed w ithin a set
period.
A device which sets up a reinforcem ent only w hen some such specification has been
m et can be added to any schedule. Thus, in FI 5 drl 6 a response is reinforced approx
im ately every 5 m inutes but only when it follows the preceding response by at least 6
seconds. In V I 5 drl a response is reinforced on a V I schedule with a m ean of approxi
m ately 5 m inutes b u t only w hen a few responses have m et a given speed requirem ent.
In crfdrl 3 every response is reinforced w hich follows the preceding one by at least
3 seconds. (T he com pletion of the preceding reinforcem ent m ay or m ay not be used1
in establishing the criterion for the following response. If it is not, the schedule is
technically F R 2 drl 3.)
459
460
VARIABLE INTERVAL WITH DIFFERENTIAL REINFORCEMENT OF LOW RATES

V I 1 drl
In one experim ent, drl 6 (no response reinforced unless preceded by a 6-second IR T )
was added to an early V I 1 perform ance, 3 sessions after erf. Figure 565 shows the
last of this perform ance before the arrow in R ecord A. A roughly linear over-all rate
of 1.1 responses per second prevailed. T he drl 6 contingency postponed the next re
inforcem ent until a, about 10 m inutes later. (T he largest interval in the V I series was
200 seconds.) For the rem ain d er of the session the over-all rate fell continuously,
reaching approxim ately 0.35 response per second by the last segment of the record at
b. A final perform ance after 9 sessions on this V I 1 drl 6 is shown in R ecord B, where
the rate of responding is constant a t 13 responses per minute. If the bird were respond
ing exactly once every 6 seconds, the rate would, of course, be 10 responses per m in
ute. Local features of the curve in R ecord B show th a t the extra 3 responses per
m inute are the result of occasional small groups at higher rates.
A second bird showed a similar effect of V I 1 drl 6. Record C shows a final perform
ance after a sim ilar history. T he over-all rate of responding is higher, and the ten d
ency to pause after the reinforcement is m ore pronounced.
Figure 566A begins with the perform ance under V I 1 drl 6 seen in Fig. 565B. A t
the arrow the drl was decreased to 1 second. T he rate of responding increases slowly
during the rem ainder of the session, the final rate in the session being about doubled.
W ith further exposure to drl 1, the rate continues to increase, reaching the final p er
form ance shown in R ecord B after 11 sessions. T he last segment in R ecord B shows a
linear perform ance at slightly less th an 0.8 response per second, com pared with the rate
461
300
RESPONSES
DIFFERENTIAL REINFORCEMENT OF RATE
of 1.1 reponses per second for the bird on V I 1 before drl. T h e second bird again
showed a slightly higher rate. R ecord C shows its perform ance on V I 1 drl 1 after a
sim ilar history. T h e over-all rate is also low er th a n th a t u n d e r V I 1 w ithout drl, a l
though it should be noted th a t the V I 1 w ould probably have continued to rise.
T he drl was again increased to 6 seconds and final performances for both birds are
shown in Fig 567A and B. R ecord A shows an increase above the earlier perform
ance in Fig. 565B, w hile R ecord B m ay show a slight decrease below the earlier p e r
form ance of Fig. 565G. In Record A a burst of about 75 responses a t a rate a p p ro
priate to the variable-interval schedule w ithout drl occurs a t a, displacing the curve
vertically. These break-throughs occur frequently at this stage, possibly because of
frequent transitions between various rate contingencies.
T h e size of the drl was then increased in the following session to 12 seconds, an d
Records C and D show performances 5 sessions later. In R ecord C the over-all rate of
responding falls to 7.5 responses per m inute, and in R ecord D, to 5.8 responses per m in
ute. If the bird were responding a t a rate exactly satisfying the drl contingency, the
rate would be 5 responses per m inute. T he brief bursts of responding at b, c, and ^ r e
semble the earlier perform ance of the same bird in Record A.
462
Extinction a fte r VI 1 drl 1 2
T he m agazine was disconnected at the start of the session following the final p e r
form ance on V I 1 drl 12 shown in Fig. 567C. Extinction was carried out for 16 hours.
T h e entire curve (shown in Fig. 568) contains 800 responses, a n d shows a sm ooth an d
orderly decline to a near-zero value. T he only m ajor deviation is a t a, where the origi
nal V I 1 drl 12 rate reappears for about 15 minutes.
VI 4.5 drl 2
An experim ent in which a drl 2 was added to a well-developed V I 4.5 has been m en
tioned in C h ap ter Six, w here dosages of sodium pentobarbital were regulated in term s
of rate under V I. Figure 569 shows one transition from V I 4.5 to V I 4.5 drl 2 in th a t
experim ent. T h e adjusting drug procedure was not in force a t this stage. R ecord A
Fig. 5 7 0 .
463
Lim ited-hold reinforcem ent a fte r VI drl
is a sam ple of the final perform ance under V I 4.5 after a long history of V I with a n d
without continuous adm inistration of sodium pentobarbital. Record B shows the 10th
session under V I 4.5 drl 2. T he drl produces a fairly stable over-all rate of respond
ing of about 0.5 response per second. T his is twice the rate which ju st satisfies the
drl contingency at every response, b u t reinforcem ents are not significantly postponed.
Each session begins characteristically at a relatively high rate, and occasional periods
of sustained responding occur at rates considerably higher th an the prevailing rate, as
a t a an d b. An occasional shift to a low er-than-usual value is exemplified at c.
Slight pausing or slower responding just after the reinforcem ent reflects the low dens
ity of short intervals in the variable-in terval schedule. A second bird showed a sim ilar
transition.
Limited hold after drl
T he lim ited-hold contingency discussed in C hapter Six was accidentally put in force
against a background perform ance w hich involved the differential reinforcem ent of
slow responding. A bird h a d been on a chained V I 3 V I 3 drl schedule. Figure 570
begins w ith the low rate established on the V I 3 drl p a rt of this schedule. T h e limitedhold contingency was, however, in effect. T he first 3 reinforcements, at a, b, and c,
are widely spaced, because the low rate from the drl makes the occurrence of a response
in the 0.5-second period of lim ited hold unlikely. Following the 3rd reinforcem ent,
at c, a rapid acceleration occurs to the V I rate formerly shown by this bird in the a b
sence of the drl. This is a m uch m ore rapid transition th an th a t which results from
restoring V I after V Idrl. T he low rate and pattern of responding under V ld rl m ay
m ake the lim ited-hold contingency especially effective by providing m axim um contrast
betw een occasions for reinforcem ent an d nonreinforcem ent while sustaining the be
havior at the over-all level required if reinforcements are to occur to m aintain the
464
behavior. A slight differential reinforcem ent of rapid responding is assum ed to have

occurred at a, b, and c, on a scale m uch too small to show in the record.
FIXED INTERVAL WITH DIFFERENTIAL REINFORCEMENT OF LOW RATES
W ell-developed perform ances on several values of F I have been shown in Fig. 252,
260, 261, 264, an d 266. W e studied the effect of the differential reinforcem ent of low
rates on these performances. R einforcem ent occurred whenever the bird paused for
a t least 6 seconds after the designated fixed interval h ad elapsed. A 5-m inute T O fol
lowed all reinforcements.
FI 1 drl 6 TO 5
T he record of the transition from F I 1 to F I 1 drl 6 was lost because of an a p p a ra

tus failure. A well-developed perform ance is evident in a later figure (Fig. 576).
Fig. 5 7 1 .
Transition to FI 2 d rl 6 TO 5 and later perform ance
FI 2 drl 6 TO 5
Figure 571A contains the transition to F I 2 drl 6 T O 5. T h e session begins w ith a

perform ance appropriate to FI 2 T O 5, a n d the rate is above 1 response every 6 sec
onds at the end of the first 2-m inute interval. R esponding continues a t the term inal
value under the previous FI 2 schedule until a break in the perform ance satisfies the
drl contingency for a reinforcement a t a, after 900 responses a n d 20 minutes. T he sec
ond interval segment following the reinforcem ent at a begins w ith a pause of almost
2 m inutes, but the rate exceeds the critical value and reaches the term inal value a p p ro
priate to the FI perform ance. T he rate of responding falls m om entarily a t b, and the
second reinforcem ent is received after approxim ately 5 m inutes an d 100 responses.
By the end of the session, responding occurs in bursts separated by pauses w hich occa
sionally satisfy the drl requirem ent, and reinforcements begin to occur m ore frequently.
By the 3rd session the term inal rate during the interval has fallen to 0.4 response per
second. R ecord B shows a final perform ance from the m iddle of the 12th session after
a total of 230 reinforcem ents under FI 2 drl 6 T O 5. T he over-all rate has fallen to
0.15 response per second, w ith a pause after the reinforcem ent an d acceleration to a
465
term inal rate of 0.2 response per second. A lthough the term inal rate of responding is
slightly higher th a n the 10 responses per m inute w hich ju st satisfies the drl require
m ent, slight variations in the local rate provide instances where 6-second pauses occur.
At c, d, and e responding continues for from 10 to 15 m inutes before a pause long
enough to m eet the drl requirem ent occurs. T he differential reinforcem ent has o b
viously been effective in elim inating the high term inal rates under FI 2, although inter
val curvature remains.
FI 4 drl TO 5
Figure 572A shows the transition from F I 4 T O 5 to F I 4 drl 6 T O 5. T he drl con

tingency greatly lengthens the intervals betw een 3 of the first 4 reinforcem ents. Fol-
300
RESPONSES
lowing the reinforcem ent a t a, however, the over-all rate of responding rem ains low.
T he next 4 reinforcem ents are delayed by pauses ra th e r th a n by an excessive rate.
Following the pause after reinforcem ent at b, the rate increases to 0.4 response per sec
ond; the reinforcements a t c and d are not postponed, because responding now occurs
in small bursts.
T he rate during the next few sessions rem ained generally low, w ith frequent in
stances of reinforcements postponed because no responses occurred. Eventually, the
over-all rate of responding increased and becam e stable from segment to segment.
R ecord B shows a segm ent from a daily perform ance after a total of 460 reinforce
m ents on FI 4 drl 6 T O 5. T he over-all rate of responding has fallen to approxim ately
0.15 response per second, and most reinforcem ents now occur on schedule. T he over
all p a tte rn of responding is linear, w ith some slight interval scalloping. A small burst
follows the reinforcem ent at e, a n d a pause w ith com pensation follows reinforcem ent
at f
466
FI 8 drl 6 TO 5
Figure 573A shows the transition from FI 8 T O 5 to F I 8 drl 6 T O 5. This is the first
p art of the session. T he 1st interval begins as usual and accelerates to the term inal rate
of responding, which is m aintained for approxim ately 10 m inutes. No pauses as long
as 6 seconds occur, and the 1st reinforcem ent does not occur until the rate falls off, at a.
T h e 2nd a n d 3rd reinforcem ents, at b a n d c, occur after sim ilar segments. T h e rein
forcement a t c is followed by a 6-m inute pause and a slow acceleration to a lower term i
nal rate. T he rate is still too high to satisfy the drl 6 requirem ent, however, and the
reinforcem ent a t d occurs only after 23 m inutes. T h e rate is n ear zero during the next
segment, b ut recovers during the last 2 reinforcements of the record, at e and f . T h e
reinforcem ent a t/o c c u rs w ithin 1 m inute of the designated schedule, largely because of
the grain of the record.
By the 5th session of FI 8 drl 6 both the over-all and term inal rates have declined; and,
in general, the perform ance consists of a long pause after the reinforcem ent and a slow,
shallow acceleration to a low term inal rate. T he start of the session (Record B) shows
a generally higher rate th an the rem ainder of the session. R ecord C illustrates the
final perform ance on the schedule after 14 sessions of exposure. T he higher rate a t the
start of the session has become m uch less pronounced, an d most of the intervals are
scalloped, w ith an even lower term inal rate of responding th an th a t in Record B. M ost
reinforcem ents are received w ithin 10 m inutes. T hree break-throughs a t higher rates
of responding occur at g, h, and i. (Note th a t the reinforcem ent at the beginning of
the segment at e was delayed by an apparatus failure rather th an by the b irds failure to
satisfy the drl.)
FI 16 drl 6 TO 5
Figure 574 shows the transition from F I 16 T O 5 to FI 16 drl 6 T O 5. H ere, the first
perform ance w ith drl is not very different from the preceding F I schedule, because
th e term inal rate h a d contained a sufficient n um ber of 6-second pauses to avoid severe
467
postponem ent of reinforcement. T he drl still reduces the term inal rate, however, as
R ecord B, a segment taken 2 sessions following R ecord A, shows. As the rate of re
sponding falls, the grain of the record becomes smoother. This sm oother grain works
against satisfying the contingency. T he intervals between reinforcements become
longer, reaching 30 m inutes after the reinforcem ent at a a n d 32 m inutes after the rein
forcem ent at b. T he period of curvature becomes m ore extended, as in the segment
ending with the reinforcement at c. Record D shows the 8th session. (Record C is for
a later session described below.) T h e pause after the reinforcem ent is now longer.
T he term inal rate of responding has fallen from 0.25 to 0.3 response per second, an d the
scallop extends through most of the interval segment. T he term inal rate of respond
ing is now low enough so th at most reinforcements occur practically on schedule. T h e
slow-down contingency shows a further effect 4 sessions later (R ecord E), w here the
term in al rate has fallen to the order of from 1.5 to 1.8 responses per second. T h e
changes within the interval segments, however, are no longer so smooth as those in R ec
ord D. F u rth er exposure to the slow-down contingency results in an even more ir

regular perform ance by the 13th session, as R ecord C shows. (N ote again th a t the
segments in the figure have been arranged, for best reduction, in a different order.)
W hile the over-all rate has increased considerably because of sustained periods of
higher rates of responding, as at d, e , f and g, reinforcements are not greatly postponed,
because of the m arked oscillations in over-all rate and the frequent instances of 6-second
pauses in the fine grain of the record.
FI 32 drl 6 TO 5
A final perform ance on FI 32 drl 6 T O 5 was recorded for one bird after a welldeveloped perform ance on F I 1 drl 6. Figure 575 shows the 11th session on the
FI 32 drl 6 T O 5 schedule.
Transition from FI drl 6 TO 5 to FIT0 5
W hen a final perform ance on each of the five values of FI drl 6 h a d stabilized, the
drl was removed, and the transition to a simple F I was observed. (The bird chosen for
468
Fig. 5 7 5 .
FI 32 d rl 6 TO 5 a fte r ten sessions
the final perform ance on each value of FI drl 6 was the same as th at in the first tra n
sition from erf. W e have no case to report for the transition to F I 2, since the bird orig
inally a t this value h ad been used in studying F I 32 drl.)
F I 1. T he final perform ance after 13 sessions on FI 1 drl 6 T O 5 shown in Fig. 576
has already been m entioned. T he over-all curve is roughly linear, with a slight ten d
ency for a higher rate at the start of the interval. T he drl contingency is satisfied for
most reinforcements by the rough grain o f the record, which includes m any 6-second
pauses, rath e r th a n by a low over-all rate. W hen the slow-down contingency is re
moved in the next session (Record B), the over-all rate increases rapidly to 1.5 responses
per second by the 5th reinforcem ent in the session. T his rate m ay be com pared w ith
an over-all rate of 0.4 response per second in the previous session with drl. T ow ard
Fig. 5 7 6 .
Transition from FI 1 d rl 6 TO 5 to FI 1 TO 5 and la te r perform ance
469
th e end of the segm ent shown in R ecord B, a period of lower rate of responding devel
ops at the start of the interval segment, a n d a term inal rate of almost 4 responses per
second is reached w ith few pauses or deviations in rate. In the next session (Record C)
the fixed-interval segm ent becomes m ore sharply scalloped, an d reinforcem ent u n i
form ly occurs after a sustained term inal rate has been reached. By the 9th session
(R ecord D) the over-all rate has declined as the pause and acceleration following the
reinforcem ent have become m ore m arked. T he term inal rates range from 2 to 3.5 re
sponses per second.
F I 4. Figure 577A shows the final perform ance under FI 4 drl 6 T O 5 when the bird
was placed on th a t schedule for the second tim e after being tested a t other values.
T h e over-all rate is higher th an it was d u ring the first exposure to this schedule, d e
scribed in Fig. 572. T h e contingency was being satisfied in p a rt by a rough grain.
Records B a n d C show the developm ent of the perform ance on F I 4 after drl was dis
continued. In R ecord B the term inal rate increases alm ost im m ediately. T he over
all perform ance rem ains roughly linear, a n d strong scalloping develops. By the 9th
session in R ecord C, the over-all rate of responding has fallen even farther. N one of
th e interval segments begin w ith substantial pauses, an d two (at a and b) begin w ith
the full term inal rate from the preceding interval in spite of the T O 5.
F I 8. W hen the drl contingency was om itted from the F I 8 perform ance, the first
session, shown complete in Fig. 578, shows little change. Both the pause a t the start of
the interval segm ent an d the m agnitude of the term inal rate of responding increase
som ew hat. C om pare, for exam ple, the segments a t a, b, and c w ith the earlier p er
form ance in Fig. 573. By the 4th session on FI 8 T O 5, shown in Fig. 579A, the term i-
470
Fig. 5 7 9 .
Later perform ance on FI 8 TO 5 a fte r drl
nal rate of responding has increased further, with a corresponding decrease in the length
of the pause at the start of the interval. By the 6th session (R ecord B) the over-all
rate has increased m arkedly, with the highest rates of responding occurring at the m id
dle of the interval segments, as a t a, b, a n d c, so th a t the curves take on an S-shaped
character. T h e perform ance 10 sessions after rem oval of drl is shown in R ecord C.
T h e over-all rate is now fairly high. T h e pause following the reinforcem ents is either
absent or brief, but the successive segments show consistent curvature to term inal rates
varying from 0.8 to 1.5 responses per second.
F I 16. T h e effect of rem oving the drl contingency on F I 16 is shown in Fig. 580,
where the course of developm ent is represented by 2 consecutive interval segments each
from the 1st, 2nd, 4th, and 9th sessions. (T he order of the records is inverted to p e r
m it a m ore com pact arrangem ent.) Record A, taken from the end of the 1st session,
shows about the same rate of responding as w ith drl; b u t local oscillations in the rate,
as at a an d b, are becom ing m ore m arked. A sm ooth a n d continuously accelerated
scallop emerges by the end of the 2nd session (R ecord B); b u t by the 4th session (R ec
ord C) the curve shows a sm ooth b u t rap id acceleration to a m uch higher term inal
471
rate th a n th a t which occurred heretofore. Finally, by the 9th session (R ecord D) the
period of lower rate of responding during the first p a rt of the fixed-interval segment
becomes m ore extended, and the shift to the term inal rate of responding m ore abrupt.
F I 32. T he effect of rem oving the drl contingency on FI 32 is shown in Fig. 581,
w here the order of the segments has also been inverted. Each record contains 2 suc
cessive segments from the end of the respective session. No effect of the rem oval of
th e drl contingency is evident a t the end of the 1st session (R ecord A). T h e over-all
curve reveals a sustained interm ediate constant rate w ith a very slight increase d u r
ing each interval segment. By the 2nd session (R ecord B) the term inal rate has in
creased above the over-all rate u n d er drl an d the scallop has become m ore m arked.
This trend continues in the subsequent sessions in Records C through G for the 3rd, 4th,
6th, 7th, an d 10th sessions, respectively. Interval segments frequently begin with a
pause of the order of 5 m inutes; th en the rate of responding increases to approxim ately
Fig. 5 8 0 .
Transition from FI 16 d rl 6 TO 5 to
FI 16 TO 5 and later perform ance
Fig. 581.
Transition from Fl 32 drl 6 TO 5 to FI 32 TO 5 and later perform ance
473
0.75 response per second, w hich is m ain tain ed w ith rough grain a n d some oscillation
until the next reinforcement.
A COMPARISON OF EXTINCTION AFTER FIXED INTERVAL A N D FIXED
INTERVAL WITH DIFFERENTIAL REINFORCEMENT OF LOW RATES
W e exam ined the effect of differential reinforcem ent of low rates on the fixedinterval perform ance by taking extinction curves after final perform ances h a d been
recorded on FI 1, 4, 8, 16, a n d 32 w ith drl a n d by taking a second set of extinction
curves for the same birds following the developm ent of a norm al FI pattern after the drl
contingency had been removed.
F I 1. Figure 582 contains an extinction curve after F I 1 T O 5 (Record A) and an
earlier extinction curve after F Id rlT O 5 (R ecord B). T h e curves reflect the im m edi+I5R'$IN55win
ately preceding performances. R ecord A begins a t the high term inal rate of the FI 1
perform ance and falls off sharply with frequent re-occurrence of brief periods a t a high
rate. R ecord B continues the low, sustained interm ediate rate of the final perform
ance under FI 1 drl 6 in Fig. 576A. T h e over-all rate falls very slowly during the ses
sion, averaging 2.5 responses per second for the last h a lf of the curve com pared w ith
3.5 responses per second for the first part.
F I 4. Figure 583 contains extinction curves after F I 4 T O 5 and FI 4 drl 6 T O 5. In
Record A the extinction after F IT O 5 shows m arked S-shaped features, with the rate
oscillating betw een almost zero an d the term inal fixed-interval rate. Extinction after
F ld rl (R ecord B) begins at a m oderate rate which declines continuously an d slowly,
showing only m inor oscillations.
F I 8. E xtinction curves in Fig. 584 after F I 8 w ith a n d w ithout drl tell the same
story. Extinction after FI 8 w ithout drl produces a curve consisting of 3 m ajor seg
m ents a t rates near the term inal fixed-interval rate, separated by long pauses a n d pe-
300
RESPONSES
Extinction a fte r FI 8 TO 5 and FI 8 drl 6 TO 5
D IF F E R E N T IA L R E IN F O R C E M E N T O F RA TE
475
riods of slow responding. In contrast, extinction after F ld rl begins a t a m oderate rate

sustained for m ore th a n 1 hour, after which the rate oscillates between very low and
m oderate values.
F I 16. Figures 585 an d 586 are a com parison of the extinction curves after FI 16
w ith a n d w ithout drl, respectively. T h e beginning of extinction after FI is shown in
Fig. 585A an d continued in Fig. 586. T h e curve begins w ith approxim ately 4000 re
sponses at the high term inal rate from the previous F I 16 reinforcem ent. T h e rate
then falls to an interm ediate value, after w hich the original high rate of responding re-
Fig. 5 8 6 .
C ontinuation o f Fig. 5 8 5 A
476
turns once m ore, at a. T he rate then falls for the rem ainder of the session, reaching
zero for 20 m inutes after b. M uch later in the session, after 7 hours in which the rate
was very near zero, the rate of responding increased tem porarily, at a in Fig. 586. T he
rate did not reach the high FI term inal value, however, and the grain is rough. Be
ginning at b the curve oscillates w ith a fairly stable frequency. T he highest rate is of
the same order of m agnitude as at a. T he persistence of interm ediate rates of respond
ing in prolonged extinction m ay show some survival of the effect of the earlier drl
perform ance. Figure 585B shows the parallel extinction curve after FIdrl. T he ses
sion was shorter an d the record is com parable only w ith Record A w ithout the con
tin u atio n in Fig. 586. T he over-all rate begins a t approxim ately 0.25 response per
second, approxim ately one-sixth the beginning rate in R ecord A. T he over-all rate
falls slowly with 2 m arked oscillations. N ear the end of the session, sustained respond
ing occurs at the same rate as th a t at the start of the session. T he highest rate during
extinction occurs a t c, at 0.4 response per second.
F I 32. Figure 587 shows extinction curves after F I 32 w ith and w ithout drl. Both
curves begin w ith the rate ap p ro p riate to preceding perform ances. T he rate in R ec
ord A falls fairly continuously, while in R ecord B, after drl, the decline is less orderly.
T he starting rate falls rath er abru p tly from 0.4 response per second to about 0.2 re
477
sponse per second a t a. This rate is then m aintained approxim ately for the rem ainder
of the session, w ith local deviations.
FIXED RATIO W ITH DIFFERENTIAL REINFORCEMENT OF L O W RATES
Reinforcem ent occurs in F R drl when the bird pauses for a given period after h av
ing com pleted the fixed ratio. Differential reinforcem ent of low rates on a ratio sched
ule is difficult because of opposing tendencies generated by the schedule: the schedule
produces high rates ju st before reinforcem ent, largely because com pleting the ratio is
reinforcing. T he schedule is sim ilar to ta n d F R F I in th a t the m ain features of F R are
preserved while the differential reinforcem ent of high rates norm ally present in the
schedule is elim inated. T h e drl has the ad d ed property, however, of postponing the
reinforcem ent until the rate falls below a critical value. T h e schedule theoretically
should generate an oscillating state, at least w ith small ratios. A high rate postpones
reinforcem ents an d increases the n um ber of responses per reinforcem ent. As the rate
falls, the num ber of responses per reinforcem ent approaches the value of the fixed ratio,
a n d reinforcem ent a t this value again tends to increase the rate of responding; and
so on.
The e ffe ct o f d iffe re n tia l re in forcem en t o f lo w rates on
an established fix e d -ra tio p e rfo rm an ce
Early exposure. In one experim ent a perform ance on a m oderate size of fixed ratio
was first established. T he differential reinforcem ent of low rates was then added.
Figure 588 contains a transition from F R 150 to F R 150 drl 3. R ecord A shows the
last 2 excursions on F R 150, after 11 sessions during which the size of the fixed ratio was
advanced from 20 to 150 following erf. T h e final perform ance consisted of a pause
Fig. 5 8 8 .
Transition fro m FR 1 5 0 to FR 1 5 0 drl 3
478
after reinforcem ent up to 2 or 3 m inutes a n d a shift to a term inal rate, either ab ruptly
or w ith a small am ount of curvature. T he following session in R ecord B begins w ith
1 reinforcem ent on F R 150 at the arrow . T he schedule then becomes F R 150 drl 3.
T h e term inal rate continues for m ore th a n 600 responses before a 3-second pause sets
up the reinforcem ent a t a. A lower term inal rate is assum ed a t b. T he rest of the
session is characterized by an over-all rate of slightly m ore th an 1 response per second
w ith rough grain. At c, d, and e the earlier rate and sm oother grain return. These
ra te changes are independent of the reinforcem ent. R ecord C contains a segment
from the 3rd session after the introduction of the drl 3. T he perform ance is roughly
linear at about 0.6 response per second. T he rate frequently shifts widely, as a t f an d
g, a n d the drl 3 requirem ent is being satisfied m ainly by the rough grain.
Figure 589 shows a transition from F R 60 to F R 60 drl 6. T he bird h ad h ad only
3 sessions on F R 60. A sam ple of the perform ance appears before the arrow. A p
proxim ately 600 responses were then em itted at the ratio term inal rate before a slight
decline satisfied the added drl contingency. T he same high rate appears again at a, b,
an d c, separated by segments a t m uch lower rates. L ater in the session an over-all
rate of approxim ately 0.4 response per second is reached, w ith most reinforcements be
ing received w ithin 20 or 30 responses after the ratio has been completed. Occasional
short spurts of rap id responding occur m idw ay th ro u g h th e session, as a t d, e , f an d g.
O th er exam ples appear after the over-all perform ance has become linear. T he 2nd
session on F R 60 drl 6 (Fig. 590) shows an overnight loss of the control by the drl con
tingency. T he record begins w ith a sustained ru n at the term inal fixed-ratio rate a t
a; the first 6-second pause occurs some 850 responses later, a t b. T hereafter, the drl is
effective in preventing the emergence of higher rates of responding, except at c and d.
D IF F E R E N T IA L R E IN F O R C E M E N T O F R A TE
479
At the arrow the size of the ratio was reduced to 30. T he rate falls to 0.3 response per
second by the end of the session, an d a pause a n d period of sm ooth curvature develop
after reinforcem ent. Some segments, a t e, f and g, show quite smooth and continu
ous curvature. T he perform ance has adjusted to the decrease in size of ratio, and re
inforcem ents now occur m ore frequently a n d after fewer num bers of responses th an
under the previous F R 60 schedule.
Further exposure tofixed ratio with differential reinforcement o f low rate. Tw o birds were re
inforced on F R d rl 6 for 60 sessions w ith a total of nearly 450 hours on the schedule.
D uring this period the drl contingency was occasionally rem oved, and the value of the
fixed ratio occasionally changed. No single type of perform ance emerged as a gen
eral pattern.
T h e extent of the variation in perform ance throughout the experim ent is illustrated
in Fig. 591, which contains excerpts from various stages. T he segments in the figure
do not occur in any order an d sim ply represent the kinds of rate changes observed.
Records A and B show a linear perform ance in which only a very slightly lower rate
follows reinforcem ent. T he rate exceeds the critical value an d segments occur con
taining large num bers of responses because the drl requirem ent is not met. Records C
a n d D illustrate another type of perform ance which appeared frequently. T he fixed-
480
ratio segments begin w ith a low or zero rate. T ow ard the m iddle of the segment the
rate becomes high; th en an a b ru p t shift usually occurs to interm ediate rates, which
are m aintained until a pause occurs th a t satisfies the drl requirem ent. T he extent and
rate of the fast responding vary considerably. These kinds of rate changes are often
interm ixed w ith linear performances. T he perform ance appears to show a rate re
late d to each of the com ponents of the schedule a scallop for the F R , a low term inal
rate for the drl. Records E and F illustrate a perform ance w here somewhat similar
changes in the rate occur independently of reinforcem ent. These sometimes include
continuous rate changes.
All these kinds of changes ap p ear at various stages in the perform ances of all the
birds studied on FR drl. T he sequence does not ap p ear to be orderly, however. Fig
ure 592, for exam ple, shows a single session on F R 150 drl 6 after 170 hours o f rein
forcement. All varieties of rate changes during the ratio segment described in connec
tion w ith Fig. 591 m ay be found.
In C h ap ter Eight it was noted th a t on ta n d F R F I the bird m ay not be able to
m ain tain a sustained perform ance on a large fixed-ratio com ponent, but th a t a pro-
300
RESPONSES
D IF F E R E N T IA L R E IN F O R C E M E N T O F R ATE
481
longed pause and period of acceleration followed reinforcem ent, rath er th an the com
plete a n d a b ru p t cessation of responding in a simple FR . Figure 593 shows a sim ilar
effect in the perform ance after 207 hours of reinforcem ent on F R at ratios of from 30 to
100 with a 6-second drl. This perform ance preceded a complete cessation of respond
ing. W e subsequently reinstated the perform ance on this schedule by reducing the
size of the ratio com ponent. T h ro u g h o u t nearly all segments in Fig. 593 the rate
increases fairly continuously. Therefore, it usually becomes too high to satisfy the drl
requirem ent when the num ber of responses in the ratio has been emitted. T he large
n u m b e r of responses per reinforcem ent a n d the absence of any reinforcem ents after
sm all num bers of responses produce prolonged pauses after reinforcem ent, as in a
strained fixed ratio.
The effect o f removing the drl contingency. Figure 594 shows the effect of rem oving the
drl contingency for the bird whose earlier perform ance on F R 150 drl 3 has been shown
in Fig. 588. T he 1st excursion shows the perform ance after 17 sessions of reinforce
m ent on the F R d rl schedule, w ith a n over-all ra te of approxim ately 0.6 response per
second, an occasional brief pause following the reinforcem ent, and frequent shifts in
th e local rate. W hen the 3-second drl contingency was rem oved, a t the arrow , the
local an d over-all rates of responding gradually increased over the next 7 excursions
of the recording pen. T h e over-all rate of responding becam e 1.5 responses per sec
ond d u rin g the excursion (at c), a n d the term in al rate of responding, alm ost 2 re
sponses per second (at d). T h e term in al F R ra te of responding varied considerably
d u rin g the rem ainder of the session, b u t it reached a value as high as 4.5 responses
per second, as at g. Along w ith the increase in the term inal rate, a b ru p t shifts to zero
or near-zero rates of responding began to occur following the reinforcem ent, as at a,
b, an d e, or elsewhere, as at f and h.
T h e following session, shown in Fig. 595, begins w ith a recovery of some of the
properties of the previous FR drl performance. Following the reinforcement at a,
482
10 MINUTES
Second session on FR 150 after FR 150 drl 6

however, a fixed-ratio perform ance is reached com parable w ith the perform ance a t
the end of the previous session. As the session proceeds, however, the term inal rate
of responding becomes delayed, as the pause an d period of acceleration following the
reinforcem ent becom e a sta n d a rd feature of the segm ent. T h e sm ooth curv atu re
a t b and c and extended interm ediate rates preceding the term inal rate foreshadow sub
stantial pauses a t the final segments in the figure. T h e rem oval of the drl contin
gency has generated prolonged pauses after reinforcem ent and elsewhere, instead of the
previous perform ance, w here responding was sustained continuously a t low rates.
Figure 596 shows a sim ilar transition for a second bird in the experim ent. T he drl
contingency was rem oved a t the start of the session. T he bird h ad had a history of
170 hours of reinforcem ent on various fixed ratios up to 150 with differential reinforce
m ent of responses following 6-second pauses. T he over-all rate of responding accel
erates steadily, reach in g a n e a r-te rm in a l perform ance by the 3rd excursion of the
figure, w ith an over-all ra te of ab o u t 2.2 responses per second a n d a term in al fixedratio rate of 3 responses per second. As in the previous transition, a continuous d e
velopm ent of a pause following the reinforcem ent occurs as the fixed-ratio performance
Fig. 5 9 6 .
Transition fro m FR 150 drl 6 to FR 1 5 0 to FR 150 d rl 6 (second bird)
483
develops, reaching the m axim um size a t a. A t the arrow the drl contingency was
again introduced. T h e subsequent perform ance parallels the earlier transition from
F R to FR drl. At b a sustained period of fast responding occurs, although most fixedratio segm ents no longer show the effect of the previous F R reinforcem ent. T h e
large n u m b er of responses per reinforcem ent now occurring as the result of the high
rates developed u n d e r F R w ithout drl now produces pausing after reinforcem ent,
w ith one particularly extensive example.
T he rem oval of the drl contingency in a th ird bird produced a m uch slower in
crease in rate. T he entire 1st session on F R 150 w ithout drl 6 showed no change over
a prevailing F R drl perform ance. T he same perform ance continued on a 2nd session
w ithout the drl contingency, as the 1st segm ent in Fig. 597 shows. No change in
perform ance has occurred, although the n u m b er of responses per reinforcem ent cor
responds to the fixed ratio precisely. H igher rates of responding begin to em erge
during th e next 3 excursions of the recorder, particularly a t a an d b; and following the
reinforcem ent at c, a norm al fixed-ratio perform ance emerges with an over-all rate of
approxim ately 3 responses per second and local rates of 3.3 responses per second. T he
norm al fixed-ratio perform ance is sustained for alm ost 4 excursions of the recorder,
w hen there is a sudden shift to the previous drl perform ance, a t d. T his is continued
through 4 fixed-ratio segments, an d is followed by the norm al fixed-ratio perform ance
again. For the rem ainder of the session, the pause a n d period of acceleration fol
lowing the reinforcem ent increase, leading to a pause of over 20 m inutes. T h e entire
session contains 120 reinforcements, approxim ately twice the norm al num ber for the
bird; a n d some of the pausing a n d lower rates of responding tow ard the end of the ses
sion m ay be due to satiation rather th an a norm al developm ent of the schedule.
In the following session, shown in Fig. 598, the first 2 reinforcements on F R 150 w ith
out the drl continue the perform ance developed in the previous session. W hen drl 6
is again added at the arrow, the transition to a typical F R d rl perform ance is somewhat
more rapid th an the 1st transition described in Fig. 589.
484
Fig. 5 9 9 .
Transition from erf to FR 6 0 drl 6
The d eve lo p m e n t o f an FRdrl p e rfo rm a n ce im m e d ia te ly fo llo w in g erf
Figure 599 shows a 1st session on F R 6 0 drl 6 after erf. T h e pen resets a t re in
forcem ent. In the 1st segm ent in the graph, an extinction curve from the preceding
erf, nearly 200 responses are em itted before the rate falls enough for a 6-second
pause to occur. T h e rate continues to fall d u rin g the 2nd segm ent, an d a 2nd re in
forcem ent is received after approxim ately 80 responses, w ith the help of the rough
grain. T h e next 2 ratios show very low rates, p robably because of extinction a t this
value of FR. But a m ore stable perform ance soon emerges. A break-through of a
higher rate occurs a t a\ but otherwise the drl contingency sustains a generally constant
rate. N ote in particu lar the segm ent at b. This rate prevails for the rem ainder of
the session. All reinforcements are received w ithin approxim ately 10 responses of the
com pletion of th e ratio. T he figure resem bles a norm al transition from erf to F R
except th a t the linear phase is prolonged.
T he same schedule was continued for a total of 584 hours; Fig. 600 shows the fu r
th er developm ent of the perform ance. In R ecord A, after 13 hours, the rate tends to
486
increase tow ard the end of the ratio segment. This increase is slight b ut continuous
a t a a n d e, b u t large following a pause, as a t b a n d c. A t d a n d f the rate postpones
reinforcem ent far beyond the n um ber of responses in the ratio. In R ecord B, after 26
hours, the rate of responding is roughly linear, with reinforcements occurring at roughly
the fixed-ratio value. In R ecord C, after 54 hours, a pause an d acceleration after
m any reinforcem ents appear. A t g a high rate of responding im m ediately follows a
long pause. In R ecord D, after 63 hours, the pause after the reinforcement becomes
longer, an d the rate increases throughout most of the fixed-ratio segment. In spite of
th e high rate, how ever, pauses occur frequently enough so th a t the n u m b er of re
sponses does not greatly exceed the size of the fixed ratio. A t 108 hours, in R ecord E,
the pause a n d acceleration following the reinforcem ent have increased the rate b e
yond the point where the drl contingency is m et, and segments with very large num bers
of responses appear. In Record F, at 151 hours, the performance is tem porarily
stable, w ith a pause an d slow increase in rate to a low value. In R ecord G high rates
of responding em erge d u ring the m iddle of the ratio segm ent, as a t h an d i. T he
segm ents in R ecord H , after 239 hours, show a general increase in ra te th ro u g h the
ratio segment, with a tendency for a higher rate of responding to appear tem porarily in
the m iddle of the segment. In R ecord I, 10 hours later, the rate of responding is high
w hen the ratio is com pleted, an d most segments show a subsequent negative accelera
tion, with reinforcem ent occurring when the rate of responding has fallen. Two of the
m ore conspicuous S-shaped curves a p p e a r a t j a n d k. R ecord J , after 285 hours,
shows a wide v ariety of rates, in cluding a positively accelerated segm ent a t m a n d a
negatively accelerated segm ent a t I. In spite of the generally increased local rate,
the num ber of responses in the fixed ratio is not being greatly exceeded because of the
grain. By R ecord K , after 329 hours, a different perform ance emerges which results
in a m uch higher frequency of reinforcem ent. T he rate tends to be highest im m edi
ately after reinforcem ent and falls off during the rem ainder of the segment, com plet
ing p a rt of the fixed-ratio requirem ent a t a high rate b u t slowing dow n subsequently
w hen the drl requirem ent is operative. T he fine grain of the record shows responses
occurring in ra p id bursts sep arated by discrete pauses, a n d a reinforcem ent such as
487
th a t at n occurs because of the grain ra th e r th a n the low over-all rate. M a rk e d

exam ples of negatively accelerated segm ents are evident am ong linear perform ances.
In spite of the high frequency of reinforcem ent u n d er this type of perform ance, it is
not stable. Records L, M , a n d N, taken after 362, 376, a n d 382 hours, respectively,
show a variety of rate changes during the ratio segments, including nearly all of those
already described. Responding still occurs in bursts, however. This is perhaps the
only progressive developm ent in the experim ent.
T he ratio was then increased to 240, the drl rem aining a t 6 seconds, for 60 hours.
T h e result is shown in Fig. 601, where the pen does not reset after each reinforcement.
T h e b ird sustains a perform ance on this size of ratio show ing a low over-all rate,
rough grain, an d wide oscillations in the over-all rate. T he high rate a t a appears to
com pensate for the preceding pause, b u t a t b the increase precedes a period of slow
responding (at c). Little or no curvature exists after reinforcement, despite the roughly
fixed n um ber of responses per reinforcem ent.
A second bird on F R 60 drl 6 satisfied th e drl contingency by a rough grain m uch
earlier in the experim ent. T h e final perform ance showed m any m ore instances of
high rates at the beginning or in the m iddle of the ratio segment. T h e fixed ratio is
thus counted out in a short tim e, and the subsequent retardation achieves reinforce
m ent. Figure 602 sum m arizes the experim ent. In the 2nd session (R ecord A) the
over-all and local rates have already fallen; a typical perform ance consists of a pause
a n d slow acceleration to a slightly S-shaped curve. A single instance occurs of the
emergence of a high rate, possibly a residue of the previous erf. This rate could also
result from the reinforcing effect of increasing the count. In Record B, after 15 hours,
th e over-all ra te is low a n d alm ost linear, w ith reinforcem ents occurring ap p ro x i
m ately every 10 minutes. Record D shows two instances in which a high rate early
in th e ratio segm ent com pletes the ratio req u irem en t a n d the rate then falls off to
satisfy the drl before reinforcement.
Following the session represented by R ecord D, the drl was rem oved briefly, w ith
the result shown in Fig. 603. T h e first four lines represent the 1st session of F R 60.
T he over-all a n d local rates increase steadily. T h e 1st segm ent on the 2nd session
w ithout drl (beginning at a) starts w ith a long pause a n d extended acceleration, as in
the previous session; but in the 2nd fixed-ratio segment the rate increases beyond the
final rate in the previous session. T ow ard the end of the session, beginning at b, the
long pause following reinforcem ent reap p ears for 4 segm ents, after w hich a rap id
acceleration leads to an even higher over-all rate. T h e portion of the record at the
end of th e 2nd session beginning a t c shows 12 reinforcem ents occurring w ithin 15
minutes. T he 3rd session w ithout the drl, beginning at d, shows no further increase in
the rate of responding, although some local rates are considerably higher than any in
the previous sessionsfor exam ple, a t e , f g, a n d h. In spite of the absence of the drl
contingency, the grain continues to be rough. Responses occur in bursts separated
by frequent pauses of the order of 6 seconds. At i, tow ard the end of the 3rd session,
the drl contingency is reinstated, b u t it has little effect. T he 3 sessions of reinforce-
488
Fig. 6 0 2 .
FR 6 0 drl 6 portions taken over

3 3 5 hr (second bird)
m ent w ithout th e drl h a d not been effective in progressing to w ard a fixed-ratio p e r

form ance; in spite of the new schedule, the bird was still perform ing appropriately
to drl. T h e final portion of the figure shows the start of the next session under F R 60
drl 6, in which the low over-all rate seen at the start of the figure returns.
W ith the drl again in effect, the fixed-ratio segments becam e roughly negatively ac-
Fig. 60 3 .
489
Temporary omission of the drl 6
celerated, producing a higher frequency of reinforcem ent, as R ecord 602E, after 39

hours after erf, shows. In R ecord F, after 47 hours, a constant rate is too high to
satisfy the drl requirem ent even with a rough grain, and several hundred responses per
reinforcem ent are em itted. R ecord G, after 52 hours, contains sm ooth, negatively
accelerated segments, which become m ore m arked in Record H , after 68 hours. In
R ecord I, after 71 hours, the curves are accelerated, and the rate is too high at the
tim e the fixed ratio is counted out to satisfy the drl requirem ent, so th a t the segments
contain a large num ber of responses. In R ecord J , a t 160 hours, the performance con
sists of a high rate early in the fixed-ratio segm ent, followed by a shift to a lower rate
w hich satisfies the drl contingency. Sim ilar perform ances are shown in Record K, at
164 hours; Record L, at 168 hours; and R ecord M , a t 173 hours. In one segment, for
exam ple, the whole fixed ratio is em itted at almost a norm al fixed-ratio rate, and a sud
den drop to a lower rate then satisfies the drl requirem ent. T he rem aining perform
ances in the figure, Record N at 181 hours, R ecord O at 190 hours, Record P at 210
hours, R ecord Q ,a t 211 hours, R ecord R a t 299 hours, and R ecord S a t 335 hours,
show a variety of perform ances. Some segments show a high frequency of reinforce
m ent because of high rates of responding early in the ratio followed by a b ru p t shifts
to lower rates w hich satisfy the drl requirem ent. O thers show interm ediate rates
w here the grain of the record satisfies the drl requirem ent as soon as the fixed ratio is
counted out. Still others show extended scallops which considerably exceed the n u m
ber of responses in the ratio.
Figure 604 shows a com plete session following R ecord P in Fig. 602. R einforce
m ents are received alm ost on schedule because m an y ratio segm ents begin a t a high
rate of responding a n d drop to a low rate. In any case, the rough grain is im p o r
tan t.
Following the session represented in R ecord S in Fig. 602, we explored the size of
490
the ratio at larger values after brief reinforcem ent on F R 30 and 40 with the drl contin
gency. Reinforcem ent was continued on larger fixed ratios with drl 6 for almost
200 hours, d u rin g w hich there was alm ost no progress tow ard a stable perform ance.
Figure 605 contains segments from the various sessions. R ecord A on F R 220 drl 6,
after a total of 390 hours on various F R drl schedules, shows a m oderate over-all rate of
responding. T he grain is rough and the segments negatively accelerated, either w ith
sm ooth curvature, as at b, or a break-through to a higher rate early in the segment,
as at a. R ecord B, at 401 hours, shows roughly linear segments except for the pause a t
c followed by a m arked com pensation in rate. R ecord C, after 402 hours, shows a
Fig. 6 0 4 . FR 6 0 drl 6 showing negatively

accelerated segm ents and rough grain
som ew hat different type of pause a n d com pensation, a t d. Records D a n d E, a t 420
an d 424 hours, respectively, show instances of higher rates of responding im m ediately
after the reinforcem ent, as at e , f a n d g. O th e r segments, as a t h a n d i, show m arked
local rate changes, rough grain, and short bursts of rapid responding. R ecord F, a t
449 hours, shows m arked oscillations in rate w ithin the ratio segment. W hen the r a
tio was reduced, high initial rates w ith strong reta rd a tio n , S-shaped curves, a n d a
som ew hat lower m axim um rate appeared, as in R ecord G, after 534 hours. D uring
th e rem aining 83 hours of the experim ent the perform ance varied over a very wide
range. In R ecord H , after 617 hours, a long pause follows reinforcem ent, a n d re
sponding is slow a n d steady. T his is a re tu rn to a perform ance w hich prevailed
very early in the experim ent.
150 R's
492
In a sim ilar experim ent 2 birds w ith a very long history of F R were exposed to
F R d rl, w here the drl was first of the ord er of a few milliseconds. Such a drl co n tin
gency was easily satisfied by the existing rate and grain. No change in perform ance
was observed during the period of exposure. T he drl was then gradually increased
an d had very little effect until the drl reached 1 second, w hen negative curvature a p
peared. Reinforcem ents were received roughly on schedule. W hen the drl was then
a b ru p tly ch anged from 1 second to 2 seconds, the change in perform ance was m ore
pronounced, as in Fig. 606. T h e drl contingency now postpones m any reinforce
m ents; a n d before the end of the session, m ark ed changes in rate a p p e ar in the ratio
segments. A rough grain emerges, doubtless because of the drl.
Fig. 6 0 6 .
FR 1 6 0 drl 2 after FR 1 6 0 drl 1
Fig. 6 0 7 .
493
Later performance on FR 1 6 0 drl 2
F igure 607 shows representative sam ples from th e 6th, 8th, a n d 11th sessions on
F R 160 drl 2. T his is one of the m ost consistent a n d sustained exam ples we have
observed of a perform ance under F R drl in which p a rt of the ratio is ru n off at a norm al
ratio rate, with negative acceleration then leading to a low term inal rate th at, aided
by grain, satisfies the drl contingency.
E xtinction a fte r FRdrl
Figure 608 contains an extinction curve after 36 hours of reinforcem ent on F R 60

drl 6 at the stage of developm ent following R ecord B in Fig. 600, where the prevailing
perform ance was a fairly stable low rate. T h e curve shows a fairly regular decline
from an interm ediate rate to a near-zero rate by the end of the figure. T he high rates
previously shown in F R a n d occasionally on F R d rl do not appear. T h e curve sug
gests extinction after V I ra th e r th a n F R , a n d little evidence exists th a t the fixed-ratio
com ponent of the F R drl schedule has any effect.
A sim ilar extinction curve was tak e n ju s t after R eco rd M in Fig. 600, after m ore
th a n 376 hours of reinforcem ent on F R d rl. T h e over-all rate declined c o n tin u
ously, no high rate appearing.
VARIABLE INTERVAL W ITH DIFFERENTIAL REINFORCEMENT OF H IG H RATE
Figure 11, C hapter T hree, has already described a device designed to close a cir
cuit w hen a given n um ber of responses has occurred w ithin a designated tim e interval.
494
It has been used for the study of the differential reinforcem ent of high rates (drh).
T he num ber of responses required at a given ratio and the rate could be varied m ore
or less independently. In the following experim ent several values were used. T h e
conditions are relatively arbitrary an d will not be specified in detail, since the following
experim ents are in te n d e d m erely to d em o n strate the general effects of d rh ra th e r
th a n to m ake a p aram etric analysis of the variables.
In one experim ent 2 birds were reinforced on an arithm etic 1-m inute V I schedule
after erf. Figure 609A shows a stable perform ance after 8 sessions on V I 1. W e
changed the schedule of reinforcem ent to V Id rh by w ithholding reinforcem ent on V I
until the rate of responding was high enough to satisfy the d rh contingency. To the
extent th a t reinforcem ents are postponed because the rate of responding is not high
eno u g h w hen a reinforcem ent is set u p , th e m ea n in terv al (an d p articu la rly , the
shortest interval) of the V I schedule is increased. By advancing the value of drh pro
gressively in small steps, the rate on V I can be increased w ithout an appreciable in
crease in the m ean interval of reinforcem ent.
R ecord B in Fig. 609 shows a perform ance after 10 sessions on V Idrh. T he over
all rate is betw een 1.5 a n d 2 responses per second, com pared w ith slightly less th a n 1
response per second in R ecord A w ithout drh. T hree sessions later, after the n u m
ber of responses required for the drh is increased slightly, the rate increases to betw een
2.5 a n d 3 responses p er second (R eco rd C). T h e fine g rain o f th e record shows
short bursts at very high rates separated by pauses.
495
Figure 610 shows a later stage of exposure to V Id rh . In R ecord A, after 26 ses

sions, reinforcem ent is co n tingent on a higher rate , sustained for a larger n u m b er of
responses th a n in Fig. 609. T h e over-all ra te is betw een 3 a n d 3.5 responses per
second. Fifteen sessions later, in R ecord B, the required rate has been increased, b u t
the n u m b er of responses a t th a t rate rem ains the same. T h e over-all rate reaches
4 responses per second, while local rates in sustained runs w ithout pauses reach 5 re
sponses per second. N ote the absence of interm ediate rates of responding. A sec
ond bird on the same procedure showed the same result.
Fig. 6 1 0 .
Later performance on VI 1 drh
E xtinction a fte r V Idrh
Figure 611 shows an extinction curve after V Id rh u n d er a value of d rh w hich m ade

it difficult for the bird to m aintain a norm al frequency of reinforcement. T he 5 seg
m ents before the reinforcem ent a t a show a perform ance which never satisfied the cu r
rent drh. For some tim e after a, reinforcements are fairly often received a t the same
drh. T h e m agazine was disconnected a t th e arrow a n d the session continued for 24
hours. At least 80% of the responses in the curve are em itted by the end of the second
line in the figure, a little m ore th a n 1 h o u r after extinction began. M ost of the later
responses occur a t a rate com parable w ith the high rate under drh. T h e interm e
diate over-all rates at d; e, and f are composed of short bursts a t high rates separated by
pauses. A sustained burst of responding at 4 responses per second occurs a t the end of
496
Fig. 61 1.
Extinction after VI drh showing intermediate rates
the session, at g. Interm ediate rates of responding occur a t b and c, as the rate changes
from the high over-all rate in the second line to the near-zero rate during the rem ainder
of the session.
Figure 612 shows a n extinction curve after 51 sessions of exposure to V I 1 d rh w hen
the bird m aintains an almost norm al rate of reinforcem ent. Extinction is begun a t
the arrow , an d the resulting curve consists of periods of sustained responding at the
n orm al V Id rh rate altern atin g w ith increasingly longer pauses. T he periods of sus
tain ed responding grow shorter as extinction proceeds. T h e actual rates of re
sponding, even a t the end of the session, are the same as under V Idrh.
RESPONSES
300
,
,
J
/
______ r
J
,
'
Fig. 6 12 .
Extinction after VI drh showing no intermediate rates
498
DIFFERENTIAL REINFORCEMENT OF RATES WITH PAC IN G 1
W hen the rate of responding required for reinforcem ent is specified by the setting
of an u p p er lim it, as w ith drl, the bird has only to respond a t any rate lower th a n the
requirem ent to be reinforced. W hen a lower lim it is specified, as in drh, reinforce
m ents occur w henever the rate is at any higher value. H ow ever, both an u p p er an d
a lower lim it can be specified. A response m ust occur a t least x seconds since the
preceding response, an d not m ore th a n y seconds since the preceding response. T he
num ber of successive responses which m ust m eet these specifications m ay also be
specified. T he technique is called pacing. A response m eeting such requirem ents
m ay be called a paced response.
A pacing contingency is m ade effective in th e following way. T h e b ird is p u t on
V I 1 until its perform ance is stable. Every paced response is then reinforced until
th e rate alm ost always satisfies the contingency.
VI with pacing
T h e pacing contingency is added to V I 1 in a com bined schedule. In one experi

m ent the contingency required th a t 2 successive responses should n ot occur m ore th an
2 seconds or less th a n 1.5 seconds after the preceding response. This schedule m ay
be w ritten: V I 1 pacing (2) 1.5-2.0. Figures 613A and B show stable samples of paced
behavior under V I 1 pacing (3) 0.8-1.3. W hen the pacing contingency was removed,
its effect survived for several sessions. Figure 613C shows the 1st session w ithout the
pacing contingency. A 1st response, reinforced a t a high rate at a, reinstates the V I
perform ance, which is now reinforced on schedule. Nevertheless, before the short ses
sion is over, the rate has fallen to the value prevailing under the pacing contingency.
T his general ch aracter prevailed for several sessions. R ecord D in Fig. 613 is for the
4th session, and some slight survival m ay be seen in the 6th session in R ecord E. T he
7th session, in Record F, shows a stable high level of perform ance under V I 1.
1 T hese experim ents w ere carried out by Dr. A lfredo V. L agm ay, now o f the U niversity o f the Philippines,
to w hom we are grateful for perm ission to report the results here.
Fig. 6 1 4 .
499
Break-through o f a high rate

under pacing
T he bird was then shaped to a lower pacing rate u n d er conditions of V I 1 pacing

(2) 1.5-2.0. Figure 614A shows a stable perform ance. As the rate is forced to lower
values by pacing, high rates occasionally break through. An extrem e instance of this
is shown in Fig. 614B at a. T he record is for the session im m ediately following R ec
ord A a n d shows, in general, the same type of perform ance. Nevertheless, at a several
h u n d red responses are em itted at the rate appropriate to the original V I seen in Fig.
613F.
F igure 615A shows a still low er ra te m a in ta in e d u n d e r V I 1 pacing (3) 2.0-2.5.
Reinforcem ents are characteristically followed by pausing, a n d occasional periods of a
higher ra te are a p p a re n t, as a t a, b, a n d c. W hen the pacing contingency was re
m oved, th e retu rn to a V I 1 perform ance was m uch slower th an in Fig. 613. Figure
615B is for the session following th a t of R ecord A a n d shows a fairly sm ooth accel
eration to a higher over-all rate. This curve is m arked by conspicuous instances of the
em ergence of the V I rate, as a t d a n d e. Records C and D are for the sessions im
m ediately following R ecord B a n d show a progressive increase to a V I perform ance.
T he survival of the extrem e pacing contingency is nevertheless clear in irregularities,
as a t f an d periods of slower responding, as a t g.
Fig. 6 1 5 .
S low return to VI a fte r VI pacing
500
Fixed ratio with pacing
A schedule of F R 3 5 p acin g (2) 1.5-2.0 was su b stitu ted for a V Ip a c in g schedule

after a stable pacing perform ance h a d developed. T h e experim ent was designed to
see w hether the constancy of num ber of responses a t reinforcem ent an d pacing con
ditions would begin to produce an effect by m aking the production of num ber reinforc
ing. Figure 616 shows the developm ent from the 5th through the 50th session.
R ecord A is for the 5th session on F R 35 pacing after V I 1 pacing. R esponding is still
m ainly a t the paced rate, although some pausing or slow responding appears after
reinforcem ent. This condition prevails through the 6th session (R ecord B), although
the term inal rate is increasing slightly. E ventually, this condition makes it difficult
to satisfy th e pacing contingency a n d th e n u m b e r of responses a t reinforcem ent inRs/SEC
L A -/L A A /
y [y iA A A ^ A ^ ^ tA y L A y [^ y L A
c
. ^
- - v 4 /L -
A A A A A A J A ^JJÂ J\A A M A A /Jy

v
A AM V U W \A A \A A A J[a AAa A JA A J
A A /\jk ftA A A A A j'A A ^ V ]J ^ ^ / ^ j '
A A A A A A A A A A A A A A A S M ^ M a^
^ a A a % a A a /W\a ^ A \ m m
Fig. 6 1 6 .
D e v e lo p m e n t o f an FR p e rfo rm
ance w ith pacing
creases greatly, as R ecord C shows for the 8th session. T h e increase in the num ber
of responses per reinforcem ent eventually produces a reduction in rate which corrects
this change, an d by the 15th session (R eco rd D) th e n u m b er of responses a t re in
forcem ent has again becom e fairly constant. Slight negative c u rv atu re appears to
w ard the end of m any of these ratio segments, however, an d by the 17th session, in
R ecord E, the curves have come to resem ble strongly those under F R drl. (See the
earlier section in this ch apter.) T he rem ain in g records in the figure are from the
19th, 23rd, 29th, 33rd, a n d 50th sessions. T h e perform ance shows the same kind of
instability seen under FR drl.
Figure 617 shows a com plete session (considerably longer th an usual) from the se
ries shown in Fig. 616. T he session occurred betw een Records D and E in the earlier
figure. As w ith F R drl, various types of perform ance come an d go th roughout the
session.
AM A A A A A A A A A A A A A A A A J\J lA J u \
501
^ A A- A
A A A A \A \A A Â A A J\/ Â A A A A L A A A A A A A A A /A A rA A A A s
V A A A A A \A A jV ^ j\y\A \yvyV A \yiA \^^yvA \ yÂ
W hen V Ipacing was again restored after F R pacing, the earlier perform ance was,
in general, recovered. H ow ever, the pausing after reinforcem ent persists a n d is usu
ally to some extent com pensated for by a short period of rap id responding. Figure
618 shows a 3rd session on V I 1 pacing (3)1.1-1.4 after the long series of experim ents
ju st described on FR pacing. N ote the consistent pause after reinforcem ent and, in
most cases, a com pensatory ru n which restores the curve to approxim ately the extrapo
lation of the preceding segments.
THOUSANDS OF
RESPONSES
502
Fig. 6 1 9 .
Extinction a fte r V lpacing
Extinction a fter Vlpacing
Figure 619E shows a n extinction curve tak en after a w ell-established V lpacing.

T h e principal portions of the curve d u rin g w hich the bird was responding are shown
a t A, B, C, an d D for the segments indicated. If responding occurs at all, it occurs
a t approxim ately the earlier paced rates. Segment D contains some interesting pauses
a n d com pensatory increases in rate.
Chapter Ten
MULTIPLE SCHEDULES
INTRODUCTION
A m u l t i p l e s c h e d u l e consists of two or m ore alternating schedules of reinforcem ent

w ith a different stimulus present during each. T h e schedules m ay alternate simply or
a t random . Schedule changes are usually m ade after reinforcem ent, although this
condition is not essential. T h e fact th a t perform ances under the various com ponents
of the m ultiple schedule are usually quite independent of each other provides a tech
nique for arran g in g control perform ances w ithin a single subject a n d a single session.
For exam ple, a variable-interval schedule as one com ponent of a m ultiple schedule
m ay be used as an index of m otivational conditions whose effect upon the other com
ponent schedules is being studied. As alread y noted, a m ultiple schedule is desig
n ated w ith the word m ult followed by designations of the schedulesfor example,
m ult F R 50 F I 10.
MULTIPLE FIXED-INTERVAL FIXED-RATIO SCHEDULES
Development and final performance
Development after erf. A m ultiple schedule m ay be p u t in force im m ediately after
erf. F igure 620 shows p a rts of th e first 3 sessions of th e tra n sitio n from e rf to m u lt
F I 1 0 F R 2 0 . R ecord A shows the first 3 hours after erf. T he first 2 reinforcements
a t a an d b hap p en ed to occur on the F I 10 schedule while the key was red. T hey re-
503
504
fleet m ainly the extinction of the erf, d u rin g w hich the key h a d been both red an d
green from tim e to tim e. Im m ediately after the reinforcem ent a t b, the key becomes
green, an d the 3rd reinforcem ent a t c occurs on the F R 20 schedule. A negatively
accelerated segment follows during the next FI 10 segment on a red key, and reinforce
m ent occurs after a long pause a t d. T hree F R 20 segments then occur with the key
green at the highest rate so far observed. T h e F I 10 segm ent w hich follows begins a t
th e same high rate. Beginning w ith the fixed-ratio segm ent a t e, all fixed ratios in
th e presence of the green key-color are ru n off a t a fixed-ratio rate. M eanw hile, re
sponding im m ediately after reinforcem ent declines progressively during the FI 10 seg
ments, w hen the key is red. A corresponding increase in rate appears tow ard the end
of the interval. By th e end of R ecord A the ra te in the presence of the red key is
essentially constant throughout the interval segment.
After 1 \ hours of exposure to the m ultiple schedule (R ecord B), a slightly lower
ra te im m ediately after reinforcem ent appears in the interval color. T he rate is con
stan t at 3 responses per second w hen the key is green. In R ecord C, after 13 hours
of reinforcem ent on m ult FI 10 F R 20, the 3rd session following erf, a m arked pause fol
lows reinforcem ent under the fixed-interval color if the preceding schedule was fixedratio, as at h an d i, b u t not w hen 2 intervals follow each other, as at f an d g. N ote the
small bite betw een g a n d h.
By the 7th session following erf, a fairly stable perform ance under the m ultiple sched
ule has developed, as seen in Fig. 621. T he fixed ratio h ad been increased from 20
to 50 responses 7 hours earlier. A m ark e d fixed-interval scallop occurs a t c a n d g,
w here the preceding schedule is fixed-ratio; b u t a t e, h, a n d i, w here the preceding
schedule is also fixed-interval, the rate is alm ost constant th roughout the segment, ex
cept for a slight curvature im m ediately after reinforcem ent. In general, the extent
o f th e fixed-interval scallop depends u p o n the n u m b e r of preceding fixed-ratio seg-
M U L T IP L E SCH EDULES
505
ments. For exam ple, the pause an d scallop in the segments a t c a n d g, following 3
ratios, are m ore pronounced th a n those a t a, d, a n d f , following 1 ratio. A t b a burst
of approxim ately 50 responses occurs at the fixed-ratio rate in the m iddle of a fixedinterval segm ent in the presence of the fixed-interval color. Figure 622 shows a m uch
later perform ance for the same bird on m ult FI 1 0 F R 5 0 , 131 hours, or 39 sessions,
after erf. P ausing a n d acceleration now follow reinforcem ent, even in interval seg
m ents preceded by interval segm ents. Scalloping d u rin g fixed-interval segm ents
preceded by fixed-ratio segments is m ore m arked, however; a n d in m any cases, as a t a
a n d c, a pause a n d low rate extend th ro u g h most of the segment, the term inal rate
not being reached. Scallops in fixed-interval segments preceded by fixed-interval seg
m ents v ary from a b rief period at a slightly low er rate of responding, as at b, to a
substantial pause, as at d. T he over-all rate has fallen progressively during the devel
opm ent of the perform ance on the m ultiple schedule, largely because of the increase
in the extent of the pause and scallop in the fixed-interval segments.
Fig. 6 2 2 .
M u lt FI 10 FR 50, 39 sessions a fte r erf
T h e second bird in the experim ent showed a sim ilar transition to a sim ilar final
perform ance.
F inal perform ances were also recorded for 2 birds after a n extended history on
m ult FI 5 F R at various fixed ratios. Figure 623A shows a segment after a history of
44 sessions of a m ultiple fixed-interval fixed-ratio, during which the perform ance
was probed w ith T O s after reinforcem ent in several sessions; an d in several other ses
sions the schedule was FI 5 only. In the perform ance in Fig. 623A the rate under the
fixed-ratio schedule is 4.5 to 5 responses per second, com pared w ith a term inal rate
of 3.5 to 3.7 responses per second in the F I 5 schedule. T he difference is by no m eans
so great as in the previous experiment. U nder the FI 5 reinforcement the pause an d
acceleration after reinforcem ent are the sam e w hether or not the preceding segm ent
was interval or ratio. N ote the knee a t a. T he perform ance in R ecord B was re
corded 21 sessions later, w hen the fixed ratio h ad been increased to 100. T h e extent
of the pause u n d er the FI 5 reinforcem ent has increased, a n d the term inal rate has
fallen. T h e rate under the fixed-ratio schedules has increased. T he FI rate a t b, for
exam ple, is 2.5 responses per second, while the F R rate a t c is 4.5 responses per sec-
506
ond. In R ecord C, 7 sessions later, the fixed ratio has been increased to 175, a n d the
perform ance rem ains sim ilar to th a t in R ecord B. In Record D, 3 sessions later, the
fixed ratio was increased to 225. R ates of approxim ately 4 responses per second now
ap p ear during the fixed-interval segments (at d a n d e), b u t fall to a lower value b e
fore reinforcem ent., A slight pause appears a t the start of some ratio segments. In
R ecord E, 5 sessions later, the fixed ratio has been increased to 300. T he n um ber
o f responses being em itted in th e ratio a n d in terv al segm ents is roughly of th e sam e
order, although the rate of responding in the ratio segm ents is 4 responses per sec
ond, com pared w ith rates of the order of 2.5 responses per second a t the end of the in
terval segments. A second bird in the experim ent showed a similar performance.
T h e fixed ratio was th en increased to 375. Figure 624 shows the perform ance 15
sessions after Fig. 623. T he m ultiple control is still being m aintained, with term inal
interval rates of approxim ately 2 per second a n d ratio rates of m ore th an 4 per second.
Fig. 6 2 4 .
Early perform ance on m ult FI 5 FR 375
507
N early all fixed-interval segments begin w ith a substantial pause and contain some
acceleration to the term inal rate. Pauses at the start of the fixed-ratio segments are
either absent or only a few seconds long. Some signs of loss of m ultiple control b e
gin to a p p e a r a t a a n d b. H ere, after a short pause in th e interval color, responding
begins at the ratio rate before it shifts to a lower rate and resumes the norm al interval
p attern . A n extrem e exam ple is m arked by the arrow . T h e transitions from the
pause to the term inal rate in the interval segments are becom ing m ore abrupt.
R einforcem ent was continued on m ult F R 375 F I 5 for slightly m ore th an a m onth,
after which the perform ance shown in Fig. 625 developed. T he term inal rate in the
interval has increased so th a t all segments are approxim ately of the same length and it
is difficult to differentiate between ratio and interval segments. Ratio segments have
been m arked by dots at the start of the segment. Perform ances in some segments are
clear-cut, as, for exam ple, the ratio segm ents at a a n d / a n d interval segments a t b
a n d d. O n th e o th er h a n d , some ratio segm ents, as a t c, e, a n d g, resem ble interval
segments.
A very different set of values was chosen in an o th er experim ent in w hich the ratio
was low (26) an d the interval long (18 m inutes). Figure 626 shows a final perform
ance after a history of 357 hours of reinforcem ent on m ult F IF R w ith occasional in tra
m uscular injections of sodium pentobarbital. As in some of the previous figures, ex
ten d ed scallops occur in interval segm ents preceded by ratio perform ances (as a t a, c,
a n d d), while interval segments preceded by other fixed-interval perform ances, as a t e
a n d f show a constant rate of responding th ro u g h o u t the 18-m inute segment. T h e
term inal ratio rate is of the order of 5 responses per second; the m axim al rate during
a fixed-interval segment is 0.73 response per second.
Development o f mult F I 10 F R 70 after extended F R 70. Tw o birds, whose late r p e r
form ances will be described in C hapter Fourteen under M ultiple C hains, developed
508
a m u lt F I 1 0 F R 7 0 p erform ance after previous exposure to F R 70 alone. Figure

627 shows the 1st transition to the m ultiple schedule. T he 2 com ponents of the m ul
tiple schedule are recorded separately. In R ecord A the session begins u n d e r the
fixed ratio in the presence of a red key. T he key-color is novel and has the usual effect.
T h e bird takes more th a n 10 m inutes to emit the ratio and receive the reinforcement at
a. T he 2nd ratio segm ent ending a t b was program m ed next. It is negatively accel
erated. T he key then changed to blue, the color present during the earlier F R 70,
a n d the 1st interval segm ent in R ecord B was recorded. T h e 3rd ratio segm ent was
th en executed. It shows positive acceleration to a high term inal rate. T h e ratio
ra te increases progressively for the rem ain d er of the session, w ith an im provem ent in
grain. T h e fixed-interval perform ances interspersed w ith these fixed-ratio perform
ances are negatively accelerated, as in a transition from erf, b u t the curves show some
effect of the previous fixed-ratio reinforcement. R ecord G contains a segment of the
interval perform ance from the 2nd session on the m ultiple schedule, after a total expo
sure of 6 hours. H ere, the perform ance is roughly linear, with some slight pause or
lower ra te of responding following reinforcem ent, as at c a n d d, w here fixed-ratio seg
ments preceded. T he 3rd session, exemplified by R ecord D, after 10 hours of rein
forcem ent on the m ultiple schedule, shows m arked scalloping w henever the interval
509
segments are preceded by ratio segments, a n d an alm ost constant rate otherwise, as a t
the last 2 reinforcements in the record. M eanw hile, the fixed-ratio performances re
m ain sim ilar to the last segments of R ecord A.
M u lt F IF R in the rat. T hree rats gave a sim ilar perform ance on m ult FI 5 F R 20.
T he previous history included m ult erf extinction FI 10 (to establish a stimulus for T O ),
F R 2 0 , an d F R 3 0 . In the m ult F IF R the com ponent schedules appeared in simple
alternation. T h e general illum ination flashed on an d off once per second when the
schedule was fixed-ratio, a n d rem ain ed steady w hen the schedule was fixed-interval.
T h e results m ust be qualified by the in ad e q u a te am o u n t of food used as reinforce
m ent. As already noted, it was discovered later th at m ore stable perform ances would
result w ith 0.1 to 1.5 grams per reinforcem ent instead of 0.05 gram , as in this experi
ment.
Figure 628 contains segments from the first 38 hours on m ult F I 5 F R 20, during
which a fairly stable performance developed. T he first exposure to the m ultiple sched
ule (R ecord A) shows the effect of the preceding fixed-ratio reinforcement, which b e
comes less pronounced as the session progresses. By the end of this 1st session (Record
B) clear evidence exists of m ultiple control. Responding begins im m ediately after re
inforcem ent during the flashing light w hen the schedule is fixed-ratio; and a pause w ith
subsequent acceleration after reinforcem ent occurs w hen illum ination is steady an d
the schedule is fixed-interval. In R ecord C, after 11 hours of reinforcem ent on the
m ultiple schedule, the m ultiple control is clear. T he flashing light controls a rate of
approxim ately 2.5 responses per second u n d er the fixed-ratio schedule. T he steady
light produces a pause of up to 2.5 m inutes at the beginning of the interval and a term i
nal rate of 1.5 responses per second under the interval schedule. Occasionally, as at a,
the rat responds ratio-wise im m ediately after reinforcement, but then pauses and accel
erates to the term inal interval rate. T ow ard the end of the session the rates during
the fixed-interval stimulus fall to low values, as at b and c, where responding does not
begin until near the end of the segment a n d the term inal rate is not reached. R ecord
510
D shows a late stage of developm ent, after a total of 38 hours on the m ultiple sched
ule. T he term inal rate under the fixed-ratio schedule is of the order of 3 responses per
second, com pared with slightly over 2 responses per second a t the end of the interval
segments. T h e fixed-interval performances are uniform from segment to segment w ith
a substantial pause extending th rough most of the interval a n d a fairly a b ru p t shift
to the term in al rate. Instances occur a t d an d e w here the ratio rate appears briefly
during the interval segment.
Figure 629 shows a m uch later perform ance for this rat on m ult F I 5 F R 40.
A second ra t in the experim ent showed roughly the sam e developm ent of a m u lti
ple perform ance except during the early stages, w here the grain was very rough. A
w ell-m arked m ultiple control developed in the 2nd session, a n d later perform ances re
semble th a t of Fig. 628C or 628D. A th ird ra t showed a sim ilar course of develop
m ent except th a t a very low over-all rate developed after the 3rd session, a n d long
pauses appeared, particularly a t the beginning of the fixed-ratio segments. L ater, the
perform ance returned to the patterns of Fig. 628D.
Extinction a fte r multiple FIFR
Extinction after a m ultiple schedule m ay be carried out in a num ber of ways, includ
ing the following:
( 1) T h e response is extinguished in the presence of the stim ulus correlated w ith one
schedule, the m ultiple schedule is re-established, and the response is then extinguished
in the presence of the stimulus correlated w ith the other schedule.
(2) T h e response is extinguished in the presence of the stim ulus correlated w ith one
schedule, a n d th en in the presence of the stim ulus correlated w ith the other schedule.
(3) E xtinction is carried out while the stim uli are rotated. For exam ple, after the
num ber of responses in the fixed ratio has been em itted, the key changes to the color
associated w ith the fixed-interval reinforcem ent; this key-color is present for the d u ra
511
tion of the fixed interval. T h e color th en changes if the next schedule w ould have
been a ratio, or rem ains the same if another interval occurs in the series.
(4)
R einforcem ent m ay be continued on one schedule, while extinction is carried
out in the presence of the second stim ulus appropriate to the other schedule. T he last
alternative is technically either m ult FR ext or m ult Flext.
Several types of extinction curves were produced by a bird which had had an exten
sive history on m ixed fixed-interval fixed-ratio schedules (see C h ap ter Eleven), and
th en on a m ultiple schedule in w hich th e stim ulus correlated w ith the com ponent
schedules was either a steady or flashing green key-light. Figure 630 shows a stable
perform ance on the m ult F I 10 F R 125 schedule in the portion of the record before the
arrow. Fixed-ratio a n d fixed-interval schedules alternate w ith each other, an d the
perform ance is stable and sim ilar to those w hich have already been described. At the
arrow the reinforcem ent was om itted, an d the key-light was simply changed to the
p a tte rn correlated w ith the fixed-ratio schedule. For the rem ain d er of the session
the hatches indicate changes in key-light p attern rather than reinforcements. T he fact
th a t the resulting extinction curve shows m arked deviations from the previous m u l
tiple F IF R perform ance indicates th a t the m ultiple perform ance depended upon the
reinforcem ent as m uch as on the stimuli on the key. T he rate does not drop to zero
w hen the stimulus correlated with the fixed-interval schedule appears at a. Instead,
th e rate of responding from the previous fixed-ratio segm ent is m aintained for a short
tim e, after w hich it falls off abruptly to an interm ediate value w hich then increases
slightly as a norm al fixed-interval segment. In the interval segment at d the rate
continues to fall throughout. Fixed-ratio performances also change m arkedly with the
omission of reinforcement. At b the segment begins at the rate from the previous in
terval perform ance, w hich is m aintained for h a lf of the ratio before the rate increases ;
a n d in the segment at c an interm ediate rate is m aintained throughout.
512
Fig. 6 3 1 .
Transition from mult FIFR to mult ext FR
Tw o sessions after the extinction curve in Fig. 630, we carried out partial extinction
of the m ultiple schedule by discontinuing reinforcements in only the fixed-interval
color. T he session shown in Fig. 631 begins w ith a stan d ard m ultiple perform ance
as in Fig. 630. Beginning at the arrow, reinforcem ents a t the end of the 10-minute,
fixed-interval period were om itted. T he stimulus simply changed to the other p attern
at the end of the designated interval. Im m ediately, the rate during the 1st ratio seg
m ent (a) falls below the usual value, w ith rough grain. T he next fixed-interval seg
m ent ( b) shows a constant interm ediate rate throughout. For the rem ainder of the ses
sion the over-all rate of responding in the presence of the fixed-interval stimulus falls
off progressively. T he norm al character of the fixed-interval perform ance is some
times preserved, as at c, d, and k. Exam ples are present, however, of the pattern disin
tegrating, as at e and h, w here substantial responding follows reinforcem ent before the
rate falls a n d accelerates to the term inal value. At g the initial rate in the interval
segm ent is alm ost equal to the fixed-ratio rate, and it declines continuously during the
rem ain d er of the segment. T h e p a tte rn of responding is disrupted, even though each
perform ance is preceded by the reinforcem ent of the preceding fixed ratio. T he fixedratio perform ance is also affected by the extinction. Segments at f , i, and j are run off
at the term inal inter val rate. T ow ard the end of the session, along w ith fixed ratios at
lower rates of responding, as a t I, a pause m ay follow reinforcem ent, as a t m and n,
although the term inal rate is norm al. T he ability to sustain a large ratio in a m ultiple
schedule m ay be due in p art to induction from the fixed-interval reinforcements. A l
though extinction in the presence of the fixed-interval stimulus is not complete by the
513
end of the session, it has reached a low value and the ratio perform ance begins to
strain.
Five sessions after Fig. 631, a th ird type o f extinction was carried out, in w hich the
stimulus previously appropriate to the fixed-ratio schedule of reinforcement was pres
ent continuously for the first p art of the session, and the stim ulus previously correlated
w ith the fixed-interval schedule of reinforcem ent was present during the latter part.
This session, shown in Fig. 632 and 633, begins with m ultiple FI 10 F R 125; and the
perform ance before the arrow in Fig. 632 is typical of this schedule. Following the re
inforcem ent at the arrow , the m agazine was disconnected. T he stim ulus rem ained
appropriate to the ratio com ponent. T he extinction curve which follows shows re
sponding at the earlier high fixed-ratio rate, as at a, c, e, g, i, j , and k. It also shows
periods of zero or near-zero rate. Sustained responding also occurs, however, appro-
Fig. 6 3 3 .
C ontinuation o f Fig. 6 3 2
priate to fixed-interval reinforcem ent, as a t b, d , f an d h. T h e transitions from one

rate to another are relatively abrupt.
T h e same extinction session is continued in Fig. 633. At the arrow the stim ulus
has changed to the p a tte rn previously correlated w ith fixed-interval reinforcem ent.
E xtinction then continued for 4^ m ore hours, during w hich approxim ately 1000 re
sponses were em itted. T he curve is com posed largely of 3 wave-like changes, each
resem bling a fixed-interval segment with a fairly smooth acceleration to a term inal rate
of approxim ately 0.75 response per second. T he small n u m b er of responses in this
p a rt of the curve indicates th at the previous extinction in the presence of the fixed-ratio
stim ulus has affected the am ount of behavior available in the presence of the fixedinterval stimulus.
A nother bird was extinguished in the presence of the fixed-ratio stimulus after prior
extinction in the presence of the fixed-interval stimulus, after 191 sessions on various
values of m ult F I 5 FR. T he complete extinction curve is presented in Fig. 634C in re
duced form. T he recorded form is shown a t A a n d B. T he extinction in the pres
ence of the fixed-interval stim ulus (R ecord B) shows various interm ediate rates !as well
as 1 period of very rapid responding for more th an 2000 responses. T he 2nd extinc
tion curve, in the presence of the fixed-ratio stimulus, consists of 2 large segments at the
514
Fig. 6 3 4 .
Ext a fte r mult FIFR: interval stimulus fo llo w e d by ra tio stimulus
ratio rate. The decline to a near-zero rate is not im m ediate, but the high fixed-ratio
rate of responding is abruptly resumed.
Increasing the over-all rates on a multiple schedule by reinforcement
o f the separate members
Figure 635A shows a given stage in the developm ent of a m ult FI 1 0 F R 6 5 for a
bird whose perform ance has already been considered (Fig. 620, 622, 631, 632, and 633).
T he scalloping in the interval is w ell-m arked, except th at intervals which follow in te r
vals show no strong tendency to scallop (b a n d c). M eanw hile, the ratio perform ance is
Fig. 6 3 5 .
Increase in over-all rate by separate reinforcem ent o f com ponents

o f a m ultiple schedule
515
irregular, showing pauses, as at a, a n d an unusually low term inal rate. W e tried to

raise the rate by reinforcing continuously on the ratio schedule and color (abandoning
th e m ultiple schedule tem porarily) and by advancing the size of the ratio. O ther ses
sions contained only reinforcement of the interval schedule under the appropriate color.
This im proved the m ultiple perform ance, as R ecord B in Fig. 635 shows. T he ratio
is now 130 ra th e r th a n 65, a n d although some pausing occurs, as a t h, the rate is now
above 3 per second. T he term inal rate in the interval is also higher th an th a t in R ec
ord A. T he pausing in the interval is also m ore pronounced, so th a t the scalloping is
more m arked (d' e, and g). T he bird is still not scalloping, however, when one in ter
val follows another (f).
This tre a tm e n t of a weak F R on a m ultiple schedule builds up the ratio in the

absence of contrary inductive processes from a m ultiple FI schedule. T here appears to
be a reverse influence upon the interval performance in the form of a higher term inal
rate after the ratio has been so built up.
Reversal o f stimuli
T h e perform ance before the arrow in Fig. 636 developed after 12 hours of reinforce
m ent on m ult FI 5 F R 80, with a blue light on the key during the F I 5 reinforcem ent
a n d a red light on the key during the F R 80 reinforcem ent. At the arrow the stim uli
in the m ultiple schedule were reversed. T he stim ulus which had been present d u r
ing the fixed-interval reinforcem ent now occurred when the schedule of reinforcem ent
516
was fixed-ratio, and vice versa. T he first stim ulus after the reversal was the former
fixed-ratio color, now present during fixed-interval reinforcem ent. T h e fixed-ratio
rate of responding is m aintained for several h u n d red responses, w ith a slight decline in
the rate a t a and again at b ju st before reinforcement. At the end of the FI segment
a t c the schedule of reinforcem ent was changed to fixed-ratio, and the stim ulus on the
key changed to the form er interval color. T he fixed-interval segment following the
reinforcem ent a t d began w ith a brief period of responding at the fixed-ratio rate, fol
lowed by a shift to an interm ediate rate for the rem ainder of the interval. A high
rate developed almost im m ediately in the presence of a new fixed-ratio stim ulus (e) ;
an d responding decreased progressively during the fixed-interval segment, as the effect
of the previous fixed-ratio reinforcem ent wore off and the fixed-interval reinforcements
took effect. Before the end of the session the new stim uli were in alm ost com plete
control of the fixed-interval an d fixed-ratio performances. T he fixed ratios were now
being ru n off at a constant high rate of responding, as a t f an d i, a n d m any of the
fixed-interval segments showed smooth and extended curvature, as at g and h.
R einforcem ent was continued on the reversed m ultiple schedule for a total of 33
hours. Figure 637A shows a well-developed performance. A high rate prevails in the
presence of the fixed-ratio stim ulus, an d a m arked scallop and acceleration to a lower
term inal rate in the presence of the fixed-interval stimulus. Scalloping is less m arked
in those intervals not preceded by fixed-ratio segments, as at a. A t the start of the
next session (R ecord B) the stim uli in the m ultiple schedule were again reversed, re
tu rning to conditions before the reversal in Fig. 636. T he 2nd transition took place
m uch m ore quickly and resulted in a more orderly performance. A norm al fixed-ratio
perform ance em erged beginning at b, w ith a rate of m ore th a n 5 responses per sec-
517
ond. T he reversed fixed-ratio rate was m aintained through most of the first 6 interval
segments, but at c a scallop began to form. C urvature developed progressively tow ard
the end of the session. An effect of the previous fixed-ratio reinforcem ent survived in
several fixed-interval segments which began at the ratio rate a t d, e, and f .
The effect o f a sudden change o f stimuli on a multiple FIFR schedule
In the experiments on m ultiple F IF R already described the interval color was occa
sionally changed when the bird was in the m idst of a ratio performance. Figure 638
shows some typical effects. The beginning of the ratio segments are m arked by the dot
over the reinforcement. In Record A the arrow indicates the change to the FI color
after a rath e r rough ratio perform ance. T he ratio rate continues with only a slight
drop for about 30 responses. T h e rate then breaks; after a pause, it accelerates to es
sentially the ratio value at a. It then drops to the interval rate a t b. Two other exam
ples in w hich the ratio rate holds for a short tim e in the interval color occurred in
the 1st session of such probing.
T hereafter, the change was im m ediate, as Curve B shows. A t each arrow a change
was m ade from the ratio to the interval color. An interval scallop followed im m edi
ately. Both of these interval segments show a slight drop to a lower rate before the in
terval is completed (at c and d j.
At e the ratio rate continues for alm ost 100 responses before the bird shifts ab ru p tly
to the interval rate, as at f and g. In R ecord C the interval color was also introduced
at the arrow during the pause at the beginning of a ratio. H ere, the interval term i
nal rate was not reached quite so abruptly.
Figure 639 shows a change for another bird together with an additional effect. T he
record begins w ith 2 interval perform ances (a a n d b) and then changes to a ratio. A t
the first arrow the key was changed to the interval color. T here was an im m ediate
518
drop to the term inal interval rate, w hich continued until the reinforcem ent a t the sec
ond arrow. Thereafter, the ratio color prevailed an d extinction was perm itted to take
place. This procedure gives some indication of w hat w ould have h appened a t the
first arrow if the bird h a d been perm itted to continue to respond u n d e r the ratio color
b u t w ithout reinforcement (as was the case because of the change to the interval p e r
form ance). T he figure also supplies an exam ple of extinction in the ratio color of a
m ultiple F IF R schedule. In general, the m ajor responding occurs a t the ratio term i
nal rate, although some negative curvature is present as th a t rate falls off a t c and e,
executed m ainly by a roughness in grain. T h e retu rn to the high rate is usually very
ab ru p t, as a t d and f .
Change to the Fi stimulus during a strained ratio on a multiple FIFR
A pigeon which had difficulty in holding m ult F I 5 F R 275 provided an opportunity

for testing the effectiveness of the interval color during prolonged pauses on the F R
com ponent. Figure 640 shows an exam ple, together w ith the current state of the p er
form ance. Tw o interval segments w ith a m oderate term inal rate are followed by
a strain ed ratio at a. In the next interval only ab o u t a h u n d red responses occur
before reinforcem ent. A nother exam ple of this occurs later at d. A second ratio
shows a long pause at b. At the first arrow the color of the key was changed to th at
appropriate to the F I schedule. Responding began im m ediately and accelerated to
the term inal interval rate. At the second arrow the ratio color was replaced. An ex
ceptionally long pause followed before the ratio wa com pleted a t c. Evidently the
conditions producing the pause at the beginning of the ratio segment clearly involve the
stim uli appropriate to the ratio perform ance a n d are not due to em otional or m oti
vational conditions or fatigue.
Figure 641 shows other examples of this effect. R ecord A is a single ratio period
betw een reinforcem ents in which a m om entary change was m ade to the interval color
at one point. Responding began im m ediately and accelerated to the interval term i
nal rate. In a similar example at Record B acceleration was slower and produced only
Fig. 6 4 1 .
519
C hange to the Fl stimulus d u ring a strained ra tio
a m oderate rate before the ratio color was restored. O th er exam ples of the same p ro
cedure appear in Records C, D, and E.
Figure 642 illustrates a further exam ple of the effectiveness of the interval color while
th e ratio is badly strained. T h e session began (top line) w ith an interval perform
ance, followed by the ratio color in which a few responses were em itted in the region of
a. A pause of about 80 m inutes then occurred. At b the interval color was in tro
duced, and a fairly typical perform ance followed and was reinforced a t c. A return to
Fig. 6 4 2
Change to the FI stimulus du rin g a strained ra tio
520
the ratio color then produced another pause of about 40 m inutes, at the end of which
a change to the interval color again produced an interval perform ance w ith reinforce
m ent at d. T he color then rem ained ap p ro p riate to an interval, and another rein
forcem ent was received a t e. A change to the ratio color was then followed by a pause
of about 45 m inutes. As the session continued (not shown in the figure), the color
was changed to th a t ap p ro p riate to the interval a n d the first response was reinforced.
T he response occurred w ithin a few seconds of the change. A retu rn to the ratio color
produced an 83-m inute pause; at the end of this the interval color was restored, an d
a response occurred in a few seconds a n d was reinforced. A retu rn to the ratio color
produced an o th er pause of 35 m inutes w hen the session was ended. T he only re
sponding in the ratio color th roughout the entire session occurred at a. T h e interval
color, however, produced appropriate interval perform ances whenever introduced.
The interpolation o f blocks o f FR in mult FIFR
T he in terrelation of com ponent m em bers in a m ult F IF R is most obvious in the

effect upon the interval scallop of a preceding ratio perform ance. W e exam ined this
relationship by inserting blocks of ratios in an FI schedule after a history of m ult F IF R .
O ne bird w ith a substantial history of m ult F I 5 F R 40 was p u t on FI 5, a n d a block
of 9 F R reinforcem ents under the ap p ro p riate form er F R color was interpolated in
each of 2 sessions. Figure 643 shows the results. In both sessions the interval follow
ing the interpolation contains an exceptionally long pause (a a n d f ) and, consequently,
only a small num ber of responses before reinforcem ent (b an d g). A lthough some evi-
521
dence exists of running-through after reinforcem ent on this schedule before interpola
tion of F R under stimulus control, running-through becomes very conspicuous im m e
diately after the interpolation. Slight running-through occurs a t a and b, and 3 very
m arked examples occur later at c, d, and e. O n the 2nd day, there is running-through
z X f a. conspicuous case a t g, an d slight exam ples a t h a n d i. An acceleration to a
brief ru n at the ratio rate appears following g.
FI a fter mult FIFR
O ne characteristic of m ult F IF R is th a t the F I perform ances are smoothly scalloped

a n d give the appearance of FI w ith T O after reinforcem ent. T h e F R com ponents
serve as TO s for the interval schedule. W e checked on this by changing to straight
FI 5 after m ult FI 5 F R 40. Figure 644 shows the entire session. T he previous history
included 68 hours of m ult FI 5 F R 40-80. In the figure the last ratio appears at a.
At this tim e the interval shows smooth scalloping. Beginning a t the arrow the sched
ule was FI 5. Fairly consistent scalloping continues for some time. At b, however,
a small run-through appears, an d at c, a w ell-m arked knee. T he pause at d is brief
a n d is followed by a very long ru n at essentially the ratio rate a t e. L ater, an an o m a
lous linear low rate occurs at f and the following reinforcement is followed by a brief
prim ing ru n of a ratio-like character a t g. A t h there is a complete running-through,
characteristic of FI w ithout T O . R unning-through is com m on thereafter. Some in
stances are w ell-m arked, as at i, an d extrem ely rapid following short pauses, as at j .
A nother im m ediate assum ption of the term inal rate occurs a t k, a w ell-m arked runthrough followed by an interval curvature a t I, and a run-through w ith negative accel
eration m arked by a bite, as at m. Some of this behavior m ay be attrib u ted to the
earlier ratio com ponent of the m ultiple schedule, a n d suggests prim ing in a m ixed
F IF R . (See C h ap ter Eleven.) O th er parts of it suggest F I w ithout T O , and show
clearly enough th a t the F R m em ber of the m ultiple schedule was functioning as a
T O with respect to the interval m em ber.
Fig. 6 4 4 .
Transition from m ult FIFR to FI
522
T he possibility should not be overlooked th a t the high rate and constant count at re
inforcem ent in the ratio has an effect in suppressing the behavior a t the beginning of
the interval m em ber. Thus, interval scalloping on m ult F IF R is probably m ore pro
nounced an d begins w ith longer pauses th an on F IT O .
The disruptive effect o f TO 8 on F I 5. R ecord A of Fig. 645 shows a perform ance a
few sessions after th a t in Fig. 644 on FI 5. A substantial scallop occurs after a pause
or slight running-through after reinforcem ent. At the start of the following session
(R ecord B), an 8-m inute T O followed each reinforcem ent for the first tim e in the b ird s
history. T he record begins w ith a T O at e, d u ring w hich some responding occurs.
A great deal of responding occurs during the T O following the next reinforcem ent at f .
A high rate is m aintained throughout the first 2 segments, and the pause and scallop
following the reinforcement are completely elim inated. Slower responding follows the
reinforcem ent a t g for a short time, but this lower rate is not consistent until h.
Thereafter, a lower rate of responding during the first p a rt of the interval develops pro
gressively except at i, where a high rate of responding is sustained throughout the in

terval. R ecord C presents a segm ent of the perform ance after 11 hours of further
exposure to the T O . Occasional scallops are quite smooth, with some pause following
th e reinforcem ent, as a t j , k, a n d I. B ut m any cases of rough grain a n d irreg u lar
changes in rate are also present.
Figure 646 shows a perform ance after 37 hours w ith T O after reinforcem ent. A
consistent scallop develops progressively during the session after a high rate was sus
tain ed thro u g h o u t the 1st interval. T h e pause after reinforcem ent is b rief or com
pletely absent. In the following session th e T O was rem oved. R ecord B shows the
perform ance. T he effect of rem oving the T O is not nearly so severe as the first in
troduction of the T O . Smooth FI performances soon occur, although responding m ay
occur im m ediately after the reinforcem ent, an d the term inal rate m ay be sustained
throughout the interval.
T he extrem e disruption of the FI perform ance by T O appears to be due to the his
tory of m ult F IF R , since none of the other cases of F IT O produced so radical a change.
523
This is further evidence for the view th a t the fixed-interval scallop in the m ultiple
schedule is based on different factors from those in a p u re fixed-interval schedule.
Hence, although the performance in Fig. 645A resembles th a t on FI 5, the events which
the bird uses in the fixed-interval perform ance are different from those in a simple
FI 5 schedule.
The effect o f an 8-minute TO on mult FIFR
300
RESPONSES
Figure 647A shows a fairly stable perform ance on m ult F I 1 0 F R 5 0 . V ery little
scalloping occurs w hen one interval follows a n o th e r (at a a n d e), except w hen the
preceding in terv al has shown a low rate, as a t d. T h e term in al ratio rate is
higher th a n the interval rate, b u t some pausing or slow responding are evident a t the
start of the ratio, as at b and c. An 8-m inute T O was then added imm ediately after
each reinforcem ent in the following session (Record B). This began w ith a long in ter
val at f at the end of which a response was reinforced after only a small num ber of
responses h ad been em itted. Some responding to the key occurs in the 8-m inute T O
524
a t g, a n d a ratio was then ru n off a t slightly less th a n its usual rate at h. A further
small am ount of responding in the T O appears at i, after which an interval was run off
a t a fairly low rate. F u rth er responding occurs in the T O a t j, after w hich a ratio
was executed at below the usual rate. T h e rem ainder of this record shows a high over
all rate at approxim ately the term inal value of the preceding interval, w ith only a
slow developm ent of scalloping after reinforcem ent tow ard the end of the session, as at
k, I, m, an d n. T he ratio perform ances hold a t approxim ately their preceding rate,
with only an occasional exam ple of pausing, as at o.
E ventually, a fairly adequate m ultiple-schedule perform ance is obtained in spite of
th e 8-m inute T O after each reinforcem ent. Figure 648A shows the 8th session on
the schedule. Here, the intervals are ru n off with smooth scallops which begin at a
fairly high rate, suggesting the scallops in Fig. 646 for F IT O . T he ratios at this point,
Fig. 6 4 8 .
Transition from m ult FIFR w ith TO a fte r reinforcem ent to mult FIFR
in general, show pausing before responding begins, as at a, b, an d c. N ote th a t the

presence of the T O produces scalloping even when one interval follows another for
exam ple, a t d. O n the following day (R ecord B) the T O was om itted, and the earlier
perform ance was restored w ithout difficulty. N ote th a t now little or no scalloping
appears w hen one interval follows another, as a t e. In general, slight pausing still oc
curs before responding at the ratio color, b u t it is of the order prevailing before the
appearance of the T O .
A nother bird exposed to the same set of schedules showed a lower over-all rate. A
low term inal rate is characteristic of the interval, as Fig. 649A shows. T he ratio rate
is high a n d shows little or no pausing. A n 8-m inute T O added to each reinforce
m ent shows little effect (R ecord B). (The ratio at a contains too m any responses, pos
sibly because of responding in the T O , although the record does not clearly indicate
it.) Little change occurs in the tendency to scallop w hen one interval follows another,
although some such tendency already exists in Record A. O n the following day the
T O has a m ore disrupting effect, as Fig. 650A shows. T he characteristic ratio rate a p
pears at a in the 1st interval, although the usual term inal rate is reached before rein
forcem ent at b. A gain at c the ratio rate appears in the interval color, b u t again the
ra te drops to th e interval rate before reinforcem ent. A sim ilar break-through ap-
M U L T IP L E SCH ED U LES
Fig. 6 4 9 .
525
The e ffect o f TO 8 on mult FIFR (second bird)
pears at d, a n d the high rate is m aintained until reinforcem ent a t e. A sim ilar b rea k
through of the ratio rate also occurs at f . By the end of the 9th session w ith the T O
the perform ance is quite similar to th at prevailing before T O was introduced. Record
B shows a single excursion at this time. Except for the higher rates in the interval
at g, h, a n d i, the general characteristics of Fig. 649A are apparent. U pon the rem oval
of the T O , there was an almost im m ediate retu rn to the earlier performance, as R ec
ord C indicates.
MULTIPLE SCHEDULES A N D DISCRIMINATION
M ultiple schedules involve the process of discrim ination. T he organism reacts in

a different way to two stim uli when they are app ro p riate to schedules having different
effects. T he comm onest techniques for discovering w hether an organism can dis
crim in ate betw een two stim uli use m ultiple schedules of w hich one com ponent is erf
an d the other ext. All rig h t responses are reinforced; all w rong responses are
extinguished. S ta n d ard techniques usually use a percentage of choices betw een a l
tern ativ e responses. T he use of the rate of responding on an ap p ro p riate schedule
to m ake the change in probability of the responses clearer has been reported by Dinsm oor (1952). T h e use of extinction as one com ponent is not necessary, although it
usually brings about a rapid change.
526
W e used a variation on this technique in studying the visual acuity of the pigeon.
A transparent key was m ounted over R onchi rulings illum inated from behind and p re
sented in horizontal a n d vertical positions as the 2 stim uli in m ult V Iext. In the
following experim ent the rulings were 32 to the inch an d clearly discrim inable by the
pigeon. T he procedure consisted of changing stim uli on a variable-interval sched
ule; independent series of intervals were used to program the durations of presentation
o f the rulings in vertical an d horizontal positions. In the horizontal position a re
sponse was reinforced at the end of the interval. In the vertical position the stim ulus
autom atically changed to horizontal a t the end of the interval.
T h e experim ent begins as m ult V IV I, w ith reinforcem ents on both stim uli until a
stable perform ance is developed on both. These are separately recorded. W hen
th e discrim inative procedure is th en established, th e rate declines u n d e r the stim ulus
no longer correlated w ith reinforcem ent. T h e process m ay be slow. Figure 651
shows th e over-all change in rate on the nonreinforced com ponent d u ring 11 experi
m ental sessions of various durations. T he small circles represent m ean rates for each
session. T he initial rate of the 1st session, lost in averaging, is indicated by the dashed
line a. Besides the over-all negative curvature shown in Fig. 651, the daily session
also shows a m arked decline. T he segment at A in Fig. 651 is shown in Fig. 652A, th a t
I---------4---H-O--U-R--S---------I,
,
Fig. 6 51.
O ver-all rate change during the first 11 sessions o f mult VIext
527
a t B, in Fig. 652B. N ote the strong negative acceleration w ithin most of the v ari
able intervals of Fig. 652A.
A sim ilar procedure was also used in an experim ent in w hich the discrim inative
stim uli were w hite a n d red general illum ination lights in the experim ental box. Fig
ure 653 shows the V I perform ance in th e presence of th e w hite house-lights 35
hours after the beginning of the discrim ination. This is essentially the V I perform ance
prevailing before the discrim ination was undertaken. Figure 654 shows the corre
sponding perform ance under red illum ination in the sam e session. Since the rate in
the white light was slightly higher at the beginning of the experim ent before a dis
crim ination was possible, the difference between Fig. 653 a n d 654 is even less significant
than it appears. In other words, the general illum ination is not controlling substan
tially different rates, even after 35 hours of differential reinforcem ent on m ult V IV I.
In an effort to check the im portance of the location of a color discrim ination, we
used a new p air of stim uli a n d p u t the colors directly on the key, using colored bulbs
behind the key as usual. T he perform ance for the 1st session on the w hite key was
sim ilar to th a t in Fig. 653. T h e green key was a novel stim ulus, however, an,d no re
sponses were m ade to it during the entire 1st session. T he same condition prevailed
in the 2nd session. O n the white key the perform ance shown in Segments 1, 2, and 3
in Fig. 655 was recorded. M eanw hile, no responses were being m ade to the green
529
key. Because of th e effect of a novel stim ulus, the discrim ination was in effect com
plete, w ithout differential reinforcem ent. T he relation of the stim uli was th en re
versed: all responses to the w hite key were extinguished while reinforcem ents were set
u p in case responding should begin on the green key. Figures 655 and 656 show the
result. T he portion of the record at a in Fig. 656 is for the green key in its earlier posi
tion as correlated with extinction. A t the arrow the key-color changes to white, a n d
responding begins as a direct continuation of Segm ent 3 in Fig. 655. T h ro u g h o u t
the rest of the session the response undergoes a fairly progressive extinction, as the rest
of Fig. 656 shows. M eanw hile, in altern ate periods, responses continue not to be m ade
to the now green key a t Segm ent a in Fig. 655. At b, how ever, a 1st response to the
green key occurs and is reinforced. Tw o fairly short intervals at c and d, a t the end
of which the response is also reinforced on the green key, produce an acceleration; an d
d u rin g the rem ain d er of the session on the V I green com ponent of the m ult V IV I,
substantial responding occurs, as the balance of Fig. 655 indicates. A tendency tow ard
extinction occurs in th e longer intervals, b u t the ra te holds generally a t ab o u t the
over-all level of the perform ance on the w hite key in the first 3 segments of the session.
W e m ay conclude from the m uch m ore ra p id process of extinction on the w hite key
shown in Fig. 656 th at the slowness of the process apparent in Fig. 653 and 654 m ust be
due to the position of the lights. T he color of the general illum ination is evidently
m uch less im portant to the pigeon th an the color of the key which it strikes.
MULTIPLE VIVI
Tw o birds w ith a history of V I after erf were placed on m ult V I 1 V I 3. T h e stim

ulus color was changed from blue to orange on an irregular schedule not correlated
w ith reinforcem ent. A different m ean value of V I prevailed on each. After 11 ses-
Fig. 6 5 7 .
Final perform ance on mult VIT VI 3 and mult VI 1 VI 6.5
Fig 6 5 8 .
Reversal o f discrim ination on mult VI 1 VI 6.5
531
sions the rates on the two were clearly different a n d stable, as evident in Fig. 657A
a n d B. R ecord A shows the whole session on a blue key on V I 3; R ecord B shows the
whole session on a n orange key on V I 1. Some points of changover are m arked in
th e record by diagonal dashes, although all were not clearly recorded. L ater, the
larger m ean interval was increased to 6.5. After 25 sessions on this m ult V I 1 V I 6.5,
the perform ance was as shown in Records C a n d D. T he V I 1 perform ance (R ec
ord D) rem ains essentially as in R ecord B, except th a t at the beginning of the session a
higher rate is com m only observed for the first few m inutes. T he rate on the larger
m ean V I has held to essentially the same value as on the sm aller at R ecord A, possibly
because of some in d u ctio n from th e o th er schedules, although, as we have seen in
C h a p te r Six, the V I rate generally resists change.
As a check on the extent of the stim ulus control, the colors were reversed. T h e
orange key, w hich h ad previously been correlated w ith V I 1, was now correlated w ith
V I 6.5. Figure 658 shows the first a n d last sessions u n d e r the reversed conditions.
Records A and B give the first an d last perform ances on V I 6.5 under the control of the
orange key. Some extinction of the higher rate previously associated w ith V I 1 oc
curs. R ecords C a n d D, w hich give the beginning a n d final sessions on the blue key
u nder V I 1, show a steady adjustm ent to the new schedule.
MULT F R 6 0 F R 2 00 WITH PROLONGED EXPOSURE TO EACH

CONTROLLING STIMULUS
T h e frequency w ith which stim uli are rotated in a m ultiple schedule is, of course,
arbitrary. In the experiments already described stimuli were changed frequently, u su
ally a t reinforcem ent. W e investigated the possibility of a m ultiple schedule where
a single stim ulus is in control for a full session on 3 birds w ith a long history on F R 60,
on a blue key. At the beginning of a new session the key-color was orange, and the
reinforcem ent was on F R 200. A fairly good ratio pause an d cu rvature app ro p riate
to F R 200 developed before the end of the session. O n the following session the key
was again blue and the schedule was F R 60. V ery little disturbance from the preced
ing day on F R 200 was evident. O n the 3rd day, the schedule was again F R 200 on
the orange key; on the 4th, F R 60 w ith the blue key; a n d so on. T he F R 200 schedule
perform ance developed rapidly. T he over-all rate declined som ew hat throughout
each session; this decline could be due to the intervening days on F R 60, b u t it is also
fairly characteristic of ratio perform ances. Figure 659 shows the condition on the
12th and 13th days of this procedure. R ecord A gives the entire session w ith the blue
key on F R 60; Record B gives the following entire session on F R 200 with the orange
key. N ote the decline in over-all rate d u rin g th e session on F R 200. N ote also the
long pauses controlled, even a t the beginning of the experim ent, by the orange (F R
2 0 0 )key.
532
MULTIPLE FIFI
In m any experim ents on m ultiple schedules a m ere change in stim uli seemed to
have some effect a p a rt from the correlation w ith schedules. W e studied the effect of
a simple change in a m ult F I 15 FI 15. T he key could be either green or white, chang
ing color after reinforcem ent at random . A schedule of FI 15 was in force in each
case. Figures 660 a n d 661 show a stable perform ance u n d e r these conditions for the
white and green keys, respectively. T he reinforcements after which the key changed
color have been m arked with dashes above the record. A lthough the characteristic is
not inevitable, scalloping tends to occur when the color changes an d running-through
w hen it does not. In other words, a different key-color, even though on the same
schedule, functions as a T O .
In a total of 6 sessions for this bird, the intervals following changeover began with
one of three types of perform ance as follows: pause, 12%; run-through and pause, 74%;
a n d com plete run-through at term inal rate, 14%. In the same sessions the intervals
not following changeovers showed: pauses, 32%; ru n -th ro u g h a n d pause, 28%; com
plete run-th ro u g h a t term inal rate, 40%. A second bird showed a m uch m ore con
sistent picture. Intervals following changeovers showed: pause, 90%; ru n -th ro u g h
a n d pause, 7%; an d com plete ru n -th ro u g h at term inal rate, 3%. Intervals following
no changeover showed: pause, 0%; ru n-through and pause, 75%; and complete runthrough at term inal rate, 25%.
533
W hen the schedule was changed to FI 12 on one key-color, the perform ance changed
only slightly. W hen one schedule was ch anged to F I 8 while the other rem ained
a t F I 15, how ever, a m ultiple control clearly developed, as Fig. 662 a n d 663
indicate. Figure 662 shows the FI 8 intervals, and Fig. 663, the FI 15. Pausing an d
slow responding are m uch less extended in the shorter interval. Figure 662 contains
several instances of ru n n in g -th ro u g h w hen the color does not change. Figure 663,
however, has no such case. O n the contrary, a w ell-m arked scallop in the subsequent
interval often develops w ithout a color change.
534
T he interval on the white key then was reduced to 4 m inutes. Figure 664 shows the
1st, 9th, a n d 30th sessions for F I 4 and Fig. 665 the 30th session for FI 15. T he FI 15
rem ains stable and is not affected by the reduction on the white key. T he F I 4 (Fig.
664) im m ediately elim inates the pause after reinforcem ent, a n d only a slightly low
ered rate for the first few responses after reinforcem ent rem ains. T he developm ent
is slow. In R ecord B, after 9 sessions of the new m ultiple schedule, a short pause as
well as a m ore extended c u rv atu re are ap p aren t. H ow ever, consistent pausing an d
m arked cu rv a tu re ap p ro p riate to F I 4 do not develop u ntil R ecord C, after 30 ses
sions of the m ultiple schedule.
At this size of interval little relation exists betw een w hat happens im m ediately after
reinforcem ent a n d the condition of the preceding interval. This m ay be due to the
size of the interval, or to the prolonged exposure to the m ultiple schedule.
T h e o th er b ird showed a very sim ilar perform ance, b u t w ith som ew hat shallow er
scalloping a n d a tendency to develop a m ore prolonged linear term inal rate. T h e
over-all ra te was higher so th a t the n u m b er of responses per interval ten d ed to be
larger, b u t conditions of curvature were quite similar.
Fig. 6 6 5 .
535
M u lt Fl 1 5 Fl 4 : 3 0 th session o f Fl 15 record
We then reduced the shorter interval further. After 11 sessions on m ult FI 2 FI 15

th e perform ances in Fig. 666 were recorded. Excellent scalloping occurs on the F I 15
(R ecord A) and on FI 2 (R ecord B). In general, the depth of the scalloping in F I 2
depends upon w hether there has been a change from the other schedule. Changeovers
ten d to be followed by m ore m arked scallops, although this is not always the case.
T he FI 15 perform ance shows persistent scalloping w hether or not a change in key-color
has ju st been m ade. In contrast, a simple FI 15 schedule shows occasional runningthrough unless a T O occurs after reinforcem ent.
W e then tested the effect of a very large F I com ponent in a m ult FIFI. T he FI 15
com ponent on the green key rem ained unchanged. T he interval on the white key was
increased to 56 minutes. After 41 sessions a w ell-m arked m ultiple perform ance h ad
stabilized, as Fig. 667 a n d 668 indicate. These figures show the full session, except for
the p art of the last interval on FI 56 missing from Fig. 668. W ell-m arked FI 56 scal
lops ap p ear in this figure, as at a, b, d, a n d f . Also, one instance of running-through a t
th e term inal rate occurs a t c, and one instance of slight running-through w ith a very
shallow scallop at e. T he intervals at g a n d h, not shown com plete, have very few re
sponses. T he FI 15 scallops in Fig. 667 altern ate w ith this perform ance a t random .
Fig. 6 6 6 .
M u lt FI 15 FI 2 a fte r 11 sessions
Fig. 6 6 7 .
Fig. 6 6 8 .
M u lt Fl 15 Fl 5 6 : Fl 15 record
M u lt FM 5 Fl 5 6 : Fl 5 6 record
537
These are generally typical of the interval, a n d some scalloping occurs w hether or not
the schedule has ju st changed. A m arked instance of such scalloping appears at c,
an d m inor instances at b a n d d. T he low rate in the region of reinforcem ent a t a is oc
casionally evident at this stage of the experim ent. T h e actual order of occurrence
of the segments in the 2 figures is shown by num bers.
Mult FI 2 FI 10
Performances were recorded on m ult FI 2 FI 10 in an experim ent in which the discrim inability of the correlated stimuli was m anipulated. T hree birds were used which
h a d a long history of FI w ith fading counter. (See C hapter Five.) Figure 669 is a
record of the perform ance on both stim uli, the 2 stim uli being grossly different. T he
key was red during the 2-m inute intervals, a n d blue during the 10-m inute intervals.
In the figure a fairly sm ooth scallop an d a high term inal ru n n in g rate are com m on in
Fig. 6 6 9 .
M ult FI 2 FI 10 w ith maximum stimulus contrast a fte r 4 7 sessions
the shorter intervals. P articu larly good exam ples are a t a, c, d, a n d i. T h e F I 10

com ponent shows long pausing after reinforcement w hen the schedule has just changed,
b ut also shows substantial pausing when the preceding interval was 10 minutes. R e
sponding after a long pause is often triggered very late, a n d shows a rap id sta rt w ith
negative acceleration. Instances of this are at b, e, g, a n d h. Elsewhere, fairly smooth
acceleration is observed, as a t j and k.
W hen some red light was added to the blue, both over-all and term inal rates are
lower. L ate triggering a n d sharp negative acceleration in the longer intervals are
occasionally quite m arked, as a t a, b, an d d in Fig. 670, 20 sessions after Fig. 669. An
instance of this in the short interval at c follows an o th er short interval in which the
reinforcem ent was delayed by an exceptionally long starting pause.
Figure 671 is the final picture on this schedule after m any changes in color con
trast. H ere, pausing in the shorter interval is frequently quite long, as in b, c, a n d d.
T h e longer intervals often show relatively late triggering, as a t a a n d e. T h e te r
m inal rates are now high again, however, an d the longer intervals show a large
n um ber of responses before reinforcem ent. In general, the stimulus control is good.
10 MINUTES
Fig. 6 7 0 .
Fig. 6 7 2 .
M ult Fl 2 FM 0 w ith less clear-cut stimuli
M u lt FIT FI 4 FM 6 FI 6 0 TO 5 a fte r 4 2 sessions
539
M ult Fl at four values
Tw o birds were exposed to a m ult F I 1 F I 4 FI 16 FI 60 T O 5 for 42 sessions of 8

hours each im m ediately after erf. Little evidence of m ultiple control developed. T h e
schedule had the general effect of a V I. T h e key-colors (red, blue, white, and green,
respectively) appeared to have little effect. Figure 672 is a record of a complete session
after 42 sessions. T he two periods of low, irreg u lar responding tow ard the end of
th e session are in the presence of the green key on F I 60. Slightly lower rates on this
com ponent appear earlier in the record.
D uring this p a rt of th e experim ent a tim e out of 5 m inutes followed each reinforce
m ent. Later, the T O was abandoned and the shortest interval om itted. T he sched
ule was th en m ult FI 4 F I 16 FI 60. U n d e r these conditions some m ultiple control
developed. Figure 673 shows the perform ance after 22 sessions or 176 hours of this
schedule. T he record begins w ith the end of a 4 -m inute interval on a blue key, d u r
ing w hich the bird responds at a high rate (at a). T h e key then becam e w hite, a n d a
16-m inute interval passed before the next reinforcem ent, at b. A lthough m uch of
this perform ance is at a rate close to th a t of the 4-m in u te segm ent, some acceleration
occurs a t the beginning. A considerable irregularity is characteristic of the 16-m in
ute intervals. T he color was then changed to yellow and a 60-m inute interval begun.
A lthough some running-through occurs (after b), the bird breaks into a long an d fairly
smoothly accelerated scallop appropriate to the 60-m inute schedule. Reinforcem ent
occurs at c. T he key was then blue again, an d a high rate was m aintained for most of
a 4-m inute interval. T h e color th en changed to yellow; a n d after a slight scallop
ing, a fairly high rate developed in the following 60-m inute interval. T his high rate
leads to some extinction and breaking at . T he net result is a not-very-sm ooth m ulti
ple perform ance; nevertheless, a distinguishable difference exists in the effects of the
3 key-colors.
T he 16-m inute interval was th en dropped out (the schedule then being m ult F I 4
540
Fig. 6 7 4 .
M u lt FI 4 FI 6 0 a fte r 12 sessions
FI 60 on blue a n d yellow keys, respectively); Fig. 674 illustrates the perform ance after
12 sessions. This figure shows three 4-m inute intervals at a, b, an d d, an d three 60m inute intervals at c, e, an d f . An unusual feature of the short interval has developed.
After a short pause, the rate assumes a value appropriate to the interval; b u t a sharp
negative acceleration occurs before the interval is com pleted. This condition is p a r
ticularly clear in a and b but can also be detected in d. It becam e practically inevi
table on the short interval for both birds. W e have no explanation for it. N ote th at,
in general, possibly because of th e com plicated history of the experim ent, the p e r
form ance is quite erratic.
L ater, the intervals were changed to 1.5 an d 16 m inutes, an d the colors appropriate
to them in the 1st schedule were reinstated. T h e schedule was now m ult FI 1.5 FI 16
on red a n d w hite keys, respectively. Figure 675 shows the perform ance during the
sixth 8-hour session. T h e scalloping a p p ro p riate to the 16-m inute interval is m uch
m ore uniform and characteristic of th at schedule. Negative acceleration in the short
interval still appears, particularly at a, b, c, d, and e; it is noticeable in every instance
except a t f , w here the rate rem ains low throu g h o u t. T h e sam e characteristic was
m arked in the behavior of the other bird.
T he experim ent showed good m ultiple control on m ult F IF I when the stimuli were
sufficiently different, a poor b ut distinguishable m ultiple control on a m ult F IF IF I, an d
failure to achieve m ultiple control on a 4-ply FI after a long exposure.
Fig. 6 7 5 .
M u lt FIT .5 FI 16 a fte r 6 sessions
541
M u ltiple FI 3 0 tandem Fl 3 0 FR 10
W e attem pted to bring the ratio end effect under stim ulus control by placing 2
birds on a 30-m inute F I w ith 2 key-colors changing a t reinforcem ent a t ran d o m a n d
th en by adding one tan d em F R 10 u n d er one color. T h e birds h ad h ad an extended
history of m ult FI FI w ith different values. At the beginning of the experim ent both
key-colors (green a n d white) were appropriate to F I 30. T he tandem ratio was added
to the F I 30 w hen the key was white. T h e tan d em F R h a d little if any effect. T h e
term inal rate was already about 2 per second a t the start of the experim ent, and re
m ained at about th at value throughout. However, the ratio grain developed conspic
uously. Figure 676 shows a representative session after approxim ately 200 hours on
m ult FI 30 tan d FI 30 F R 10. R ecord A is the perform ance on the green key on FI 30.
L ong pauses occur after reinforcem ent a n d also a steady, sm ooth acceleration to a
term inal rate, which m ay be m aintained for as m any as 2000 responses before reinforce
m ent. R ecord B is the alternating perform ance on tan d F I 30 F R 10. Some rough
grain, producing a lower term inal rate, appears, especially a t a. Some indication exists
th a t the curvature along which the bird approaches a term inal rate is sharper on the
.
.
.
1
tan d e m color. T h e over-all perform ance is sim ilar to th a t in R ecord A. N ote th a t
the perform ance is relatively free of ru n n in g -th ro u g h . T h e b ird seldom h a d an y
tendency to begin responding im m ediately after a change o f key-color. T h e second
542
bird occasionally continued at the term inal rate for the succeeding interval w hen the
color h appened to rem ain the same. As in other m ultiple experim ents, changing the
color has some of the effect of a tim e out. In the present experim ent the schedule is
not actually changed w ith the change of key-color, since the tandem F R did not have
a differential effect.
MULTIPLE VIVIdrh
T h e 3 birds previously studied on V Id rh w ere placed on a m ultiple schedule at

th e com pletion of th a t experim ent. W hen the key was red, reinforcem ent was V I 1
drh; w hen the key was white, the schedule was V I 1 w ithout drh. T he change from
one color to another occurred a t reinforcem ent. Tw enty-one sessions later all 3 birds
h ad developed m arked differences in rate on the 2 key-colors.
Figure 677 shows a com plete session. T h e bars m ark periods on the w hite key,
w here the rate is consistently lower. T he white-key segm ent frequently begins with a
short continuation of the rate prevailing on the red key, notably a t a, b, an d c.
Fig. 6 7 7 .
M u lt VI 1 drh VI 1 a fte r 21 sessions
W hen the perform ance shown in Fig. 677 h a d been developed, a drl was ad d ed on
the w hite key. T he schedule was now m ult V I 1 drh V I 1 drl. T he drl contingency
was easily satisfied in the early stages of the next session, which is illustrated in Fig. 678.
(W e lost p a rt of a segment in recording the top record.) Fairly long runs occur on
the w hite key, as at a an d c. Nevertheless, before the end of the session, a substantially
lower rate has developed under the drl contingency, as at e, g, and h. N ote th a t the
tendency to begin at a high rate on the w hite key, as at b, d, an d f , still exists.
Eventually, the drl rate becomes quite sim ilar to a simple V ld rl perform ance. Fig
ure 679 shows the last session. T he change to a low rate is now practically im m e
diate, a n d the rate extrem ely stable. T he drh m eanw hile continues to show runningthrough. This perform ance is due to the gap-setting on the schedule which requires
a certain m inim al num ber of responses for reinforcement. T here is no appreciable dif
ference in the d rh perform ance as the effect of the altern atin g perform ances on drl.
W hen the drl contingency was rem oved from the white key, the bird returned essen
tially to th e perform ance w hich prevailed before it was added. Figure 680 shows
Fig. 6 7 8 .
543
First session on mult VI 1 drh VI 1 drl
th e 3rd session after the rem oval of drl. N ote th a t w ith the higher over-all rate on the
w hite key (segm ents m arked by bars), the tendency to begin responding a t a high
rate, as a t a, d, an d e, returns. At this stage a tendency also exists to return to this high
rate (as a t b an d c) during the white phase, which now has no differential-rate rein
forcement.
Fig. 6 7 9 .
Fig. 6 8 0 .
M ult VI 1 drh VI 1 drl a fte r 8 sessions
Return to mult V IdrhVI a fte r mult V IdrhV Idrl
545
MULTIPLE FIdrhFI
After reaching performances as in Fig. 680 on m ult V IV Id rh , 2 of the birds in the

preceding experim ent were placed on m ult F I 15 F I 15 drh. Figure 681 shows the
changeover for 1 bird (not in the preceding figures). T he first stim ulus presented
was the red key, which h ad previously been correlated with V Id rh and is now corre
lated w ith F Idrh. T he rate begins at the usual V Id rh value a t a. At b, after extinc
tion due to the change to FI 15, the rate drops spontaneously to a lower value th an
th a t observed in the previous V I w ithout differential reinforcem ent. This lower rate
Fig. 6 8 1 .
First session on mult FI 15 FI 15 drh a fte r mult VIVIdrh
546
suggests a retu rn to a perform ance appropriate to the m uch earlier V Idrl. T he short,
rap id ru n at c satisfies the rate contingency, a n d reinforcem ent is received. T he stim
ulus then changes to the color formerly appropriate to a V I without differential rein
forcement, and a low rate is m aintained for 2 intervals on the F I 15 schedule.
A lthough no differential-rate contingency is in force, the rate actually shown is consid
erably below th a t previously prevailing on m ult V IV Id rh an d suggests again a re
tu rn to the earlier condition of V Id rh V Idrl. After the key changes again to red a t d,
a high rate prevails until a reinforcem ent is received a t th at rate, a t e and again a t f .
J u s t before f a substantial square break occurs which is comm on on drh when reinforce
ments are infrequent. T he schedule continues on the red key a t drh, an d a 3rd re
inforcem ent is received at g. W hen th e key-color changes to white, the rate drops
im m ediately to the lower value, a n d a reinforcem ent is received a t h. Shortly th ere
after, a m om entary break-through occurs of the high rate at i, but a sharp return to
th e low value follows. W hen the higher ra te ag ain breaks th ro u g h at j , a reinforce
m ent is received at this rate a t k (although there is no drh on the white key). This
single reinforcem ent at a high rate suffices to bring out a full interval of responding on
the w hite key very near the rate appropriate to the red key u n d er the previous con
tingencies. H ow ever, some rough grain produces a general low over-all rate near
th e end of th e interval w hen a reinforcem ent is received at /. T h e schedule then
changes to F Id rh on the red key, a n d the experim ent is term inated before the interval
is complete.
T h e final state to w hich this schedule was carrie d is illu strated in Fig. 682, w hich
shows a perform ance after 25 sessions for the same bird as in Fig. 681. T he session
starts w ith a pause appropriate to the FI schedule, an d the 1st interval is concluded a t
a at a high term inal rate. R esponding begins soon afterw ards; it continues w ith only
some decline in the term inal rate at the very end of the interval before reinforcement
is received at b, w here the d rh contingency is satisfied by a short run. A long pause
th en intervenes before a re tu rn to a high term inal rate, an d a break occurs a t c b e
fore the interval h ad been completed. T he drh contingency is not satisfied by the sub
sequent performance until the point of reinforcement at d. T he key-color then changes
to white, and a fairly sm ooth scallop follows on the nondifferentially reinforced sched
ule. A knee is evident a t e, an d the grain is not very sm ooth; however, the curvature
is clear. Tw o other intervals follow on this key-color; a n d alth o u g h the rate rises
a n d the scalloping becomes less m arked, some evidence of interval curvature still exists.
At f the key-color changes to red, and a pause appropriate to the F I schedule follows.
T he high rate of responding which sets in is not m aintained until the com pletion of the
interval, an d a long pause follows a t g, the reinforcem ent being received only when
th e short ru n at h satisfies the d rh contingency. T he key-color then changes to white,
a n d a rough scallop follows in which m arked knees suggest the break-through of the
d rh rate. W hen the color changes to red a t i, a long pause intervenes followed by a
ra p id rate w hich achieves reinforcem ent a t the end of the first horizontal section of
the figure. T he schedule continues w ith the red key, a n d a good instance of w hat has
547
developed by way of an interval scallop under the d rh contingency follows at j to k.

O n th e w hite key 2 shallow interval-scallops th en follow; upon the re tu rn to the red
key, reinforcem ents are received a t I, m, a n d n after very little responding during the
interval. Tw o intervals on the w hite key rep eat the previous perform ance, a n d the
subsequent behavior follows the same p attern . In general, the perform ance on the
w hite key is not very different from a standard F I performance. Roughness exists, a n d
induction from the other schedule appears to em phasize the knees which are occasion
ally seen on such a schedule. P ausing after reinforcem ent is only m oderate. E ach
interval begins fairly soon at a substantial rate. T h e pauses on the drh contingency
are extrem e, however, a n d the over-all interval p a tte rn , insofar as one develops, is a
relatively square curve. T h e grain is rough a n d reflects the small ratio runs re
sulting from the gap-setting on th e d rh ap p aratu s. (T h e actual interval on d rh de-
548
pends upon when the bird satisfies the drh contingency, and m ay be longer th an 15
m inutes.)
Figure 683 shows the earlier developm ent of the perform ance on the w hite key
w ithout drh on F I 15, separately recorded. R ecord A is for the 3rd session. Consid
erable induction from the drh com ponent appears. This induction becomes p a rticu
larly m arked in Segm ent B from the 5th session. Segm ent C is from the 7th session;
Segm ent D, from the 8th. Some pausing after reinforcem ent, particularly when the
preceding interval has been on drh com ponents, appears at a in Segm ent E for the 9th
session, a n d again m ore conspicuously at b and c in Segm ent G for the 14th. (Seg
m ent F is from the 13th session.) A tendency not to scallop rem ains when the preced
ing interval has been on the w hite key, as a segm ent from the 16th session a t Segm ent
H dem onstrates. T he segm ent also shows, as do o ther segments in the figure, the
occasional break-through of the higher rate, as a t d a n d e, through induction from the
d rh contingency. W ell-m arked interval scallops a p p e ar by the 17th session w hen
the preceding schedule has been drh, as 3 segments a t I show. Note, however, th a t a t
f the reinforcem ent on the w hite key is followed by a quick re tu rn to a high rate.
R ough grain still appears in the early stages in the scallop, as well as pauses followed
by rapid runs, as at g, h, and i.
549
Figure 684 shows the developm ent of the F I perform ance w ith drh. T he rate was
originally quite high, as R ecord A for the 3rd session shows. R einforcem ents are
being received m uch less frequently th an under the preceding V I 1 schedule, however,
a n d the over-all rate falls. Short runs corresponding to the gap-setting in the d rh
a p p a ra tu s appear. In the 4 th session, shown at R ecord B, the ra te is low er a n d has
m arked step-wise breaks. Record C is for the 1Oth session. T he actual progress of
th e perform ance a t this tim e depends upon a rb itra ry settings of the d rh contingency,
w hich were changed as the perform ance perm itted. As the requirem ents were in
creased, the runs becam e longer. Some curvature ap p ro p riate to the interval begins
to develop. This curvature is particularly evident a t a, R ecord D (for the 16th ses
sion), and becomes fairly consistent by the 25th session a t R ecord E, which im m ediately
precedes Fig. 682.
T h e o ther b ird on this schedule show ed a m uch better developm ent of an interval
perform ance w ithout differential reinforcem ent, b ut great straining on the d rh con
tingency. A fairly early developm ent on F I alone is shown for the 13th session in Fig.
685. H ere again, after changing from the other color, the interval shows good scal
loping, as at a, b, c, d, a n d e; but elsewhere, only slight evidence of scalloping exists.
T he only substantial example of a break-through of the drh rate is in the interval begin
ning at b, although slight examples ap p ear in the intervals beginning at c and d.
L ater, these disappear, and a very satisfactory perform ance on FI alone is m ain
tained, as Fig. 686 shows for the 19th session. After changing from the other color,
the interval scallop is w ell-m arked; otherw ise, it is not. T h e rate is often so low fol
lowing a change of schedule, as for exam ple at a, b, and c, th a t the reinforcem ent oc
curs at a low count. This condition produces a second-order effect, if the schedule does
550
not change, in w hich 2 successive intervals together describe a smooth scallop (as from
a to b, d to e, and h to i).
M eanw hile, the perform ance on FI 15 drh under m ultiple control and alternating
more or less at random with FI 15 shows rath er long ratio runs corresponding to the
gap-setting, but also prolonged periods of no responding. Figure 687 illustrates a fairly
representative example, alternating with the performances on FI 15 alone in Fig. 686.
T hus, Segm ents d a n d e in Fig. 686 occur a t b in Fig. 687. T h e scallops a t f a n d g in
Fig. 686 occurred at point d; and the single-interval scallop at ;, at point e. In gen
eral, after a change in schedule, a period of low responding follows which suggests a r a
th e r rough in terv al curve; b u t w hen no change has occurred from the preceding
schedule, the interval curvature m ay be lacking as before a and c.
MULTIPLE VIVI PACING
Dr. L agm ays experim ents on pacing described in C h ap ter N ine included an ex
p loratory a tte m p t to establish m ultiple control. A bird w hich h a d been paced a n d
then reinforced on V I 1 without pacing was again subjected to V I 1 pacing (3) 1.5-2.0.
T he key h a d previously been w hite, b u t it was red w hen the pacing contingency was
551
re-introduced. Figure 688 shows the effect. Possibly because of the color change, a
pacing perform ance is quickly recaptured.
M ultiple control was then attem pted, with the white key controlling V I alone d u r
ing one-half of th e session a n d the red key controlling paced V I during the other half.
Figure 689 describes th e first 2 sessions on this m ultiple schedule. T h e 1st session
(R ecord A) begins w ith the key red. T h e b ird occasionally breaks through w ith the
V I ra te alone, b u t a fairly a d e q u ate p aced perform ance appears. W hen th e key
changes color a t the arrow, a high rate on V I is observed. T he following session (R ec
ord B) begins on th e w hite key on V I, a n d a change to the red key a t the arrow leads
alm ost im m ediately to a paced perform ance. In later sessions the m ultiple control
is less effective. A m arked break-through of the unpaced rate is apparent in Fig. 690A
a t a, and 2 sim ilar break-throughs, in a later session, in R ecord B at b and c. A sec
ond bird gave very sim ilar performances.
Fig. 6 8 8 .
Return to VI 1 pacing p rio r to mult

V IV Ipacing
552
MULTIPLE SCHEDULE WITH THREE COMPONENTS
As a dem onstration of w hat m ay be done w ith m ultiple schedules containing m ore

varied com ponents, an apparatus was designed to reinforce on the following schedules :
FI 6, V I 1 drl 4, a n d F R 5 0 or F R 70. T he controlling key-colors were w hite, red,
an d blue, respectively. Figures 691 an d 692 show stable perform ances of 2 birds. In
Fig. 691 the session begins with a brief period of drl at a. O ther periods of drl under
th e control of the red key ap p ear at c , f i, /, o, r, an d elsewhere. T he 2nd schedule in
the session was FI 6 and the perform ance appears at b. O ther interval performances
are at e, h, k, n, q, and elsewhere. T he first F R 50 appears a t d; an d other instances, a t
g, j, m, p, and elsewhere. T he other bird showed a sim ilar perform ance except th a t
the over-all rate tended to be m uch higher in drl. T he session in Fig. 692 begins a t a
fairly low over-all rate. T hree interval segments, a, b, a n d c, contain very few re-
553
sponses.
F R 70.
Otherw ise, the perform ance is sim ilar to th at of Fig. 691.
T he ratio here is
A MULTIPLE SCHEDULE WITH FOUR COMPONENTS
Tw o birds w ith a long history of m ultiple an d m ixed schedules were placed on a

m ult FI 2 F I 11 F R 50 F R 250 schedule. In the early stages, blocks of reinforcem ents
on each schedule were arranged u n d er appropriate stimuli. T he birds adjusted
quickly. Figure 693, for exam ple, shows appropriate perform ances on all 4 sched
ules as they were presented in blocks of 4 or 8. L arge intervals are m arked I; small
intervals, i; large ratios, R; and small ratios, r. T h e 4 large intervals beginning a t
a show fairly good scalloping and an interm ediate term inal rate. T he rate oscillates a
good deal at this time. At b a block of 4 short ratios is run off with only slight paus
ing and at a high rate. Four large ratios beginning at c show some pausing but a high
term inal rate. Beginning at d, 4 short intervals show an essentially linear perform -
554
ance a t th e lowest of th e 4 rates. T h e general p a tte rn is co n tinued th ro u g h o u t the

session.
L ater, the 4 schedules were presented a t random . T his procedure caused some
trouble, and the larger ratio was tem porarily om itted from the set. T he other 3 sched
ules were distinguished accurately. Figure 694 shows the 7th session on m ult F R 5 0
FI 2 FI 11, w ith long pauses a n d scalloping in m any long intervals, only slight pausing
and a high term inal rate in the small ratios, and a short pause or no pause and an inter
m ediate term inal rate in the short intervals.
T he large ratio was then added to the series and the birds exposed to m ult F R 5 0
(red) F R 250 (white) FI 11 (blue) FI 2 (yellow) for a larger num ber of sessions. Figure
695 gives a perform ance after 32 sessions. T h e separate schedules now ap p e ar in
short blocks of 2 or 3 each, a n d ap p ro p riate perform ances are exhibited. T h e other
b ird showed essentially the sam e result; Fig. 696 gives its final perform ance on the
four-fold m ultiple schedule after 38 sessions. Here, the short ratio shows no pausing
a n d a high term inal rate. T he large ratio shows a high term inal rate, although a
good deal of difficulty occurs a t the beginning and later fairly smooth acceleration, as
a t a and c. T he small interval now shows some pausing and occasional smooth c u r
vature. T he large interval shows a long initial pause, with very irregular developm ent
555
of the term inal rate. Some instances show a fairly sm ooth positive acceleration, as at
b ; others show late triggering followed by negative curvature, as at d. In general,
however, both birds dem onstrate 4 separate perform ances u n d e r the control of 4 keycolors.
A MULTIPLE SCHEDULE IN V O LV IN G EIGHT CONTROLLING
STIMULI A N D EIGHT SCHEDULES
In order to get a larger num ber of clearly discrim inable patterns, small figures were
projected on the key. A figure could be either a triangle or a circle, red or blue, a n d
large or small. T he 8 possible combinations of these properties were correlated with
8 separate schedules as follows. T h e figures were red on all interval schedules, green
on all ratio schedules. They were triangles when there was no differential reinforce
m ent of rate, and circles w hen the schedule included drl. T hey were large w hen the
interval was 16 or the ratio 100; they were small when the interval was 2 or the ratio
40. This set of relations was designed to perm it a separate control via color, shape, or
size, ap art from the specific schedule.
Four recorders were arranged to record separately: (1) large FR s, (2) small FRs,
(3) large FIs, and (4) small FIs. Each recorder contained a m arker indicating w hen
the drl contingency an d its appropriate stim ulus were present. Thus, 8 different seg
ments could be separately recorded, although each schedule with and without drl a p
peared in a single record. T he order of presentation was sem i-random , and only 1
recorder ran at a given time.
T hree birds w ent directly from erf to V I 3, d u ring w hich all 8 stim uli were p re
sented in random order. Some color preference was observed. W ith one bird, for, ex
am ple, the blue pattern generated a rate of about 1.25 responses per second; the red
p attern , about 0.95. W hen the V I perform ance h ad stabilized, except for this dif
ference, the final eight-fold m ultiple schedule was introduced at once. T he results
556
for the bird showing the best stimulus control will be reported in terms of each of the
com ponent schedules.
First bird
Figure 697 shows the perform ance in the 5th session. (Each session contained 56
reinforcements.) At Record A the parts of the record under the bars were executed on
a small blue circle a n d the schedule was F R 40 drl. T he drl is effective relatively late,
a n d a large num ber of responses is generally em itted. T he short portions of the rec
ord not under bars were executed on small blue triangles and a simple F R 4 0 . In
Record B the bars m ark portions of the record executed on large blue circles and are
for F R 100 drl. T he drl here is m ore effective th a n th a t in R ecord A; an d despite the
larger basic ratio, reinforcements are often received sooner. T he portions of the rec
ord not under bars are for the standard F R 100 on a large blue triangle. Record C
gives the performance on the larger interval, the bars m arking the drl contingency. In
general, a low rate w ith only slight curvature is generated by the large red circle cor
related with FI 10 drl, while the standard FI 10 perform ance on the large red triangle
does not yet show very m arked in terv al cu rv atu re. N ote th e single instance of a
b reak-through of a rate app ro p riate to a ratio perform ance a t a. R ecord D contains
the perform ance on the small red circle (not m arked) under FI 1 drl or on the
sm all red triangle on straight FI 1. T h e drl is alm ost ineffective. T he stim ulus con
dition during the horizontal segment cannot now be identified. A t this stage the bird
557
shows the effect of drl under the control of the size of the spot on all schedules except the
smallest interval.
T hree sessions later the stim ulus control has developed further. Figure 698A shows
th e large ratio with and w ithout drl, a n d the control is obvious. O n the shorter ratio,
however, in R ecord B, the drl contingency begins to be felt only after several times
the num ber of responses have been em itted. This factor could be due to the higher
rate of responding controlled by the appropriate stim ulus (the size of the p attern) or to
induction from Record A. In the shorter interval, a t Record C, the drl contingency
is now clearly effective, although not invariably so. For exam ple, the higher rate
breaks out a t a u n d er the drl stim ulus. O n the large interval (R ecord D ), scalloping
is now m arked both w ith an d w ithout the drl contingency. Fairly norm al FI 16 scal
lops show a t b a n d c, an d the adjacent drl segments also show scalloping. At d an
interval passes with very few responses, although the drl contingency is not in effect;
a n d the following interval at e w ith the drl shows a higher rate. H ow ever, a still
higher rate appears first at f and later at g when the drl contingency is not in effect.
Figure 699 gives the final perform ance after 44 sessions on the eight-fold m ultiple
schedule. T he stim ulus patterns have been suggested, solid figures representing blue
patterns. R ecord A illustrates the characteristic high rate under thte drl until the
sm aller ratio has been counted out. At R ecord B the drl on the larger ratio produces
an app ro p riate low rate from the very beginning of the ratio. A t R ecord C on the
558
Fig. 6 9 9 .
8 -p ly m u ltip le schedule a fte r 4 4

sessions
small interval the drl is now clearly effective; but the rates are still too high for im m edi
ate reinforcement, and intervals of perhaps 10 or 20 m inutes often elapse before a re
inforcem ent is received. At R ecord D the large interval shows scalloping u n d e r drl,
as at b; but a higher term inal rate develops in the absence of the drl contingency, as
at a. T h e fact th a t the drl is m ore effective on the larger th a n On the sm aller interval
indicates th a t with the small ratio and the small interval the more frequent reinforce
m ent conflicts with the drl contingency.
In an attem pt to check the cross-induction effects, we om itted the drl contingency
Ayvwt^x
Fig. 7 0 0 .
6 -p ly m ultip le schedule (o m ittin g

FR 4 0 d rl and FR 100 drl)
559
from the sm aller ratio. This omission left a seven-fold m ultiple schedule. After 6
days, the drl was om itted from the larger ratio, leaving a six-fold schedule. T he effect
of this omission, contrary to expectation, was to decrease the m axim al rate on all sched
ules. T he rate of the small ratio declines steadily throughout the 18 days in which
th e F R 4 0 d rl was om itted, a n d a sim ilar change occurs after omission of F R 100 drl
in the F R 100 curve. T h e FI curves also show a loss of the high term inal rate. T he
final state 18 days after the usual eight-fold m ultiple perform ance is shown in Fig.
700 a n d 701, w hich should be com pared w ith Fig. 699. Figure 700C shows the final
condition of F R 4 0 , which should be com pared with the small samples of this sched
ule included in Fig. 699A. R ecord D is the final condition of the F R 100, which
should be com pared w ith the 4 samples of this in Fig. 699B. R ecord A is the final
perform ance on both F I 1 and FI 1 drl; the curve lacks the high term inal rate in the ab-
Fig. 7 0 1 .
B.
A.
6 -p ly m ultiple schedule: FI 16 record
Return to 8-ply m ultiple schedule
sence of drl in Fig. 699C. Figure 701A shows the resulting perform ance under FI 16,
to be com pared w ith Fig. 699D, the high term inal rate of which has now disappeared.
T h e following day the original eight-fold schedule was reinstated with im m ediate
effect. Thus, Fig. 700B represents the change in the perform ance at Record A as a re
sult of re-introducing the drl on the 2 ratio schedules. T he effect is to increase the
term inal rate. R ecord E shows the effect on the smaller ratio, including the drl. T he
portions which are not drl are steeper th a n those in R ecord C. R ecord F is the com
bined perform ance for F R 100 an d F R 100 drl, and the portions from F R 100 alone
are steeper th an those in Record D. T he effect on the larger interval is not so complete
in the 1st session; nevertheless, it shows an increase in the term inal rate, as a in Fig.
70IB reveals. N ote th a t a t b, under the drl stimulus, a fairly high rate appears.
In a later stage of the experim ent the eight-fold m ultiple was continued; an d the
larger ratio, both w ith and w ithout drl, was increased to 185. This increase produced
considerable straining and long positive accelerations in the ratio on drl, but scarcely
any pausing on the non-drl m em ber. Figure 702 shows an exam ple of the perform-
Fig. 702.
703.
8-ply m ultiple schedule 14 sessions a fte r large FR was increased to 185:

FR and FRdrl record
8-p ly m ultiple schedule a fte r 24 sessions: FR and small FI records (second bird)
561
ance on the large ratio w ith a n d w ithout drl. T he F R d rl feels the strain and the
straight F R does not. This condition could be the indirect result of the larger m ean
ratio produced by the failure to satisfy the drl contingencies as soon as the scheduled
ratio is counted out. Note the occasional im perfect m ultiple control at a and b, where
the bird slows down under non-drl conditions. At this stage the drl on the small r a
tio continues to produce runs at a high rate of from one to three times the ratio; this
high rate is followed by a sharp break to a low rate, which, however, often fails to re
ceive reinforcem ent for several m inutes. T he rough grain satisfies the drl contingency
in this case.
Second bird
A second bird on the same program developed m ultiple control to the extent seen
in Fig. 703 a n d 704 after 24 sessions. Figure 704 reveals an alm ost com plete suppres
sion of rate by drl. O n FI 16 alone, however, a high rate is assumed imm ediately upon
presentation of the appropriate p attern , an d continues w ithout interruption for m any
thousands of responses. Tw o exceptions to this perform ance occur late in the ses
sion, at a a n d b, w here the bird prim es itself into a drl perform ance on the non -d rl
pattern. M eanw hile, on the short interval (Fig. 703A), a high rate appears in the
absence of drl, a n d a som ew hat m ore interm ed iate rate u n d e r drl. T he sm all ratio
(Fig. 703B) shows the same characteristic as in the perform ance by the other bird: the
ratio is ru n off a t a high rate, the count being som ew hat exceeded before a low rate
appears. In the larger ratio (R ecord C) the drl contingency is generally effective from
the beginning, and the whole ratio is ru n off at a low rate, except for an occasional
break-through, as at a.
W hen the drl contingencies were om itted on the ratios, the same effect was observed
as w ith the other bird: the ratio rate fell to approxim ately h a lf the preceding value.
T he high rate in the larger interval was also reduced. W hen the drl contingencies
were added in the following session, high rates appeared at both intervals, and the ratio
perform ances returned essentially to the condition of Fig. 703.
After returning to the eight-fold m ultiple schedule, this bird developed considerable
confusion : it appeared to respond m ore to its own behavior as a controlling stim
ulus th an to the patterns on the key. Figures 705 and 706 show an interm ediate state
after 74 sessions on the m ultiple schedule, including the six-fold and seven-fold cases.
T h e sm all-interval perform ance in Fig. 705A generally shows a single high rate in
the absence of drl, as at b, c, e , f a n d g, b u t a tendency to break into this rate while u n
der drl, as at a and d. T he small ratio with drl (Record B) continues to show a high
rate before slow-down, although in some instances the ratio is not counted out before
the drl becomes effective (for exam ple, at h and i). T he small ratio w ithout drl r e
m ains high. O n the larger ratio (R ecord C) the perform ance w ithout drl is standard;
b ut w ith the drl it shows considerable roughness of grain, which now evidently satis
fies the drl contingency. T he contingency makes itself felt, however, at the beginning
of the exposure to this stimulus. M eanw hile, on the large interval (Fig. 706) m uch
Fig. 7 0 5 .
8 -p ly m ultiple schedule show ing confusion a fte r 7 4 sessions.

and small FI records
FR
563
of the perform ance shows a fine step-wise grain, which m ay appear either w ith or w ith
out the drl contingency. (T he record resets a t reinforcem ent.) Some periods of drl
show the appearance of the high term inal rate characteristic of FI 16 alone, as a t a,
a n d b. In contrast, the FI 16 schedule w ithout drl frequently shows periods of low re
sponding, as at c and d.
T h e final picture for this bird shows an even m ore erratic stim ulus control. In Fig.
7 07A the perform ance on the small ratio continues to show m arked negative curva
ture. A high rate occasionally breaks through on the large ratio under drl (Record
564
B), either at the beginning of the segment, as a t a, or throughout, as a t b. T he p e r

form ance on the short interval (R ecord C) suggests strong induction from the ratio
schedule. T he interval characteristically begins with a few responses a t a high rate;
a n d regardless of drl conditions, is com pleted at a very low steady rate, occasionally
w ith a break-through, as at c. O n the large interval (R ecord D) some suppression oc
curs in the presence of drl; b u t the norm al F I 16 is being ru n off a t a m uch lower rate,
a n d the perform ance is m arked by strong step-wise grain, also suggesting induction
from the ratio, as a t d and e.
Third bird
T h e th ird bird never achieved good control u n d er the eight-fold m ultiple schedule.
H igh ratio rates developed early, and the large-ratio drl showed a tendency to begin
a t the norm al ratio rate, contrary to the result for the other 2 birds. T he large in te r
val never developed a high term inal rate, although some control of the drl was evi
dent. T he sm aller interval developed drl control, b u t the curves were irregular.
At one point a technical defect resulted in an extensive extinction curve in the stim
ulus appropriate to the large interval w ithout drl. Figure 708 shows the entire ses
sion. T he 1st interval under drl indicates fairly adequate control. Extinction begins
at the arrow , at approxim ately the previous over-all rate for F I 16 alone. T h e record
shows characteristic break-throughs of the ratio rate, which become m ore extensive as
pauses develop later in the interval. E arly break-throughs are seen a t a, b, c, d, a n d
elsewhere. Later, after long pauses, extensive break-throughs appear at e, f g, h, an d
elsewhere. T h e rate during these break-throughs rem ains constant throughout the ex-
565
perim ent, although the over-all rate otherwise falls off fairly sm oothly from a value
such as th a t following b to th a t at i, and then j and k.
MULTIPLE F110 AVO IDANCE R S 3 0 S S 3 0
In the technique for studying avoidance behavior developed by Sidm an (1953), an

aversive stimulus (e.g., a shock) is presented periodically. T he arb itrary time between
shocks is the shock-shock interval (SS). A response, such as pressing a lever, post
pones the next shock by another arb itra ry interval (RS). W e shall designate a given
schedule in this way: avoid SS 30 RS 30, w here the intervals are expressed in seconds.
T hree rats w ith a history of F I a n d m ix F R F R (see C h ap ter Eleven) were placed
on avoid R S 30 SS 30 for 15 sessions. T h e schedule was th en changed to m ult FI 10
avoid RS 30 SS 30. T he stim ulus ap p ro p riate to the fixed interval was a buzzer; the
stim ulus ap p ro p riate to avoidance was the absence of the buzzer. A perform ance
ap p ro p riate to the FI 10 developed very slowly in spite of the earlier history. Figure
709 shows perform ances u n d er both stim uli after 260 hours (58 sessions). A wellm arked scallop has developed in FI (R ecord A); a characteristic low stable rate a p
pears in avoidance (Record B). T he hatches m ark shocks not avoided, and the dashes
above the curve indicate the points at which the schedule changed.
T h e condition in Fig. 709 was not stable, however. T he schedules were being
changed in simple alternation, after reinforcem ent on FI and after 10 m inutes on the
avoidance schedule. Since responding was fairly constant a t both points, m any a c
cidental contingencies betw een a response an d change of schedule arose. T o some
extent the perform ance on the F I schedule was being punished by a change to the
avoidance stim ulus, while responses on the avoidance contingency were being rein
forced by a change to the F I stim ulus. E vidence of the effect of the spurious co n tin
gency under the avoidance stim ulus appears in the form of bursts of responding which
are not ap p aren t on avoidance alone. Figure 710 shows two examples. Records A
an d B are for a single session recorded as in Fig. 709. R ecord C is an exam ple from
an o th er session, w here the avoidance schedule rem ained in force for a longer m ean
period. At a a n d b hundreds of responses em erge at a rate appropriate to the interval
schedule. T he accelerations are smooth.
In an effort to check the n a tu re of these bursts of responses, we discontinued the
Fig. 7 0 9 .
M u lt FI 10 a vo id RS 3 0 SS 3 0 a fte r 5 8 sessions
566
Fig. 7 1 0 .
M ult FI 10 avoid RS 3 0 SS 3 0 showing periods o f higher rates

in the avoidance stimulus
shock, a n d the avoidance behavior underw ent extinction. This procedure h a d the
effect of reducing the low sustained rate in Fig. 71 OB an d C, but the bursts of responses
rem ained in the perform ances of all three rats. Figure 711A shows a perform ance on
the FI com ponent for a second ra t in the experim ent. It does not show so sm ooth a
scallop as th e preceding figure; th e rate is m uch higher a n d greater irregularity is
evident. This ra t also showed low sustained responding on the avoidance schedule,
w ith break-throughs; but when the shock was discontinued (Record B), the sustained
rate declined slowly, and responding ceased for long periods, as a t b, c, e, g, and h. But
the rate is by no m eans zero, an d occasional bursts of responding a t a rate appropriate
to the interval schedule appear, as at d and f
An occasional period of slow responding
w ith rough grain also occurs, as a t a. This behavior could be the result of reinforce
m ent by the change to the FI schedule stim ulus, as experim ents in C hapter Twelve
will show. It could also be induction or the im perfect stim ulus control seen in the
8-ply m ultiple perform ance.
,
567
MULTIPLE DEPRIVAflON
Tw o m agazines were installed in one box; one contained food a n d the other deliv
ered small am ounts of water. (A dipper brought a small am ount of w ater into posi
tion w here the bird could drink it. T h e actu al am o u n t d ru n k depended upon the
shape of the beak and the drinking behavior of the bird.) In a study of the interaction
of types of deprivation, 2 m ultiple schedules were established. In one case a response
was reinforced with w ater (on a schedule) whenever the key was green and with food
w henever it was white. In a second case, 3 stim uli were correlated w ith (1) w ater, (2)
food, and (3) either food or w ater a t random . T he color correlated w ith the third pos
sibility was blue.
T hree pigeons were placed on V I 3 w ith w ater reinforcem ent. D uring the early
stages of the experim ent, technical problem s connected w ith the w ater reinforcem ent
were solved, and the history during this period is rath er complex. L ater, records were
obtained of the perform ance when the color was appropriate to w ater reinforcem ent
(with no food deprivation). (In each case a large dish was available in the experim en
tal box at all times containing the substance of which the pigeon was not a t the m om ent
deprived.) Some interaction exists am ong these two forms of deprivation. A
thirsty bird cannot eat a n d is therefore presum ably in some stage of food deprivation.
T he color on the key changed in simple alternation at this stage; later in the experi
m ent the presentation was random . C hanges occurred every 4 m inutes w ithout re
spect to reinforcement.
Figure 712 shows a relatively late stage o f m ult V I 3 (water) V I 3 (food) under w a
ter deprivation. R ecord A shows a perform ance w ith w ater reinforcem ent on V I 3 oil
a green key. Record D is the alternating perform ance on the white key, where a re
sponse is reinforced w ith food on V I 3; b u t the bird has undergone no food deprivation
an d is supplied w ith food in the box. N ote th a t substantial responding is present in
R ecord D, although the rate is low. R ecord B is the V I 3 perform ance for a second
568
bird w ith w ater reinforcement; a n d R ecord C, the alternating perform ance under food
reinforcement. Both rates are relatively higher th an those of the first bird. Again,
responding on the food-reinforced key is substantial. T he points a t which the schedule
changed have not been m arked.
Shortly after these curves were recorded, the birds were given free access to w ater
a n d were deprived of food. Figure 713 shows a set of records com parable w ith those
of Fig. 712 w ith the deprivations reversed. R ecord A is a V I 3 performance for food
reinforcem ent; an d R ecord D, the altern atin g perform ance u n d er the color reinforced
by the operation of the w ater m agazine (w ater being freely available in the box). R ec
ord B is the perform ance of a second b ird on food reinforcem ent, while R ecord C is
the a ltern atin g perform ance u n d er the color reinforced by the operation of the w ater
m agazine. (T he presentations are now random ized rath e r th an alternated.) A sim
ilar condition prevails: responding occurs on the color ap p ro p riate to the reinforce
m ent for which no corresponding deprivation exists.
T he operation of the w ater m agazine possibly has conditioned reinforcing properties
from its sim ilarity to the operation of the food m agazine. W hen the w ater m agazine
was disconnected, responding to the green key dropped alm ost to zero, except for a n
initial burst of responses a t the beginning of each session for one bird. A m ore
plausible explanation is th a t the bird is still form ing a discrim ination betw een the keycolors. M uch later in the experim ent (see, for exam ple, Fig. 718) near-zero rates are
observed on the satiated color.
W e attem pted to exam ine the perform ances under the 2 stim uli under sim ultaneous
deprivation in both fields. T h e experim ents are com plicated, however, by the large
am ount of interaction betw een the 2 deprivations. T h e effect of a given degree of w a
ter deprivation will be affected by the extent of the food deprivation a n d vice versa.
In general, the V I curves obtained u n d er this procedure are quite irregular. T hey
show tendencies to drop to interm ediate rates, a n d the curvature after reinforcem ent
569
m ay be quite variable. Record A in Fig. 714 shows a performance on the key-color re

inforced with water; and Record B, th at on the key-color reinforced with food. T he
over-all rates are com parable. N ote th a t R ecord B begins with about 20 m inutes of
responding at a low rate d u rin g several presentations of the food key-color. This
perform ance frequently appears in food records, and m ay indicate th at until the bird
has received a few w ater reinforcements, it is unable to eat or at least eats a t a rate sug
gesting a m uch lower level of food deprivation th a n th a t which actually prevails.
After a sm all am ount of w ater is received, the full effect of the food deprivation is evi
dent. T he rate which breaks through at a and again later in the record is m uch higher
th an the m ean over-all rate.
In Fig. 715, for a second bird, a higher level of deprivation in both fields is suggested
by the higher over-all rate. R ecord A shows responding on the green (w ater) key;
and Record B, on the white (food) key. In this case, food behavior is not suppressed a t
the beginning of R ecord B as the result of concurrent w ater deprivation. N ote th a t
both records show an occasional drop to a lower rate, as at a and b.
In Fig. 716, for the third bird, the rate of the green (water) key at Record A shows
a decline during the session th a t is probably due to satiation. T he change in rate from
a to b is of the order of 4:1. C oncurrently, responding on the key-color reinforced w ith
food shows a very low level of responding until the last few m inutes of the session (R ec
ord B). T h e bird h ad been deprived of food as well as w ater, b u t the com bined d e p
rivation has here produced low rates on the food key.
An earlier stage with the same bird shows a decline in rate during the session on w a
ter reinforcem ent, while a high rate is m aintained under food reinforcement (Fig. 717).
570
Fig. 7 1 5 .
M ult VI 3(w ater) VI 3 (fo o d ) under food and w a te r dep riva tio n (second bird)
T h e over-all rate of responding on the w ater color changes from a high value a t a to an
extrem ely low value at b. A brief period of slow responding occurs on the food key
a t c, which m ay represent the suppressing effect of the concurrent w ater deprivation.
W hen both deprivations were substantial and fairly high rates were being shown on
both key-colors, a glass of w ater was placed in the experim ental box. Tw o birds con
sumed some tim e in drinking; this tim e appeared as breaks in responding on the food
key, which happened to be present then. W hen the w ater color was on the key, how-
Fig. 71 7 .
571
M ult VI 3(w ater)V I 3 (fo o d ) under fo o d and w a te r d e p riva tio n show ing satiation
under w a te r reinforcem ent (third bird)
ever, little or no change occurred in the food rate. Figure 718 shows an exam ple.
R ecord A is for w ater; R ecord B, for food. T he session begins w ith a high rate on the
food color at b, and an interm ediate rate w ith some irregularity on the water color a t
a. At the arrows, w ater was introduced. T he rem ainder of the session shows very lit
tle responding on the w ater color. Responding on the food-reinforced key continues,
although w ith increasing irregularity. T he periods of low responding at c, d, and e all
begin after a changeover from the w ater color. T hey possibly indicate some carry
over of the low level of responding on the other com ponent of the m ultiple schedule.
In a som ew hat sim ilar experim ent the 3 birds were given free access to food and
to free access to w ater
572
deprived of w ater until stable rates were established. T h e deprivations were then re
versed. Figure 719 shows the results. Each curve in the graph represents 2 daily
experim ental sessions. A t the left of th e vertical line the prevailing condition was
w ater deprivation w ith full access to food. Perform ances after 24 hours of food d epri
vation are at the right of the line; but a supply of w ater is now available in the ex
perim ental box for the first time. Because of the w ater deprivation, the body-weights
were near the 80% level in spite of the free access to food. T hus, a 24-hour food d e p
rivation had severe effects.
Each bird is represented in Fig. 719 by 2 curves, the segments being recorded in
ran d o m altern atio n in the sam e session. Consider, for exam ple, the p air of curves E
and F. Curve F shows responding on the w ater key on both days; E shows respond
ing on the food key. O n the day represented to the left of the vertical line a substan
tial, though irregular, rate is m aintained on the w ater key (Curve F). Responding to
th e food key was negligible (C urve E). T h e following day, u n d e r reversed d ep riv a
tions, th e w ater record falls to alm ost zero, w hile the food record shows a fairly steep
slope. (The first few m inutes in all curves to the right of the line show low rates. T h e
birds presum ably spent most of this tim e in drinking a n d exploring the cup of w ater.)
T h e other 2 pairs of curves show a sim ilar result, except th a t the key-colors are less
effective controllers of rate. Substantial responding occurs on the w ater key after
w ater satiation (rig h t-h an d portions of Curves A a n d B) an d on the food key under
access to food (left-hand portions of Curves C and D).
T h e interaction betw een food an d w ater deprivation was illustrated by another ex
perim ent, reported in Fig. 720, w here the num bers of responses to the food a n d w ater
keys during the first hour of each daily session were plotted. T he 2 points at the 1st
session show a rate of approxim ately 6000 responses per hour on the food key and fewer
th a n 100 responses per hour on the w ater key. A t this tim e the conditions were free
access to w ater and 80% body-weight controlled in the usual way. Im m ediately after
this 1st session, w ater was withheld. An increase in the rate on the w ater key is not
m arked until the 4th session; a n d by this tim e the rate on the food key has dropped to
below 2000 responses per hour. This lower rate m ay be due in p a rt to additional
food given because the body-weight is lowered by the dehydration; b ut it also probably
reflects a lower level of activity under a given schedule of food deprivation as the re
sult of the concurrent w ater deprivation. L ater in the experim ent, a substantial but
varying rate appears on the w ater key. T h e only w ater now being received is th a t
used for reinforcem ent d u rin g experim ental sessions. In general, there is a rough
tendency for the rate on the food key to rise w hen the rate on the w ater key falls. This
tendency is particularly noticeable in the 13th session. W hen the bird is given free
access to w ater again in the 14th a n d 15th sessions, the rate on the food key rises to
roughly its original value.
In the later stage of the experim ent, 3 key-colors were used. O n one the response
was reinforced with food; on another it was reinforced w ith water; and on the third,
w ith either food or w ater in an unpredictable series a n d equally often. T he perform -
RESPONSES
OF
THOUSANDS
.7 1 9 .
Transition from w a te r d e p rivation and free access to fo o d to fo o d deprivation

and free access to w ater
574
Fig. 7 2 0 .
Interaction between fo o d and w a te r deprivation
ance on the th ird key will depend to some extent upon the preceding history. If, for
exam ple, w ater deprivation has been m oderate b u t food deprivation severe, perform
ance on the m ixed key is likely to show the effect of reinforcem ent w ith food a n d to re
tain signs of this effect later w hen conditions of deprivation are changed. Eventually,
however, the key reinforced w ith both food a n d w ater assum ed a value w hich a p
pears to be a sim ple arith m etic m ean of the rate on the other 2 keys u n d e r any given
set of deprivations. Figure 721 shows an exam ple in which both deprivations were
substantial. R ecord A is the perform ance on the green key reinforced w ith water.
R ecord B is the performance on the white key with the food reinforcement. An im por
ta n t feature is the tendency, already noticed, for the rate to decline tem porarily fol
lowing a changeover from the w ater key, as at a. R ecord A shows a m oderate rate
u n d er w ater reinforcem ent an d w ater deprivation. T he slope of R ecord G on the key
reinforced w ith both food a n d w ater is close to the m ean for the other two over-all
slopes.
Figure 722 shows an exam ple of this averaging w hen one deprivation is slight. O n
the w hite key the bird is reinforced w ith food an d responds rapidly (R ecord A); its
rate tends to fall shortly after changeover from the w ater key, as at a. R ecord C illus
trates a very low rate u n d e r w ater reinforcem ent u n d e r free access to w ater. T he
slope of R ecord B on the key reinforced w ith both food a n d w ater is again close to the
m ean of the other 2 curves.
Fig. 7 2 1 .
M u lt VI 3 (fo o d )V I 3(w a te r)V I 3
(fo o d or w a te r) under fo o d and w a te r d e p ri
va tio n
and free access to w ater
Fig. 7 2 3 .
M u lt VI 3(food)V I 3(w ater)V I 3 (fo o d o r w ater)
under fo o d de p riva tio n and fre e access to w ater
577
Figure 723 shows continuous cum ulative curves on the 3 key-colors, changing at r a n
dom throughout the session and w ith free access to water. Curve A shows responding
on the key-color reinforced with food u n d er an existing high food deprivation. Curve
C shows the perform ance on the key-color reinforced w ith w ater under free access to
w ater. A ny responding here is probably to be a ttrib u ted to either conditioned rein
forcing properties of the operation of the w ater m agazine or an incom plete key-color
discrimination. Curve B shows responding to the key-color reinforced with either food
or w ater in a random order an d an equal n u m b er of times. A dotted curve has been
draw n to show the m ean of Curves A a n d C. T he curve on the color reinforced w ith
both food a n d w ater is som ew hat higher th a n the m ean of the other curves, b u t is
still a fair approxim ation.
PERFORMANCE O N MULTIPLE SCHEDULES AFTER BRAIN LESIONS
Figures 384 and 385 dem onstrated a perform ance on FI 2 after brain lesion. T h e
bird was subsequently reinforced on F R 60 for 18 sessions on a blue key, followed by 1
session on F R 185 on an orange key. O ther sessions on F R 60 on the blue key followed.
Figure 724B shows the perform ance in the next session when the schedule was F R 5 5
on the blue key. This perform ance was recorded 39 days after an operation, effects of
which were determ ined post m ortem as follows: Autopsy revealed large cyst in the
forward half of the brain, 9 m m in depth an d 3 mm in diam eter. This extended all
the way from the top to the base of the right cerebral hemisphere. Microscopic exam
ination shows m arked loss of tissue on one side w ith evidence of tra u m a of h alf the re
. .
.
l.
m aining b rain .
T he perform ance is appropriate to the schedule, w ith an over-all rate of about 3 re
sponses per second an d pauses after reinforcem ent varying from zero to a few seconds.
578
In th e following session, R ecord A, th e key was o ran g e a n d the schedule F R 185.

O nly brief pauses follow the first 3 reinforcem ents. T h e pauses increase, however, to
th e ord er often produced by a fixed ratio of this size. T h e rate a t w hich the pauses
develop and the lack of any disturbance of the usual fixed-ratio pattern suggest th at
a great deal of stim ulus control has already developed from the two exposures to F R
185 on the orange key. W e cannot say w hether or not the fact th at this b ird s p e r
form ance is adequate on m ult F R F R and not on F I represents a lesion which produces
a specific deficit. A retu rn to the fixed-interval schedules of reinforcem ent u n d e r
w hich atypical curves were previously recorded w ould possibly now produce behavior
appropriate to these schedules.
In a control experim ent on the effect of b rain lesions on perform ances on m ultiple
schedules, 2 birds which h ad established stable perform ances on m ult F R F I were sub
jected to a sham operation w here the surface of the b rain was exposed. Post-m or
tem microscopic exam ination showed slight injury to the b rain surface. O ne of these
birds h ad shown the perform ance in Fig. 725A on m ult FI 10 F R 125 before the opera
tion. This rath e r large ratio is ru n off w ith only slight pausing or curvature a t the b e
ginning, and the interval scallops are quite consistent and usually smoothly accelerated.
T he first experim ental session 13 days after an operation begins as in Fig. 726. T h e
m ultiple control is present from the start. R atio perform ances show some curvature,
as a t a, although instances ap p ear in which the entire ratio is ru n off at a high rate,
579
as at b. T h e interval cu rvature has been disturbed by an in terru p tio n of the experi

m ent and by the incidental effects of the operation. T ow ard the end of this 1st ses
sion, not shown, a fairly clear interval curvature begins to emerge. T h e perform ance
on the m ultiple schedule 3 sessions later is shown in Fig. 725B, which m ay be com
p a re d w ith R ecord A in the sam e figure, tak en before the sham operation. In com
p aring the 2 records, note th a t interval a n d ratio were alternating in Fig. 725A and
changing m ore or less a t random in Fig. 725B. A simple altern atio n generally gives
a m ore stable performance. T he other bird in the experim ent also showed recovery
of a m ultiple perform ance in the 4th session following operation.
Chapter Eleven
MIXED SCHEDULES
M i x e d s c h e d u l e s are the same as m ultiple schedules except th at no stimuli are cor
related w ith the com ponent schedules. T hus, u n d er m ix F R 100 FI 10 the organism
is reinforced either on F R 100 or on F I 10, an d there is no difference in the stim uli
present in the two cases. Every m ultiple schedule has a corresponding m ixed sched
ule. T h e m ethods of pro g ram m in g the com ponents a re also the sam e in the two
cases.
MIXED FIXED-RATIO FIXED-RATIO SCHEDULES
Experiment I
In an early experim ent 2 birds were reinforced on mix F R 190 FR 30 after erf. O ne
of them developed the interm ediate perform ance shown in Fig. 727A. T h e small
ratio shows a slight pause an d a high, w ell-sustained term in al rate. T h e large ratio
shows inflections corresponding roughly to the size of the sm aller ratio. E ach long
segment contains a t least one, the distance of which from the preceding reinforcem ent
is slightly m ore th an the small ratio. Segments with two inflections are common. As
m any as three m ay occur, as at a. Evidently, the b ird s own behavior serves as a stim
ulus in the m anner of a m ultiple schedule. T he emission of approxim ately the n u m
ber of responses in the sm aller ratio prim es a pause ap p ro p riate to the larger ratio.
This perform ance becomes clear later, w hen the larger ratio is divided into 4 almost
equal parts. L ater on, after 56 hours, the bird develops an alm ost perfect prim ing
from the short to the long ratios shown in Record B. After almost every reinforcement,
there is a ru n slightly in excess of the sm aller ratio, followed by a m arked break which
m ay extend to as m uch as 5 m inutes, followed in tu rn by a single sustained ru n for
the balance of the long ratio. Almost no pausing occurs in the short ratio, but a c h a r
acteristic long pause develops in the long ratio when the controlling stim ulus has been
set up by the initial run.
Figures 728 an d 729 show various stages in the perform ance of the other bird. Fig
ure 728A shows m ultiple inflections sim ilar to those in Fig. 727A except th a t the
580
Fig. 7 28.
M ix FR 3 0 FR 1 80 showing m ultiple inflections
582
changes in rate are not so abrupt. Figure 728B reveals better prim ing into the b a l
ance of the longer ratio. T he first break and inflection are m ore m arked th an any oc
curring later in the long ratio, for example, at a or b. T he wavelike character of the
record is dam p ed d uring the long ratio. T he factors responsible for the oscillation
are to some extent controlled by the distance the bird has progressed from the p re
ceding reinforcem ent. This dam ping is still evident in R ecord C, as for exam ple a t
c, a n d the first pause after reinforcem ent is m uch m ore m arked th an any break oc
curring later in the long ratio. H ere, for exam ple, from d to e, is an over-all curva
ture appropriate to the longer ratio. In Fig. 729A the first break occurs later. This
record was taken 48 sessions after the first exposure to the m ixed schedule. In m any
instances, as for exam ple, at a an d b, the break occurs after the m iddle of the longer
ratio has been passed. Record B shows another tem porary phase containing S-shaped
curves, as a t c an d d. T he curve a t d, for exam ple, is sim ilar to those in Fig. 727B
except th a t in place of sharp breaks, the curvature is fairly smooth.
Note the consistent difference betw een the 2 birds in Fig. 727 and 729. T he first
effect after reinforcem ent is a p p ro p riate to the sm all ratio in Fig. 727 (no pause) a n d
to the larger ratio in Fig. 729 (relatively long pause after reinforcement).
M IX E D SCH EDULES
583
Experiment II
Tw o other birds w ith a history on m ultiple an d m ixed schedules were reinforced

on mix F R F R , where the size of the larger ratio varied from F R 2 5 0 to F R 380; the
sm aller rem ained a t F R 60 throughout, a n d the proportion of large to small ratios was
varied. A 2.5-m inute T O followed every reinforcem ent th roughout the experim ent.
T he ratios were recorded separately, the pen resetting at reinforcement.
Figure 730 shows 4 stages in the developm ent of a mix F R F R performance. R ecord
A shows the larger ratio after 20 sessions on m ix F R 60 F R 360; R ecord B, after 22 ses
sions; R ecord C, after 29 sessions; an d R ecord D, on F R 60 F R 380, after 30 sessions.
N ote the m ultiple breaks a t a and b. T he term inal rate in the large ratio has fallen by
Record D (at c, d, e, etc.).
T h e other bird developed a m uch m ore consistent perform ance on m ix F R 60 F R
360. Figure 731 shows the large ratios in th e 23rd session. T h e pap er speed has
doubled. N ote the sharp breaks an d the relative consistency of pausing after a n u m
ber of responses greater th a n the short ratio has been ru n off. T here is no pausing
after reinforcem ent, except a t d a n d f
T h e breaks occur considerably beyond the
F R 5 0 point; the break a t c, for exam ple, occurs after about 150 ra th e r th an 50 re
sponses after reinforcem ent. T h e rem aining ratio after this pause is of the same
order as th a t before it. T he bird is, in essence, being reinforced 50% of the tim e on a
ratio varying around 150, in addition to the reinforcem ent on F R 6 0 . In 3 cases, at
a, b, and e, the larger ratio is run off w ithout a noticeable change in rate.
T he sharp break in Fig. 731 is only a tem porary phase. U n d er various m an ip u
lations of th e percentage of large a n d sm all ratios, the perform ance shown in Fig.
732 was generated. H ere, the over-all rate has declined, an d m ultiple breaks fre
quently appear, as at b, c, an d d, w ithin a single long ratio. O ccasional sharp breaks
still occur, as at f ; b u t even so, the balance of the interval shows some negative ac
celeration and poor grain. Note at this stage the occasional appearance of pauses after
reinforcem ent, as at a and e. Little relation exists between the rate and a preceding
reinforcem ent or pause. T he perform ance suggests a variable ratio with rather rough
grain, as one m ight expect from the inform ation in the preceding paragraph.
Figure 733 shows a still late r perform ance for the sam e bird on m ix F R 60 F R 360
after an intervening history of straight F R 360. T he separate long-interval segments
have been stacked for better reproduction. T he break occurs m uch closer to an ac
tu al reading of 60 responses, but the acceleration a t the break is seldom abrupt. T he
balance of the long interval begins to show a n over-all slight positive curvature. H ere,
responding on the short ratio prim es the bird into a perform ance ap p ro p riate to
the long ratio. These curves should be com pared w ith those under mix F R F I with re
spect to the accuracy of the prim ing runs.
!
Figure 734 presents a complete session for this bird at this stage, in which both ratios
are shown in the order in which they occurred. T he shorter ratios are generally com
pleted w ithout pausing, and only an occasional pause occurs after reinforcem ent, as
D
Fig. 7 3 0 .
Four stages in the developm ent o f mix FRFR
584
300 RESPONSES
Fig. 7 3 2 .
Later perform ance on mix FRFR (second bird)
585
586
at a a n d b. A w ell-m arked negative b reak occurs a t a n u m b er of responses equal to

approxim ately twice the sm aller ratio, as at c, d, and elsewhere. T he balance of the
long ratio m ay contain additional breaks, as at e and g; or it m ay continue with a fairly
good acceleration suggesting a larger ratio perform ance, as at f
Figure 735 contains a single session for th e first b ird, recorded in two ways, a t a p
proxim ately the stage shown in Fig. 730C. R ecord G gives the whole perform ance.
(O ne hour has been om itted in which only a response or two was em itted.) Pausing
due to the shorter ratio is fairly well-developed, a n d m ultiple breaks occur, as at m
a n d n. T he segments on the larger ratio have been separately recorded a t R ecord
A, and on the smaller, a t Record B. Corresponding performances are as follows: a and
i, b and j, c a n d o, d an d p, e a n d h , f an d k, an d g and I.
W hether pauses will appear on mix F R F R depends upon the m ean of the ratios, an d
hence varies w ith the sizes of the ratios a n d the proportions in which they are used
in the program . A bird which h ad been on a mix F R 60 F R 360 for m any sessions, in
which only two instances of the longer ratio occurred for every ten of the shorter (yield-
M IX E D SCH EDULES
587
ing a low m ean ratio), paused little or not a t all, as Fig. 736A shows. W hen the
proportions of the two ratios were changed so th a t three of the larger ratios occurred for
every nine of the smaller, pausing quickly appeared after reinforcem ent, as at e, f g,
an d h, an d soon after the short ratio h ad been com pleted, as at a, b, c, d, and elsewhere.
Figure 737 reports a full daily session, in w hich for the first tim e the schedule is
m erely 360 after a history of mix F R F R . A lthough some trace of a break exists after
the shorter ratio, particularly late in the session, as at c, g, and h, the m ain change is
th e developm ent of pauses after reinforcem ent, as a t a, b, d, e , f a n d elsewhere. H ere,
the discontinuation of the reinforcem ent on the shorter ratio perm its extinctions
Fig. 7 3 5 .
Com plete d a ily session on mix FR 3 6 0 FR 6 0 (first bird)
588
Fig. 7 3 7 .
First session on FR 3 6 0 a fte r mix FR 6 0 FR 3 6 0
of the prim ing break; b u t it generates a higher m ean, which produces pausing after
reinforcem ent.
A nother bird, changing for the first tim e to F R 360 after mix F R F R , gave an entirely
different result, as Fig. 738 shows. H ere, the increased m ean ratio produces severe
straining and em phasizes the breaks surviving from the earlier schedule. After m ore
th an 3 hours on F R 360, breaks occur frequently at the appropriate points, at g, h, i,
Fig. 738.
First session on FR 3 6 0 a fte r m ix FR 6 0 FR 3 6 0 (second bird)
M IX E D SCH ED U LES
589
j , k, I, a n d elsewhere, although reinforcem ent has not been received a t the short ratio
since the preceding session. N ote th a t frequent reinforcem ents have been re
ceived after short ratios after pausing, as for exam ple a t a, b, c, d, e, an d f
Because
this bird was showing m ultiple inflections, the removal of the short ratio from the sched
ule did not greatly alter the conditions of reinforcem ent except to change the m ean
ratio, or a t best the percentage reinforcem ent on short ratios. U nder these conditions
the pausing becomes more, rath er th an less, pronounced.
This bird still showed breaks a t the end of the fourth session on F R 360, as Fig. 739A
illustrates. T he perform ance is still characteristic of F R 3 6 0 in Fig. 738, and breaks
still occur, as a t a, b, c, an d d. W hen short ratios were again added to the schedule to
com pose a m ix F R 60 F R 360 on the following session, the perform ance in Fig. 739B
Fig. 7 3 9 .
Return to mix FR 3 6 0 FR 6 0 a fte r 4 sessions on FR 3 6 0 (first bird)
began. T h e reduction in the m ean ratio reduces the pausing after reinforcem ent,
an d any substantial pausing after the com pletion of the short ratio also disappears.
An S-shaped ch aracter appears in m any segments, clearly seen a t e , f g , a n d h, which
points to surviving stimulus control.
T he change in Fig. 739 is from breaks to fewer or less sharp breaks. T h e other
bird showed the opposite effect. After 4 sessions on F R 360, almost no indication exists
of the shorter ratio, as Fig. 740A shows. C onsiderable straining occurs, b u t the r a
tios are generally ru n off quickly once they have begun. O n the following day, the
schedule was changed to mix F R 360 F R 60 in which 1 long ratio occurred for every
4 short ratios. This procedure tended to reduce the m ean ratio a n d led to an almost
complete absence of pausing after reinforcem ent, as Record B shows. T he reinforce-
590
Fig. 7 4 0 .
Return to mix FR 3 6 0 FR 6 0 a fte r 4 sessions on FR 3 6 0 (second bird)
m ent on the shorter ratio, however, began to reinstate the b reak after the short ratio
ru n in the long ratio, as is evident at a, b, c, d; e, and elsewhere.
M ix e d fix e d -ra tio fix e d -ra tio schedules in rats
Two rats w ith a history of 62 hours on F I 10 were reinforced on mix F R 160 F R 20.
A slight indication of self-generated stim ulus control appears in the 1st session, shown
com plete in Fig. 741 A. Brief pausing after reinforcem ent becomes consistent, an d
pauses after a n u m b er of responses ap proxim ately eq u al to th e shorter ratio begin to
ap p ear, as at a, b, a n d c. By the end of th e next (3-hour) session, m ultiple breaks
a re com m on, as R ecord B shows. T h e term in al ra te has increased, a n d the pauses
during the long intervals are larger th an those after reinforcement.
L ater stages are shown in Fig. 742. R ecord A is from the 15th hour; Record B, from
th e 24th; R ecord C, the 33rd; an d R ecord D, the 43rd hour. D am ped m ultiple
breaks appear in R ecord C, and a fairly consistent single early break appears in R ecord
D.
Fig. 7 4 2 .
591
Later developm ent o f mix FR 2 0 FR 160 in the rat
A com parison o f m ult FRFR and m ix FRFR
Tw o birds w ith a long history at different values of FI were reinforced on F R 5 0

for 3 sessions. Except for slightly different rates, their perform ances were quite com
parable. In the following session, one bird was placed directly on m ult F R 300 F R
50, while the other was placed on mix F R 300 F R 50. T he latter held the m ixed sched
ule well, and several values of ratio were interchanged in successive sessions w ithout
producing long pauses at any point in the bird s performance. O n the m ultiple sched
ule, however, the other b ird broke alm ost im m ediately a n d h a d great difficulty in
holding the schedule, even though the longer ratio was later considerably reduced.
Figure 743A illustrates a fairly stable perform ance on m ult F R 120 F R 50. This is the
end of the 7th session at several values of the larger FR. D uring this p art of the ex
perim ent, the key-colors were red a n d w hite on th e shorter an d longer ratios, respec
tively. In the following session the key-color was blue (the color used on the earlier
straig h t F R ), a n d the ratios rem ained 50 a n d 120. R ecord B shows th a t this m ixed
schedule is held well, although considerable pausing occurred on the m ultiple sched
ule w ith the same values of ratio. M ixed and m ultiple schedules were then alternated
in a nonsystem atic fashion, an d the values of the larger ratio were changed. T h ir
teen sessions after Fig. 743B the bird was sustaining a perform ance on a m ix F R 50 F R
240, as Fig. 744A shows. At the beginning of the following session, the same ratio
Fig. 7 4 4 .
Transition from mix FR 5 0 FR 2 4 0 to mult FR 5 0 FR 2 4 0
593
values prevailed, b u t the red a n d w hite keys h a d replaced the blue to provide a m u l
tiple schedule. T he quick developm ent of severe pausing after reinforcem ent in the
longer ratios is clear in R ecord B. This bird showed, w ithout exception, an ability
to hold mix F R F R , although long pauses appeared in the larger ratios on the com
parable m ultiple schedule.
T he other bird, w hich w ent directly to the m ixed schedule after 3 sessions on F R
50, held the perform ance well, both a t the original settings of 50 an d 300, a n d a t v a ri
ous other values of the larger ratio during the next few sessions. W hen the key-color
was changed from blue to red a n d white for purposes of a m ultiple schedule, the bird
also held a variety of values of the larger F R fairly well, although some pausing a n d
curvature usually occurred on the longer ratios. Figure 745A displays a final perform
ance on m ult F R 50 F R 240. Here, most of the longer ratios, under stimulus control,
show a m arked break and some scalloping. In the following session, the key-color was
blue, a n d the schedule was therefore m ixed. T h e over-all rate rises im m ediately,
an d the pausing after reinforcem ent is considerably reduced.
M ultiple and m ixed schedules were com pared sim ilarly in another experim ent on
2 birds with a history of approxim ately 2000 reinforcem ents on m ix F R F R and 900 re
inforcem ents (in 10 sessions) on m ult F R F R . Figure 746A illustrates a perform ance
594
on m ult F R 2 7 5 F R 5 0 w ith a T O 8 after each reinforcem ent. T h e larger ratio is

breaking severely. At one point, 50 m inutes of no responding have been om itted from
th e graph. M eanw hile, the short ratio is ru n off w ith no pausing. A fter 2 m ore
sessions of this m ultiple schedule, the controlling stim uli were rem oved, a n d a m ixed
schedule was th en in force. T he bird adjusted to this quickly; R ecord B shows the
perform ance after 4 sessions. T h e curves are essentially linear, except for a slight in
dication of a break in the long ratio shortly after the com pletion of a portion equal
to th e short ratio, as at a a n d b.
All 3 birds show th a t m ixed schedules are m ore easily held th an m ultiple w ith com
p a ra b le values. T he m ixed schedule has the effect a p p ro p riate to the mean ratio.
T he m ultiple case shows breaking on the longer ratio, com parable w ith th a t in straight
fixed-ratio performances.
E xtinction a fte r m ix FRFR
W e obtained examples of extinction during the experim ents just described, usually
th ro u g h failure of the ap p aratu s. As a rule, the perform ance in extinction depends
upon the perform ance which has been reached on the mix F R F R schedule. In Fig.
747, for exam ple, the perform ance prevailing a t the m om ent appears a t the left of the
record, from a to b. Small single breaks occur after reinforcem ent in the long ratio.
(The ratios were not being counted accurately at this time. T he apparatus eventually
broke dow n at d, and no further reinforcem ents were forthcom ing.) T he perform
ance in extinction shows, in general, runs approxim ately equal to the short ratio, as a t
c>e>f > K b J'>and elsewhere. Occasionally, a more sustained performance occurred
characteristic of th a t prevailing in th e b alance of the long ratio, as a t d an d /. T h e
b eginning of the ru n , from k to I, shows the d a m p in g of th e oscillation w hich this
bird had already exhibited in Fig. 728.
Figure 748 gives an extinction curve for the bird shown in Fig. 734 and elsewhere.
R ecord A shows the first 3 excursions on the previous day u nder m ix F R 360 F R 60 T O
M IX E D SCH EDULES
Fig. 7 4 7 .
595
Extinction a fte r mix FR 50 FR 2 5 0
2.5. T hey show the characteristic break after a num ber of responses roughly equal
to two of the sm aller ratios. This break is occasionally om itted, as a t a. At the begin
ning of the session (shown in R ecord B), the ap p aratu s was not reinforcing, an d an
extinction curve was recorded. It begins w ith a ru n of over 2000 responses, ending a t
b. A ru n of m ore th a n 1000 responses occurs later, ending a t c. T h e rest of the rec
ord is com posed of runs roughly of the ord er of one or two tim es the sm aller ratio,
a n d these ten d to show the sam e negative curv atu re as the breaks ap p earin g after
reinforcem ent in the perform ance in R ecord A.
Figures 749 an d 750 present sim ilar extinction curves for two other birds. Each
figure contains a sam ple of a perform ance on m ix F R 275 F R 50 a t R ecord A, and an
extinction beginning at the start of the following session a t R ecord B. T he grain of
each extinction curve resembles th a t of the longer ratios on m ix FR F R .
596
Fig. 7 4 9 .
Extinction a fte r mix FR 2 7 5 FR 5 0 (first bird)
E ffect o f p e n to b a rb ita l on m ix F R 6 0 F R 3 6 0
A bird whose perform ance a t another stage in the experim ent described in Fig. 737
later showed a consistent slight break in the long intervals on m ix F R 360 F R 60. An
exam ple appears at a in Fig. 751. After the reinforcem ent at b, 5 milligrams of p en
to b arb ital ( N em butal ) was injected intram uscularly. This dosage was sufficient to
abolish practically all responding for m ore th an 1 hour. A response occurs at c, and
a few a t d an d e. T he first post-injection ratio is eventually com pleted a t f nearly 2
hours after injection. This reinforcement is followed again by a long pause of nearly
20 m inutes, after w hich a full ratio is ru n off characteristically a t g. T h e next re in
forcem ent is also followed by a short pause at h, b u t a fairly typical ratio perform ance
M IX E D SCH EDULES
597
then ensues. T he 7 long ratios in the rest of the schedule during this session do not
show the prim ing breaks which are characteristic of the sessions both before a n d after
the session shown. This factor m ay be due to the hyper-excitable phase which p e n
tobarbital generally produces at approxim ately this stage. A slight break occurs at i.
MIXED FIXED-INTERVAL FIXED-INTERVAL SCHEDULES
M ix FI 3 3 0 sec FI 3 0 sec1
T he variable schedules discussed in C hapters Six a n d Seven are essentially extrem e

exam ples of m ixed schedules. Some of the series used, especially those w ith m any
short intervals or ratios, produce a prim ing ru n after reinforcem ent sim ilar to the m ix
F R F R above. Conversely, a m ixed schedule m ay have the effect of a variable
schedule until the behavior can stabilize in a perform ance ap p ro p riate to the com po
nent m em bers.
A pigeon which h ad produced fairly good scalloping on F I 2.5 (Fig. 127) returned to
a linear perform ance suggesting a V I schedule w hen first exposed to m ix F I 30 sec
F I 4. Figure 752A shows a sam ple from th e e n d of th e first 2-hour session. In the
following session, greater irregularity began to a p p ear; the rate has some tendency
to be slightly higher following reinforcem ent, b u t drops after p erhaps 100 responses, as
at a and b in R ecord B (from the m iddle of the second session, or after 3 hours of ex
posure to the schedule). T he effect is already more pronounced by the 4th hour at the
end of this 2nd session shown in R ecord C. By the end of the 3rd session, or after
6 hours of exposure to the schedule, occasional instances occur of a m uch m ore m arked
prim ing, as at c in R ecord D. Records E, F, a n d G show the final segments of the
next 3 sessions, after 8, 10, and 12 hours, respectively, of exposure to the schedule. By
the tim e R ecord G is reached, a w ell-m arked prim ing exists after the shorter in te r
val, an d a falling-off into a curvature a p p ro p riate to a longer interval, as a t d, e, n d f
1 T h e experim ents on m ix FIF1 represented by Fig. 752 to 754 w ere c arried out by M r. G eorge V ictor, to
w hom the authors express th eir indebtedness.
598
Fig. 75 2 .
Developm ent o f mix FI 3 3 0 sec FI 30 sec during the first 7 sessions
T h e segm ents a t R ecord G are from th e m iddle of th e session. By the end of the
session, a rather linear perform ance has tem porarily reappeared, with only slight p rim
ing, as at g a n d h in R ecord H. R ecord I presents the perform ance a t the end of
th e next session, after 14 hours of exposure to the m ix F IF I. H ere, fairly consistent
prim ing occurs after reinforcement, except at i, and a sustained lower but slightly ac
celerating rate in the longer intervals.
T he bird was further exposed to the same m ix F IF I schedule; a n d after a total of
36 sessions, or 72 hours, the perform ance was of the character shown in Fig. 753.
599
H ere, very w ell-m arked prim ing runs occur, long pauses after the prim ing runs, and,
in general, a m uch higher over-all rate. A T O 2 was then added after every reinforce
m ent. T he first effect was to elim inate the perform ance characteristic of the
m ix F IFI. Figure 754A, which follows directly upon the session in Fig. 753, is for the
end of the 1st session. A slightly higher rate after reinforcem ent begins to reappear.
I n the 3rd session (R ecord B), after 6 hours, a perform ance characteristic of the m ixed
schedule is m ore m arked. R ecord D shows the final state observed in this experim ent
a t th e end of 14 hours. H ere, double prim in g runs begin to a p p e a r in greater n u m
bers. T he T O appears to elim inate w hatever stim uli the bird h ad been using for the
perform ance on the m ixed schedule, b u t the new schedule w ith T O has a rap id effect
a n d is eventually sim ilar to th a t w ithout T O .
M ix FI 5 FI1
First bird. In another experim ent 4 birds were exposed to mix F I 5 F I 1 for near
400 hours each. W e have shown the developm ent of an app ro p riate perform anc.
by selecting pairs of excursions from representative points throughout the process in
Fig. 755. T he pairs of excursions in the figure are samples from the following hours of
exposure: A, 11; B, 27; C, 28; D, 66; E, 72; F, 114; G, 131; H , 153; I, 170; J , 224;
K, 254; L, 289; M , 312; N, 373; O , 378; P, 388; Q , 391; a n d R , 395. In R ecord
600
B pausing is already developing after reinforcem ent, b u t the over-all rate is quite irreg
ular. In some instances the curvature appears to correspond w ith the interval, b u t
this effect is not uniform . In R ecord C th e perform ance is nearly linear, except for
pauses after reinforcem ent. T h e effect of the m ix F IF I begins to be felt in R ecord
D, w here the evidence is clear, as a t a a n d b, of a decline in rate following an increase
in rate after reinforcement.
R ecord H begins to show a sustained perform ance in the long segments after the
prim ing run, though m ultiple breaks are evident, particularly a t a later stage, in R e c
ords L and M. Fairly well-described scallops appropriate to the longer interval are
clear in Record N and become pronounced a t R ecord O , where a considerable negative
curvature appears before the longer intervals are completed. T he bird recovers from
this perform ance to almost a linear type of record a t R ecord P, which is not very differ
ent from, for example, the perform ance in R ecord D m any hundreds of hours earlier.
T he curvature in R ecord O has generated a new set of contingencies which destroy this
perform ance. L ater, negative curv atu re returns, as in R ecord Q , in which the d e
cline during the longer intervals is occasionally fairly abrupt. A brupt changes in rate
are m ore clear in R ecord R. M ultiple inflections during the long intervals still oc
casionally occur a t this stage.
601
Figure 756 shows a com plete daily session for this bird early in the developm ent
of the perform ance under m ix F IF I between Records C and D of Fig. 755.
Second bird. A second bird showed m an y of the sam e effects. Figure 757 gives
sam ple records showing some of the principal features of the perform ance of this bird.
At R ecord A a slight pause follows reinforcem ent, b u t responding is otherwise fairly
linear. At R ecord B some inflection in the long intervals suggests a prim ing from the
10 M IN U T E S
Fig. 7 5 9 .
Decline in over-all rate during the session
603
short-interval perform ance. A t R ecord C m arked inflections begin to appear, as a t

a. In R ecord D the prim ing ru n leads directly to a single lin ear rate ra th e r th a n
th rough a second period of acceleration. R ecord E shows an exam ple of 2 in
flections in a single long interval, at b an d c. R ecord F shows negative acceleration
in the long interval, particularly at d and e, which is less ab rupt in R ecord G. This sec
ond bird showed another effect, however a sudden shift to a second stable over-all
rate. T his shift a p p e are d for the first tim e in the session shown in Fig. 758A. T h e
high rate w ith which the session begins is m uch above the norm al running rate of the
bird. It reappears elsewhere during the session; changes from one rate to another are
usually fairly a b ru p t a n d not too clearly related to reinforcem ent. This was later
found to be a consistent p attern . T h e two rates are in a ratio of ab o u t 3:1. R ecord
B shows a sam ple from the end of the 4th session after Fig. 758A. It was found upon
direct observation th a t d u ring the recording of the lower rate the bird was pecking on
the panel a t the side of the key in such a w ay th a t the response was not recorded.
T hese double rates have been observed elsewhere, a n d are usually associated w ith
m ixed schedules.
Figure 759 shows a general tendency for this bird to start fast and to retard through
out the session. T his was a useful fact, because the characteristic features of m ix
F IF I are in p art a function of the over-all rate. A single daily session shows several of
them , w ith other conditions rem aining fairly constant. W hen the rate is highest, the
only im p o rtan t feature is the pause after reinforcem ent, the rate otherwise being high
a n d uniform . A slight break first appears a t a, b u t the over-all high rate is m ain
tained consistently until reinforcement. T he first long interval a t a slightly lower rate
(b) shows 4 scallops, as the effect of the m ix F IFI. M ultiple inflections persist for sev
eral intervals following. Later, the first scallop a n d recovery in the interval become
the only im portant deviations, as at c, d, e, an d f . As the rate continues to fall, the te r
m inal rate declines, although a high rate m ay break through occasionally in a wellm arked interval scallop, as at g and h. At this stage the rate in the shorter interval has
become low, and together w ith the pause, produces only a small num ber of responses
a t reinforcem ent in the shorter intervals. P erhaps this is why not m uch prim ing oc
curs in the longer interval. A conspicuous exception occurs a t z; this perform ance
resem bles the occasional double scallop seen in the F I perform ance w hen responding
is in terru p ted by some incidental disturbances an d the pause suffices to set off a sec
ond scallop.
Third bird. A th ird bird went directly to m ix FI 5 FI 1 from erf. T he first 3-hour
session showed a slow positive acceleration, in w hich 700 responses were em itted. O n
the 2nd day, no responses were em itted during the first \ \ hours. T h e first response
was reinforced, as indicated at a in Fig. 760. A smooth acceleration then followed for
th e second time. Note the runs at b and c.
T h e th ird bird also showed a tem p o rary phase in which 2 rates prevailed. Figure
761 describes the effect a t its height. In general, however, the bird gave an extremely
uniform linear perform ance throughout the experim ent. Practically all of the c h a r
acteristic features already noted ap p ear in Fig. 761. An anom alous drop to the lower
RESPONSES
300
10 MINUTES
Fig. 7 6 0 .
Second session on mix FI 5 FIT a fte r erf (third bird)
605
rate following reinforcem ent a t a is also reflected in the general tendency for the over
all rate to assume one of two values throughout the session. Several instances occur in
w hich approxim ately the usual num ber of responses in the shorter interval are em itted
after reinforcem ent; then, a drop occurs to a low rate for the balance of the interval,
as a t b, d, and f . M ultiple inflections during the long interval ap p ear a t c and e. This
record is for an entire daily session, 120 hours after the transition from erf to mix F IFI.
Fourth bird. T he fourth bird showed the same characteristic features. Figure 762
shows a daily session 118 hours after erf. T he record is an especially good dem onstra
tion of sharp prim ing runs after reinforcements; they contain somewhat more responses
th a n those usually em itted in the short interval, followed by sh arp declinesoften to
zero before the intervals are completed. M any instances of m ultiple inflection occur
in a single long interval. T he decline during the experim ental session is characteristic
of this bird and resembles th a t of Fig. 759.
M ix FIFI w ith counter o r clock
M ix F R F R and mix F IF I are possibly the best evidence we have th a t the emission
of a given num ber of responses at a given rate m ay serve as an effective stimulus in con
trolling subsequent behavior. T he addition of a clock or counter to such a schedule
therefore should have obvious significance.
Tw o birds were reinforced on m ix F I 10 F I 2 w ith the ad d ed counter already d e
scribed. T h e spot of light on the key grew to 1 inch as its m axim al length as 300 re
sponses were em itted. Figure 763 shows a final perform ance after 44 hours. In
general, the long interval shows a break after emission of a n um ber of responses th at is
606
roughly twice the num ber shown in the shorter interval. Instances of these breaks
are a t d, e, g, h, i,j, k, m, n, o, p, q, a n d r. O nly slight breaks ap p ear early in the record,
at a an d b\ a n d the rate changes only slightly, in lieu of a break in the long intervals,
a t / a n d I. T he extent of the scalloping in the balance of the interval is fairly uniform ,
an d m uch greater th an the slight scalloping after reinforcem ent in the shorter in ter
vals or at the beginning of longer intervals.
U nfortunately, we have no perform ance on mix F I 10 F I 2 w ithout counter for this
bird for purposes of comparison.
W hen the counter added to m ix FI 10 FI 2 showed fading (see C hapter Five), it
had a less consistent effect. Figure 764 illustrates the final perform ance of the bird
shown in Fig. 763 with the fading counter. O ne fairly stable rate is considerably above
the value required to increase the size of the spot, an o th er is clearly below, a n d one
is at approxim ately the value needed. T he second p a rt of the longer interval tends to
607
take on positive curvature. T he drop in rate is no longer a b ru p t where the num ber
of responses characteristic of the shorter interval has been em itted in the longer in te r
val. For exam ple, the curves at a, b, c, an d d are fairly smoothly S-shaped.
W e showed th a t th e ad d ed counter is responsible for these features of the perform
ance, rath er th a n prolonged exposure to m ix F IF I alone, in a later session when the
counter was removed. Figure 765A shows the first 3 and the last 3 excursions from the
1st session. T he first effect of rem oving the counter is to remove practically all changes
in rate. By th e end of the session, some slight pausing occurs after reinforcem ent,
although pauses are usually preceded by a few responses, as a t a a n d b. N othing like
a standard mix F IF I perform ance appears at any tim e during the session.
O n th e following day, a counter was introduced in w hich the spot reached full size
only after 1200 responses. A break in the long interval is im m ediately reinstated (at
c in R ecord B), b u t it is greatly postponed because of the slow change in the counter
reading. T he second long interval contains a sim ilar postponed break at d. L a
ter in the session, w ell-m arked breaks occur; but, in general, they appear relatively late
in the long intervals. T he scalloping after reinforcem ent is a t tim es quite m arked
an d smooth, as a t e, f g, an d h, although later instances occur in which the pause is
m uch shorter an d the return to a high rate m uch m ore rapid.
W hen the counter was replaced by a clock, w hich reached its m axim um value in
10 m inutes (equal to the longest interval), the behavior showed some change. Figure
766 shows a session beginning after 41 hours on this schedule. A som ew hat m ore
consistent pausing after reinforcem ent a n d a som ew hat m ore a b ru p t assum ption of a
higher rate are characteristic, although m an y instances also occur in the longer in
tervals in w hich a fairly sm ooth scallop begins after the b reak in rate, as in a, b, c, d, e,
and f
N ote th a t these scallops a p p ro p riate to the longer interval are m uch m ore
m arked th a n in the shorter interval. T hey are not, however, characteristic of a clock
perform ance. At the beginning of the excursion m arked g the apparatus was changed
608
Fig. 7 6 6 .
FI 10 FI 2 + clock a fte r 41 hr fo llow ed b y FI 2 + clock
during a brief period of T O , so th a t all subsequent intervals would be 2 m inutes w ith

clock. T he first 5 intervals following show curvature com parable w ith th a t which has
been prevailing up to this point u n d e r m ix F IF I; b u t the perform ance quickly goes
over to a fairly good short FI w ith clock.
Figure 767 shows a final perform ance w ith m ix F IF I w ith clock after 95 hours. In
the session in R ecord A most of the breaks in the long intervals are a b ru p t a n d are
followed by sm ooth curvature, for exam ple, at a an d b. O n the following day, w ith a
slightly higher over-all rate b u t otherwise under the sam e conditions, the longer in
tervals ten d to show a sm ooth decline in rate to a som ew hat m ore interm ediate value
(R ecord B). Instances are a t c, d, e, a n d f
A single instance of a n a b ru p t break a p
pears at g. This difference from one day to the next is unexplained. It suggests th a t
the factors responsible for the sh a rp b rea k are subtle a n d d epend upon some te m
p o rary condition produced by the schedule. Figure 354 shows the same clock on F I
10, w ithout the effect of the m ixed FI 2.
In another experim ent the schedule was changed from F I 10 w ith clock to mix F I
2 FI 10 w ith clock. A good perform ance app ro p riate to the new schedule developed.
This is a p p a re n t after 31 hours of exposure to the schedule in Fig. 768. A n accel
eration occurs a t a app ro p riate to the frequent reinforcem ent a t this reading of the
clock. T he bird then drops to a lower rate a t b appropriate to the reading on the clock
Fig. 7 6 7 .
M ix FI 10 FI 2 + clock a fte r 9 5 hr
609
lying betw een the 2 intervals. It assumes a high rate again a t c app ro p riate to the
clock reading at the end of the longer interval. In general, all the long intervals follow
this p attern. T here appears to be too m uch induction from the two readings asso
ciated w ith reinforcem ent to perm it the bird to stop responding altogether between the
2- an d 10-m inute intervals.
In an earlier stage of the experim ent this bird h ad been on a simple F I 10 w ith clock
(after a history of m ix F IF I w ith counter). T h e resulting clock perform ance aids in
th e in te rp re ta tio n of Fig. 768. Figure 769 illustrates the developm ent of F I 10 w ith
clock. The records shown are the term inal portions of 4 successive days, Records A, B,
C, a n d D, a n d the m iddle portion of the 5th day a t R ecord E. T h e term in al excur
sions on the 5th day are above the usual rate a n d not typical of the d a y s performance.
T h e figure shows the developm ent of a fairly representative perform ance on F I 10 w ith
clock. T he pause a n d term inal rates are m uch greater th an those in Fig. 768, and
th e curvature is m uch sharper.
A nother bird w ith an unusually ragged perform ance on F I 10 w ith clock was tran s
ferred to mix FI 10 F I 2 w ith the sam e clock. Figure 770A shows a sam ple of the
term in a l perform ance on the sim ple FI. R ecord B gives the first p a rt o f the session
on m ix FI 10 FI 2 w ith clock. Scarcely an y responding occurs in the 2-m inute inter
val because of the earlier clock history. R einforcem ents, as at a, b, c, d, an d e, are fol-
610
lowed by further periods at a low rate as a function of the previous clock. T he first
result is th a t responses are reinforced a t low readings on the clock. This effect shows
a t f g, a n d h, w here the rate a t the low reading has increased substantially. Even
th e scallop a t i no longer begins w ith th e prolonged pause characteristic of the clock
performance.
As w ith the bird in Fig. 768, although this bird is capable of a good clock perform
ance on FI 10, it is unable to develop an appropriate perform ance on mix FI 2 FI 10
with clock. Figure 771 shows the term inal perform ance in this experiment. T he bird
tends to respond rapidly, for exam ple, a t I, when the clock reading is th a t which c h a r
acteristically prevails in the shorter interval. T h e ra te drops again to zero w hen
th e clock reading exceeds this value, a n d is therefore ap p ro p riate to no reinforcem ent
(at m). It returns to a higher value of responding w hen the clock again reaches a
reading associated w ith reinforcem ent a t n. However, the term inal rate a t n is not
characteristic of a good clock perform ance; and it will be apparent th at although this
over-all pattern is roughly followed in m any long intervals (g, h, i,j, k, and o), it is by no
m eans inevitable. At a the bird fails to stop responding w hen the clock is m idw ay
betw een readings for the shorter and longer intervals. At b it retards at th at time, but
fails to develop a good term inal ru n app ro p riate to a final reading of the clock. Seg-
611
m ent c resembles Segment d, an d the em ergence of a good term inal rate w ith benefit of
clock a t d is missing. In the interval at e, no high rate develops a t any tim e. A t /
a term inal rate is reached, but it is not m aintained until reinforcement. M uch longer
exposure to the schedule w ith this clock m ight have produced a sharper perform ance
sim ilar to th at a t I, m, an d n, w ith better term inal rates.
Figure 772A shows a perform ance u n d e r m ix F I 5 F I 2 w ith nonfading counter.
L ittle departu re exists here from a single ru nning rate; but the longer intervals contain
breaks w hich ap p ear after the num ber of responses em itted is equal to two or m ore
times the num ber characteristically occurring in the shorter interval. Good examples
are a t a, b, c, a n d d. T he other long intervals show few significant rate changes. T h e
record was taken 23 hours after exposure to the m ix F IF I schedule w ith counter.
M uch later in the experim ent, after a fading counter h ad been used, the bird was
again placed on a mix F I 5 FI 2 w ith counter. T he perform ance is quite stable a n d
sm oothly accelerated. R ecord B gives a sam ple. H ere, the long intervals ch arac
teristically show an inflection at a point corresponding to about twice the num ber of
responses in the shorter intervals.
Figure 773 shows a fairly consistent perform ance on m ix F I 5 F I 1 w ith fading
counter. T he history of the bird includes extended reinforcem ent on mix F I 5 F I 1
w ith nonfading counter. A period of relatively rapid responding follows each rein
forcem ent and, in the longer intervals, a sharp break to essentially a constant inter
m ediate term inal rate then occurs. T he com pensatory effect of the runs after re
inforcem ent m ay be seen by sighting along these curves.
612
T he effect upon the rate is clearest w hen the change is m ade from a fading to a
nonfading counter and vice versa. Figure 774 shows one instance of each. In R ecord
A, 4 excursions w ith a fading counter show the general characteristics of Fig. 773. A t
the arrow the fading element was removed, and the rate im m ediately rises to a m uch
higher value. T he character of the perform ance is not greatly changed, although the
difference betw een the rate ju st after reinforcem ent an d the running rate is som ew hat
reduced. In R ecord B, on the following day, the first 4 excursions show the perform -
Fig. 7 7 4 .
M ix FI 5 FI 1 : A .
Transition from fa d in g to nonfading counter
B. Transition from n o n fading to fa d in g counter
613
ance w ith the simple counter. At the arrow the fading elem ent was introduced. A t
this point the spot was presum ably large, an d a t the current rate, m ust have rem ained
large until reinforcem ent. Thereafter, however, the fading elem ent is im portant, and
a n im m ediate drop to a lower rate for the balance of the figure is clear. W ith this
drop the w ell-m arked character of Fig. 773 w ith the fading counter also returns.
W hen the values on the m ixed schedule were changed to 10 an d 2 rath e r th a n 5
a n d 1 m inutes, respectively, the tran sitio n followed a p redictable course. A p e r
form ance appropriate to the new schedule emerged fairly clearly. Figure 775 shows
the 13th hour. Some of the longer intervals reveal m arked breaks, followed by scal
loping for the balance of the interval, as a t a, c, d, f h, a n d i. Occasionally, a knee
occurs in the early p art of the interval, notably at b, e, a n d g, which m ay be to some
extent due to the earlier shorter intervals. T he ease w ith w hich the bird develops
a break ap p ro p riate to mix FI 2 F I 10, w here no very m ark ed b reak h ad appeared
Fig. 7 7 5 .
M ix FI 10 FI 2 + counter 13 hr
a fte r mix FI 5 FI 1 + counter
after prolonged exposure to FI 1 FI 5, suggests either th a t the size of the interval or

the extent of the change of the spot on the key is im portant.
Larger values in mixed fixed-interval schedules
A pigeon w ith an extended history of FI and V I was exposed to mix FI 20 FI 4. F ig

ure 776 shows a fairly early perform ance. T he long interval at b is convenient for
inspection. It begins with a scallop sim ilar to th at in the short interval, as at a, and the
rate then drops off smoothly to a stable value, suggesting a V I rather than a large F I
perform ance. Tw o other birds on this schedule passed through phases of this sort.
(N ote th a t in this figure the m ovem ent of the pen indicating reinforcem ent was not
always com pleted. T he return m ovem ent was accom plished during perhaps the next
614
10 or 20 responses. T he over-all curvature, of course, is not affected by this defect

of recording.)
W hen a counter was added, the increase in over-all rate obscured the effect of the
m ixed schedule. A sample perform ance appears a t the beginning of Fig. 777. W hen
a fading elem ent is added to the counter, a t the arrow , the rate is currently so high
th a t the counter reading is not affected. Im m ediately after reinforcem ent at a, where
the counter reading is m inim al, the fading elem ent is effective for the first time. T he
result is to reduce responding for some tim e to a value below th a t a t w hich the spot
will grow. T he bird emerges from this low rate in a smooth acceleration, a n d reaches
Fig. 7 7 7 .
M ix FI 2 0 FI 4: transition from counter to fa d in g counter
M IX E D SCH EDULES
615
a term inal rate at b, at which the spot grows at a fairly rapid rate and reaches its m axi
m al size before reinforcem ent. A very sim ilar perform ance follows for the next
interval, w hich is also long. A short interval is then term inated at c. D uring a series
of 3 short intervals, some acceleration occurs; and the long interval which follows b e
gins at a hig h er-th an -u su al rate a n d shows the sam e sm ooth acceleration to the high
term inal ra te a t d. At no point in these long, sm ooth accelerations does the begin
ning m ovem ent of the spot seem to have any effect.
Extinction a fter mix FIFI
D uring a series of experim ents in w hich the prevailing schedule was m ix F I 20 FI 4

with counter, 1 bird was extinguished for a t least 7 hours on 2 occasions w ith the slit
at large. Figures 778 and 779 show the resulting curves. In both figures, some slight
shifting from one rate to another appears, often quite a b ru p t; but the general linear
ch aracter of the behavior through these long experim ental periods is clear, as is also
the tendency for the over-all rate to fall off fairly sm oothly d u ring the period. Both
records show a slight w arm -u p , a quick acceleration to the term inal ra te in Fig. 778,
a n d a gradual increase occupying the 1st excursion of Fig. 779. T he curves suggest ex
tinction after V I. As has already been pointed out, V I is a species of m ixed schedule.
Fig. 7 7 9 .
Second extin ctio n curve a fte r m ix FI 2 0 FI 4 + counter
616
M ixed FR extinction1
Tw o birds w ith a history of erf a n d 20 sessions of F R 5 0 were placed on a sched

ule in w hich, in ran d o m order, a response was reinforced after 50 responses, or no re
sponse was reinforced during an extinction period of 20 m inutes. Later, the extinc
tion period was extended to 1 hour. T h e com ponent schedules were separated by a
10-second T O ; some such period is necessary to distinguish betw een the end of ex
tinction and the beginning of the ratio.
Figures 780, 781, and 782 give the developm ent of an appropriate perform ance for
1 bird. (Some inaccuracy of tim ing m ay be detected in these figures. T he points u n
der consideration should not have been affected, however.) In Fig. 780 the segments
occur in the order shown from top to bottom . T h e 1st reinforcem ent was received
after a ratio a t a. D uring the next 20 m inutes the extinction shown a t b followed.
T hereupon, after a 10-second T O , a second extinction curve followed at c, also for 20
m inutes. After a 10-second T O a t d, a ratio was ru n off a t e. This was of course
not controlled by a current stimulus, so th a t b, c, and e are essentially consecutive stages
of extinction after the single ratio reinforcem ent a t a. Now, however, a reinforce
m ent occurs at f upon com pletion of the ratio. T h e extinction curve shown at g then
follows, after which 2 ratios were reinforced at h and i. These were followed .in tu rn
by a som ew hat m ore sm oothly declining extinction curve a t j , which, in tu rn , was fol
lowed by a ratio at k a n d another extinction curve a t I. T h en , after a 10-second T O ,
a further period of extinction followed at m. This was not preceded by a ratio re in
forcem ent and essentially continues the curve a t I.
Figure 781 shows the later developm ent in the same session and for 3 sessions follow
ing. T h e first segm ent in R ecord A follows im m ediately upon the last segm ent in
Fig. 780. T hereafter, every 4th extinction period is represented. T he experim ental
session lasted 14 hours; C urve 6 shows the last extinction th a t day. T he following
session (R ecord B) shows a m arked change in the extinction curvature. T h e ratio
perform ances at this stage, though om itted from the figure, are suggested by the p rim
ing runs at the beginnings of the extinction curves shown. T he 1st extinction in the
2nd session contains a large n u m b e r of responses, b u t Segm ents 2, 3, 4, a n d 5 show a
rapid drop in the am ount of extinction responding. Some responding returns on the
following day in Record C, b u t again it rapidly drops to a fairly low over-all rate d u r
ing the extinction period. In the 4th session, R ecord D, th e period of extinction was
som ew hat shorter, because of difficulties in tim ing; b u t it shows a drop to essentially
the final perform ance under such a schedule. Segm ent D2 shows an excellent exam
ple of a prim ing ru n followed by a n alm ost com plete absence of responding for the
balance of th e extinction period. Instances of m ultiple prim ing runs develop before
the end of the period w ith some consistency; excellent exam ples are ap p a re n t in Seg1 W e a re indebted to D r. M erle M oskow itz for carrying out this experim ent.
Fig. 7 8 0 .
First session o f mix FR 5 0 e x t 2 0

a fte r FR 5 0
618
Fig. 7 8 1 .
Segments from the first 4 sessions

on m ix FR 5 0 e x t 2 0
m ent 10. T h e perform ance w hen the schedule is F R 5 0 is represented by the seg
m ents in R ecord E, which are, of course, sim ilar to the first responses during the extinc
tion period.
T he fu rth er developm ent of mix F R 5 0 e x t is followed in Fig. 782A. T he figure
shows p a rt of a session about 3 hours long on a single recording. In the next session
(R ecord B) the m ultiple breaks become rare. Several sessions later, w ith the extinc
tion period now lengthened to 30 m inutes, the perform ance shown in R ecord C was re
corded.
A second bird confirms all the features of the preceding experim ent, w ith ab o u t the
sam e speed of developm ent. Figure 783A shows the perform ance of this bird w ith
619
R s/SEC
4$
_y_Lr
Fig. 7 8 2 .
m
T
Later perform ance on mix FR 5 0 ext
an even longer period of extinction. (T he T O is now 6 m inutes rather than 10 sec

onds.) This is the second session in w hich the extinction periods have been 1 hour
long, following an earlier period with 30-m inute extinction. M ultiple breaks are clear.
S quare changes in rate are obvious, an d the prim ing runs a t the beginning of the
session have, in general, the same m agnitude. Later, in a somewhat shorter extinction
period (50 m inutes), the m ultiple runs tended to disappear. A stage in w hich this
phase is well-developed appears for 4 consecutive extinction periods from a later session
in R ecord B. This bird showed a tem porary exceptional phase in the earlier develop
m ent of the perform ance. Figure 782D represents a period in the exposure to the
schedule com parable with th at in Fig. 782A. T he running rate is not quite so high
an d shows some slight curvature in the prim ing runs, as a t a a n d b, becom ing m uch
m ore m arked on the following day, shown in R ecord E. T h e ratio rate has fallen
ab ru p tly , as at c, a n d m arked c u rv atu re appears. T his is best seen in the breaks oc
curring later during extinction, p articularly a t d. T he bird recovered from this c h a r
acteristic w ithin a few days, and eventually showed the perform ance seen in Record F,
which is for 2 sessions preceding Fig. 783.
620
Fig. 7 8 3 .
M ix FR 5 0 ext 6 0
MIXED FIXED-INTERVAL FIXED-RATIO SCHEDULES

M ix F I1 0 F R 4 0 1
A pigeon w hich h ad been reinforced on m ult F R 40 FI 10 w ith green and white keycolors, respectively, for 63 hours had developed the perform ance shown in Fig. 784A.
T he ratios are all ru n off at high rates. But the intervals show considerable variation
in curv atu re a n d in the num ber of responses; an d an interval following an interval
usually begins a t the preceding term inal rate w ith no scalloping. At the beginning of
the following session, the key-light was changed to orange and the schedule was then
mix F R 40 FI 10. T he new color has some disturbing effect, as the first p a rt of Record
B shows, b u t a reinforcem ent is quickly reached a t a, a n d a second is received at b,
1 W e are indebted to D r. C lare M arshall for carry in g out this experim ent.
621
also on the ratio com ponent. T h e key-light rem ains unchanged, of course, b u t the
schedule now shifts to FI 10; and during this interval nearly 1000 responses are em itted.
T h e ra te varies som ew hat. A second interval follows, a n d a reduction to a lower
rate ensues. T he next reinforcem ent is on the ratio schedule, which causes a return to
a fairly sustained perform ance d uring the 3rd interval ending w ith a reinforcem ent
a t c. By this tim e reinforcem ents have occurred on the orange key on essentially a
V I or V R basis, a n d a sustained perform ance com pletes th e experim ental session.
R ecord C follows R ecord B in the sam e session after 3 hours, the intervening perform
ance having been om itted from the figure. Some tendency to pause or to reduce the
ra te after reinforcem ent begins to develop. This is ap parent, for exam ple, a t e, and is
accu m u lated into an over-all lower rate in the region of d an d f
T h e first w ell-m arked effect of mix F R F I appears only after 3 sessions on the sched
ule. T he whole 4th session is shown in Fig. 785, which begins 13 hours after the in
tro d u ctio n of the m ix F R F I. T he early p a rt of the session shows essentially a linear
perform ance. But beginning at a reinforcem ent tends to be followed by an increase
in rate, an d this rate tends to fall off after som ething of the order of two or three h u n
d red responses have been em itted. T h e decline in rate takes various forms, as at b,
c, a n d d, a n d it becomes progressively sh a rp e r la te r in th e session, as a t e, f a n d g.
C urvature begins to occur after the prim ing run.
In a session 26 hours after the introduction of the m ixed schedule, the prim ing after
reinforcem ent is practically inevitable; and at this stage the break into the curve a p
p ro p riate to the interval m em ber of the schedule is either smoothly curved or fairly
sharp. Figure 786 shows the perform ance. P rim ing runs curve off smoothly, as a t a,
or b, or fairly sharply, as a t c or d.
Eventually, a sharp break always occurs, an d a t a count approxim ately twice th at of
th e ratio. Figure 787 gives a daily session, w hich begins after 95 hours on m ix F IF R
(the ratio having been reduced to 27). H ere, alm ost all breaks are sharp, a n d the
perform ance during the balance of the interval tends to show an initial long pause, fol
lowed by a smooth acceleration.
623
A second bird only partially confirms the records ju st described. T he earlier m u l

tiple perform ance was somewhat sharper, leading to a higher term inal rate than th a t
in Fig. 784. T h e general c h a ra c te r of Fig. 784C was confirm ed in the 1st session.
T h e perform ance in Fig. 786 was also d u p licated in its essentials after about the same
num ber of hours on the schedule. Shortly thereafter, however, the second bird b e
g an to show rep eated breaks in the interval segm ent. T his anom alous perform ance
can be explained by the fact th a t the b ird was occasionally pausing after reinforce
m ent. Pausing after the first break in the interval segment can therefore trip off a sec
ond ratio run. Exam ples of pauses after reinforcem ent a p p e a r in Fig. 788 at a, c,
d , f a n d elsewhere. W hen the bird prim es itself into a pause, as, for exam ple, a t b,
e, a n d g, it m ay emerge from this pause w ith a ratio perform ance rath er th an an in
terval perform ance. T h e actual prim ing effect shows dam ping, ju st as in some of the
m ix F IF R experim ents described elsewhere. Good examples of dam ping are a t e an d
g. T he tendency to pause an d to respond w ith another ratio perform ance declines as
th e interval progresses. A lthough some segments show a prim ing ru n leading into a
fairly smooth sustained rate until reinforcement (at h ), the occasional instances in which
pauses are followed by reinforcem ent after roughly 40 responses m ake it impossible
for th e bird to develop the pause an d sm ooth acceleration characteristic of the FI seg
m ents in Fig. 786.
Figure 789 shows a segment of the following session. H ere, reinforcements are oc
casionally followed by pauses, as at a and b, an d these pauses are invariably followed by
624
ratio runs containing a num ber som ew hat greater th an 40. T h e ru n m ay be followed
by an o th er pause followed by an o th er run, as a t c, or it m ay prim e directly into an
in term ediate constant rate, as at d.. Occasionally, the ru n after reinforcem ent prim es
directly into an interm ediate rate, as a t e. T h e figure shows a variety of co m b in a
tions of these effects.
Figure 790 illustrates the best m ix F IF R perform ance for this bird. It contains
several exam ples of pauses im m ediately after reinforcem ent, after w hich a ratio of 40
responses is reinforced, as at a, e, i,j, an d n. It also has examples in which such pauses
occurring im m ediately after reinforcem ent are followed by an interval reinforcement,
as at h, m, a n d o. P articularly tow ard the end of the session, instances still occur, as in
Fig. 789, in w hich a ru n after reinforcem ent is followed by a pause which then prim es
625
into an interm ediate term inal rate, as at p a n d q. However, the figure shows several
instances in which there is a prim ing run of approxim ately twice the ratio of 40 followed
by a m arked pause followed by a ru n for the balance of an interval. T here is often
fairly good positive curvature, suggesting the m ix F IF R perform ance of Fig. 787, as a t
b, c, d , f g, k, an d I. T o this extent the b ird is responding in a fashion which we m ight
call appropriate to mix FIFR . (This schedule had an equal num ber of interval and
ratio reinforcements, an d it m ight have been possible to bring the perform ance around
by increasing the n u m b er of ratios. This increase would presum ably have reduced
the pausing after reinforcem ent and w ould have reduced the variation in the actual
contingencies.)
M ix FIFR in rats
T h re e rats studied on m ult F R F I, as alread y described, were transferred to mix

F R F I at the same values in the presence of the stim ulus previously correlated w ith the
ratio. In the earlier m ultiple-schedule experim ent, the 2 stim uli had been alternated
rath er th an presented at random ; this was also done in the m ixed case. Figures 791
a n d 792 contain 2 examples of first sessions on m ix F R 2 0 F I5 . T he result shows
clearly th a t the interval perform ance on the preceding m ultiple schedule was due as
m uch to the prim ing stim ulus from the preceding ratio perform ance as to the external
stim ulus control, because all 3 rats very quickly developed an essentially m ultiple
perform ance on the mixed schedule. In other words, they quickly dropped any
prim ing run at the beginning of the interval, and showed good interval scalloping in a l
tern atio n w ith good ratio segments. N o prim ing ru n is needed, since the ratio p e r
form ance which invariably precedes serves as the necessary stimulus. (The third ra t
developed the perform ance a little m ore slowly, b u t reached the sam e final patterns.)
W e easily checked this interpretation by changing the schedule to a random presen-
626
tatio n of ratios a n d intervals. All 3 rats were u n ab le to m ain ta in a m ultiple-type

perform ance on the random m ixed schedule. Figure 793 shows an exam ple for the
rat in Fig. 791. T he preceding perform ance on an alternating m ixed schedule was es
sentially like th a t shown in Fig. 791. H ere, however, we see repeated runs of approx
im ately 1^ to 2 tim es the num ber of responses in the ratio occurring thoughout the
interval, w ith an occasional longer ru n ap p ro p riate to the end of an interval. Before
627
the end of the session, a fair approxim ation to a m ix F R F I develops. This is inter
spersed w ith instances in which the ratio begins w ith a long pause at e and h, and hence
w here an interval is occasionally run off w ith a fair perform ance w ithout prim ing, as
at a. T he com m oner case, however, is th a t in which a prim ed ru n leads to a com ple
tion of the interval segm ent in a m anner typical of a m ixed schedule, as a t b, c, d, f ,
and g. (This schedule was in force for only a short tim e; the reinforcem ent was inade
q u a te to produce a norm al ratio perform ance. Since pauses frequently occur be
fore the ratio run, a percentage reinforcem ent of ratio-sized runs follows. This m ain
tains the strained condition originating under the inadequate reinforcement.)
U pon returning to a m ultiple schedule, however, the rat is able to develop an d hold
a reasonably good perform ance. Figure 629 has illustrated an exam ple. In order
to get this type of perform ance, however, it was necessary to am plify the controlling
stim uli by adding to the flashing or steady light a buzzing sound or the absence of such
a sound. U n d er this com bined stim ulus, a fairly good m ultiple perform ance devel
oped; b u t it is m arked by a n occasional prim in g ru n a n d a ra th e r rough g rain in the
interval scallop. T he term inal rate observed in the interval is not m arkedly higher
th an th at of the final running rate in the ratio.
The effect o f drugs on mix FIFR1
The effect o f sodium pentobarbital on mix FRFI. A bird h ad developed a stable perform
ance on m ix F R 4 0 F I 10. It was injected intram uscularly w ith 2 m illigram s of so
dium pentobarbital at the arrow in Fig. 794. T h e im m ediate effect was a disturbance
of the 1st interval perform ance a n d the elim ination of prim ing into an interval scallop
1 T h is experim ent was conducted in collaboration w ith D rs. P. B. D ews a n d C lare M arshall.
d eb ted to D r. M arsh all for carrying out the experim ent.
W e are in
628
Fig. 7 9 4 .
The e ffect o f 2 mg o f p e n to b a rb ita l on m ix FIFR
in the 2nd a n d 3rd intervals, a t a. Some evidence exists of a disturbance of the in

terval curvature a t b, c, and d, but the perform ance is probably norm al at e. T he h ori
zontal portion of th e interval curve to w a rd the end of the session seems to be p ro
longed, suggesting a later depressive effect.
In a sim ilar experim ent a few sessions later, 4 m illigram s of sodium pentobarbital
were injected intram uscularly at the arrow in Fig. 795. T he effect appears almost im
m ediately as a very rough grain a n d a tendency to d e p a rt from the usual term inal
rate. R atio perform ances were also disturbed (at a and b). Little evidence is present
of any prim ing into the scallop appropriate to the interval for a num ber of reinforce
m ents; the first evidence appears at c, while at o' a fairly norm al case exists. T h e bird
recovers tow ard the end of the experim ental period, giving good perform ances tow ard
the latter h alf of the session. H ere, the excitatory effect of the drug can be detected
for about 2 hours. It takes the form of a disturbance in grain and the rem oval of the
pausing ap p ro p riate to the intervai schedule, w ith some disturbance even im m edi
ately after reinforcement, where the ratio is usually run off rapidly. Again, a later pos
sible depressive effect appears in the long pauses a t e and f .
629
In th e session following Fig. 787 the bird was

injected w ith 10 m illigram s of synhexyl (a m arijuana-like drug) a t the arrow in Fig.
796. T he lack of a prim ing ru n im m ediately after the injection can be attributed to
the handling of the anim al. N o reliable effect appears until the intervals a t a, b, c, an d
d. T he effect of this am ount of the drug is not great. T hree days later, however, the
sam e bird was injected w ith 20 m illigram s o f synhexyl, as shown in Fig. 797. T h e
interval im m ediately following injection is disturbed; b u t this disturbance m ay be due
to handling. A m uch m ore profound effect occurs later; the disturbance of the grain
of the record begins at a b ut becomes m arked a t b and c. This p attern resembles the
effect of sodium pentobarbital evident in Fig. 795 a t a a n d b. A later m arked depres
sant effect o f synhexyl appears in the intervals d, e, g, and h, in spite of the fairly n o r
m al ratio runs at f . Some disturbance in grain occurs throughout the balance of the
session.
The effect o f synhexyl on mix F R F I.
630
MULTIPLE FR PRIMED FI
First bird
T he transition from m ultiple to m ixed schedules was first studied with the use of a
prim ing stimulus. O n m ult F R 120 FI 10, for example, the key-color was green during
the fixed ratio an d flashing green (1 flash per second) for 2 m inutes at the beginning
of the interval. D uring the rem ainder of the interval the light was steady green. U n
der these circumstances no actual stim ulus difference exists between the ratio and the
last 8 m inutes of the interval. B ut the flashing light d u rin g the first 2 m inutes of the
interval prim es the organism into an interval scallop, and this stim ulus m ay control
the bird for the balance of the interval.
A b ird h a d been on m ult F R 120 F I 10 in w hich the stim ulus for the ratio was a
green key, a n d the stim ulus for a n interval was a red key. In the present experim ent
the ratio stim ulus rem ained green, b u t the interval, as already noted, was also green
w ith a flashing prim e for the first 2 m inutes. Figure 798 shows the first session in
w hich this schedule was first presented after the earlier m ult F R F I. T he first 3 ratios
are ru n off in the form er ratio color. (Some slight delay a t the start of the experim ent
was not unusual.) T he first ap p earan ce of the flashing light follows reinforcem ent
631
a t a. It suppresses the behavior; but the bird recovers fairly quickly an d assumes the
ratio rate, w hich is m ain tain ed except for slight pausing u ntil the lst-interval re in
forcement at b. Except for a short period of flashing light, following a, all of this occurs
on the green key. T he following reinforcem ent occurs after F R 125. T he segment
shows some disturbance resulting from the previous long ru n at the ratio rate and in the
ratio color. Responding begins before the light stops flashing; and during the in ter
val which follows m ore th an 1000 responses occur at the ratio rate, but with some rough
grain. Reinforcem ent is then received at c. M ore m arked breaking begins to occur
at d, e , f and g, and the interval which follows the reinforcement a t h shows a very rough
grain. T he first developm ent under the prim ing schedule appears to be the sup
pression of behavior d u ring the flashing light, arising from the fact th a t no reinforce
m ent occurs while the light is flashing or for some tim e thereafter. T he reinforcements
a t i,j, and k are followed by intervals and hence by flashing lights for a 2-m inute p e
riod. T he bird slows down during the flashing light and returns im m ediately to a high
rate w hen the flashing term inates. Figure 799A shows the 3rd hour of the 1st ses
sion. R esponding in the flashing light is fu rth er reduced, instances occurring here a t
632
b, d , f an d i.
These are followed by a high rate rem iniscent of the earlier m ultiple p e r
form ance, a t c, e, g, an d j. R einforcem ent never occurs a t these high rates, because
they tend to ap p e ar ju st after the cessation of the flashing light in the interval. Note,
however, the appearance of a second over-all rate, notably a t a, h, and k, at an in te r
m ediate value resulting from the unusual schedule of reinforcem ent now prevailing on
the steady green light.
F urther developments appear by the 2nd hour of the 3rd session, 9 hours after the b e
ginning of th e schedule, as R ecord B shows. H ere, the rate is reduced d u rin g the
flashing light, as at m\ and these low rates generally are followed by the highest rates o b
served in the session, as a t n, o, an d p . M ost of the tim e, however, the bird is ru n
ning at an interm ediate rate (for exam ple, a t q) resulting from the residual schedule on
the steady green light.
These general features persist for some tim e. T h en , an o th er change occurs in th a t

a steady green light following reinforcem ent begins to control the higher rate a p
p ropriate to the inevitable fixed ratio existing u n d e r these conditions. In R ecord C,
which is for a later p a rt of the 3rd session a t the 12th h our on the schedule, the p e r
form ance under the steady green light following reinforcem ent is strengtheneda t r, s,
and X. Lower rates nevertheless occasionally appear, as a t v an d w. Scattered re
sponses sometimes ap p ear during the flashing light, as, for exam ple, at t andjy; b u t the
low rate here is usually followed by a high rate rem iniscent of the earlier ratio. In
th e figure exceptional instances show the em ergence of this high rate later in the in ter
val after the interm ediate rate has already been assumed (as a t u).
Figure 800A shows the beginning of the 2nd hour in the following session. T h e
steady green key ju st after reinforcem ent now controls a high constant rate. A low
rate appears d u ring the flashing green light w hich prim es an F I scallop a t b. T h e
633
fixed-ratio ra te breaks th ro u g h following th e term in atio n of the flashing light a t a.

A high rate of responding is now being reinforced in the steady green light because
o f the restoration of the ratio perform ance. This factor leads to frequent breakingthrough of the ratio rate later in the balance of the interval. B reaking-through had not
appeared earlier because high rates were not being reinforced. R ecord B gives ex
am ples of this stage from the 17th hour in the 5th session. Reinforcem ents of F R 125
on a steady green light at high rates occur a t f h, i, and j. T he flashing green light
produces a lowered rate a t c, e, g, an d k. A som ew hat lesser reduction in rate appears
d u rin g the flashing light a t I an d m. These suppressions are generally followed by
periods of very rap id responding, w ith exceptions a t k an d n. B ut rap id runs drop to
the interm ediate rate shown more characteristically in Record A. After the rate has
fallen to this in term ed iate value, how ever, periods of rap id ru n n in g m ay now break
through, as at d. This is the by-product of the im proved contingencies of reinforce-
Fig. 8 0 0 .
Segments fro m the fo u rth , fifth , and sixth sessions on m ult FR 120 prim ed FI 10
m ent on the steady green light under the ratio schedule, which is now being prim ed in
its tu rn by reinforcem ent not followed by a flashing light.
A further developm ent of all these characteristics appears in R ecord C, where a rein
forcem ent followed by a steady green light uniform ly produces a typical ratio p e r
form ance, as at r, v, w, z, a n d elsewhere. T h e flashing light after reinforcem ent now
produces some running-through and a rath er ragged low rate rather than a complete
suppression. Examples are at o, s, t, u, and elsewhere. These periods of slow respond
ing are now not inevitably followed by ratio runs. In fact, such runs are becoming
exceptional, as at p an d q. W e have here the beginning of a perform ance in the in ter
val appropriate to the interval schedule. A good exam ple is the interval between x
a n d y , where the preceding ratio was ru n off a t a high rate a t w.
A perform ance on the m ult F R 125 prim ed FI 10 appears in Fig. 801, showing the
7th, 8th, and 9th sessions in Records A, B, a n d C, respectively. R ecord A contains
appropriate perform ances in both interval and ratio. Slips occasionally occur, as
a t a, w here in an interval segm ent the bird reverts back to the interval rate after em it
ting approxim ately 50 responses at the ratio rate; a t b, where the term ination of the
634
flashing light produces a small burst o f responses, after w hich the bird assumes the
p ro p er interval perform ance; a n d at c, w here a b urst of about 150 responses
is em itted a t th e ratio rate upon term in a tio n of the flashing light, after w hich the
bird reverts back to the interval rate. R ecord B shows 2 examples of slips. At e a
whole ratio is ru n off at the interval rate; an d at f an interval begins norm ally, but ac
celerates into the fixed-ratio rate for approxim ately 50 responses before returning to
the interval rate for the rem ainder of the interval. M ore m arked deviations occur in
Record C, w here 3 ratios, at g, h, an d i, occur at the fixed-interval rate. In general,
however, the m ultiple control a t this stage of the schedule is good. N ote th a t the in
tervals have c u rv atu re w hether the preceding schedule was fixed-interval or fixedratio. This is unlike the perform ance u n d e r a straight m ultiple schedule.
A t this stage of the experim ent, T O probes were used to exam ine the process, an d
further developm ent was to some extent disturbed. L ater, however, this bird showed
the m uch m ore advanced state of the prim ed m ult F R F I evident in Fig. 802. This
is the 25th session on the prim ing schedule. (An anom alous reinforcem ent hatch a p
pears in the record at b ; this m ay have been m erely an erroneous m arking operation or
an actual reinforcem ent.) A lthough some exceptions occur, the typical perform ance
at this stage m ay be described as follows. W hen a reinforcem ent is followed by the
steady green light, a stan d ard ratio perform ance is ru n off, as a t f , g, an d h. W hen a
Fig. 8 0 2 .
T w e n ty-fifth session on mult FR prim ed FI
635
reinforcem ent is first followed by 2 m inutes of flashing light a n d then by the steady
green light, however, a curve app ro p riate to a n interval perform ance appears, as a t i,
j , a n d k. T he exceptions are exemplified at a, c, d, and e in the early p art of the session.
H ere, the acceleration developing d u ring the interval sets off too high a rate; since
the key-color is appropriate to FR , this rate continues for approxim ately the num ber of
responses in the ratio, only to prim e the bird into the linear perform ance prevailing
u n d er the earlier, ra th e r anom alous schedule u n d er the steady green light. It should
be noted th at mistakes on the steady green light take the form of running at a rate
a p p ro p riate to a ratio perform ance on an interval schedule ra th e r th an a t an in ter
val rate during a ratio. It should be noted, too, th a t the runs which break through
in the interval schedules are usually as long as the ratio or longer. A n exception a p
pears at e.
Second bird
T h e 1st day on w hich a second bird changed from a m ult F R F I w ith green and
red lights to m ult F R prim ed FI with a green light flashing for 2 m inutes as a prim e was
lost in recording; b u t the 2nd day revealed a rather different perform ance, as Fig. 803A
shows. T he over-all rate is higher, and the bird was not disturbed by the flashing light
to the sam e extent. At a, for exam ple, some responding occurs d u ring the flashing
period a t alm ost exactly the ra te of flashing. In other experim ents a degree of stim
ulus control has been observed under which responses can be evoked or stopped w ithin
a fraction of a second by m an ip u latio n of the light. H ere, the bird was evidently
striking the green light each tim e it a p p e are d in the 1-per-second flashing. As soon
as the steady green is present w ithout flashing, the ratio rate is m ain ta in e d for m ore
th a n 1000 responses a t b, an d the following ratio begins w ith a slight pause, at c. T he
636
following interval a t d shows responding during the flashing light and an almost im
m ediate re tu rn to the ratio rate for the b alance of the interval. T h e effect upon the
following ratio was to lengthen the pause at e. T he following interval a t f shows re
sponding during the flashing light and again a return to the ratio rate; but the following
interval shows a prolonged pause, possibly because of the straining due to the m ain
tenance of the ratio rate during the entire interval. T his pause is followed, however,
by a retu rn to the ratio rate near the end of the interval a t g. T he following in te r
val again contains m ore th a n 1000 responses. T he pauses at the beginnings of in te r
vals now usually exceed those in ratio s for exam ple, a t h, i, a n d j. T he flashing
light begins to function not only for its d u ratio n b u t for some tim e following. W hen
responding resumes, however, the rate tends to be at the ratio value.
O n the following day the perform ance is repeated, as Fig. 803B shows, except th a t
the ratios now show some strain, as a t k a n d I, while the intervals start w ith pauses
or low rates. In one case (m) the initial pause is followed by a high rate. W here the
perform ance in Fig. 802 showed a ratio perform ance b reak in g into the interval p e r
form ance, the reverse is true here. T h e term inal interval perform ance on steady
green carries into the ratio.
Figure 804 shows a late perform ance for this second bird u n d er a prim ing sched-
Fig. 8 0 5 .
637
The e ffe c t o f reducing the size o f the FR com ponent on m ult FR prim ed FI
ule before probes were introduced. T he ratio perform ance has suffered badly. R a
tios are frequently ru n off a t the term in al interval rate. T h e intervals tend to be
m arked by prolonged pauses but, in general, to end at the same rate as the ratios.
Figure 808 shows a still later perform ance for this bird. A t this tim e, T O probes
were being used, a n d the perform ance m ay show the effect. H ere, however, a m uch
better curvature has developed in the interval, although the ratios are still being ru n
off at m uch below a norm al ratio rate, w ith rough grain and curvature.
In an effort to im prove the ratio perform ance for this bird, the ratio was reduced
from 125 to 50. Figure 805 shows th e 1st day on the new schedule. A t the begin
ning of this session, difficulty still exists in starting the ratio; but before the end of the
session, the ratio rate is accelerating to a value higher th an the term inal rate in the in
terval. O n the following day the 2 rates clearly break apart; an d by the 3rd session
w ith the shorter ratio, a reasonably satisfactory perform ance under m ult F R prim ed F I
appeared, as evident in Fig. 806. H ere, the steady green directly after reinforce
m ent controls a high rate appropriate to a ratio. Flashing green after reinforcement
sets up a pause which prim es the bird into a prolonged period of very slow respond
ing, or no responding a t all, which leads to a fairly standard interval segment. Excep
638
tions ap p e ar a t a, b, a n d c, w here the ratio rate breaks th rough in the interval; an d

at d, w here th e pause following reinforcem ent leads im m ediately to the ratio rate, a t
which a num ber of responses equal to approxim ately 3 times the ratio is em itted b e
fore the rate drops to the value characteristic of the term inal-interval perform ance.
This figure confirms Fig. 802 in the essentials of the prim ed F R F I perform ance and in
dicates th a t the earlier failure to conform was due to the selection of too high an FR .
Effect o f a TO probe on m ultiple prim ed FI
T he first b ird in the experim ent ju st described showed a t one stage on m ult F R
125 prim ed F I 10 (between the stages represented by Fig. 801 and 802) a perform ance
in which ratio segments contained either a high rate, a smooth interm ediate rate, or,
occasionally, a high rate dropping to a lower value during the ratio. T he interval p e r
form ances a t this tim e generally show prolonged periods of low rate after reinforce
m ent, w ith fairly smooth acceleration to an interm ediate linear term inal rate. Probes
consisting of 1-m inute T O s were introduced approxim ately m idw ay in an occasional
interval. T he first two showed some disturbance; shortly after the probe term inated,
responding accelerated to the ratio rate for a brief period. Thereafter, however, the
probe h a d rem arkably little effect. T h e d u ratio n of the T O was later increased to 5
m inutes. T hree instances at 5 m inutes ap p ear in Fig. 807, w here, a t the arrows, a
T O 5 was introduced m idway in the interval curvature or shortly after acceleration h ad
begun. In each case the interval was com pleted in essentially a norm al fashion in
spite of this period of TO .
As we h av e ju s t seen, th e second b ird h a d d ev elo p ed a p e rfo rm a n c e u n d e r m u lt
M IX E D SCH EDULES
639
F R prim ed FI which showed difficulty in the ratio. Nevertheless, w hen 5-m inute T O
probes were introduced into the fairly w ell-developed interval scallop, as shown a t
the arrow s in Fig. 808, little effect upon the over-all curvature was noticeable. A t a
th e rate is som ew hat reduced, but it quickly returns to a value characteristic of the
interval. At b, c, a n d d little or no change is evident in over-all rate. W hatever is re
sponsible for the interval curvature a t this point survives a 5-m inute period of T O .
Effect o f reducing prim ing stimuli on mult FR primed FI
First bird. W e co n tinued the experim ent on prim ing the interval in a m ultiple
schedule by reducing the duration of the flashing green light, which served as the p rim
ing stimulus. Shortening the flashing green light from 2 m inutes to 1 m inute had no
effect on the performance. After several days, the duration of flashing was dropped to
30 seconds. Figure 809 shows the effect. T he record begins w ith a well-developed
prim ed m ultiple perform ance w ith sm ooth curvature in each interval (except for an
anom alous break a t a). Beginning at the arrow , the prim ing stim ulus lasted only 30
seconds. T he 1st interval a t b is run off normally. T he 2nd, however, shows break-
640
th ro u g h of th e ratio rate (at d). N ote th a t th e bird occasionally ra n the ratio at the
term in al interval rate (as a t c). Som e disturbance of the p rim in g stim ulus is also
evident at g. T h e m ain result appears to be in the ratio perform ance. T h e ratio at
e, w hich is ru n off in the steady green light, im m ediately following the reinforcem ent,
shows very rough an d irregular grain. T h e ratio at f is run off with curvature a p p ro
priate to the term inal parts of an interval. Such curvature m ay be due to the fact
th a t the rate did not accelerate to the norm al term inal rate in the preceding interval.
T he prim ing stim ulus was later reduced to 15 seconds, an d the schedule was con
tinued for 3 sessions. Figure 810 shows a final perform ance with a 15-second prim e in
the interval. Except for the slight burst of responses at b an d the low rate at a, this
d u ratio n of the prim in g stim ulus is effective in p roducing a m ultiple perform ance.
These segments were taken from the m iddle of the session, however. At the beginning
o f the session the bird showed irregularities. Sim ilar irregularities early in the ses
sion appeared the next day.
At the beginning of Fig. 811 the prim e is still 15 seconds long, and it fails to bring
th e bird into the interval perform ance a t a an d b. T he b ird prim es itself by ru n n in g
off from 1 to 2 ratios a t the ratio rate. A t c, however, the performance typical of the
later p a rt of Fig. 810 has developed.
At the arrow in Fig. 811 the prim ing stimulus was reduced to 5 seconds. T he im
m ediate effect was a failure to produce the pausing appropriate to the beginning of the
interval, w hich runs off at alm ost the term inal rate throughout the interval, as
at d. A nother interval (at e) is sm oothly executed, an d a ratio is then ru n off
641
norm ally a t f . T he prim ing stim ulus fails to w ork in the interval w hich follows a t g,
an d the bird prim es itself into an interval scallop by running a t the ratio rate for about
300 responses. T he 5-second prim e also fails to operate effectively a t h, i,j, and k.
W hen the prim ing stimulus was rem oved altogether, the schedule becam e, of course,
a m ix F IF R . T his occurred 2 sessions after the ineffective 5-second prim e of Fig.
811, a n d is shown in Fig. 812. H ere, all intervals begin w ith self-prim ing runs, after
which the bird decelerates fairly quickly b u t smoothly to a pause followed by a fairly
typical interval scallop. M eanw hile, the ratios are for the most p a rt ru n off at stan d
a rd rates, although exceptions appear at a and b. (For a further development of this
Fig. 8 1 2 .
Transition from mult FR prim ed FI to mix FIFR
M IX E D SCH EDULES
643
type of schedule, see the section on mix F IFR .) Figure 813 describes the final perform
ance for this bird 6 sessions after Fig. 812. T h e ru n after reinforcem ent before the
break to a low rate has becom e som ew hat shorter, an d the break is occasionally m ore
a b ru p t, as a t a.
Second bird. W hen the second bird was tested on a shorter flashing light as a prim e,
trouble appeared at the 30-second duration. T he bird showed a tendency to resume
the ratio rate at the term ination of the flashing light, or shortly thereafter, and to count
out approxim ately one ratio before prim ing itself into the interval scallop. O cca
sionally, a stan d ard interval perform ance was observed; and, occasionally, the short
ru n after the pause led into the term inal-interval rate for the balance of the segment.
Some of these effects a p p e ar in Fig. 814. L ater, the bird showed difficulty w ith a
prim e of 15 seconds a n d alm ost com plete disruption w ith one of 5 seconds, w hen the
perform ance was essentially the same as on a m ixed schedule.
INTERPOLATED SCHEDULES
A relatively short period on 1 schedule m ay be inserted into a background sched

ule which occupies the m ain p art of the session . W hen no stimuli exist appropriate to
the 2 schedules, we shall speak of the short schedule as in terp o lated . It has the
effect of a com plex probe. In a case to be described the schedule during most of the
session was FI 15, b ut for a certain n u m b er of successive reinforcem ents it was FI 1.
How often a brief period on another schedule is interpolated per session depends upon
the length of a session and the tim e needed to observe the consequences. Points to be
noted include the changeover to the interpolated schedule, the developm ent of a p e r
formance on the interpolated schedule, and the changeover to the background sched
ule. O f course, m any com binations are possible. W e have studied the interpolation
of a short FI in a long FI, a small F R in a long FI, and a small F R in V I.
FI inter FI
Tw o birds previously used in an experim ent on m atching colors were placed on

FI 15 for 30 hours. T he daily sessions were 8 hours long for one, a n d 6 hours long for
the other. A brief period of FI 1 was th en in terp o lated in the m iddle of each ses
sion. In general, this period contained 16 reinforcements, although the num ber varied
somewhat.
For one bird the results of the early interpolations were lost through poor recording
and ap paratus defects. By the 15th interpolation, however, a w ell-m arked p a tte rn
h a d em erged, w hich appears in Fig. 815A, show ing the im p o rta n t m iddle section of
the days run. T he bird is not pausing substantially after reinforcem ent on FI 15 in
spite of rath e r long exposure to th at schedule.
T he interpolated schedule as a whole seems to have some of the effect of a VI. T h e
early reinforcem ents on FI 1, the interpolation beginning a t a, also show only slight
pausing, but a m arked scallop develops during the interpolation. U pon the return to
644
Fig. 8 1 5 .
FI 15 inter FI 1 a fte r 14
interpolations
the 15-m inute interval, the scallop appropriate to the short interval begins, at b. It
runs past the usual point of reinforcem ent a n d th en breaks sharply into a com pensa
tory pause, a t c. T he com pensatory n a tu re of this pause is suggested by the fact th a t
it is not characteristic of the longer-interval performance. T he bird then returns to an
essentially linear FI 15 performance. Some variation in the over-all rate is evident,
however. A m arked deviation appears in the record at d; where after a progressive d e
cline in rate, the curve accelerates to the term inal slope in the short interval. R ec
ord B shows roughly the same perform ance for the following session. Scallops develop
on the interpolated schedule, and a ru n appropriate to F I 1 appears at the beginning
of the first subsequent F I 15 at e. T his ru n is followed by the same break and, in this
case, w hat appears to be a late com pensatory increase at f . T he perform ance is re
peated in the following session (R ecord C), b u t R ecord D does not show any m arked
inflection upon entering the subsequent 15-m inute interval at g. T he earlier per
form ance reappears in the following session, however, as R ecord E shows. In this case
645
CO
zo
CL
cn
UJ
ir
o
o10
Fig. 8 1 6 .
FI 15 inter FI 1: entire session a fte r 2 0 interpolations
a period of rath er slow responding follows before the retu rn to the norm al FI 15 rate
a t h. Figure 816 shows the entire following session. T he early breaks at a a n d b are
exceptional for this bird; and, in general, no m arked, consistent pausing appears after
reinforcem ent. Scallops developed during the interpolated period; these are followed
by a substantial ru n a t c, which is followed again by a break to a low rate before the
re tu rn to the stan d ard FI 15 rate at d. T h e record clearly shows th a t pausing occurs
on F I 1 b u t not on F I 15, a n d th a t the pause on F I 1 m ust be reinstated each day
through reconditioning in the early p a rt of the interpolated schedule. Figure 816 oc
cu rred beween Records E an d F of Fig. 815. An over-all disturbance is evident as
a result of the interpolation. L ater interpolations are shown in Fig. 815 F, G, H ,
an d I.
T h e latter effect was m uch m ore m arked for the other pigeon. This bird had a l
read y developed some tendency to pause after reinforcem ent, a n d the first effect of the
in terp o lated FI 1 was to sum m ate this tendency to produce a decline in rate. Fig
ure 817 A shows the first interpolation. T h e short intervals beginning a t a to c an d d
red u ced th e rate very nearly to zero, from w hich a m uch higher ra te recovers, a l
though w ithout consistent scalloping. (Some negative acceleration actually appears,
particularly in the 11th and 12th intervals.) W hen the schedule returns to 15 m in
utes, the norm al perform ance is quickly reinstated, at e. T he tendency to respond at a
higher rate after reinforcem ent, noted p articu larly at f , disappears in the later p a rt
of the session not shown, w hen there is a slight tendency to pause after reinforcem ent.
T h e 2nd interpolation, in th e following session, has the sam e effect. It leads to a
pause of about 5 m inutes, at g. This pause is followed again by a recovery on the
short-interval schedule a n d a retu rn to a stable rate on FI 15 thereafter. In subse
qu en t sessions, pauses a n d scallops after reinforcem ent on FI 15 develop further; but a
perform ance appropriate to FI 1 also emerges, although scalloping develops late. T he
7th interpolation is illustrated in Fig. 817C; a n d the 11th, a t R ecord D. By this tim e
300
RESPONSES
M IX E D SCH EDULES
647
th ere is a considerable tendency to pause on the 15-m inute schedule; b u t such pauses
a p p e a r only occasionally on F I 1, notably a t h in this record.
W here an interp o lated FI 1 against F I 15 h a d som ething of the effect of V I for the
first bird, the perform ance of the present bird suggests a m ix F IF I. As early as the
3rd interpolation, long runs occur at 2 rates on F I 15 alone which are com m on in mix
F IF I. A substantial portion of the 3rd session is shown in Fig. 818 (follows Fig. 817B).
N ote the sudden em ergence of a m uch higher rate of responding on F I 15, a t a an d
b. T h e effect is m ore com m on after the block of reinforcem ents on F I 1. T h e in ter
polated perform ance is followed by a som ew hat higher rate at c, which then declines
to the lower of the 2 ru nning rates a t d. T h e rate rises again to the exceptionally high
value at e. T he negative curv atu re after reinforcem ent shown a t f g, h, a n d i re
sembles th a t at c and appears to be the result of the interpolated F I 1.
L ater, the perform ance resem bles th a t o f the first bird m ore closely, except th a t
the rate following the interpolation is generally lower for some tim e. T he 20th, 21st,
22nd, a n d 23rd interpolations for the second bird are given in Fig. 819, which m ay
Fig. 8 1 9 .
FI 15 inter FI 1: 20th to 2 3 rd interpolation (second bird)
648
Fig. 8 2 0 .
FI 1 5 inter FI 1: 31 st in te rp o la tio n (second bird)
be com pared w ith Fig. 816. W ell-m arked scallops a p p e ar on F I 1, b u t they develop
again each day. T he FI 15 curve now shows a longer an d m ore m arked scalloping.
T he first long interval after interpolation shows prim ing, b u t the ensuing rate for 15 to
30 m inutes following the in terpolation is consistently lower. A t this tim e the b ird
still shows wide variations in ru nning rate.
Figures 820 a n d 821 show the final perform ance for the second b ird , a t the 31st
interpolation. At the beginning of Fig. 820, w ell-m arked F I 15 scallops appear, as a t
a a n d b, w ith a n occasional exam ple of ru n n in g -th ro u g h w ithout scalloping, as a t c
a n d d. Some evidence of a second-order effect appears in 3 intervals, e,f, and g. T h e
perform ance is no longer characterized by frequent, sharp shifts from one running rate
to an o th er. T o w ard the end of the session, how ever, the perform ance is som ew hat
m ore variable. Figure 821 shows this perform ance, together w ith the interpolation.
A n early ru n appears, suggesting the FI 1 perform ance, a t a. This run m ay be due
649
to a tem poral discrim ination of the tim e in the session w hen the F I 1 interpolation is
norm ally m ade. It is followed, however, by a norm al interval scallop a t b. T he next
interval shows a n a b ru p t break before reinforcem ent at c, which would suggest some
straining at this point. T he following interval begins early; the rate again declines to
a low value at d, b u t exhibits some recovery at e before reinforcem ent. U n d e r the
FI 1 interpolation, scallops develop quickly. T he usual FI 1 perform ance occurs at the
beginning of the first 15-m inute interval at f which prim es the bird with an interval
sequence. T he balance of the session shows oscillation in rate, rath er rough grain, a n d
frequent breaking to lower rates.
FI inter FR
In a continuation of the experim ent ju st described, the interpolation m em ber was

changed from FI 1 to F R 3 0 and, later, F R 5 0 . U nder F R in te rF I the slight scal
loped characteristic of the early short interval disappears.
T he small ratio is held eas
ily. Some prim ing begins to appear elsewhere in the daily session. T he 4th interpo
lation of F R 30 is given in Fig. 822, containing the whole session. Figure 816 showed
the last perform ance for this bird w ith an interpolated FI. In the present figure the
over-all rate has risen sharply. T h e background perform ance is roughly linear. L it
tle evidence appears of any change in rate after reinforcem ent until the reinforcem ent
at a. This lack of change m ay show some control exercised by the position in the daily
session, because the following 3 reinforcements a t b, c, and d also show brief, short p rim
ing runs. T he interpolation runs off rapidly at a m uch higher over-all rate th an the
background FI 15, a n d a t the end a short prim ing occurs into a very low rate a t e. A
very m uch depressed rate for several intervals following the interpolation is ch aracter
istic of this bird. T h e extent of this depression, w hich in this pigeon is m ore th a n an
h our long, appears to be reduced as the experim ent progresses. For the first 3 re in
forcements after interpolation, no consistent prim ing run occurs; but later in the session,
every reinforcem ent is followed by a su b stan tial ru n a t th e in terp o lated ratio rate.
These begin at f a n d extend through the last reinforcem ent a t g.
W hen F R 50 inter FI 15 was continued after the stage shown in Fig. 822, well-
650
m arked interval scallops began to appear in the background schedule. M any of these
begin w ith short ratio runs suggesting a m ixed schedule. Figure 823 shows the 11th
session following Fig. 822. T he first reinforcements, a through d, are followed by fairly
standard interval scallops. T he fifth, a t e, is followed by a prim ing run, however,
w hich is repeated again at f and g w ith only slight pausing. T h e following reinforce
m ent a t h is followed by a substantial pause, as is also the next a t i; a n d the intervals
show the term in al rate throughout. T he reinforcem ent a t j produces an irregular
ru n , and the interpolated high rate follows throughout the segment. All of this m ay be
a tem poral discrim ination in anticipating the interpolated schedule. T he ratio r e
inforcem ents are ru n off at a still higher rate, however. A prim ing break into the
interval pause appears at k, and the next 4 intervals show a low sustained rate. A brief
prim ing ru n follows the reinforcement a t /. At m, evidence of a prim ing effect is u n
clear, because the bird runs through the subsequent interval. A t n, however, a wellm arked prim ing ru n occurs followed by a sm ooth scallop. Inconsistency of the p e r
form ance is shown at o, where a sim ilar scallop follows w ithout a prim ing run.
A lthough some progressive change in the size of the first prim ing ru n is to be ex-
651
pected as the result of prolonging exposure to the interpolated schedule, it is probably

significant th a t the bird shows the clearest break a t the end of the prim ing run at an
average of 77 responses on F R 3 0 as against 123 responses on F R 5 0 . T he run is
roughly equal to \ ratios.
T h e prolonged period of slow responding which follows the interpolated F R for this
bird persisted at approxim ately the same point in the session when the interpolation
was w ithheld. A lower rate was evident for 4 sessions after the block was w ithdraw n.
Figure 824 shows the 3rd session. A single interval reveals a low rate early in the ses
sion, at a, but the perform ance then continues in a fairly standard fashion for FI 15 until
b, after w hich 5 intervals show a low rate from w hich the bird g rad u ally recovers.
T he 6th interval, beginning a t c, is fairly norm al. T he over-all curvature of the session
strongly resembles th a t of sessions with interpolated FR.
Figure 825 indicates the perform ance of the second bird in the 5th session with an
in terp o lated F R 30. T he p a rt of the session shown in the figure was followed by 6
hours during which a n over-all rate of about 1200 responses per hour was m aintained.
652
Fig. 8 2 5 .
FI 15 inter FR 30: 5th in te rpolation (second bird)
T he figure shows some early instances of negative curvature following reinforcement,

as at a a n d b, although prim ing is not pronounced in the early p a rt of the session. T he
in terp o lated block begins w ith a fairly characteristic depressed rate after reinforce
m ent a t c; b u t the 1st reinforcem ent on the ratio (at d) produces a fairly characteristic
ratio perform ance except for ra th e r m arked scalloping. T he last reinforcem ent
on the ratio schedule at e is followed by a prim ing run, leading into a very low rate at f .
T he following 2 intervals show an unusually low n u m b er of responses a t reinforce
m ent. In this respect the record confirms the perform ance of the first bird; however,
little evidence exists of sharp prim ing after reinforcem ent a t this stage. T he next 2
Fig. 8 2 6 .
FI 1 5 inter FR 30: 15th, 16th, and 17th in te rp o la tio n (second bird)
M IX E D SCH EDULES
653
or 3 hours show in general a low over-all rate, w ith wide oscillations in running rate.
Figure 826 shows the 15th, 16th, a n d 17th sessions u n d e r interpolated F R for the
second bird. Portions of each session before an d after the interpolated block of ratios
are shown. M ore evidence of random prim ing exists, b u t it is by no m eans so clear
as in Fig. 822, for example. T he first reinforcements on the ratio schedule, a t a, c, an d
e, suffice to generate typical ratio perform ances during the interpolations; a n d in each
case a prim ing ru n follows the last such reinforcem ent, a t b, d, an d f leading into a
good interval scallop. O ccasional prim ing runs follow other reinforcem ents after the
interpolation. Especially in Record B, a general depression in rate follows the in te r
polated block.
W hen the interpolated ratio was increased to 50, the effect upon the following p rim
ing ru n was im m ediate. Figure 827 illustrates the 3 sessions following Fig. 826C.
Portions before an d after the interpolated ratios are again shown. H ere, the prim ing
runs at a, b, a n d f are m uch m ore extended th an those a t b, d, and f in Fig. 826. If
the ru n at a is significantly larger th a n h itherto, it m ust be wholly because of the p re
ceding block of ratios, since this was the first day the ratio h ad been increased.
Prim ing runs elsewhere in the session are at c, d, and e.
W hen the interpolation was withheld, the bird being reinforced simply on FI 15 for
654
the whole session, m any instances of prim ing continued to be seen. Figure 828 reports
the whole session following Fig. 827C. W ell-m arked prim es ap p ear a t a, b, c, d, e , f
a n d j. W h at m ay be regarded as late prim es a p p e ar a t g, h, an d i, although these
resemble the knees often seen in interval scallops. N ote th at this b ird does not show a
pattern for the whole session suggesting the tem poral effects of Fig. 823.
By the 5th session after the rem oval of the interpolated block of ratios, only occa
sional evidence of a prim ing ru n appears. Figure 829 reports the first p a rt of this ses
sion; it shows possible primes at a a n d b, b u t elsewhere m erely a rath e r ragged FI 15
perform ance. T he session was continued for 1^ hours beyond the p a rt shown in the
graph. A bout 4000 responses were em itted during this tim e, w ith the same general
character shown.
V I 5 inter FR50
T h e background schedule was th en changed to V I 5 ( geom etric ) for a few ses

sions. A block of 10 reinforcements on F R 50 was interpolated every hour. T he p ro
gram m ing equipm ent developed trouble, a n d some of the m aterial was lost; but Fig.
830 shows 1-hour segments of all recorded instances of the interpolated F R , with some
o f the behavior before an d after in the order of its occurrence. T he F R rate is not
m uch higher th a n th a t of the V I, an d not quite typical of an F R 5 0 perform ance.
After each interpolation, a prim ing ru n of from 80 to 180 responses occurs, followed,
in general, by a ra th e r sharper break th a n th a t appearing elsewhere in the V I p e r
form ance. Instances of prim ing occur elsewhere, p articu larly in the later segments.
N ote the ra th e r poor V I perform ance, ap p aren tly due to the frequent interpolation
of the block of FRs.
T h e second bird was reinforced on this schedule for only 4 interpolations during 2
sessions. Tw o of these show m arked prim ing runs followed by substantial decreases,
m ore m arked th an in Fig. 830. T he other 2 interpolations were followed by only
slight increases in rate, which were not clearly prim ing runs. T he experim ent was not
continued long enough to see w hether this o ther b ird w ould, in general, give the p e r
form ance of the present figure.
655
A MULTIPLE SCHEDULE COMPOSED OF FR A N D MIX FRFR
W e a tte m p te d to bring the characteristic prim ing effect of the m ix F R F R under

stim ulus control, an d to separate it from a straight F R u n d e r the control of a different
stimulus. T hree rats which h ad been on FI for 62 hours following erf were then re
inforced on mix F R 2 0 F R 8 0 . Figure 173 showed later perform ances on FI. T he
beginning of the new procedure was given in Fig. 741, w here traces of the earlier FI
656
perform ance were evident. R ecord B showed exam ples from the end of the 2nd ses
sion after 6 hours of exposure to th e m ultiple schedule. T h e larger F R h a d been
increased to 160. Figure 742 showed the subsequent developm ent over a period of 13
sessions. T h e position of the first break in the long ratio becam e m ore stable a n d oc
curred after fewer responses as the experim ent progressed.
This mix F R F R schedule was continued u n d er the control of one stimulus (steady
light in the experim ental box); and, a t the same tim e, u n d e r a different stim ulus (a
flashing light), responses were reinforced on the longer ratio alone. If successful, this
procedure should yield a perform ance on the longer ratio showing step-wise breaks
u n d e r th e steady light a n d no breaks u n d e r the flashing. In the tim e devoted to the
experim ent (100 hours) this was achieved w ith only 1 of the 3 rats. Figure 831 illus
trates the next-to-last session for this rat. A indicates the steady light under which re
inforcem ent occurred after 20 responses or after 160. B indicates the flashing light
under which reinforcem ent occurred at the end of 160 responses. A fairly good m u lti
ple control is evident. In general, u n d e r Stim ulus B the ratio is ru n off w ith some
c u rv a tu re a n d possibly a som ew hat low er term in al ra te , b u t w ithout gross breaks.
U n d e r Stim ulus A th e longer intervals usually show breaks. T h ere are four excep
tions. Tw o breaks occur under Stim ulus B which are not justified by the schedule;
they could be regarded as induction from the other stimulus. In three cases a break
fails to occur under Stim ulus A. (A slight break appears in one of these.) O u t of
42 long intervals the curvature is appropriate to a mix F R F R or to a single F R in 37 or
38 instances. O n the following day, w hen a sim ilar perform ance prevailed, 7 m is
takes occurred u n d er Stim ulus B, a n d 2 or possibly 3 u n d er Stim ulus A. W hen the
657
large ratio was dropped to 135, and the experim ent continued otherwise in the same
way, the effect disappeared.
O ne of the other rats m ay have failed to show this effect because the over-all rate was
high a n d little breaking occurred as the result of the mix F R F R . Note th a t on this
schedule very few short ratios are reinforced.
Chapter Twelve
CHAINED SCHEDULES
T w o o r m o r e responses m ay be chained together if the first produces a stimulus in
the presence of which the second is reinforced. Chains of any length are conceivable.
T he separate m em bers of the chain are usually identified by the topography of the
response. A nother type of chain is possible in which a response of a single topography
is em itted u n d e r different stim uli, these stim uli being produced in succession by re
peated reponses. In this case the m em bers of the chain are identified by the reinforce
m ents w hich they produce or the stimuli under which they occur. It has been
assum ed th a t some such chaining occurs in a well-developed fixed ratio in which each
response m ade in the presence of a given n um ber of responses already em itted is re
inforced by an increase in th at num ber.
In a chained schedule the bird first responds in the presence of one stim ulus on a
given schedule a n d is reinforced by a change in the stim ulus under which it is then re
inforced, usually on a different schedule. U ltim ately, a response is reinforced with
food. C hain ed schedules are u n d e r the stim ulus control characteristic of m ultiple
schedules, b u t they differ in th a t only one eventual reinforcem ent w ith food m aintains
all m em bers of a schedule.
A dem onstration of a chained schedule is produced by beginning w ith a m ultiple
schedule containing extinction as one com ponent, for exam ple, m ult FI 1 ext, in which
the periods of extinction are term in a te d according to a ran d o m distribution of in te r
vals sim ilar to th a t used in V I 1. T he reappearance of the stim ulus controlling F I 1 is
not correlated w ith a response, nor does a T O occur. H ow ever, the stim ulus a p p ro
priate to the FI 1 schedule can now be m ade to ap p ear w hen the organism occasionally
responds d u ring the extinction stimulus. Any increase in responding during the p a rt
of the schedule previously showing extinction dem onstrates the reinforcing effect of the
stim ulus appropriate to FI.
CHAIN VIFI
Chain V I I FI T1
In one such experim ent a bird was reinforced on m ult F I 1 ext. T he final perform
ance generated is shown in Fig. 832A an d C; but, for the m om ent, only the 1st seg1 See Ferster (1953) for a n application o f c hained schedules to the problem of delayed reinforcem ent.
658
C H A IN E D SCH ED U LES
659
m ent of R ecord A is relevant. These perform ances show a high rate of responding on
FI 1 (G), w ith little or no pausing after reinforcem ent, a n d a very low rate on extinc
tion (A), both under appropriate stimuli. A t a, in R ecord A, the production of the
stim ulus appropriate to FI 1 was m ade contingent upon a response u n d er the other
stim ulus on a V I 1 schedule. T h e first such contingency was felt im m ediately after
point a. A single 1-m inute interval was then executed. T h e burst of responding after
a was recorded in the retu rn to the stim ulus form erly appropriate to extinction. T he
second production of the FI 1 stim ulus occurred at b. Subsequently in the session, the
rate in R ecord A increased steadily. In general, responding tended to be most rapid
after the return from the F I 1 schedule.
Records B an d D show the 2nd session. A fairly stable V I perform ance (B) has
been generated by reinforcem ent consisting sim ply of a change to the key-color on
which reinforcem ents are scheduled in F I 1. R ecord D gives the perform ance on the
latter key.
Tw o other birds were reinforced for 40 sessions on m ult F I (blue key) extinction (red
key). T he 2 schedules were then chained together by letting responses on the red key
produce the blue key in V I 1, a response on the red key being reinforced a fixed interval
later. A substantial rate on the red key developed very slowly. Several values of FI
(1, 1.5, 2, a n d 5 m inutes) were introduced into this general p attern. Stable condi
tions on some of these are shown in the following figures.
Figure 833A shows a perform ance on V I 1 (red key) reinforced, as m arked, by the
production of the blue key. Record B shows the alternating perform ance on FI 1 on
the blue key. T h e over-all rate is higher in R ecord A, the first m em ber of the chain,
th a n in R ecord B, the m em ber reinforced w ith food. W hile pauses occur in both
cases, they are m ore m arked in B, a n d the transition to a term inal rate is m ore abrupt.
Figure 833C shows a perform ance on V I 1 (red key) reinforced, as m arked, by the pro
duction of the blue key. Record D shows the alternating perform ance on FI 1.5 on the
blue key. T he over-all rates are here of the same order of m agnitude. T he V I sched
ule in R ecord C is generally successful in elim inating the pause after reinforcem ent,
while the segments in R ecord D usually show FI pauses.
660
Increases in the size o f the FI component on chain V I 3 FI
In another experim ent, 2 birds were stabilized on chain V I 3 FI in which the

value of F I was som ew hat system atically varied. These birds h a d h ad an extensive
history of m ult V I 3 FI 3 in which, of course, both com ponents were separately rein
forced w ith food. A transition to the chained schedule was m ade by the omission of
the food reinforcem ent on V I 3; instead, the production of the stim ulus appropriate to
F I 3 was substituted. Both birds substantially lost the perform ance on V I 3 form erly
prevailing u n d er the m ultiple schedule before developing a chained performance.
First bird. After 60 sessions on the chained schedule, the fairly stable conditions in
Fig. 834 prevailed. R ecord A is the V I 3 perform ance, w here each reinforcem ent,
as m arked, consists of the production of the stim ulus u n d e r which FI 2 was reinforced
661
with food. R ecord B gives the alternating FI 2 perform ance. T h e over-all rates are
perhaps a little lower for the second m em ber of the chain. T h e V I perform ance
showed considerable irregularity and a tendency to adopt the term inal FI rate. Slight
pausing occurs after reinforcem ent on the V I schedule, an d substantial pausing an d
curvature, on the F I schedule.
Chain VI 3 FI 5.5
T h e F I interval was then extended to 5.5 m inutes an d the new schedule rem ained in
force for 48 sessions. Figure 835 shows the final perform ance. T he FI 5.5 exhibits
some negative cu rvature, giving an S-shaped c h a ra c te r to m an y interval segments.
Pauses follow most reinforcements, but instances also appear of running-through and of
fairly rap id acceleration to the term inal rate. T he perform ance u n d er the first sched
ule in the chain shown a t Record A is very irregular, with a low over-all rate leading
to m arked acceleration during the session.
T he F I interval was then further increased to 7.5. Figure 836 describes the p e r
form ance 15 sessions later. T he FI schedule shows a lower term inal rate th an th a t
in Fig. 835 and considerable irregularity in the interval curvature. M arked instances
o f negative acceleration before reinforcem ent appear. (T he som ew hat variable per-
662
form ance on F I in such a schedule m ay be due in p a rt to the irreg u lar V I schedule of

the preceding m em ber. T h e interval perform ance is sometimes executed after a very
short interval on the first com ponent of the chain a n d in other instances after a long
interval.) In the session following Fig. 836 the original value of F I (2 minutes) was re
stored. T he rate on the first m em ber of the schedule increased im m ediately. T en
sessions later, the stable perform ance ap p a re n t in Fig. 837 was recorded. T h e F I 2
perform ance at R ecord B is very sim ilar to th at in Fig. 834B, while the V I perform ance
C H A IN E D SC H ED U LES
663
on th e first m em ber shows a m uch sm oother grain. Some pausing after reinforce
m ent is now developing on the V I schedule. T h e over-all rate on the first m em ber of
th e chain is now slightly higher th a n th a t on the second and is higher th an the rate
previously observed on V I 3 in Fig. 834.
Second bird. W e studied a second bird on the same program ; in general, its perform
ance confirms the essential points just m ade for the first bird. Figure 838 shows the
stable perform ance on the original c h ain ed V I 3 F I 2. Figure 839 illustrates a p e r
form ance 50 sessions after the change to V I 3 FI 5.5. T h e V I perform ance on the first
m em ber of the chain is somewhat m ore severely disrupted than it was for the other bird.
T he over-all rate is still of the sam e order as th a t occurring under F I 2 in the second
m em ber of the chain. Figure 840 shows the final perform ance on chained V I 3 FI 7.5.
T h e FI perform ance closely resembles th a t of the other bird. T h e V I perform ance
shows a tendency to ap p ro ach the term inal F I rate a n d m uch m ore prolonged pauses
th a n those in Fig. 836. N o effort was m ade to recover the original perform ance w ith
this bird.
664
CHAIN FIVI
In another experim ent on 2 birds the 2nd (food-reinforced) com ponent of a chained
schedule was m aintained at V I 3 th roughout the experim ent. T h e 1st m em ber re
inforced by the change to the stim ulus ap p ro p riate to the 2nd m em ber began a t FI 2,
b u t subsequently becam e FI 3.5, F I 5, and then FI 7.5. Figures 841, 842, 843, and 844
illustrate the results. Figure 841 shows a stable chained perform ance after 60 ses
sions on ch ained FI 2 V I 3. T he F I 2 shows pauses after reinforcem ent, sm ooth c u r
vature in m any instances, and some second-order effects in spite of the intervening
periods on the other key. T he V I perform ance shows no consistent relation between
local rate a n d recent reinforcements an d a well-sustained high over-all rate. O n
FI 3.5 in Fig. 842 the interval perform ance shows increasingly longer pauses a n d some
variatio n in the term inal rate. T h e figure was recorded after 16 sessions on chained
FI 3.5 V I 3. T h e perform ance on the V I com ponent rem ains essentially as in Fig.
841. O n F I 5.5 (Fig. 843), pauses after reinforcem ent on the interval schedule are
often exceptionally long, w ith a n a b ru p t change to a term inal rate, as a t a an d c, or an
ab ru p t increase followed by negative curvature leading again into the term inal rate, as
a t b. An occasional interval is ru n off a t an interm ediate rate, as a t d. T h e over-all
rate has not changed its order of m agnitude. T he perform ance on the 2nd com ponent
on V I 3 (B) rem ains unchanged. W hen the 1st com ponent is FI 7.5 (Fig. 844), the FI
perform ance shows increasingly longer pauses and frequently a b ru p t changes to a rate
n e a r the term in al rate. T he term in al rates th ro u g h o u t the experim ent ten d to be
lower th an norm al on an FI schedule alone, and the pauses are also longer than norm al
throughout. T he V I schedule on the 2nd com ponent in Fig. 844 rem ains essentially
unchanged (B).
A repetition of the experim ent w ith the second bird produced stable performances
Fig. 8 4 1 .
Final perform ance on chain FI 2 VI 3
Final perform ance on chain FI 3.5 VI 3
Fig. 8 4 3 .
Final perform ance on chain FI 5 .5 VI 3
Fig. 8 4 4 .
Final perform ance on chain FI 7.5 VI 3
300
RESPONSES
300
R ESPONSES
Fig. 8 4 2 .
RESPONSES
300
^
Chain Fl 3.5 VI 3
Fig. 8 4 6 .
Chain Fl 5.5 VI 3
Fig. 8 4 7 .
Chain Fl 7.5 VI 3
300
RESPONSES
Fig. 8 4 5 .
Fig. 8 4 8 .
667
Chain FI 2 VI 3
w hen the first com ponent was FI 3.5, FI 5.5, and FI 7.5, shown in Fig. 845, 846, an d
847, respectively. T he second V I com ponent (B) rem ains unchanged th rough
out, but the over-all rate declines severely on the initial chained com ponent on FI.
Pauses become extrem ely long, often delaying reinforcem ent, an d the term inal rates
are exceptionally low. W hen we tried to recover a higher rate on the 1st com ponent
of the chained schedule by reducing the interval to FI 2 after Fig. 847, a higher over
all rate failed to return. Figure 848A illustrates the perform ance. In Fig. 848B, the
V I perform ance also begins to show irregularity.
CHAIN VIFR
F our birds were reinforced on F R after erf u ntil the perform ance stabilized. A
discrim ination was then set up. Responses continued to be reinforced on F R 50 on a
blue key; but the key changed at reinforcem ent to orange, a n d no responses were re
inforced until, after an interval which varied in the m anner of a V I 1 schedule, the key
again changed to blue. D iscrim inations developed quickly in 3 of the 4 birds. Fig
ure 849 gives the 1st session for 1 bird. T he perform ance on the blue key appears a t
R ecord A (a defect in the recorder was responsible for the break a t a); a n d the declin
ing rate on the orange key a t R ecord B. T he latter shows considerable irregularity
an d a frequent retu rn to the ratio rate, as a t b a n d c, during extinction. T he perform
ance on the blue key on the 6th day appears at R ecord C; and on the orange key, a t
R ecord D. T he rate is essentially zero on the orange key. T his perform ance contin
ued for the rest of the 10 sessions devoted to discrim ination. (T he exceptional bird
continued to respond at the beginning of th e experim ental session on the orange key,
but its rate dropped to essentially zero before the end of each session.)
O n the 11th day the change from orange to blue was m ade contingent upon a re
sponse to the orange key. This contingency produced a chain in which a response on
th e orange key was reinforced on V I 1 by a change to a blue key. T he 50th response
on the blue key was then reinforced w ith food. Figure 850 shows the im m ediate effect
for 2 birds. Both birds developed substantial rates of responding on the orange key
668
during the session, as in Records B a n d D. R esponding on the blue key continued, as

Records A and C show.
T he exceptional bird already noted showed the perform ance of Records A and B in
Fig. 851 on the last day of the discrim ination. R ecord A gives the responding on
the blue key; R ecord B shows the characteristic high rate on the orange key at the b e
ginning of the session, leading to a near-zero rate before the end of the session. W hen
the responses on the orange key were reinforced by the production of the blue key,
an increase in rate occurred as in the other birds; but this increase was added to the
original high rate in the session. Records C and D show the resulting perform ance.
All birds were m aintained on the ch ained V I 1 F R for 60 sessions, d u ring which
the values of the ratio were changed. In 2 birds the ratio was advanced to the point a t
which severe breaking after reinforcem ent was observed. Figures 852 and 853 show
s,a o s i
670
the course of the perform ance. In Figure 852, Records A, C, a n d E show perform ances
on the blue key on ratios beginning at 75 a t R ecord A and ending w ith 300 a t R ec
ord E. These are for the 10th, 35th, a n d 60th sessions in the experim ent. T h e d e
velopm ent of m arked breaks on F R 300 is evident, as a t a a n d b. T h e perform ance
on the orange key which is reinforced by a change to these ratio schedules varies signifi
cantly. At R ecord B, in the perform ance on the same day as R ecord A, pauses a p
pear after reinforcem ent; some rough grain and positive acceleration occur during the
longer intervals. Pausing a n d scalloping becom e m ore m arked by the 35th session
a t R ecord D (corresponding to the perform ance at R ecord C on the ratio) ; an d in the
60th session, w hen the ratio has been extended to 300 a n d is breaking, the rate on
th e first m em ber of the chain becomes very low, as R ecord F reveals.
Essentially the sam e points m ay be m ade for the second b ird in Fig. 853. Pausing
after reinforcem ent in the V I schedule appears at Records B, D, and F. T he corre

sponding ratio performances are a t Records A, C , an d E, respectively; these are for the
10th, 23rd, a n d 34th sessions. R ecord F begins to show m arked pausing an d
scalloping sim ilar to th a t in Fig. 852D. W hen, as a t R ecord G in the 54th session, the
ratio of 300 produces m arked breaks, the V I perform ance is reduced to the low level
a t R ecord H.
T h e ratio was not advanced for the other 2 birds, so th a t straining did not develop.
H ow ever, pauses after reinforcem ent in the V I m em ber of the chain appear. R ec
ords A a n d B in Fig. 854 are for the 15th session. T h e perform ance on the ratio (R ec
ord A) shows no pausing a t th e sta rt of th e ratio run; b u t the perform ance on V I 1 at
R ecord B shows pauses after reinforcem ent, generally followed by some com pensatory
increase in rate, w ith negative acceleration d u ring the longer intervals. Records C
and D are for the 20th session. R ecord D still shows irregularities in the V I perform-
671
ance, w ith pausing after reinforcem ent. By the 57th session, however, show n at
Records E and F, the ratio and V I perform ance are norm al.
T h e results w ith the fourth bird in the series were similar.
In another experim ent, 2 birds w ith a history of m ult F R F I an d m ult Flext were re
inforced on chain V I 3 F R for 57 sessions during w hich the F R was increased from 20
to 70. Figure 855 shows perform ances from the first p a rt of the last session. Records
A a n d B give th e first com ponents of th e chain on V I 3. Some pausing after rein
forcem ent occurs, a n d some variation in rate. In general, however, a fairly good V I
perform ance is evident. Records C a n d D show the corresponding perform ances on
the second com ponent under F R 70. Except for an occasional pause after reinforce
m ent, the ratio is well-sustained.
CHAIN FRVIdrl
In an experim ent to be described later (see Fig. 862) a stable perform ance was estab
lished on chain V I 2.5 V I 2.5 drl 4. T he 1st m em ber of the chain on a blue key was
th en m ade F R 3 5 , while the schedule on the 2nd com ponent on the orange key re
m ained V I 2.5 drl 4. T he sequence of events was as follows. O n the blue key the 35th
response was reinforced by a change to an orange key on w hich a response was re
inforced on V I drl 4. Figures 856A and B show a final perform ance after 27 sessions
o f this schedule. T he ratio performance at Record B is quite irregular, and the ratio
rates are exceptionally low for so sm all a value. This condition developed gradually
d u rin g the 27 sessions. Initially, th e ratio perform ance was practically n orm al; a
sam ple from the 2nd session appears at R ecord C. M eanw hile, the responding on the
orange key shows some slight decline d u ring the session, a n d the over-all rate is m uch
lower th a n th a t in Fig. 862. T he ratio grain could be due to interference from m e
d iating behavior on the drl schedule. Nevertheless, the perform ance illustrates the
possibly unexpected condition th a t responding m ay occur a t a fairly high rate when
the reinforcem ent consists of the production of a stim ulus controlling a low rate.
672
T h e same result was obtained w ith a second bird, also on chain V I 2.5 V I 2.5 drl 4.
W hen the schedule on the first (blue) key was changed to F R 50, a very high rate im
m ediately ap p eared on the ratio (a high rate existed on th a t key u n d e r V I 2.5). T he
perform ance for the 1st session on the ratio m em ber appears in Fig. 857C, which corre
sponds to Fig. 856C. L ater, the ratio perform ance showed trouble, as it did w ith the
o th er bird. Figure 857B shows the perform ance on the ratio after 28 sessions of the
chain FRVTdrl, during w hich the value of the ratio was dropped to 35. Record A
contains the very low stable perform ance u n d er V I 2.5 drl 4. T he rate has fallen be
low th a t of Fig. 868, partly because of the prolonged exposure to the drl contingency.
C H A IN E D SCH EDULES
673
CHAIN VIVI
Tw o birds were reinforced on m ult V I 1 (blue key) ext (orange key) until a n e a r
zero rate developed on the orange key. T h e periods of extinction varied in the m a n
ner of a V I 1. T he 2 schedules were th en linked together in a chain; responses on
the orange key produced the blue key on a V I 1 schedule. Figure 858 illustrates the
developm ent of a substantial rate on the orange key in 3 sessions. R ecord A shows
th e perform ance on the orange key on the last day of the m ultiple schedule. O nly a
few responses occur during the session. R ecord B is for the 1st day upon w hich a
response to the orange key produces the blue key on a V I 1 schedule. T he rate in
creases slightly during the session, an d a t least one sustained ru n occurs (at a ) at essen
tially the rate on the blue key. Records C and D show the 2nd and 3rd days of the
chained schedule; a higher rate develops a n d in R ecord D conspicuous pausing occurs
after reinforcem ent, as at b, c, d, an d elsewhere. R ecord E shows the V I 1 perform
ance on the blue key on the last day of the m ultiple schedule corresponding to R ecord
A. R ecord F shows the perform ance on the blue key (2nd com ponent) on the 3rd
d ay of the chained schedule corresponding to R ecord D. T h e appearance of an u n
usually high rate at e dropping a t f to the norm al rate was exceptional a n d was not evi
dent in the other bird or on the om itted days betw een Records E and F for the pres
ent bird.
Fig. 8 5 8 .
Developm ent on chain VI 1 VI 1
674
T h e level of deprivation was then varied. Figure 859 shows performances at 2 levels
for the same bird as above. T he weight during the sessions shown at Records A and
B was approxim ately 10% above th a t in Records C an d D. R ecord A shows respond
ing on the orange key; and R ecord B, on the blue key 35 sessions after the beginning
o f the chained schedule. N ote the slightly higher rate on the blue key, the absence of
any pausing on the blue key, and a rath er rougher grain on the orange key, m uch of
this contributed by slight pausing after reinforcem ent. Records C and D show the
perform ance a t the lower body-w eight, 43 sessions after Fig. 858. T he rates here are
substantially higher th an those at Records A a n d B. R esponding to the orange key
is m ore irregular, and the rate is again relatively high for the blue key.
Figure 860 shows a sim ilar set of records for a second bird. Records A a n d B illus
tra te performances on the orange and blue keys, respectively, at a relatively high bodyweight. T he perform ance on the orange key is som ew hat m ore orderly, but the same
over-all rates prevail. Records C a n d D indicate the perform ances a t a 12% lower
body-w eight. T he perform ance on the blue key a t R ecord D is now quite linear an d
shows alm ost no pausing after reinforcem ent. R esponding to the orange key a t R ec
ord C shows some pausing and com pensatory running after reinforcem ent; but, in gen
eral, a high rate is also m aintained. T h e rates are of the sam e order on both the
orange a n d blue keys. A lower body-w eight produces a higher rate on both m em
bers of the chain.
Tw o birds were reinforced on m ult V I 3 (blue) V I 3 (orange) im m ediately after erf.
T h e V I rates showed a color preference for blue in the ratio of 9:7. W e then set up
a chain by reinforcing responses to the blue key w ith a change to the orange key instead
Fig. 8 6 1 .
Chain VI 3 VI 3
676
Fig. 8 6 2 .
Chain VI 3 VI 3 drl: segments from the VI 3 drl record showing the developm ent
o f a low rate
of w ith food. This procedure reduced the total density of food reinforcem ent by onehalf. T he rate on the blue key declined during 20 subsequent sessions. Figure 861
illustrates the entire 21st session. R ecord A shows responding on the previously p re
ferred blue key, reinforced, as m arked, by changes to the orange key. Record B shows
responding on the orange key, reinforced, as m arked, w ith food. After food rein
forcement, the key becam e blue immediately.
A drl 4 was then added to the V I schedule on the orange key. This was the 2nd
com ponent of the chain reinforced with food. T he decline in rate is extremely slow for
this contingency. Figure 862 gives the 2nd an d 3rd excursions from each of 11 suc
cessive sessions. R esponding tends to be especially rap id im m ediately after the
changeover from the blue key, as a t a a n d b. M ore m arked examples of this tendency
occur elsewhere, and possibly represent an em otional effect arising from the release
from the blue key an d the presentation of the orange key, responses to which are rein
forced w ith food. O ne result is the lengthening of the shorter intervals in the V I
schedule. C oncurrently w ith the decline shown in Fig. 862, the perform ance on the
blue key becomes very irregular. Long periods of no responding occur, the grain is
very rough, and the over-all rate declines. At times, however, a fairly high rate
emerges. Figure 863 shows the entire perform ance on the blue key corresponding
to the session from which the segments at K in Fig. 862 were taken. V ery rough grain,
677
periods of quite rap id responding, as a t a, a n d exceptionally long pauses, as a t b, c,

a n d d, occur. T h e rap id responding a t a is as extrem e as it ever becomes, a n d the
pauses a t b, c, an d d are exceptionally long. However, the record correctly reports the
general disturbance in the 1st m em ber of the chain in w hich the 2nd m em ber con
tains a drl. This disturbance could in p a rt reflect interference from m ediating b e
havior.
W hen the drl contingency was then removed, the bird returnd very rapidly to the
high uniform ra te of the orange key. Figure 864 shows the 2nd session of the new
schedule. T h e perform ance on the 1st m em ber of the chain (R ecord A) has already
recovered m uch of the character of Fig. 861A.
W hen the drl was again added to the contingency on the orange key, the rate fell off
w ithin a single session to the level reached only after 11 sessions in Fig. 862. Figure
865 shows the entire 1st session. N ote the m arked instances of a high rate im m ediately
following the change from the blue key, as at a and b. Following this change in the
contingency on the orange key, the responding on the blue key began to show the ex
trem e ranges in rate already seen in Fig. 863.
A second bird showed initially a higher rate on both key-colors under m ult V I 3 V I 3,
w ith a preference of approxim ately 9 to 8 in favor of the blue key. This perform
ance is in the sam e direction as th a t of the other bird, b u t the relation does not change
during the chaining. D uring 21 sessions in which the 2 schedules were chained, the
bird did not show a decline in rate on the blue key. Figure 866 shows the 22nd session
complete. W hen drl was added, the perform ance showed the usual decline; b u t the
rate on the blue key rem ained essentially unchanged. T he perform ance was sim ilar
to a later session under these conditions, shown in Fig. 868. W hen the drl is again
om itted, the rate rises quickly, as Fig. 867 shows. R ecord A illustrates the perform
ance on the blue key reinforced by the appearance of the orange key; and Record B,
the perform ance on the orange key reinforced w ith food. N ote the beginning of a
tendency to respond m ore rapidly on the blue key after the preceding reinforcement on
679
the orange key, as at a. Note also, in R ecord B, a m uch sharper curvature resulting
from a high rate im m ediately after changeover from the blue key, as at b and c.
W hen the drl was again added to the 2nd m em ber, the rate fell. Figure 868 gives
a sam ple of the perform ance. O n the blue key (R ecord B) a higher rate im m ediately
after reinforcem ent is now common. It is followed by a fairly smooth decline w hen
ever the interval perm its, as a t a, b, c, a n d d. N ote th a t this perform ance represents a
tendency for the rate not to rise above th a t previously seen on this key (Fig. 867A)
but to decline to a lower over-all value.
Tw o of the birds in the preceding experim ent, one on chain V I 3 V I 3, th e other
on chain V I 1 V I 1, were left in the experim ental box for 18f and 16 hours, respectively.
D uring these long sessions, substantial am ounts of food were received in reinforce
m ents. Figure 869 gives the result. R ecord A is the cum ulative perform ance of the
b ird on the 2nd com ponent of c h ain V I 3 V I 3 a n d R ecord B is for the 1st com po
nent. Satiation reduces the rate in the 1st com ponent, b u t has little effect on the sec
ond. T he same result held for the other bird. R ecord C shows the 2nd-com ponent
rate holding well, while the 1st com ponent (R ecord D) declines in satiation.
Fig. 8 6 8 .
Return to chain VI 3 VI 3 drl (second bird)
680
Fig. 8 6 9 .
Satiation curves on chain VIVI
CHAINED FIFR
Tw o birds w ith a history of m ult F IF R w ere reinforced for 30 sessions on chain

F I 4 F R , w here the F R varied betw een 20 an d 60. Figure 870 illustrates the last o b
served perform ances on chain FI 4 F R 60. R ecord A shows the perform ance on the
681
lst-com ponent F I for one bird. Pauses are longer th an those for a simple F I 4. R ec
ord C shows the corresponding 2nd-com ponent perform ance on F R 60. A few small
breaks occur, b u t the term in al ra te a n d general p a tte rn are typical of a ratio p e r
formance. Records B and D show the com parable records for the second bird.
C H AIN FRFI
Tw o birds, also w ith a history of m ult F IF R , were reinforced for 58 sessions on

chain F R F I, during which the FI values ranged from 1 to 3 m inutes while the ratios
varied from 20 to 50. Figure 871 shows perform ances for 1 bird under 2 sets of in
terval values. Records A a n d C illustrate 1st a n d 2nd com ponents, respectively, on
chain F R 20 FI 1. T h e ratio perform ance at R ecord A shows consistent pausing after
reinforcem ent, which would be quite unusual on a simple ratio of this size. T he over
all a n d term inal rates in Record A are lower th an those in R ecord C. M eanw hile,
the FI 1 perform ance in Record C lacks the pausing after reinforcem ent and scalloping
evident in a simple FI 1. This perform ance is related to the position of the FI 1 on
the 2nd com ponent of the chained schedule. Records B and D show performances on
th e 1st a n d 2nd com ponents of chain F R 2 0 F I 2. T he FI perform ance shows some
slight scalloping. But the m ain effect of the larger interval is felt in the preceding
ratio perform ance (R ecord D), w here both over-all a n d term inal rates are low. T he
second bird showed sim ilar results, except th a t it could not m ain tain a com parable
over-all rate on the ratio com ponent.
CHAIN FRFR
T he block counter described in C h ap ter Four is in reality a species of chain F R

FRFR. . . .
In the experim ent described there the color of the key could change at
prescribed points during the emission of the ratio. T he last response in the ratio on
the last color of the counter was, of course, reinforced w ith food. But the production
of this last color thereupon becam e reinforcing for the last response in the preceding
color, an d so on. In some of these experim ents the arrangem ents of colors during the
ratio produced effects which are relevant here.
682
Fig. 8 72.
Chain FR 15 FR 105 and chain FR 3 0 FR 30 FR 30 FR 30
In one case a color change occurred at only one point during the ratio. Fifteen re
sponses on a red key produced a blue key, a n d 100 responses on the blue key p ro
duced food. T h e perform ance (Fig. 872) shows a pause after each reinforcem ent, as
a t a, c, a n d e. After the change from red to blue at 15 responses, a 2nd b u t smaller
pause frequently occurs, as at b, d an d f . At g, w here the pause after reinforcem ent is
exceptionally long, practically no pausing occurs on the change in color. Separate
recording of the 2 parts of the ratio would show a low over-all rate, with strong pausing
on the red key and a high over-all rate w ith only slight pausing on the blue.
R ecord B shows an exam ple of a perform ance in w hich the key-color changes after
each block of 30 responses. T hree breaks can usually be detected in each segment.
H ow ever, the pausing after reinforcem ent is often longer th a n any pause upon change
of color. L ater changes frequently have some tendency to show less pausing. This
tendency is p a rticu larly clear in R ecord C, w hich is for the session following R ecord
B. R ecord C shows the dam p in g of the oscillation already m entioned in C h ap ter
Four, as at h. Note th a t dam ping tends to occur when a ratio is straining. (The 7 re
inforcements shown in the figure are the only ones received in a session of more th an
1 hour.) T h e dam ped ratio at i followed a long pause not shown in the graph.
T h e usual effect of changing from one set o f colors to an o th er is to break up the
control of the chain a n d to produce a tem porarily high rate. In Fig. 873A the bird is
showing a badly strained performance on the 4-color schedule just described. At the
beginning of the following session the ratio of 120 responses was subdivided in succes
sive blocks of 60, 30, a n d 30 responses. T he 1st color was th at under which responses
h a d form erly been m agazine-reinforced. A change to a previously nonreinforced
color occurs after 60 responses. After 30 responses, the previously reinforced color re-
683
tu rn s for the last 30 responses. R ecord B shows the effect. T h e pausing after re
inforcem ent characteristic of the earlier 4-color perform ance is im m ediately lost, since
the presence of the previously reinforced color leads to an im m ediate start, as a t a,
c, an d e. T h e change to the previously nonreinforced key after 60 responses produces
a slight pause, however, as a t b, d, an d f . T he pause a t 60 responses continues for
some tim e, an d is present in the last 2 ratios in the session a t j and k, although these now
also show pauses after reinforcement. At one stage the whole sequence functions as
a single color ratio, as a t g, h, a n d i. O n th e following day, the break a t 60 responses
Fig. 8 7 3 .
Transition from chain FR 3 0 FR 3 0
FR 3 0 FR 3 0 to chain FR 6 0 FR 3 0 FR 3 0
684
Fig. 8 7 4 .
Return to FR 120 a fte r chain FR 6 0 FR 3 0 FR 30
continues to ap p ear from tim e to tim e. T he increase in the over-all rate produced
by the new color schedule begins to disappear. This increased rate is characteristic of
changing a color sequence in subm ultiples of FR . R ecord C continues the same p ro
cedure as R ecord B.
After pausing h ad developed in Fig. 873C, the color previously present a t reinforce
m ent was m aintained throughout the ratio. Figure 874 shows 2 resulting perform
ances. In R ecord A a bird not previously discussed requires m ore th a n 25 reinforce
m ents to recover from the break in g on th e c h a in e d ratio a n d to reach the high
over-all rate at a. T h e pausing occurring before th a t point was characteristic of the
preceding chained ratio, similar to th at of Fig. 873C. T he other bird (Record B), from
th e preceding figure, also requires a few reinforcem ents to reach the new high rate
shown a t b. H ere, the rate declines, a n d the pauses em erge on the new schedule with
the single key-color w ithin the session. A com parable decline for the other bird a p
peared in a later session.
C H AIN VI UNCORRELATED FI (Reinforcem ent by the Production
o f a Stimulus in the Presence o f W hich Food Is Presented Unrelated
to a Response a t the End o f 1 Minute)
W e tried to show w hether the perform ance on the 2nd com ponent on a chained
schedule or the reinforcem ent received under the stimulus appropriate to th at perform-
685
10 MINUTES
Fig. 875.
Chain VIFI with the m agazine opening independently o f a response a t the end
o f the FI
ance m aintains behavior on the 1st com ponent. To this end we changed the proce
d u re on a ch a in e d V IF I so th a t food was presented on an F I 1 schedule b u t w ithout
correlation w ith a response. A t the end of 1 m inute u n d er the 2nd-com ponent color,
the m agazine operated w hether or not the bird was responding. U nder such condi
tions th e rate fell off, b u t usually n ot entirely to zero because of accidental reinforce
ments. Figure 875A and C shows the 1st day u n d er the change. T he F I 1 rate on
the preceding bona fide chain was sim ilar to the starting rate in R ecord C a t a. D u r
ing the session the rate declines m arkedly to the value a t b. R ecord A shows the
responding in the 1st com ponent on this day.
W ithin a few sessions the rate during the last half of each session dropped to a low
value. E ach new session, however, began a t a high rate followed by m arked negative
acceleration. Record D shows a perform ance, after 17 sessions, with this initial neg
ative acceleration. M eanw hile, the 1st com ponent of the chain, where the V I 1 rein
forcem ent is the production of the 2nd color, undergoes no substantial decline, as seen
in R ecord B. T he last excursion in Record B corresponds to the period of very slow
responding in the 2nd color a t c. T herefore, the conditioned reinforcem ent of the
property of the stim ulus for the 2nd schedule w ould ap p e ar to derive from its corre
lation w ith food, regardless of w hether or not the bird is responding in the presence
of the 2nd stim ulus. It should be noted, however, th a t the rate in the 2nd color does
not reach zero in this experim ent. W e could have forced the rate to zero by w ith
holding reinforcem ent every tim e a response occurred.
CHAIN FRcrfdrl
Experiment I
Tw o birds w ith a long history of F R drl described elsewhere were changed to a sched
ule in w hich the drl phase was accom panied by a stim ulus change on the key. As
686
300
RESPONSES
soon as the bird counted o u t the ratio, the key-color changed from red to purple, a n d
the first response which followed a 6-second pause on the purple key was reinforced.
T h e ad d ed stim ulus should eventually p erm it the b ird to ru n the ratio at a high rate,
a n d th en to slow dow n im m ediately for reinforcem ent. T h e resulting schedule is
chain FR (red)crfdrl(purple).
W ith both birds the result suggests some release from the previous drl effect be
cause of the appearance of the purple light.
O ne b ird in p articu lar ten d ed to respond faster w hen the purple light appeared.
T he over-all rate for the 1st day for this bird, in a session lasting 6 hours, did not in
crease as a result of the added stimulus. T ow ard the end of this period, in fact, pauses
as long as 30 m inutes appear. T h e perform ance u n d e r the ratio is essentially th a t
w hich prevailed under F R drl. Segm ent 1 of Fig. 876 shows the perform ance during
approxim ately the 1st hour of the next session. T he tendency to ru n fast w hen the
p u rp le key appears is no longer evident. E arly in the segm ent signs a p p e a r of some
stim ulus control for the drl phase in the fairly sharp breaks, as at a. In the 2nd seg
m ent, acceleration is noticeable; a n d before the end of the period the perform ance is
fairly a p p ro p riate to the new schedule. A t b in Segm ent 5, for exam ple, the ratio is
com pleted a t a fairly high rate after only a b rief startin g pause, a n d the rate then
drops ab ru p tly almost to a value sufficient to achieve im m ediate reinforcem ent. T he
other bird reached this stage m uch m ore rapidly. T h e 1st session on the new sched
ule shows signs of the tendency to accelerate upon the appearance of the new stimulus,
as a t a, b, c, and d in Fig. 877, although the effect is m uch less m arked th an with the
other bird. Otherwise, the perform ance for the first 2 segments resembles th a t under
the form er FR drl. T he acceleration to a new perform ance takes place relatively ra p
idly in the region between e and f
An excellent approxim ation of the final perform
ance under this schedule appears a t g a n d elsewhere.
687
Figure 878A illustrates the most advanced developm ent of a performance on chain
FR crfdrl in the experim ent. H ere, w ith very few exceptions, the ratio is counted o u t,
the light changes color, and in a few responses at a low rate the reinforcem ent is re
ceived. T h e ratio perform ance is now sta n d a rd for this value of the ratio (95) a t this
level of deprivation. T he first bird (R ecord B) did not develop quite so high a rate
on the ratio, a n d obviously occasionally h a d trouble in slowing down after the change
in color. Nevertheless, the over-all rate is m uch higher th an in the case where there
was no stim ulus change at the com pletion of the ratio.
An interm ediate stage of the developm ent of this perform ance for the bird in Fig.
876 an d 878B m ay help reveal the n a tu re of th e scalloping occasionally observed on
FR s. In Fig. 879, 4 segm ents are reproduced from th e end of the 10th session on
chain F R 120 crfdrl 6. Straining in the ratio is evident in the long horizontal segments
of the record; th ere is also difficulty in adjusting to the drl contingency, for exam
ple, a t a, b, c, an d d. Nevertheless, fairly good stimulus control is clear in the sharp
ness w ith which the rate falls after the ratio has been counted out. T he curious feature
of the perform ance is th a t all the ratio runs are smoothly scalloped, suggesting FI r a
th e r th a n F R curves. This factor m ay be due to some inductive effect from the low
rate u n d er the drl contingency. U n d e r this schedule the ratio is reinforced by the
change of color at its completion. T he rate at the m om ent of reinforcem ent with food
is very low. T he ratio perform ance is therefore usually reinforced when the bird is
responding rapidly, but the m ajor reinforcem ent occurs at a low rate. T he tem porary
effect is to produce a fairly smooth curvature during the ratio; this m ay be related to
the chain F R F I schedules reported earlier, w here the F R com ponent in the 1st m em ber
showed low rates and curvature. T he second bird never exhibited this degree of
s,a oQi
Fig. 8 7 8 .
Fig. 879.
Final perform ance on chain FR 95 crfdrl
Chain FR 9 5 crfd rl developing a fte r reinforcem ent (first bird)
689
straining and, in general, showed no positive curvature during the ratio perform ance
a t any stage in the developm ent of the earlier perform ance under FR drl.
Experiment II
A nother bird w ith a long history (442 hours) on F R d rl, w ith occasional tests o f the
effect of rem oving the drl contingency, was then reinforced on a chain F R 8 0 crfd rl 6.
T h e developm ent of the perform ance is illustrated in Fig. 880, where pairs of segments
have been chosen from various sessions throughout the process. Record A shows the
perform ance prevailing a t the end of the experim ent on F R drl. Beginning w ith the
next session the house lights were dim m ed upon com pletion of the ratio. D im
m ing thus served as a stim ulus w hich could control the drl contingency a n d reinforce
the FR . Som e effect begins to a p p e a r in the 3rd h o u r of th e session (R ecord B),
where the breaks at a, b, c, d, and elsewhere m ark the stim ulus change. T he condition
a t the 9th h our in a later session is given a t R ecord C; a t the 18th hour, a t R ecord
D; at the 22nd hour, at Record E; at the 33rd hour, a t R ecord F; and at the 37th hour,
at R ecord G. T he control exercised by the dim m ing of the light develops progres
sively. T he rate falls im m ediately after the ratio is com pleted. However, a good r a
tio fails to em erge u n d er these conditions in the later stages of the experim ent, a n d
the over-all ra te has some tendency to fall off, w ith signs of straining on the F R 8 0
schedule.
Figure 881 shows the final condition under the chained schedule for the com plete
last session. C onsiderable breaking occurs on the ratio, although the term inal rate is
fairly high. As usual, rath e r m arked curvature in the scallop on the ratio is corre
lated w ith breaking. T he stim ulus control is generally adequate although occasion
ally, as a t a, b, a n d c, several responses are m ade before th e criterion of a 6-second
delay is satisfied. A com parison o f this figure with Fig. 878 reveals th at the ratio rate
is here relatively low. This condition m ay be due in p a rt to the fact th a t the d im
m ing of the house lights is not very clear-cut as a stim ulus; b u t it m ay be a general
property of chains in which the 1st m em ber is an FR . (See Chain FRF1 .)
690
The effect o f reducing the difference in stimuli on a chain FR 120 crfdrl
T he stim uli used in an experim ent on chain F R crfdrl already described consisted
of a red 6-watt, 115-volt bulb behind a translucent key. A blue light of the same w att
age was added to this key to produce the purple key specified in th a t experiment.
W e tested the im portance of the difference betw een the 2 stim uli by ad d in g a p o ten
tiom eter to the circuit of the blue light so th a t its relative intensity could be reduced.
W e m ade no effort to analyze these lights accurately. Differences will be expressed in
term s of the resistance of the potentiom eter. In one series, daily runs were m ade
with the potentiom eter set at 800, 1100, 1150, 1250, 1450, an d 1500 ohms, respectively.
691
T h e change in voltage at the blue lam p undoubtedly changed the hue as well as the
brightness. O h m readings were used m ainly to reproduce a given value from day to
day.
T he threshold proved to be near the 1500-ohm setting. Figure 882A is for the
last p art of a session w ith the setting a t 800 ohms. T h e perform ance is not essentially
different from th a t in the earlier experim ent with the full addition of the blue light,
shown in Fig. 878B. O nly by the 3rd day, at 1150 ohms, from which R ecord G repre
sents the term inal p art of the session, is there evidence of a prem ature break at b an d
of a tendency not to respect the drl contingency sufficiently, as a t a. In the following
session, w ith a setting of 1250 ohms, 2 prem atu re breaks occur at c and d; this diffi
culty becomes m ore serious in R ecord E in the following session with a setting of 1400
ohms. At 1500 ohms on the following day (R ecord F), some ratios are ru n off a t a
fairly constant rate, suggesting the perform ance for this bird on F R 70 drl 6 given in Fig.
600. T he m axim um stimulus differences were reinstated a t the arrow in Record G,
an d the original perform ance was quickly recovered. This effect m ight provide a use
ful technique for the investigation of threshold values, w here the actual properties of
the stim uli were carefully controlled an d m easured, an d the m ean tim e taken to reach
the ratio used as a m easure of the effectiveness of the stimulus.
Extinction a fte r chain FRcrfdrl
After the developm ent of the term inal perform ance seen in Fig. 878B, the m ag a
zine was disconnected and the stim ulus allowed to rem ain purple after the com pletion
of the first ratio. Figure 883 shows the resulting extinction, following a short period
on the earlier schedule before the first arrow . Except for short break-throughs of the
692
ratio rate a t a an d b, the rate rem ains app ro p riate to drl u n til c. After c a rate appro
priate to the ratio emerges for about 1000 responses, declining again to a rough grain
at the lower rate. Some instances exist of a retu rn to a higher rate, notably at d, b u t
the over-all decline brings the rate to practically zero by point e. A 40-m inute period on
the same procedure has been om itted, following e, during which only 25 responses were
em itted. T h e purple light was then changed to red (second arrow). T he short burst
a t / d o e s not com plete the ratio. At g, how ever, w hereupon the light changed to
purple a n d a reinforcement was received after a short period of responding a t the drl
rate. T his reinforcem ent reinstated the original schedule quickly, as h and i
show.
T he original F R drl was m aintained for 2 full sessions, and in the following session
extinction was p erm itted to tak e place in the ra tio color (red) alone. Figure 884
illustrates the result. A bout 1000 responses are em itted under the red stimuli before
the rate falls off, sm oothly b u t ra th e r rapidly, at a. After a substantial pause, a brief
period of responding at the ratio rate appears a t b; again it falls off fairly smoothly,
this tim e to a rate roughly ap p ro p riate to drl, although the stim ulus previously corre
late d w ith drl is not present. A fter an o th er period of no responding a t c, the bird
returns again to the ratio rate a t d, a n d the over-all rate ultim ately falls off to a low
value. H ere, then, is an instance in which the rate appropriate to the drl contingency
emerges for some tim e under the stim ulus previously correlated with the ratio part.
After com pletion of the perform ance shown in Fig. 884, the stimulus was changed to
purple, the stim ulus appropriate to the drl contingency. This change reinstates some
responding, in Fig. 885, w hich follows continuously after Fig. 884. In this figure
responding at the ratio rate emerges briefly, although the color is now ap p ro p riate to
the drl contingency. T he rate falls off to a very low value before the end of the p e
riod.
A second bird with the same history was also extinguished in the color appropriate
to drl. Extinction begins after the com pletion of a ratio a t a in Fig. 886. A lthough
the ratio ra te appears briefly as a t b, c, a n d d, th e over-all rate falls off strongly to
C H A IN E D SC H ED U LES
693
practically zero before the point e is reached. A t this point the color was changed to
red (the color previously correlated w ith the ratio com ponent of the chained sched
ule). T h e ratio rate appears for approxim ately 700 responses and then breaks sharply
(at f ) to a low value. R esponding reappears briefly a t a higher rate, as a t g. Be
ginning ab ru p tly at h, a fairly substantial rate declines sm oothly to a low rate at which
only 280 responses were em itted in the final 8.5 hours of the session (not shown in
th e figure).
In another experim ent with the sam e birds, a well-developed perform ance on chain
F R 120 crfdrl 6 was extinguished w hile th e stim uli were changed every 5 m inutes.
Figure 887 shows 2 curves for one bird. In the actual experim ent the segment of R ec
ord A m arked a occurred first. At the dot above the record at a the stimulus changed.
Segm ent b in R ecord B was then recorded, the recorder in R ecord A stopping. A t
th e sm all dot above the record beyond b th e stim ulus was changed to the first setting
again, an d Segm ent c in Record A was recorded. This portion was followed in tu rn
by Segm ent d in B, Segm ent e in A, S e g m e n t/in B, a n d so on. T h e 2 curves are a p
p ro p ria te to th e respective parts of the ch ained schedule. R ecord A is a good ex-
Fig. 88 6 .
E xtinction a fte r chain FRcrfdrl in the d rl color fo llo w e d b y extinction in the FR color
A
5 MIN.
- V I
695
am ple of extinction after FR , w hen responding occurs a t the ratio value or in very
rough grain a t lower values. (Several hours of little or no responding have been
om itted from this figure as m arked.) T h e com plete curve in R ecord B, m eanw hile,
shows only th e ra te ap p ro p riate to th e drl contingency, except possibly for very brief
runs at g and h.
The e ffe c t o f certain drugs upon chain FRdrl
Tw o birds showing a fairly stable perform ance on chain F R 120 erf drl 6, as shown in
Fig. 888A, were given 0.06 m illigram of lysergic acid (LSD ) in 3 cubic centim eters
of w ater, adm inistered orally at the start of the following session (R ecord B). A con
siderable depressive effect appears im m ediately following the adm inistration of the
d ru g a t a. T he 1st ratio is com pleted after about 35 m inutes (at b). Pauses follow
ing reinforcem ent are conspicuously longer throughout the experim ental session.
(This was the only substantial effect of LSD found in a series of 4 adm inistrations of
0.005,0.005, 0.01, a n d 0.06 m illigram , respectively.) Pauses are extended a n d the
ru nning rate is reduced, but the stim ulus control of the drl contingency and the be
havior under th a t contingency rem ain essentially unchanged. T hree doses of lysergic
acid, from 0.005 to 0.02 m illigram , showed no m arked effect in the second bird.
R ecord C illustrates another exam ple of the effect of a suppressing drug on the p e r
formance. After 2 ratios had been ru n off norm ally (not shown), the bird was given
a large dose of H istadyl (an antihistam ine) by m outh in w ater solution, and was re
placed in the apparatus. D uring the next 130 m inutes only 30 responses were emitted.
These occurred in groups of 2 or 3 a n d indicated th a t the bird could respond a t
this tim e. T he perform ance shown in R ecord C then followed. T he first 2 ratios re
q uire about 15 m inutes each, b u t th e drl contingency is soon effective. Some dis
tu rb an ce is evident in the rath er m arked positive curvature in m any of the ratios in
Segment 2, but the perform ance is essentially norm al before the end of Segment 3 a t
the end of the experim ental session. (These experim ents are offered sim ply to show
the character of records influenced by depressant drugs.)
A C O M P A R IS O N OF THE R EINFORCING EFFECTS OF THE T W O
C O N TR O LLIN G STIMULI IN A MULT FIFR
In this experim ent the ap p aratu s contained 2 keys, a n d 2 colors were possible on
each. O n the right key, red and blue eventually control the ratio a n d interval m em
bers of a m ult F IF R , respectively. O n the left key the schedule was always V I 3.
R einforcem ent on this key was the presentation of one or the other of the colors on the
right key. W hen the left key was white, the right key was unlighted; but a response
to the left key w ould produce, on V I 3, a red key on the right. In the presence of the
red key a response was then reinforced on FR . T h e left key was unlighted during
the FR. W hen the left key was green, a response would, on V I 3, change the right key
from unlighted to blue, a response on the blue key being reinforced on FI 10. F our
perform ances were recorded: (1) the V I perform ance on the w hite left key; (2) the V I
696
perform ance on the green left key; (3) the F I perform ance on the blue right key; and
(4) the ratio perform ance on the red right key. For convenience, the latter 2 perform
ances were recorded in succession on a single recorder.
T he two V I rates on the different colors on the left key provide direct measures of the
reinforcing effects of the 2 controlling stim uli on the right key. In an actual experi
m ent, for exam ple, the session m ight begin with the left key white and the right key u n
lighted. R esponding on the white key would, after a variable interval, produce a red
light on the right key (the white light im m ediately going out on the left key). T he bird
would then respond on the red key and be reinforced upon com pleting 50 responses.
T h e red light w ould then go out, a n d the left key w ould be either white or green. In
either case the bird would respond on the left key. I f the key h ad become green, a
response would, after a variable interval, produce blue on the right, and the green key
RIGHT KEY
LE FT KEY
R >
RIGHT KEY LIGHT
R ----- > MAGAZINE
WHITE : V I ---------- --------- > RED : FR
GREEN : V I ----------- -------- > BLUE : FI

Fig. 8 8 9 .
Table o f contingencies
w ould darken. Responding on the blue key would, after 10 m inutes, produce a re
inforcem ent. Figure 889 represents the contingencies.
T h e 2 birds used h a d h a d a n extended history on F R . T hey were placed in a
new ap p aratu s and put directly on m ult F R F I. T he novel stim ulus in the apparatus
h ad, as usual, an effect sim ilar to a perform ance im m ediately after erf. This was
show n for 1 b ird in Fig. 627, in C h a p te r T en, w here the developm ent of the m ultiple
schedule was described.
T he 2 birds were then changed to the m ultiple-chain procedure just described. Fig
ure 890 describes a well-developed perform ance under the m ult chain V I 3 FI 10 chain
V I 3 F R 50 after 12 sessions. Record A shows the m ultiple performance on the right
key w ith interval scallops at a, c, and elsewhere and ratio performances a t b, d, and else
where. Note th a t this is a relatively low ratio rate. T h e figure also shows vertical
portions of th e curve, as a t e , f a n d elsewhere, w hich represent responding to the u n
lighted right key. Since these always precede ratio performances, they indicate th at
697
w hen the bird is responding on the w hite key a t the left (one such response producing
the red key on the right), there is some tendency to move to the right key before the
red light appears. This tendency corresponds w ith the higher rate shown on V I 3 on
the w hite left key. R ecord B shows the separately recorded V I 3 perform ance on
the white key, reinforced by the appearance of the red key on the right. A substantial
slope is m aintained throughout the session. Record C, on the other hand, shows the
very low ra te on th e left green key w hen a response is reinforced occasionally by the
production of the blue key on the right. (This p a rt of the m ultiple schedule is u n
duly long, because the periods of no responding lengthened the intervals appearing on
the V I 3 schedule. T he segment shown is only p art of the session, and was followed
by 2 hours in which only 70 responses occurred.) Roughly speaking, the bird responds
a t a substantial rate to produce the stim ulus for F R 50 b u t scarcely a t all to produce
th e stim ulus for FI 10.
W e tested the result by reversing the reinforcing conditions on the left key: where
a response to the white key had previously produced a red key on the right, it now p ro
duces a blue; and where a response to the green key on the left had previously produced
blue on the right, it now produces red. T he effect is alm ost im m ediate.
Figure 891 A, before the arrow , shows responding on the green (pre-FI) key. R ec
ord B, before the arrow, indicates responding on the white (pre-FR ) key. T he rates
before the arrow s are of the sam e order as those in Fig. 890B an d C. R ecord C gives
the perform ance on the right key of both colors on one record. At the arrows the con
tingencies were reversed. R ecord A now shows responding on the white key and R ec
ord B on the green. T he perform ance a t R ecord A declines slowly during the session ;
th at on R ecord B rises somewhat m ore rapidly to a substantial rate. A complete re
versal of rate on the colors on the left key resulted from the reversal of the reinforcing
contingencies in the change to the rig h t key. (T he num bers set opposite the seg
m ents indicate the actual recorder order of segments in Fig. 891 for p a rt of the session.)
699
O n the following day the rate on the w hite key showed some recovery from the low
value at the end of R ecord A, b u t declined quickly to a very low value for the balance
of the session. T he rate on the other color of the left key rem ained substantial.
Figure 892 shows a final perform ance on the reversed contingencies. T he V I p e r
form ance on the pre-ratio color appears at R ecord B, a n d on the pre-interval color a t
R ecord C. R ecord A contains p a rt of the m ultiple perform ance on the right key.
T he rates on all colors are slightly higher th an those in Fig. 890. T he ratios in R ecord
A now show pauses, as at b, and small knees, as at a. N ote th a t the tendency to re
spond on the right key before it is lighted has disappeared.
T h e length o f the fixed interval was th en reduced slowly u ntil the perform ance
was practically indistinguishable from th a t on fixed ratio on the m ultiple schedule.
T h e actual reinforcem ent with food is, of course, brought closer to the presentation of
the ap p ro p riate controlling key-color. T he rate im m ediately after the appearance
of the key-color is also changed. Both of these m ay affect the reinforcing properties of
the colors. Figures 893 and 894 indicate changes in the m ultiple schedule through
these various stages. Figure 893A shows a sam ple of the perform ance when the fixed
interval is first reduced to 5 m inutes, the fixed ratio rem aining 50. (The session fol
lows im m ediately after Fig. 892.) P artly because of the first shortening of the interval,
the term inal rate in the interval is reduced relative to the ratio rate in the m ultiple
schedule; the rate a t a, for exam ple, is m uch lower th an th a t at b, c, and d. In R ecord
B (3 sessions later) th e interval was fu rth e r shortened to 4 m inutes. This change
appears to have an effect upon the ratio, an d the term inal rates are scarcely different
at this, stage. T he interval was then further decreased to 3 minutes; Record C shows
a sam ple of a perform ance from the 3rd session. H ere, for the most part, the term inal
rates are indistinguishable from the ratio rates, although some pausing and curvature
are still present in the early p a rt of the intervalfor exam ple, a t f g, and h. T h e ver
700
tical line a t e a n d elsewhere shows responding on the key before the appearance of
th e color controlling the interval schedule. T he earlier instances of this w hen the FI
was 10 m inutes h ad preceded the ratio schedule only. R ecord D, p a rt of the 5th
session on F I 3, shows somewhat m ore m arked scalloping. Record E gives the 6th ses
sion on FI 3. Here, m arked scallops have returned, and the term inal rates are again
lower th a n the ratio rates. (T he rates at i a n d k are substantially lower th an those at
j and I.) W hen the FI is reduced to 2 m inutes, distinguishing between the interval and
ratio performances becomes difficult, as Record F shows.
Records A an d B in Fig. 894 show the 1st a n d 17th sessions on F I 1. In R ecord B
no distinguishable difference exists betw een interval an d ratio perform ances. L ater
sets of values were as follows: FI 2 F R 5 0 (R ecord C); FI 3 F R 5 0 (R ecord D ); F I 5 F R

50 (R ecord E); an d FI 5 F R 100 (R ecord F). T h e F R 100 perform ance in R ecord F
is inferior to th at at F R 75 in Fig. 893A.
C oncurrently with these changes in the m ultiple schedule, changes appeared in the
perform ances on the left key. Figure 895 shows various conditions on the color on
w hich a response was reinforced w ith the presentation of the blue (interval) key on
the right side. (Note th a t these are not in the order in which they occurred.) R ec
ord I is a sam ple of an early perform ance shortly after R ecord C in Fig. 892. T he re
inforcing schedule is FI 5. R ecord H shows a n exam ple of a b reak-through (at a)
a t a high rate produced by th e drop to F I 3 as the reinforcing schedule. In Records
G a n d F the reinforcing schedule was FI 5. W hen the interval was reduced to 1 m in
ute, the rate on the pre-interval key rose. R ecord E shows the performances shortly
after Fig. 894A, with the FI set at 1. A later perform ance under FI 1 appears at A at a
stage between Curves A and B in Fig. 894. R ecord C, with FI 3, follows Record D,
701
Fig. 8 9 5 .
Pre-interval VI perform ance d u r
in g changes in the size o f the FI in the m ult
FIFR com ponent o f the m ultiple chain
with FI 1. R ecord B shows a som ew hat later stage, falling betw een Records D and E
in Fig. 894.
T he perform ance on the pre-ratio key, m eanw hile, also undergoes some change b e
cause of the m odification in the size of the interval. Figure 896 gives samples. R e c
ord A corresponds to Fig. 892B. R ecord B shows the first sign of a disturbance on
FI 2; this is for the 2nd session before F in Fig. 893. F urther irregularity develops in
R ecord C on F I 1 for the 4th session of this schedule; an d in R ecord D, for the 8th.
E arly perform ances recover w hen the interval is again increased to 3 m inutes, as R ec
ord E indicates. L ater, w ith the interval holding a t 5, a slight tendency for nega-
702
Fig. 8 9 6 .
Pre-ratio perform ance during changes in the size o f the FI in the mult FIFR
com ponent o f the m ultiple chain
tive curvature develops and becomes characteristic of the perform ance. This appears
in Record F for the 15th session after the retu rn to FI 5, and corresponds to a session
a few days before F in Fig. 894. This curvature m ay be nothing m ore th an the even
tual adjustm ent to the V I schedule on the pre-ratio (and pre-interval) key.
Chapter Thirteen
CONCURRENT SCHEDULES
CONCURRENT SCHEDULES O N T W O KEYS
In s o m e e x p l o r a t o r y e x p e r i m e n t s , pigeons were given access to 2 keys on the same

wall of the experim ental cham ber. A single m agazine could be arranged to reinforce
a response to either key. A separate program m ing circuit controlled reinforcem ent
on each key. Separate recorders were also used.
C oncurrent VIVI
Figure 897 shows concurrent perform ances u n d e r V I on the 2 keys. A slightly

higher ra te appears on the rig h t key in R ecord B. O therw ise, the perform ances are
characteristic of V I.
Figure 898A and B show sim ilar perform ances for the same bird on a later session.
A slightly higher rate now appears on the left key at R ecord A. R ecord C shows a
703
Fig. 898.
Transition from concurrent VI 1 VI 1 to VI 1 on a single key
C O N C U R R E N T SCH ED U LES
705
record taken on the following session, w hen the left key was covered. Reinforcem ents
on the right key were program m ed by both circuits and, of course, occurred twice as
frequently as before. T he over-all rate is approxim ately twice th a t shown on the
same key for the preceding session.
T he experim ent was continued w ith the left key covered for 11 sessions. W hen both
keys were m ade available at the sam e tim e, the perform ances in Fig. 898A an d B were
recovered, as Fig. 899 shows.
C oncurrent FRFI
In a further experim ent the schedule on the left key was F R 125; and th a t on the
right, an F I 8. T he value of the ratio was varied from tim e to time. Figure 900 gives
an early exam ple of the performances. Record A shows the performance on FI 8 on
the right key, R ecord B the perform ance on F R 125 on the left key. No scalloping a p
propriate to the interval has developed, and the negative bursts of responding are the
inversion of the positive accelerations on the ratio perform ance in Record B. In other
words, the bird responds on the interval schedule w hen not responding on the ratio,
but the ratio perform ance dom inates. This perform ance is generally successful in
achieving the interval reinforcem ent w hen it is due, a n d perm its the anim al to exe
cute a fair F R 125 perform ance. T h e com peting key seems to emphasize a smooth
acceleration on the ratio schedule. R ecord C shows a recording of all responses on
both keys. O nly the last 5 excursions in the session are shown. T he over-all perform
ance is roughly linear, with occasional scalloping, m uch of which is probably due to
the ratio schedule. If so, responding on the interval key does not m ake up for the
pauses after reinforcem ent on the fixed-ratio schedule. Com plete compensation is not
to be expected, however, since the interval contingency on the right key causes a lower
rate th an the ratio on the left.
706
Figure 901 illustrates a m uch later perform ance w ith reduced F R on the left key
against F I 5 on the right key. H ere, the ratio perform ance a t B is m uch squarer, a n d
shows a fairly norm al tendency to fluctuate between periods of long and short pauses.
T he behavior on the interval key varies inversely. W hen the rate in R ecord B is high
in the region of a, it is low in R ecord A at b. W hen the rate is low at c in Record B,
it is high a t d in Record A. After the rate returns again to a reasonably high value in
R ecord B a t e, it falls sharply to a low over-all value a t f
T he perform ance is suc
cessful in achieving the interval reinforcem ents alm ost as soon as scheduled, a n d p e r
mits the ratio perform ance to proceed practically w ithout interference from the other
key. T h e finer step-wise grain of the interval curve is also, of course, the reciprocal of
the ratio curve.
A second bird in the transition to F R on one key a n d FI on the other showed, at an
early date, a good ratio perform ance on F R 35 and an F I perform ance illustrating the
general features of Fig. 900 and 901. R esponding on the interval occurred during
pauses on the m ore powerful ratio schedule.
W hen the ratio was strained, however, by greatly increasing it, the ratio character
of the record was lost. Some evidence exists of a tem porary tendency for the FI sched
ule to affect both keys, w ith higher rates on the F R key d u ring the scallop of the short
FI. In Fig. 902A, for exam ple (for the second bird in the experim ent), fairly good
scalloping occurs on the FI 2 key. R esponding on the larger ratio key (the ratio is
now 245) occurs when the bird is not responding on the interval key (B). W hen the
ratio was increased to 385 (Fig. 902C and D), the record occasionally showed examples
of sm ooth, inverted interval curves on the ratio key, as at a. (In this figure the left-
C O N C U R R E N T SCH EDULES
707
key reinforcem ents are also m arked on the right key. T h e interval schedule perm its
the reader to determ ine which of these were on the right key.)
This bird never showed a good step-wise break on a 5-m inute F I except w hen the
ratio on the other key was quite sm all F R 25. By varying the size of the ratio, how
ever, we could reproduce these general features of concurrent F IF R on 2 keys. Fig
ure 903 shows a late perform ance w ith F R 75 on the left key a n d F I 5 on the right key.
T he record on the left key has been reproduced in the recorded form. T he 3 seg
m ents are continuous. Segments from the record on the right key have been cut an d
reassem bled in order to indicate sim ultaneous features of the 2 curves. At a a rein
forcem ent on the interval schedule produces a pause on both keys, at b and c. In gen
eral, the pauses in the FI curves correspond to periods of high rates of responding on
the ratio curve. Instances have been m arked by the connecting lines. T hus, re-
708
sponding on the interval curve (the lower curve) occurs w hen the ratio schedule is not
com m anding a high rate. Note th a t this bird does not develop a good term inal rate
on the ratio com parable w ith th a t in Fig. 901.
Figure 904 illustrates a late developm ent of a sim ilar perform ance for a third bird
in the sam e experim ent, w ith F R 150 on the left key a n d F I 5 on the right. T h e F R
perform ance in the u p p er curve is alm ost sta n d a rd for th a t schedule. S ubstantial
responding occurs on the other key during the pauses in the ratio perform ance, partic
ularly as these develop during the experim ental session. W here the F R schedule
shows only small pauses at a, the rate on the FI key is negligible, as a t b. W here the
Fig. 9 0 4 .
C oncurrent FI 5 FR 150
pauses have become prolonged in the latter p art of the period, as at c, very sustained re
sponding occurs on the FI key a t d. A few corresponding runs and pauses in the 2
records have been connected by lines.
A fourth bird was p u t on several settings of the 2 schedules w ithout m uch effect.
It tended to show performances appropriate to a variable schedule on both keys. Late
in the experim ent, however, w ell-m arked scallops began to ap p ear on FI 2. O n a
large ratio of 385 concurrently on the other schedule, th e perform ance closely resem
bled a V I schedule except th a t some trace of the curvature in the 2-m inute scallop
ing appeared in the reverse form, as in Fig. 902C a n d D. L ater sessions showed an
even greater sim ilarity to the behavior of the pigeon in the figure.
709
C oncurrent VIFI
Tw o birds on a concurrent V I 2 F I 2 on 2 keys failed to develop m arked scalloping

on FI, although some traces are clear in the later stages in the experim ent. In Fig.
905, sighting along th e lower record will reveal scalloping on F I 2. H ow ever, scal
lops in progress are often broken shortly before reinforcem ent by the occurrence of
reinforcem ents on the other key. Lines have been draw n to connect 4 such instances.
U nder these conditions the bird responds approxim ately w ith a pattern of 2 responses
on the left key, 1 on the right, 2 on the left, 1 on the right, etc. In general, pauses oc
cur on the right key (lower record) after reinforcem ent, b u t slight runs occur on the
V I key after reinforcem ent. T he perform ance on the V I key is essentially linear, and
th e bird does not ap p e ar to show an y tendency to respond m ore often on th a t key
while periods of low responding are in effect on the FI key. This performance is th ere
fore not com parable w ith th a t in w hich a n F R schedule on one key controls rap id
responding on the other during the F I pause.
CONCURRENT SCHEDULES O N A SINGLE KEY
C oncurrent FI 10 a vo id RS 3 0 SS 3 0
T hree rats w ith a history of m ult FI 1 0 a v o id R S 3 0 S S 3 0 were reinforced on F I 10

alone for 34 hours. T he general type of perform ance seen in Fig. 906 for a full ses
sion appeared. In general, good pausing and a high term inal rate occur. An occa
sional period of low responding appears and, at a, one instance of postponing the rein
forcem ent beyond the 10-m inute interval. T he avoidance (R S 30 SS 30) schedule
was reinstated in the following session, the stimulus situation being the same as on FI 10
710
in the earlier history. Tw o schedules F I 10 a n d avoidance were now being p ro

gram m ed a t the sam e tim e but independently. Figure 907 gives a n early stage of the
resulting performance. A few shocks reinstate the fully developed avoidance behav
ior, although they now occur in the presence of the stim ulus which h a d appeared only
d u rin g the FI 10. A typical avoidance perform ance a t a fairly low, steady rate re
places the long pauses a t the beginning of the intervals. T his rate m ay have some
effect in encouraging knees, but these tended to disappear in later sessions. T he te r
m inal rate rem ains approxim ately the same. A final perform ance under this schedule
appears in Fig. 908, w here the whole session is represented. T he sustained avoidance
rate is pointed u p in a few cases by lines w hich estim ate the slope. O ccasional knees
appear. D uring the session the rat received 22 shocks for failing to m aintain the basic
avoidance rate.
W hen the avoidance contingency was changed to R S 15 SS 15, the slope in the early
p a rt of the interval rem ained about the sam e. Figure 909 gives the 3rd session on
this schedule. T he interval perform ance is som ew hat m ore orderly, possibly because
711
of longer exposure to the schedule, as well as ad aptation of em otional responses to

shock. In the session following Fig. 909 the avoidance contingency was changed
fu rth er to R S 7 SS 7. This change was m ade at the arrow in Fig. 910. T h e effect is
an im m ediate increase in the basic avoidance rate at a which is m aintained th rough
o ut the interval. Acceleration to the usual term inal rate is missing. Acceleration is
lacking elsewhere, as at b an d c. At d a late acceleration reaches som ething less th an
th e usual term inal rate before reinforcem ent. This developm ent m ay be due to con
ditioned suppressing properties of the shocks, which are now received m ore frequently.
L ater, interval segments uniform ly ended a t a high term inal rate. (See the begin
ning of Fig. 912.)
T h e suppression of the scallop a p p ro p riate to F I 10 noticeable in some instances in
Fig. 910 was m uch m ore m arked in a second rat. H ere, the change to the RS 7 SS 7
contingency was m ade at the beginning of the experim ental session. Figure 911A
gives a sam ple of the preceding day on R S 15 SS 15. T h e com plete following session
a t R S 7 SS 7 ap p ears in R ecord B. A n acceleration does not b reak th ro u g h until
very late in the session, at a, near the end of the interval. Thereafter, some suggestion
o f interval curvature appears for the balance of the period. This rat showed sim ilar
Transition from concurrent FI 10 avoid RS 15 SS 15 to concurrent

FI 10 avoid RS 7 SS 7
Fig. 9 1 1 .
Transition from concurrent FI 10 avoid RS 15 SS 1 5 to concurrent

300
RESPONSES
Fig. 9 1 0 .
Fig. 9 1 2 .
Transition from concurrent FI 10 avoid RS 7 SS 7 to concurrent

713
periods of no positive curvature during the 3 other sessions a t which the schedule re
m ained a t R S 7 SS 7. U pon retu rn in g to R S 15 SS 15, th e F I 10 behavior was re
stored.
W hen the contingency was restored from RS 7 SS 7 to R S 15 SS 15 for the first ra t
discussed above, at the arrow in Fig. 912, the rate decreased alm ost im m ediately in the
early p art of the interval. C om pare the slopes a t a, b, and c, for example, with those a t
d an d e.
A m u ltip le schedule in w hich one stim ulus controls a concurrent
FI 10 a v o id RS 3 0 SS 3 0 w hile a n o th e r controls FI 10
Tw o rats from the experim ent on concurrent FI 10 avoid R S 30S S 30 continued to

be reinforced on th a t schedule in the presence of a buzzer. In the absence of the buzzer
Fig. 91 3.
Early perform ance on m ult FI 10 concurrent FI 10 avoid RS 3 0 SS 30
they were reinforced on FI 10 alone. Figure 913, for the 5th session, shows an early d e
velopm ent of a m ultiple performance. Responses on F I 10 ap p ear in R ecord A , while
those in the absence of the buzzer on the concurrent schedule ap p ear in R ecord B.
T he 2 stim uli altern ated in a simple fashion a t reinforcem ent. T he concurrent FI 10
avoid R S 30 SS 30 evidently has the effect previously described (at Record B), while the
m ultiple controlled FI 10 is fairly norm al (at R ecord A).
These characteristics are even m ore m arked in the later developm ent evident in Fig.
914, after 31 sessions on the m ultiple schedule. (A block of 5 of these sessions was
devoted to extinction.) Note the long pauses under the stim ulus controlling the F I
perform ance in Record A, and the parallel segments of the typical avoidance rate a t the
beginning of each interval in R ecord B. In Fig. 913 the term inal rate in the con
current case was somewhat higher th an th a t in the straight FI. This difference is even
m ore ap p aren t in Fig. 914, where the term inal rates in R ecord A are of the order of
1.5 responses per second, b u t in R ecord B they are of the ord er of 2 responses per sec
ond. A slight roughness of grain is evident in the term inal straight FI rate.
714
W h en th e avoidance contingency was o m itted so th a t both stim uli now controlled

FI 10, the avoidance perform ance slowly disappeared u n d er the stim ulus previously
controlling it. Figure 915 gives a sam ple of the perform ance after 15 sessions. R ec
ord A shows the perform ance with the buzzer off on FI 10; and Record B, the a lte rn a t
ing perform ance w ith the buzzer on a t FI 10. If the shocking circuit had been
connected d u rin g R ecord B, the ra t w ould have received ap p ro x im ately 194 shocks
d u rin g th e period. In th e earlier perform ance on the form er m ultiple schedule, in
cluding the avoidance schedule, the rat received only 2 shocks in m aintaining the basic
avoidance rate during the session shown in Fig. 914.
Figure 916 is a plot of the shocks actually received by both rats on the concurrent
schedule together w ith the n um ber of shocks they w ould have received after the avoid
ance contingency was removed. T he first 7 points show the num ber of shocks a c tu
ally received on the concurrent schedule. T he points thereafter are the n um ber th at
w ould have been received if the shocking circuit h ad been connected. T he num ber
rises significantly only after 4 sessions, b u t eventually reaches a high value a t the stage
show n in Fig. 915. W hen the shock was again introduced, there was a very rapid
retu rn to the previous concurrent schedule, as the 3 last points in Fig. 916 indicate.
Fig. 9 1 5 .
Fig. 9 1 6 .
M ult Fl 10 Fl 10 a fte r mult Fl 10 concurrent Fl 10 avoid RS 3 0 SS 30
Plot o f the number o f 30-sec pauses during the concurrent Flavoid perform ance
716
Fig. 91 7.
Extinction a fte r mult FI 10 concurrent FI 10 avoid RS 3 0 SS 30 by allo w in g food

m agazine to operate em pty
T he extinction m entioned in connection w ith Fig. 914 occurred about halfway b e

tw een th a t figure and Fig. 913. T he m agazine feed jam m ed, so th at the m agazine
operated with the usual auditory stimuli, but no food appeared in the trough. T he re
sult was the extinction of the interval scallop an d the survival of the avoidance p e r
form ance. Figure 917 gives the perform ance in the 5th session. O ne FI scallop at a
in R ecord B shows surviving control of stimuli a t start of the session.
T h e balance of R ecord B is characteristic of the avoidance schedule. A lternating
segments under the other stim ulus (R ecord A no buzzer) show only a low trickle of
responses, a n d m any reinforcem ents are postponed.
The e ffe c t o f ch lo rp ro m a z in e and p e n to b a rb ita l on
a m u ltip le FI 10 (concurrent FI 10 a v o id RS 3 0 SS 30)
Figure 918A illustrates the perform ance of one rat on the schedule ju st described
under the stimulus controlling the concurrent p art of the schedule. (The perform ance
on F I alone is not shown.) This is a perform ance sim ilar to th a t of Fig. 914. In
th e following session 2.5 m illigram s of chlorprom azine was injected intraperitoneally
im m ediately before the start of the session. T h e effect was the elim ination of prac-
Fl 10 concurrent FI 10 avoid RS 30 SS 30
717
tically all activity for some tim e. R ecord B in the figure is for responding u n d e r the
concurrent FI avoidance com ponent. R ecord C is the altern atin g perform ance on
FI alone. Responses occur on both com ponents sufficiently often to produce the a l
ternation of stim uli. For exam ple, at a in R ecord B a first response changes the stim
ulus to th a t app ro p riate to FI alone; an d a som ew hat delayed response at b in Record
C changes the stim ulus back. In each case a pellet is produced as reinforcem ent.
T he pellets were eaten, and observation of the rat showed it to be awake, alert, b u t
otherwise not responding. Shocks were received on the avoidance schedule whenever
the rate fell below the critical point, as it clearly did during the first 2 segments of R ec
ord B. After 3 or 4 intervals, however (that is, after about 40 m inutes), the avoidance
perform ance is reinstated, beginning in m arked form approxim ately at c. This con
tinues for the balance of the session. V ery little responding occurred on the F I alone,
however, one interval being extended to almost 2 \ hours, w hen a response occurred
(at d) and the schedule was reversed. W hen the schedule later returned to the FI stim
ulus, a trickle of responses em erged in the interval a t e. In no case does the o rd i
nary FI 10 perform ance under food reinforcem ent appear. Evidently the drug greatly
718
reduces the behavior appropriate to the food reinforcem ent while leaving a substan
tial level of avoidance behavior.
In another session a sm aller dose of the drug was given 1 m illigram at the start of
the session. Figure 919 illustrates the performances. In Record A, 10 intervals u n
der the concurrent schedule show a ra th e r rough avoidance perform ance w ith no evi
dence of the interval scallop; a n d a t R ecord B the initial 9 intervals on F I 10 only also
show a very low rate. A fairly sta n d a rd interval scallop appears beginning w ith the
avoidance rate at a, although the rate falls before reinforcem ent is received. O n the
F I 10 schedule a single interval perform ance emerges a t b. T he rate then declines.
Several examples of interval curvature appear, however, in com bination with the
avoidance contingency on the concurrent schedule. In this experim ent the avoid
ance contingency is essentially untouched, although the perform ance becomes some
w hat m ore irregular u n d er the adm inistration of the drug. T he F I 10 perform ance
is rem oved for the first 3 or 4 hours, and makes only an irregular appearance thereafter.
A 1-m illigram dose of the same drug given to the second ra t produced an even m ore
m arked effect in rem oving the interval scallop from the concurrent schedule. O nly 3
slight traces of this appear in the 3rd hour of the session. O n the concurrent FI sched
ule alone, 2 fairly substantial interval scallops a p p e ar near the end of the 3rd hour.
L ater, an injection of 2 m illigram s of sodium pentobarbital h ad little effect upon the
perform ance on either com ponent schedule. T h e ra t h ad shown a sustained, fairly
stable condition on FI 10, in which pauses were occurring after all reinforcem ents, b u t
the durations of the pauses covered a fairly narrow range.
AN
UNCLASSIFIED SCHEDULE
An unclassified complex schedule had the following properties. T he bird was re

inforced upon the com pletion of 160 responses as in a simple FR . If the bird paused
for 2 m inutes after reinforcem ent, however, it was reinforced for the 1st response.
T hus, if the ratio began to strain so th a t th e pause after reinforcem ent exceeded 2
m inutes, reinforcem ent occurred on F R 1. T his has the same effect as an adjusting
schedule: straining is corrected by addition al reinforcem ents, not (as in the adjusting
case) by a slight change in the size of the ratio b u t by a tem porary change in the m ean
ratio. Figure 920 illustrates a series of sessions on such a schedule. R ecord A shows
a sam ple of the behavior under F R 160. T he effect of reinforcing the 1st response if
the pause after reinforcem ent exceeds 2 m inutes is alm ost im m ediate. U nfortunately,
the 1st record was poorly m ade, b u t R ecord B gives a p a rt of the 2nd session. By
this tim e a few reinforcem ents on c rfd rl 2 m in have led to responding soon after re
inforcem ent, although the term inal rate in the ratio is not im m ediately reached. A
single instance in which a response is reinforced because of a pause greater th an 2 m in
utes appears at a.
O ne of the first effects seems to be to encourage a low rate after reinforcement.
R ecord C shows a segm ent of a session following R ecord B in w hich the final com ple
tion of the ratio is m uch disturbed, and a scattering of responses fills in the interval.
719
N ote th a t a response after a pause of 2 m inutes is reinforced only w hen it is the first re
sponse after reinforcem ent. Several responses after pauses greater th an 2 m inutes
occur in R ecord C, b u t no single response is reinforced a t this time. T he following
session shows recovery; an d the spacing of reinforcem ents is now of the same order as
those in R ecord A, b u t the pauses are now filled w ith a few scattered responses.
R ecord E, a sam ple for the session following R ecord D, shows ra th e r prolonged re
sponding at a low rate. T he first response after reinforcem ent always occurs too soon
R's/SEC
MMW
A lii)
[M ill
UUU
H1J
An
JL i
ilU uL
Ir n M
Fig. 9 2 0 .
A n unclassified adjusting schedule
to be reinforced on F R 1. Record F is for the session following R ecord E. A nother

exam ple of a single response reinforced after a pause of 2 m inutes appears a t b.
A nother exam ple, shown in R ecord G at c, is followed by a m om entary interm ediate
rate of responding before the term inal rate is reached. In R ecord H , for the session
following R ecord G, the ratios are spaced out roughly as in R ecord A, b u t no pause
after reinforcem ent exceeds 2 m inutes. R ecord I at g gives a single exam ple of a rein
forcement of 1 response. All other cases have a t least one other response w ithin the re
quired 2 m inutes. T he final perform ance observed in this experim ent is represented
by Record J , w here the ratios are separated by substantial periods of time, but a few
720
responses follow soon after reinforcem ent a n d fail to satisfy the drl contingency in
cluded in the schedule.
T his experim ent appears to dem onstrate a successful technique for preventing the
ultim ate extinction of a ratio which cannot otherwise be sustained by the organism.
ADJUSTING SCHEDULES
O rganism s differ in the m axim al size of ratio u n d e r w hich they will reach a n d sus
tain an F R perform ance. These differences are a function of deprivation level,
health, general reactivity of the organism , an d so on. T h e history through which the
m axim al ratio is approached is also im portant. W e investigated a technique for d e
term in in g the m axim al ratio a b ird can hold by ap p roaching the m axim al ratio with
the benefit of a n atu ral correction of the advance in size of ratio. This procedure was
designed to avoid the usual arbitrary program of advancing the ratio. T he resulting
program is an exam ple of a general type which m ay be called an adjusting schedule.
T he special case of an adjusting ratio m ay be defined as a schedule in which the
nu m b er of responses em itted before reinforcem ent is changed progressively in term s of
some characteristic of the behavior of the bird during the preceding ratio. T he bird
begins at some value of the ratio which it can conveniently hold. This value is then
Fig. 9 2 1 .
A djusting FR
C O N C U R R E N T SCH EDULES
721
increased in term s of its perform ance. If any straining appears, the size of the ratio
is autom atically decreased. T he basis for increasing or decreasing the ratio chosen in
this experim ent was the pause after the reinforcem ent. O th e r possibilities m ight be
the total tim e to complete the ratio, the tim e to complete some fraction of the ratio, a n d
so on. Note th a t the adjustm ent m ade a t each ratio affects other ratios. Adjusting
schedules therefore differ sharply from interlocking schedules, w hen a stan d ard set of
specifications is resum ed at each reinforcement.
W e studied 3 birds on an adjusting ratio schedule for approxim ately 300 sessions;
during this tim e, we varied the criteria for changing the ratio and m ade incidental v a r
iations in the deprivation level. W ith the techniques an d criteria employed, the
m axim al ratios sustained by the 3 birds w ithout prolonged pausing were approxim ately
445, 600, a n d 650.
Figure 921 gives an exam ple of the way in which the adjustm ent perm its the bird to
set its ow n ratio appropriate to the conditions of deprivation, health, etc. T he c ri
terion of adjustm ent a t this tim e is a 25-second pause. D uring the first 25 seconds the
ratio decreases slowly unless the bird responds. If a response occurs, the ratio is in
creased by 5 responses. T he session begins with a badly strained ratio at a, b, and c.
At this point the ratio has the value of approxim ately 270. Because of the breaking,
the ratio was autom atically adjusted dow nw ard, an d reaches about 235 a t d. Still
some pausing an d curvature appear at this point, and a further reduction follows. By
e the ratio has been reduced to about 200, and here the perform ance holds well. A
slight pause results; b u t it is fairly stable, a n d the ad justm ent is therefore reversed.
T h e ratio is now slightly increased a t each reinforcem ent. By f it has retu rn ed to
about 250 responses, some pausing reappears, a n d the ratio is adjusted dow nw ard, a p
proaching 200 by g.
T he experim ent shows th a t the pause after reinforcem ent is an im portant property
of the ratio perform ance. T he adjusting ratio has some of the self-corrective c h a ra c
teristics of interval schedules, although the criteria must lie w ithin a fairly narrow range
to protect the behavior from extrem e strain.

Schedules of Reinforcement

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Schedules of Reinforcement

Загружено:

Авторское право:

Доступные форматы

Chapter One

T w o NONiNTERM iTTENT s c h e d u l e s of reinforcem ent are:

between reinforcements.) For example, a fixed ratio is increased or decreased by a

A complex program m ay be composed of two or m ore of these schedules arranged

A m ong th e in d ep en d en t variables w hich m odify this ra te or probability are some

th e experim ent is ru n continuously, it m ay be argued th a t some instances of the se

th e present point: an ap p aratu s which differentially reinforces rate of responding m ay

W hen an organism is first reinforced on a given schedule, the actu al conditions

Assum ptions about conditions at the tim e of reinforcem ent m ay be checked by an

A reasonably satisfactory form ulation of schedules of reinforcem ent em erged slowly

A t y p i c a l e x p e r i m e n t a l b o x is m ade from a picnic icebox approxim ately 30 by 12

ments. This circuit is designed to program an d record a fixed-interval schedule of

D iagram o f a typical circuit

Block d ia g ra m o f a circuit fo r mult FIFR

SCHEDULES OF REINFORCEM ENT

EACH RESPONSE MOVES

Photograph o f a cum ulative recorder

tion is achieved by o perating th e coil for th e first in sta n t of operation a t a voltage

T he records for a given bird during an experim ent are pieced

1 T h e latest m odel supplied by G erb ran d s has this resetting feature.

P reparation o f a telescoped figure

duced. A lthough the actual record is stepwise, a single constant slope m ay be

experim ent are considered, it m ay be necessary to distinguish between three rates.

Cum ulative curves showing v a ri

special aspect of the d ep en d en t variable is of p a rtic u la r interest. R ecording all be

Percent body-w eight is a useful m ethod of controlling deprivation. Tw o pigeons

T he process of em otional adaptation continues during m agazine tra in

3. T h e response of pecking the key can be shaped u p by hand. T he experi

D rl is conveniently a rran g ed w ith a

A device fo r d iffe re n tia lly re in fo rc

Tim e o u t (abbreviated as T O ) is any period of tim e during which the organism

A probe is some m om entary change of conditions against a background of a stable

T h e length of a daily session m ay be expressed either as a period of tim e or as a

Length o f exposure to schedules

T he effect of a schedule often depends u pon m om entary accidental contingencies

So far as possible, a change in conditions is m ade in the m iddle of an experim ental

I n a f i x e d - r a t i o s c h e d u l e of reinforcem ent, every rath response produces a reinforc

A response is never reinforced ju st after a previous reinforcement. T he operation

A response cannot be reinforced w ithin a shorter period of tim e than th at required

In interval schedules the probability th a t a response will produce a reinforcem ent

Stylized p lo t o f the transition from erf to FR

Slow developm ent o f FR 4 5 a fte r erf (second bird)

Transition from erf to FR 5 0

Unusual transition from erf to FR 50

Transition from erf to FR 20

Final perform ance on FR 20

Early developm ent on FR 45

LARGE CHANGES IN THE FIXED RATIO

E xtinction after fixed ratio shows th e effects of th e controlling contingencies a r

Subjects vary considerably in the extent to which interm ediate rates

of acceleration at a, the curve shows no interm ediate rates of responding. T he record

Extinction a fte r FR 120 showing sustained interm ediate rates

Extinction a fte r FR 6 0 before developm ent o f a norm al FR perform ance

First e ffe ct o f percentage

T h e 2 birds in the experim ent reacted a t different speeds to these conditions. O ne

D etail o f n egative curvature under

T h e effect of deprivation on a stable fixed-ratio perform ance was studied in the

X s 4 HOUR S ES SIO N IN WHICH

SES SIO N IN WHICH

Summary o f the effect o f deprivation on FR

FR a t 6 8 % and 8 2 % o f the inactive w e ig h t (1 2 9 RP)

SCHEDULES OF REINFORCEM ENT

O ver-all rate changes under FR a t 95% o f the inactive w e ig h t (1 2 9 RP)

Detail o f a record a t an FR a t 105% o f the inactive w eight