In general, response duration may be defined as the period of time between the beginning and end of a response. In operant research with rats, the duration of a lever press is the interval between depression of a lever and its release. When reinforcement is arranged for lever pressing of fixed durations, for example, a minimum interval of 10 s of depression, rats will eventually emit a majority of lever presses with durations that satisfy reinforcer requirements, but they will also emit many responses that are too short to meet reinforcer criteria (Hurwitz, 1954; Kuch, 1974; Lachter & Corey, 1982; Peck & Byrne, 2016; Platt, Kuch, & Bitgood, 1973; Senkowski, Vogel, & Pozulp, 1978; Stevenson & Clayton, 1970). This results in a bimodal distribution of lever-press durations, with one peak near the reinforced duration value, and a smaller peak at short durations, typically less than 1 s. Kuch (1974) noted that the bimodal distributions of durations obtained with duration-based schedules are similar to distributions of interresponse times (IRTs) typically recorded under differential-reinforcement-of-low-rate (DRL) schedules. The reasons for these bimodal distributions are not entirely understood. Here we shall define “short durations” as any lever press of a duration too short to meet reinforcement criteria.

There have been at least two previous attempts to modify these short durations. Senkowski et al. (1978) hypothesized that because approaching the hopper, a behavior they termed “goal-approach,” is incompatible with depressing a lever, short durations may be a function of the relative momentary strength of approaching the hopper. They found that decreasing the level of food deprivation under conditions where reinforcement was delivered for depressing a lever for a requisite duration resulted in a modest increase in mean lever-press durations, although the response distribution was still bimodal. In addition, they found that decreasing reinforcer magnitude had a similar effect. Platt et al. (1973) hypothesized that adding an 8-s intertrial interval (ITI) after all lever presses could interrupt a pattern by which reinforcers could strengthen short durations that were followed quickly by reinforced presses. Under this arrangement, short durations could never be followed by a reinforcer by a period less than an 8-s ITI plus any time until a response meeting the duration requirement occurred. This arrangement greatly reduced short durations.

Casual observation of lever pressing in previous investigations of response duration in our lab suggested that rats responding under reinforcement schedules requiring long lever-press durations (≥ 4 s) often emit a range of responses in addition to moving the lever with their paws. Such behavior included biting the lever and stimulus lights, sniffing a bolt, and raising the torso by full arm extension against the lever. Although some of these responses may have contributed to the downward force necessary to press the lever, it seemed possible that some may have been, at times, incompatible with lever depression. The goal of the current investigation was to further investigate short durations in rats responding under fixed-duration (FD) schedules of food delivery by combining automated recording of lever pressing with video analysis. By documenting other behaviors besides those that close the microswitch, the nature of these short durations may be better understood.

Method

Subjects

Three male Sprague Dawley rats (Taconic Biosciences, Hudson, NY), approximately 4 months old at the start of the study, served as subjects. All rats had previous experience with FD schedules in which depressing the lever for a designated interval resulted in the delivery of an edible reinforcer. Rats were housed together with unlimited access to water. They were maintained at 90% of their free feeding weights and kept under a 12:12 hr light/dark cycle. Procedures were approved by the Institutional Animal Care and Use Committee at Massachusetts College of Liberal Arts.

Apparatus

Three operant test chambers (Med Associates, St. Albans, VT) were used. Chambers were 30.5 cm long by 24.1 cm wide by 21.0 cm high. One response lever was mounted on the front panel 7 cm above the chamber floor. A force of 0.25 N activated the microswitch. A receptacle located in the center of the front panel 3 cm above the chamber floor allowed access to sweetened condensed milk (SCM)(Casa Solana Brand, Sysco Corporation, Houston, TX) provided by a liquid dipper. The dipper cup was 0.01 cc, but sweetened condensed milk adhering to the sides of the cup make this measure an estimate only. Chambers were enclosed in sound-attenuating boxes equipped with a fan to provide ventilation and sound masking. However, chamber doors were open during the sessions to allow for video recording. A house light, located in the center of the rear panel approximately 2 cm from the chamber ceiling, was illuminated during all sessions. All environmental events were controlled by a microcomputer running MED-PC software (Med Associates, St. Albans, VT) located in an adjacent room. Videos were recorded with a Canon EOS Rebel T3.

Procedures

Sessions were 45 min in length and held at approximately the same time, 5 days per week. Rats started training under a Fixed Ratio (FR) 1 schedule and were then exposed to longer FD requirements across several sessions. Under the FD schedules, holding down the lever for a requisite interval resulted in 3 s of SCM delivery; lever release was not a reinforcer requirement. If rats released the lever prior to the duration requirement, the response was recorded but resulted in no programmed consequence. We changed duration requirements between sessions and held them constant within sessions. The first duration requirement was 0.75 s. The duration requirement for the second session was 2 s. We then increased duration requirements by 1 s until reaching a requirement of 7 s. Following the 7-s requirement session, we implemented the terminal FD 10-s schedule.

Once median response duration showed the absence of any trend as determined by visual analysis, two sessions were video recorded for each rat. These were the 14th and 19th sessions of the terminal schedule for Rat 1, the 10th and 18th sessions for Rat 2, and the 12th and 18th sessions for Rat 3. All three authors viewed the videos independently. We wrote down any behavior we saw other than depressing the lever with one or both paws. As a group, we then agreed upon and defined responses other than lever pressing with paws. All scored responses are described in Table 1.

Table 1 Description of responses recorded from the video analysis for each rat
Full size table

Because behavior patterns changed substantially after the first reinforcer delivery of the session, different scoring procedures were used for different parts of each video. Two independent observers counted the number of hopper entries prior to the first reinforcer delivery. Interobserver agreement (IOA) for these hopper entries was calculated for half of the video recorded sessions by dividing the lowest number of entries counted by the highest number counted for an agreement of 85.7% (12/14 x 100).

Videos were then scored independently by two observers for the presence or absence of idiosyncratic and/or repetitive behaviors in 2-s intervals during the time between the 1st and 10th reinforcer delivery. Interval recording was used as some of the responses (e.g., biting the lever) were difficult to count accurately as they 1) they occurred often at a fast rate, 2) they were sometimes obscured by the position of the rat’s body, and 3) the beginning and end of a response was not always clear and was determined by both visual and auditory observations. IOA for the interval recording was calculated for approximately 30% of scored intervals for each rat by dividing the number of agreements (both observers scoring “+” or “-”) by the total number of intervals for an agreement of 84.2% (571/678 x 100).

Following the completion of video recording during the first phase, each rat was exposed to a fixed-timed (FT) schedule based on the mean reinforcement rate calculated from the last three sessions of the first phase. These times were 15.3, 18.0, and 16.5 s for Rats 1, 2, and 3, respectively. This was done to test if lever release was determined by timing or was under control of the SCM delivery functioning as a discriminative stimulus. This was followed by an extinction phase in which no SCM deliveries were available, a return to the FT schedule, and then a final phase of the FD 10 schedule. We implemented phase changes when median response durations showed the absence of any trend as determined by visual analysis. Table 2 depicts the number of sessions of each phase for each rat.

Table 2 Number of sessions of each condition for each rat
Full size table

Results

Figure 1 depicts the frequency distribution of response durations from the last two sessions of both FD 10 phases for each rat. All rats emitted bimodal distributions of response durations, with one peak at less than 1 s and a second peak at slightly longer than 10 s, the reinforcer requirement. These bimodal distributions were also present during the videoed sessions. Except for a few presses emitted during the first phase for Rat 2, durations rarely exceeded 12 s. For all rats, durations slightly greater than the reinforcer requirement made up the largest percentage of lever presses. The short-duration mode was minimal in the first phase for Rat 2.

Fig. 1
figure 1

Frequency distributions of response durations for the last two sessions of each FD 10 phase.

Full size image

Durations of each lever press emitted during the final five sessions of each phase are shown in Figure 2. For Rats 1 and 2, patterns of responding were similar during the FD and FT phases, with the exception that values in the long-duration modes increased under the FT schedule such that they approximated the length of the FT intervals. For these two rats, the FT schedule maintained robust lever pressing throughout the phase. For Rat 3, long durations (≥ 10 s) ceased during the FT schedule, but lever pressing persisted throughout the phase. Extinction reduced the number of lever presses emitted for all rats and eliminated long-duration presses for Rats 1 and 2. Reintroduction of the FT schedule increased the number of lever presses for all rats, and increased long-response durations for Rats 1 and 2. With the exception of session 101 for Rat 1, responding during the last FD 10 phase matched patterns obtained during the first FD 10 phase.

Fig. 2
figure 2

Durations of each lever press emitted during the last 5 sessions of each phase. Please note that the FT value for each rat was individualized as described in the Method.

Full size image

Analysis of the video recordings indicated differences in the rats’ behavior prior to and after the first reinforcer delivery. Prior to the first press of 10 s or greater, rats allocated more behavior away from the lever, spent time grooming, and made more unreinforced hopper entries. However, once reinforced responding began, patterns of behavior were consistent and were allocated primarily towards the lever and the chamber wall near the lever. Hopper entries typically only followed reinforcer delivery. Results of the video analysis for the session periods between the 1st to the 10th SCM delivery are shown in Figure 3. All rats emitted biting as well as idiosyncratic responses while depressing the lever. These included biting the lever (all rats), sniffing the chamber wall and stimulus lights, head turning, and running their nose across the top of the lever. These responses are depicted as cumulative-scored intervals. Photographs of some of these responses are shown in Figure 4. These behaviors invariably ceased at the moment of lever release, which appeared to be under discriminative control of the sound of food delivery. Hopper entry also appeared to be under strong stimulus control by the sound of SCM delivery and occurred rarely at other times. After the first reinforcer delivery, short lever-press durations appeared to be due primarily to “accidental” lever releases during vigorous biting and sniffing. The number of short lever presses between consecutive reinforcers, when they occurred, are shown above the tick marks. These biting and idiosyncratic responses were largely absent prior to the first reinforced press; however, hopper entry was more frequent. When data from both videotaped sessions were combined, Rat 1 emitted 22 lever presses and 18 hopper entries prior to the first press that met reinforcer requirements. Rat 2 emitted 6 lever presses and 10 hopper entries, and Rat 3 emitted 3 lever presses and 2 hopper entries.

Fig. 3
figure 3

Cumulative record of responses scored during the video recordings. Descriptions of the behaviors can be seen in Table 1. Ticks on the hopper-entry data path indicate SCM deliveries. Numerals above the ticks indicate the number of short-duration responses emitted between the indicated SCM delivery and the previous one. No numeral is listed if there were no short-duration responses.

Full size image
Fig. 4
figure 4

Representative idiosyncratic responses. The top photograph depicts head turning by Rat 1. The center photograph shows nose gliding by Rat 1. The bottom photograph shows Rat 3 biting the chamber wall.

Full size image

Figure 5 demonstrates within-session changes in response duration for one representative FD 10 session for each rat in which bimodal distributions were clear (Session 21 for Rat 1, Session 97 for Rat 2, and Session 89 for Rat 3). The y-axis depicts the cumulative number of lever presses with durations long enough to meet reinforcer requirements against those too short to meet reinforcer requirements. The x-axis depicts the sequence of responses rather than the passage of time. For all rats, short-duration presses were more prevalent early in the session, but the proportion of these presses gradually decreased and were overtaken in frequency by reinforced presses. Although short durations became less frequent as a function of session time, they still tended to occur in bursts of two or more. For example, for those same sessions indicated in Figure 5, Rat 1 emitted 25 short durations, and only 3 of them were both preceded and followed by reinforced responses. For Rats 2 and 3, those fractions were 6 out of 32 and 9 out of 36, respectively. The cumulative records for each rat also demonstrate streaks of reinforced responses.

Fig. 5
figure 5

Cumulative responses from a representative FD 10 session for each rat. The solid line shows cumulative reinforced responses (those lever presses with a duration ≥10 s) and the dotted line represents lever-press durations too short (< 10 s) to meet the reinforcer requirement.

Full size image

Discussion

Rats emitted bimodal distributions of response durations in a fashion similar to those reported in previous investigations in which reinforcement was arranged for holding down a response lever for a requisite interval. Releasing the lever followed by premature hopper entry, which was more common early in sessions, along with movement of the response levers due to vigorous biting and sniffing, sometimes resulted in lever release. We therefore believe that the bimodal distributions of responses documented here and in previous investigations are due to a combination of hopper observing entries and “accidental” lever releases. Based on visual analysis of what we assume to be representative sessions, hopper observing accounted for a minority of these short durations once rats began emitting reinforced presses. Given that most of the short durations were less than 1 s, the probability of these responses was low once lever holding commenced for more than 1 s.

Both our FD schedule and the types of responses we observed bear some resemblance to prior research of superstitious behavior (Skinner, 1948) or alternatively, behavior-induced by food presentation (Staddon & Simmelhag, 1971; Staddon, 1977; Timberlake & Lucas, 1985). These investigations employed FT schedules in which appetitive reinforcers were delivered at regular temporal intervals independently of an organism’s behavior, and responses other than the designated operant (e.g., wing flapping by pigeons) were noted. Of course, our FD schedule did have a response requirement. However, 10 s of lever depression, once started, produced temporal regularity not unlike that arranged under FT schedules. Once the lever was depressed, 10 s allowed plenty of time for the rats to emit other responses. We will not take a strong stand on whether such responses (biting, sniffing, head turning, etc.) were the result of adventitious reinforcement (Skinner’s analysis), or species-specific behaviors induced by food presentation, akin to Timberlake and Lucas’s (1985) or Baum’s (2012) analysis, as we do not believe our methodology sheds any particular light on the issue. However, the behaviors other than lever pressing we recorded persisted reliably and steadily throughout the 10-s lever-depression intervals and may have occurred due to the same mechanisms as behaviors recorded under FT schedules. One possible difference is that the responses we observed did not increase shortly after reinforcer delivery and then fall in probability shortly before reinforcer delivery as did the responses reported by Staddon (1977) under FT schedules. There was consistent close temporal contiguity between these responses and reinforcer delivery throughout the session. For example, every time we observed SCM delivery, Rat 1 was turning his head upside down and nibbling the corner of the response lever from the start of lever depression until the moment of SCM delivery. Likewise, Rat 3 bit the wall or stimulus lights above the lever throughout much of the 10 s durations up until SCM delivery. Biting and idiosyncratic behaviors may have been induced by food presentation, but, unlike responses emitted under a FT schedule, were restricted spatially to the lever (Rats 1 and 2) or near the lever (Rat 3) as allocating behavior elsewhere would have resulted in lever release. In this way, the biting observed in all rats, even emitted concurrently with head rotation (Rat 1) or leaning against the lever to gain access to the stimulus light (Rat 3), could have become a component of the operant of lever-pressing and maintained adventitiously by SCM delivery. There was little opportunity for breaking the correlation of SCM delivery and these responses, and we observed them persist throughout the study for each rat.

We believe our findings complement previous hypotheses regarding the prevalence of subcriteria durations. We observed some unreinforced hopper entries early in the session prior to the first reinforced press, and this accounts for some of the data in the short-duration modes. Therefore, our experiment provides some further evidence for Senkowski et al.’s (1978) hypothesis regarding hopper entry competing with lever pressing. Those authors found that decreasing motivating operations for food also decreased these presses. However, we found that such entries were rare once reinforcers were delivered, and they do not account for all short durations. Although we are unaware of prior work examining within-session changes in response duration under duration-based reinforcement schedules, there is a considerable literature showing that the rate of discrete responses increases and then decreases within an experimental session (McSweeney, 1992; McSweeney & Murphy, 2000). Although both satiation (e.g. Bizo, Bogdanov, & Killeen 1998) and habituation (McSweeney & Murphy, 2000) have been proposed as potential explanations for these patterns, the fact that our rats exhibited changes early in the session suggests that habituating to the experimental context may underline the within-session changes in our study.

Platt et al. (1973) posited that short durations could be adventitiously reinforced if they were emitted either prior to a reinforced press or between the completion of a reinforced press and reinforcer delivery. Inserting an 8-s ITI after every subcriteria press reduced the frequency of these presses. The authors suggested this reduction could be due to the ITI’s punishing subcriteria presses, or simply disrupting temporal contiguity between these presses and reinforcer delivery. They did not report data prior to the fifth reinforcer delivery, noting, like we did, that behavior early in the sessions was more variable. Therefore, subcriteria durations may have occurred for different reasons at various time points in their investigation as well. We do not know if their rats also emitted biting and idiosyncratic responses, but if so, such “lever presses” could have been decreased by ITI’s by the mechanisms Platt et al. suggested.

We believe we can largely rule out timing failure as a primary explanation for the bimodal distributions, at least for Rats 1 and 2. For those rats, lever release and hopper entry appeared to be under strong discriminative control of the sound of reinforcer delivery. During the FD 10 schedule, few lever-press durations were between 9 and 10 s, and most were between 10 and 11 s. For Rats 1 and 2, FT schedules maintained similar performance as the FD schedules; lever release often occurred at the moment of reinforcer delivery. Therefore, it appears that these two rats learned to depress the lever until the sound of reinforcer delivery, rendering timing unnecessary. Long-duration presses were not maintained by the FT schedules for Rat 3, so it is possible that timing did play a role in the maintenance of long durations for that rat under the FD 10 schedule.

Differential reinforcement of low rate (DRL) schedules and the duration-based schedule we used in the current study have some common features in terms of both the arrangement of environmental events and resultant behavior patterns they produce. In both schedules, reinforcers may be consumed only if two behavioral events are separated by a minimum time interval. In the current study, SCM was available if there was at least 10 s of lever depression, and subsequent lever release was required for hopper entry and SCM consumption. In DRL schedules, reinforcer deliver depends on IRTs and are delivered only if there is a minimum time interval between two consecutive responses (another name for a DRL is an IRT ≥ t schedule). DRLs, like FD schedules, typical produce a bimodal frequency distribution of responding, with a dominant peak near the reinforced IRT value and a second, smaller peak at short IRTs (Kramer & Rilling, 1970). In addition, DRLs often produce a pattern in which reinforced responses occur in clusters. As stated by Kramer and Rilling, “reinforced IRTs tend to follow reinforced IRTs, and unreinforced IRTs tend to follow unreinforced IRTs” (p. 233). We could present an accurate description of our data by replacing “IRTs” in the quote with “response durations.” Finally, several investigators have reported the occurrences of “collateral” and “mediating” behaviors emitted during the IRTs for the designated operants. Examples include licking by monkeys (Hodos, Ross, & Brady, 1962) and chewing by rats (Laties, Weiss, & Weiss, 1969). These responses may be similar in nature to those we noted in our video recordings. It may be interesting to compare the same organisms under both FD and DRL schedules to test if there are any systematic differences between response distributions, sequences, and adjunctive behaviors. Furthermore, programming a duration-based schedule in which the reinforcer is not delivered until lever release, instead of the depression-only requirement used in the current study, would allow for a more appropriate comparison to DRL schedules. Any differences in behavioral patterns between the two schedules may help illuminate whether these phenomena are products of the particular schedule requirements or are due primarily to periodic food delivery.

To the best of our knowledge, this study is the first to report the effects of FT reinforcement schedules on response duration. Response-independent SCM delivery maintained responding for two of the rats in a manner that was largely indistinguishable from patterns generated by the FD schedules. Similar findings have been reported when reinforcer delivery has been arranged for discrete lever presses. For example, Edwards, Peek, and Wolfe (1970) found that independent food delivery decreased response rates in rats previously exposed to fixed-ratio schedules, but the reductions were not evident until 50 or more sessions of response-independent food. Lattal (1973) found that under some schedule parameters, responding maintained under variable-time schedules was very similar to responding maintained by variable-interval schedules. Our data add to a small but growing body of research that demonstrates that response duration may be similar to discrete responding in how it responds to reinforcement schedules (Peck & Byrne 2016; Rider & Kametani, 1987).

Finally, we would like to note a limitation with our data collection. Although we recorded the duration of every lever press and the sequence in which they occurred, we did not have a time stamp for each response. Such data could provide a more detailed picture of the session dynamics. During the video analysis, we could not confidently identify instances in which lever releases were sufficient to be recorded by the computer due to the rapid oscillations during biting and sniffing. A true cumulative record would allow for precise temporal placement of short durations as well as analysis of pausing and IRTs.