Retention and Delayed Extinction of an Instrumental Response in the Toad Rhinella Arenarum Effects of Overtraining/Retencion y extincion diferida de una respuesta instrumental en el sapo Rhinella arenarum: efectos del sobreentrenamiento/Retencao e extincao diferida de uma resposta instrumental no sapo Rhinella arenarum: efeitos do overtraining.
Amphibians are an important group for comparative psychology because they represent the vertebrate transition from water to land. Systematic studies of their learning have revealed a divergence between mammals and amphibians in the so-called paradoxical reward effects (Amsel, 1992).These effects challenge Thorndike's (1911) law of effect, which stated that the consequences of a behavior control the emission of such response. Thus, rewards strengthen the response in proportion to the magnitude of the reward, implying that large reward should lead, for example, to slow extinction. However, experimental evidence shows that this law is not universal. Paradoxical reward effects are a series of experimental paradigms in which the instrumental response does not fit the law of effect. Instead, reward magnitude has an inverse relationship with the rate of extinction--hence the label "paradoxical". In such situations, predictions from the law of effect do not match experimental results. Paradoxical reward effects have been widely observed in mammals, but have not been found in other groups of vertebrates, such as amphibians, especially under widely-spaced conditions of training (e.g., one trial per day). This has led to the hypothesis of an evolutionary divergence in the emotional control of learned behavior (see Papini, 2003, 2014; Papini, Salas & Muzio, 1999).
Situations involving paradoxical reward effects have mainly been studied in amphibians using a runway procedure with spaced trials and water as reward in partially dehydrated animals (Muzio et al., 1992, 1994, 2006, 2011; Papini et al., 1995). The following paradigms have been studied in both mammals and amphibians:
Successive negative contrast
To observe this phenomenon, two groups of animals are trained, one group receiving a large reward and another group a small reward. After an asymptotic response has been reached for both groups, the large reward of the first group is reduced to the small reward of the other group. In mammals, the group that suffers this reward magnitude devaluation exhibits an impairment of performance below the level of the group that has always received a low reward. After a few trials, the response recovers and both groups tend to respond similarly (e.g., Crespi, 1942; Elliott, 1928; Pellegrini & Mustaca, 2000; Pellegrini, Ruetti, Mustaca, & Muzio, 2004). Unlike in mammals, incentive devaluation in amphibians results in a gradual adjustment of their response to the level of the unshifted group, without going through a period of poorer performance (e.g., Muzio et al., 1992, 2011). This effect is referred to as reversed successive negative contrast.
Partial reinforcement extinction effect
In mammals, animals that receive a random mixture of reinforced and nonreinforced trials during acquisition show enhanced resistance to extinction in comparison to animals that receive continuous reinforcement (e.g., Humphreys, 1939; Bouton et al., 2014). The opposite effect is observed in amphibians. Toads receiving partial reinforcement during acquisition extinguish faster than toads receiving continuous reinforcement (e.g., Muzio et al., 1994, 2006). This effect is referred to as reversed partial reinforcement extinction effect.
Magnitude of reinforcement extinction effect
This procedure is a more abrupt version of the successive negative contrast effect, in which acquisition with either a large or a small reward, animals are shifted to extinction (i.e., no reward). In mammals, extinction is faster after acquisition with a large reward than after acquisition with a small reward (Hulse, 1958; Pellegrini, 2011). In amphibians, larger rewards enhanced resistance to extinction (e.g., Muzio et al., 1992, 1994), showing a reversed magnitude of reinforcement extinction effect.
Overtraining extinction effect
In mammals, animals receiving a larger number of acquisition trials extinguish their response faster than those receiving fewer acquisition trials (e.g., North & Stimmel, 1960). An analogous situation in amphibians led to the opposite outcome: A slower extinction rate after receiving a larger than after a smaller number of acquisition trials (e.g., Muzio et al., 2006), a reversed overtraining extinction effect.
Altogether, all these results indicate that amphibians, unlike mammals, follow a simple rule in which the strength of behavior during extinction is a direct function of reward magnitude during acquisition (Muzio et al., 1992, 2011). Amsel's (1992) frustration theory claims that paradoxical reward effects involve an emotional response of frustration due to the unexpected absence or devaluation of the reward. Paradoxical reward effects reveal not only a reaction to the surprising changes in the environment, but also to the internal emotional changes that occurred as a result of the violation of reward expectancies.
The strengthening of an acquired response in amphibians has also been studied in terms of spontaneous decay (i.e., in the absence of training). In toads, an instrumental response decayed spontaneously as a function of the time elapsed since the last reinforced acquisition trial (Puddington, Papini, & Muzio, 2013). Toads maintained their response with a small relative loss of performance after the first 8 days without training. But response deterioration was significant beyond 16 days without training. When reward magnitude was reduced, however, response decay was significantly enhanced.
Following on these previous results, this experiment evaluated the effects of overtraining on the retention and delayed extinction of the instrumental response in a runway, using two different amounts of acquisition practice. Two groups were trained for 5 or 15 trials, one trial per day. After acquisition, each group spent 8 days without training before they received 8 additional extinction trials. Based on previous studies (Puddington et al., 2013), we expected that a small number of acquisition trials would impair retention of the instrumental response and increase extinction rate.
Twelve adult male toads (Rhinella arenarum) collected from ponds around Buenos Aires were used as subjects. This species is not listed as threatened (International Union for Conservation of Nature and Natural Resources, 2014). Toads were maintained according to the NIH Guide for the Care and Use of Laboratory Animals (National Research Council, 2011). Upon arrival in the laboratory, toads were subjected to a morphological examination to assess body symmetry and standard reflexes for anurans (Muzio, 1995). They were treated for a week with antibiotics (0.125 mg/0.1kg Baytril from Bayer Laboratories, once daily) and anthelmintics (0.8 mg/0.1 kg Mebutar from Andromaco Laboratories, single dose) to control bacterial and parasitic infections. During the feeding period (7 days) and purging period (7 subsequent days), animals were kept in group cages (10-15 subjects) with running water. Before the start of the experiment, animals were fed for a week with frog chow Ganave. Toads were transferred to individual plastic boxes (13 x 13 x 7 cm) with deionized water for two days, and weighed daily thereafter. Standard weights (weight of the hydrated animal with its urinary bladder emptied; Ruibal, 1962) were obtained the day before the start of pre-training. The animal's standard weights varied between 93.8 g and 155.5 g (Mean = 116.4 g, SEM = 5.6 g). The vivarium was kept at 24-25[degrees]C and 40-50% humidity, and subjected to 16:8 h light/dark cycle (lights on at 04:00 h). Constant acoustic stimulation was provided by a white noise generator (20-30,000 Hz). During training, animals were kept in individual dry plastic cages. Animals were kept at 80% of their standard weight during the entire experiment. This procedure successfully results in toads motivated to search for water (Muzio et al., 1992; Puddington et al., 2013).
We used a black Plexiglas runway, 100 cm long, 12 cm wide and 20 cm high (figure 1). The runway had a starting box (20 cm long) and a goal box (20 cm long), connected by a straight alley (60 cm long). The runway was covered with translucent Plexiglas lids that allowed constant observation of the animals through a mirror. Each compartment was illuminated by a 15-Wbulb placed over each translucent lid. The alley was separated from the start and goal boxes by a manually operated guillotine door. Inside the goal box there was a green Plexiglas container (13 x 10 x 3 cm), with deionized water and a metallic grid. The water level could be above or below the metallic grid, thus the water could be accessible or inaccessible (the latter condition was used during extinction trials).
Animals received two pre-training trials, one per day, in which they were placed in the starting box for 30 s, then both doors were opened, and the toads were able to freely explore the runway for 10 min. In the first pre-training trial, three drops of deionized water were scattered about the floor at the beginning, middle, and end of the alley. In the second pre-training trial, drops of water were placed only in the middle section of the alley and next to the second guillotine door, in front of the goal box. The container located in the goal box was always filled with accessible deionized water.
Training involved one trial per day. After the two pre-training trials, animals were randomly assigned to one of two groups (n = 6). Toads in Group 15 received 15 reinforced acquisition trials, whereas those in Group 5 received 5 reinforced acquisition trials. Each trial ended with a 300-s period of access to deionized water in the goal box. After acquisition ended, both groups spent seven days without training, and extinction started on the eighth day. All the toads received 8 extinction trials similar in all respects to acquisition trials except that the deionized water located in the goal box was inaccessible. The two groups started training at the same time; because different numbers of acquisition trials were administered (15 vs. 5), the start of extinction occurred at different moments for each group.
In each training trial, toads were placed in the start box for 30 s, and then the guillotine doors were raised. Two dependent variables were registered:
(1) Running latency (in seconds): Time from the moment the animal had its four legs in the alley and out of the start box, to the moment it entered the goal box with its four legs. This variable was recorded by the manual operation of a digital timer (1-s units) and transformed to the [log.sub.10] to improve normality and allow for parametric statistics. (2) Weight variation (g/100 g): The weight of every subject (in grams) was registered before and after each trial to estimate water consumption. The difference between these two weights was divided by the standard weight and multiplied by 100 to provide a relative measure of water uptake corrected for individual differences in body weight.
Toads were trained between 08:00 and 13:00 h. After each trial, the animals were transferred to dry cages where they remained until the next day. At least 30 min after each daily trial, animals that had lost weight during the trial were supplemented with deionized water, whereas those that had gained weight were dehydrated. This procedure ensured that animals would be at 79-81% of their standard weight at the start of each trial. During the trials, the animal was allowed a maximum of 180 s to leave the start box or a maximum of 180 s to enter the goal box if already in the alley. A maximum latency of 180 s was assigned (1) when the animal failed to leave the start box after 180 s, or (2) when the animal left the start box before the maximum 180 s, but failed to enter the goal box within the following 180 s. In incomplete trials, toads were gently guided to the goal box where they received the outcome appropriate for that trial (300 s of access to the deionized water in acquisition trials or 300 s of nonreinforced detention in extinction trials).
Analyses of variance (ANOVA) were computed to make inferences about behavioral outcomes. Specific assumptions for each test were tested before performing the analysis. The alpha level was set at 0.05. Partial eta square values were added as an estimate of effect size.
Figure 2 shows the running latencies of Group 15 and Group 5during acquisition and delayed extinction. During the first 5 acquisition trials, the toads assigned to these two conditions acquired the runway task at about the same speed. A Group X Trial anova found no interaction or group differences, Fs <1, but a significant trial effect, F (4, 40) = 9.39,p < 0.001, [[eta].sup.2] = 0.48. The terminal acquisition response (runway latency in the last acquisition trial) revealed nonsignificant differences between groups, F < 1. An independent repeated-measure anova was performed for acquisition trials in Group 15 that confirmed a trial effect, F (14, 70) = 3.78,p < 0.001, [[eta].sup.2] = 0.43. The analysis of the main effects with LSD comparisons showed that trial 15 only differed from trials 1, 2 and 3, ps < 0.01, and no differences were observed between any of the last 10 trials, with the exception of trial 10, which differed from trials 6 and 8, ps < 0.01.
Extinction led to a weakening of performance for both groups. Although figure 2 shows a tendency for extinction in Group 5 to be faster than in Group 15, a Group X Trial anova during the eight extinction trials showed significant effects only across trials, F(7, 70) = 3.75,p < 0.01, [[eta].sup.2] = 0.27, with nonsignificant effects between Groups, F (1, 10) = 2.33, p > 0.15, or for the Group x Trial interaction, F < 1.
Performance after the retention interval (8 days) was evaluated by computing the difference between the running latency in the first extinction trial and the last acquisition trial. Figure 3 shows the mean retention for each group estimated by this method. No differences were observed, F < 1.
Finally, Figure 4 shows the weight variation during all acquisition trials for both groups. A Group X Trial analysis covering the first 5 trials indicated nonsignificant effects across Groups, F < 1, Trials, F (4, 40) = 2.00, p > 0.11, or for the Group x Trial interaction, F < 1. An additional analysis between groups comparing only the last acquisition trial showed significant differences, F (1, 10) = 6.71,p < 0.03. Furthermore, a separate analysis of Group 15 during all the acquisition trials indicated a significant effect for Trials, F (14, 70) = 3.46,p < 0.001, [[eta].sup.2] = 0.41. Pairwise LSD comparisons indicated a significant difference in water uptake between the last acquisition trial and trials 1, 2, 4, 5, and 6, ps < 0.05.
The results reported here provide new information about the determinants of retention and extinction in toads. First, there was no evidence that overtraining affected retention of a runway response after a period of 8 days without training. Previous results had indicated that there was a magnitude of reinforcement effect on retention, with larger rewards resulting in better performance after an 8-day retention period (Puddington et al., 2013). In the present experiment, after 8 days without training, animals returned to the level of response they had at the end of acquisition even after a three-fold difference in the amount of acquisition practice. Second, the current results are in agreement with those of two previous studies where reversed overtraining extinction effects had been observed. Muzio et al. (2006) showed that animals that received 10 acquisition trials extinguished faster than those that received 30 acquisition trials. More recently, Puddington, Papini and Muzio (submitted), with the same acquisition procedure used here, but with immediate extinction, observed faster extinction after 5 acquisition trials than after 15 acquisition trials.
Although for this study a standard sample size was used (n = 6 in each group) some, but not all, negative results could be attributable to lack of statistical power. For example, whereas there is a tendency for groups to differ in the last acquisition trial (figure 2), retention differences were not observed (figure 3).However, between-group differences had been observed when comparing weight variation in the last acquisition trial. As is usual in these experiments (e.g., Muzio et al., 1992), toads increased their weight across acquisition trials, since physiological changes in the water absorption system occur across trials (for an example in a different training paradigm, see Daneri, Casanave & Muzio, 2011).
Table 1 summarizes previous data for toads. We studied memory decay in the same training situation used in this experiment using several retention intervals before the start of extinction (a different number of days between the end of acquisition and the start of extinction). The results of the current study and those of an unpublished study (Puddington et al., submitted) both based on an 8-day retention interval, are compared to similar studies using immediate extinction (starting a day after the last acquisition trial). Overall, these results show that whereas reducing acquisition trials weakens the response with immediate extinction, the effect of the amount of training on retention after 8 days was not observed in the present experiment. However, retention intervals did affect extinction after acquisition with different reward magnitudes (30 vs. 600 s of access to water). Therefore, it seems plausible that the amount of training and reward magnitude during acquisition affect extinction via different mechanisms. However, this conclusion should be taken with caution until data are available from a study in which these two factors are manipulated simultaneously. These results need to be complemented with new studies focusing on the neural mechanisms underlying instrumental extinction in amphibians. Such studies will contribute to understand common and divergent learning mechanisms of both retention and delayed extinction in vertebrates.
This research was funded in part by Grant UBA-CYT-P293BA from the University of Buenos Aires, by Grant PIP 0893 from the CONICET and by Grant PICT 2243 from FONCYT, Argentina, all directed by RNM.
Amsel, A. (1992). Frustration Theory. Cambridge, UK: Cambridge University Press.
Bouton, M. E., Woods, A. M., & Todd, T. P. (2014). Separation of time-based and trial-based accounts of the partial reinforcement extinction effect. Behavioural Processes, 101, 23-31.
Crespi, L. P. (1942). Quantitative variation in incentive contrast studies involving discrete-trial procedures. American Journal of Psychology, 55, 467-517.
Daneri, M. F., Casanave, E., & Muzio, R. N. (2011). Control of spatial orientation in terrestrial toads (Rhinella arenarum). Journal of Comparative Psychology, 125(3), 296-307.
Elliott, M. H. (1928). The effect of change of reward on maze performance of rats. University of California Publications Psychology, 4, 19-30.
Hulse, S. H. (1958). Amount and percentage of reinforcement and duration of goal confinement in conditioning and extinction. Journal of Experimental Psychology, 56, 48-57.
Humphreys, L. G. (1939). The effect of random alternation of reinforcement on the acquisition and extinction of conditioned eyelid reactions. Journal of Experimental Psychology, 25, 141-158.
International Union for Conservation of Nature and Natural Resources (2014). IUCN red list of threatened species (Version 2014.3). Retrieved from http://www.iucnredlist.org
Muzio, R. N. (1995). Estudio del aprendizaje asociativo en el anfibio Bufo arenarum. Aspectos comparados. (Tesis doctoral, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires).
Muzio, R. N., Pistone Creydt, V., Iurman, M., Rinaldi, M. Sirani, B., & Papini, M. R. (2011). Incentive or Habit Learning in Amphibians? PLoS One, 6(11), e25798. 10.1371/journal.pone.0025798.
Muzio, R. N., Ruetti, E., & Papini, M. R. (2006). Determinants of instrumental extinction in terrestrial toads (Bufo arenarum). Learning and Motivation, 37(4), 346-356.
Muzio, R. N., Segura, E. T., & Papini, M. R. (1992). Effects of schedule and magnitude of reinforcement on instrumental acquisition and extinction in the toad, Bufo arenarum. Learning and Motivation, 23, 406-429.
Muzio, R. N., Segura, E. T., & Papini, M. R. (1994). Learning under partial reinforcement in the toad (Bufo arenarum). Effects of lessions in the medial pallium. Behavioral and Neural Biology, 61, 36-46.
National Research Council (2011). Guide for the care and use of laboratory animals (8th ed.). Washington, D.C.: National Academies Press.
North, A., & Stimmel, D. (1960). Extinction of an instrumental response following a large number of reinforcements. Psychological Reports, 6, 227-234.
Papini, M. R. (2003). Comparative psychology of surprising nonreward. Brain Behavior and Evolution, 62, 83-95.
Papini, M. R. (2014). Diversity of adjustments to reward downshifts in vertebrates. International Journal of Comparative Psychology, 27(3), 420-445. doi: 10.1037/e615782013-001 http://search.crossref.org/?q=mauricio+r.+papini+2014
Papini, M. R., Salas, C., & Muzio, R. N. (1999). Analisis comparativo del aprendizaje en vertebrados. Revista Latinoamericana de Psicologia, 31(1), 15-34.
Papini, M. R., Muzio, R. N., & Segura, E. T. (1995). Instrumental learning in toads (Bufo arenarum): Reinforcer magnitude and the medial pallium. Brain, Behavior and Evolution, 46, 61-71.
Pellegrini, S. (2011). Stimulus magnitude effects on the extinction of conditioned consummatory responses to flavored sucrose solution in mice. Behavioural Processes, 87(2), 190-196.
Pellegrini, S. & Mustaca, A. E. (2000). Consummatory successive negative contrast with solid food. Learning and Motivation, 31, 200-209.
Pellegrini, S., Ruetti, E. M., Mustaca, A. E., & Muzio, R. N. (2004). Efectos de la cantidad y del tiempo de refuerzo sobre el contraste negativo sucesivo consumatorio (CNSc). Revista Latinoamericana de Psicologia, 36(2), 317-331.
Puddington, M. M., Papini, M. R., & Muzio, R. N. (2013). Vulnerability of long-term memory to temporal delays in amphibians. Behavioural Processes, 99, 7-11.
Puddington, M. M., Papini, M. R., & Muzio, R. N. (2017). Exposure to extinction cues as a determinant of instrumental extinction in terrestrial toads (Rhinella arenarum). Animal Cognition. In press.
Ruibal, R. (1962). The adaptive value of bladder water in the toad, Bufo cognatus. Physiological Zoology, 35, 218-223.
Thorndike, E. L. (1911). Animal intelligence. New York: Macmillan.
Received: September 04, 2016
Accepted: may 04, 2017
Martin M. Puddington
IBYME-CONICET & Universidad de Buenos Aires, Argentina
Texas Christian University
Ruben N. Muzio * IBYME-CONICET & Universidad de Buenos Aires, Argentina
* Correspondence: Grupo de Aprendizaje y Cognicion Comparada, Laboratorio de Biologia del Comportamiento. Instituto de Biologia y Medicina Experimental (IBYME-CONICET). Vuelta de Obligado 2490. (C.P. 1428) Buenos Aires, Argentina. E-mail: firstname.lastname@example.org
Caption: Figure 1. Schematic of the runway used in the experiment
Caption: Figure 2. Running latency (means [+ or -] SEMs) of two groups of toads receiving acquisition training during either 5 or 15 daily trials with access to water for 300 s followed by 8 extinction trials after an 8-day retention interval. Extinction trials in both groups are represented together although they occurred ten days apart from each other
Caption: Figure 3. Retention performance, measured as the difference between the running latency in the first extinction trial and the last acquisition trial (means [+ or -] SEMs)
Caption: Figure 4. Weight variation (means [+ or -] SEMs) for each group during acquisition trials. Variations in body weight were corrected for individual differences in body weight across trials