Relational Ability and Language Performance in Children With Autism Spectrum Disorders and Typically Developing Children: A Further Test of the TARPA Protocol.
One theory that offers a potentially useful approach to linguistic generativity is relational frame theory (RFT; Hayes et al. 2001). According to RFT, language can be understood as contextually controlled, arbitrarily applicable relational responding. In relational responding, an individual shows a generalized repertoire of responding to one stimulus in terms of another (see, e.g., Fox, Hayes et al. 2001). A range of different species have demonstrated nonarbitrary relational responding in which the relations involved are based on physical properties of the relata (e.g., Giurfa, Zhang, Jenett, Menzel, & Srinivasan, 2001; Harmon, Strong & Pasnak, 1982; Reese, 1961; Wright & Delius, 1994). For example, Harmon et al. showed that monkeys could be trained to choose the taller of two stimuli. However, relational frame theorists argue that typical members of the human species exposed to contingencies provided by their socioverbal community can learn to show an additional type of relational responding, referred to as arbitrarily applicable relational responding (AARR). This is primarily not based on nonarbitrary or formal relations between the stimuli being related but on aspects of the context that specify the relation, such that the relational response can be brought to bear on any relata, regardless of their nonarbitrary properties (see, for example, Stewart & McElwee, 2009).
One example of relational framing, perhaps the earliest and most fundamental, is word--object bidirectional relations. To illustrate, imagine that I teach a young child to emit the word astrolabe in the presence of an object that she has never seen before (taught: object--word relation). Later on, in the vicinity of that same object, I ask her to point out the astrolabe and she does it quickly and easily, without any training (derived word--object relation). It can work the other way round as well. I might first teach the word--object relation by reinforcing selection of the appropriate object from an array after I say the word astrolabe. Later, I present the astrolabe and ask her what it is, and she answers correctly.
RFT explains this ability to derive the untaught relation between a word and an object after being trained in the other direction, in a novel name--object pair, as responding in accordance with a derived coordination ("sameness") relation between the word and the object. This is the earliest form of derived relational responding to be established. We learn this pattern through exposure to certain contingencies of reinforcement, provided at a very early age. For example, caregivers will often utter the name of an object in the presence of an infant and then reinforce any orienting response that occurs towards the particular object ("hear Name A--0 look at Object B"). They will also often present an object to the infant and then model and reinforce an appropriate naming response ("see Object B--+ hear and say Name A"). In this way, the caregiver effectively teaches the child in both directions in the bidirectional relational pattern (frame). Furthermore, this informal training consistently occurs in the presence of particular contextual cues, such as the word is, the phrase name of or the presence of both a novel object and novel name. RFT suggests that after a sufficient number of name-to-object and object-to-name exemplars have been taught, these contextual cues become discriminative for the bidirectional pattern (frame) of coordination and the generalized operant response class of what we might refer to as derived "naming" (i.e., treating an object and a word as the same as each other) is thereby established. Imagine, for instance, that a child with this multiple exemplar history is told, "This is your ball.-Contextual cues, such as those listed, will now be discriminative for symmetrical responding between the name and the object. Thus, without any additional training, the child will now point to the ball when asked, "Where is the ball?" (Name A [right arrow] Object B) and will answer "ball" when presented with the ball and asked "What is this?" (Object B Name A).
Arbitrarily applicable relational responding, in accordance with sameness or coordination, appears to be the first pattern of AARR to emerge, and there is empirical research with infants charting the emergence of this pattern as well as demonstrating how it can be explicitly trained (Lipkens et al. 1993; Luciano et al. 2007). Many other patterns of AARR are learned also. As an example, consider how AARR in accordance with comparison, might be acquired. In this case, the child likely first learns to choose the physically larger of two objects in the presence of auditory stimuli, such as "bigger," and to choose the physically smaller of two objects in the presence of stimuli, such as "smaller." Then, through exposure to multiple exemplars of this type of pattern in the presence of these contextual cues, the relational response becomes abstracted, such that it can be applied in conditions in which there is no obvious formal relation; for example, after being told that "Mr. A is bigger than Mr. B," a child will be able to derive that "Mr. B is smaller than Mr. A."
Patterns of arbitrarily applicable relational responding are referred to as relational frames. RFT suggests the existence of a variety of these frames, including sameness (or coordination, e.g., "A = B"); comparison (e.g., "C is bigger than D"); opposition ("black is the opposite of white"); distinction ("this is not the same as that"); hierarchy ("a whale is a type of mammal"); analogy ("A is to B as C is to D"); deixis ("I am here and you are there"); and temporality ("spring comes before summer"), amongst others. Furthermore, RFT researchers have provided an increasing quantity of empirical evidence for the existence of this variety of patterns of framing (e.g., Barnes-Holmes, Barnes-Holmes, & Smeets, 2004; Barnes-Holmes, Barnes-Holmes, Smects, Strand, & Friman, 2004; Berens & Hayes, 2007; Carpentier, Smeets & Barnes-Holmes, 2003; Roche & Barnes, 1997; see also Dymond, May, Munnelly & Hoon, 2010; Rehfeldt and Barnes-Holmes 2009).
Despite the diversity of relational frames, three defining generative features characterize all instances: (a) mutual entailment, (b) combinatorial entailment, and (c) transformation of function. Mutual entailment describes the feature of relational framing wherein if Stimulus A is related to Stimulus B in a certain context, then a novel relation between B and A may be derived in that context. For instance, in the context of a game using arbitrary colored circles as coins (see, e.g., Barnes-Holmes, Barnes-Holmes, Smeets, Strand & Friman, 2004), if I tell a child that Coin A is more than Coin B, I can test for mutual entailment by then asking which coin is less. Combinatorial entailment occurs when at least two stimulus relations combine to allow the derivation of a novel relation. For example, if 1 tell a child that Coin A is worth more than Coin B. and Coin B is worth more than Coin C, I can test for combinatorial entailment by asking which of the two coins, A or C, is more (or less). Transformation of stimulus functions is the phenomenon whereby if Stimuli A and B participate in a relation, and A has acquired some psychological function, then in a context that selects particular stimulus functions of A as behaviorally relevant, the stimulus functions of B will be transformed in accordance with that relation. For instance, after asking the child in the example which coin he or she would prefer (i.e., an appetitive function), if the child selects the coin that is "more," this is evidence of transformation of function.
Given these generative properties of relational framing and that there are multiple different forms of framing, this behavior is potentially extremely generative. Any stimulus can be related to any other stimulus in accordance with any relational frame, including those listed above as well as others. Each time this happens, the functions of the stimuli involved are transformed. This activity can be either "receptive," whereby we respond under appropriate control to the relational framing behavior of others (i.e., listening with understanding), or "expressive," whereby our relational framing acts as a stimulus for other people (i.e., speaking with meaning). As children grow into adulthood, continued relational framing activity, both receptive and expressive, produces an increasingly complex and multirelational network involving vast numbers of different objects and events whose functions are transformed based on relations between them. In the context of the current topic, typically developing children readily learn this type of responding through exposure to the natural language environment (see, e.g., Lipkens et al. 1993; Luciano et al. 2007). However, children with ASD, amongst other categories of developmental delay, do not easily learn these patterns (e.g., Rehfeldt et al. 2007), and thus specific targeted training is required (see, e.g., Rehfeldt and Barnes-Holmes 2009).
A novel protocol for assessing and training relational framing, the Training and Assessment of Relational Precursors and Abilities (TARPA; Moran et al. 2010), tests a number of key forms of responding that are critical (from an RFT perspective) to the development of this repertoire and thus generative verbal behavior. The ultimate aim of the TARPA project is to enable assessment and training of the multiple forms of relational framing that underlie a fully generative linguistic repertoire. However, at this stage of its development, the core function of the TARPA is to assess and train relational framing at a very basic level. As suggested earlier, coordination or sameness appears to be the first and most fundamental form of framing. Hence, the current instantiation of the TARPA focuses on the emergence of coordination or equivalence.
The forms of responding on which the current TARPA focuses include (a) basic or simple discrimination; (b) nonar-bitrary (i.e., based on formal or physical properties) conditional discrimination; (c) arbitrary (i.e., experimenter-designated) conditional discrimination; (d) mutually entailed relational responding (e.g., deriving the symmetrical relation B [right arrow] A from the trained relation A B); (e) combinatorial entailed relational responding (e.g., deriving the untrained relations A [right arrow] C and C [right arrow] A by combining the trained relations A [right arrow] B and B [right arrow]C); and (t) transformation of function (which in the context of coordinate relations is also referred to as transfer of function, that is, responding similarly to stimulus A as to stimulus B, not based on explicit training of A but on the participation of A and B in a derived relation of sameness) In addition, the TARPA assesses these patterns of responding with both visual and auditory stimuli.
The aim of this study was to extend previous work on the TARPA. Moran et at. (2010) provided a test of an earlier version of the TARPA, assessing children diagnosed with ASD. One aspect of this study was the correlation of performance on the TARPA with functioning as measured using the Vineland Adaptive Behavior-Scales (VABS). Five children between the ages 6.4 and 13.6 years participated. A score on the VABS was attained for each child, and the child was then tested with the TARPA across a number of sessions. A Spearman's rank correlation test showed a strong and significant correlation (r=0.97,p=.05) between performance on the TARPA and adaptive functioning as measured using the VABS. Further tests were conducted to examine the relationship between TARPA performance and scoring for each of the subscales of the VABS. These showed that the TARPA correlated highly, though just outside significance, with the Communication subscale (r=0.947, p=.056), while showing lower, more clearly nonsignificant correlations with the Daily Living (r=0.56, p=.25) and Socialization (r=0.56, p=.25) subscales. This was consistent with what might be expected from a language-relevant protocol and was considered a good preliminary result for the TARPA. Finally, Moran et al. (2010) also showed some differences in performance based on stimulus modality. For instance, a number of participants showed weaker responding on sections involving exclusively auditory stimuli than on sections involving at least some visual stimuli, irrespective of the strength of their overall performance.
The current study, which comprised two experiments, extended Moran et al. (2010) in several respects. First, the version of the TARPA employed in this study is a more comprehensive and systematic measure. Whereas the Moran et al. protocol was limited with respect to the stimulus modalities involved in the testing of the properties of derived relations (i.e., mutual and combinatorial entailment and transformation of function; e.g., there was no training or testing of these properties with exclusively visual stimuli, and both the latter properties were tested with auditory stimuli only), the current protocol tests all three properties in each of four different modalities (i.e., all visual, all auditory, and two separate auditory--visual modalities). In addition to this, the current protocol increased the range of tests of simple discrimination and added tests of generalized identity matching in both the visual and auditory modalities. This protocol also constitutes a more systematic measure of derived relational responding. In contrast with the Moran et al. protocol, it includes tests of maintenance of trained relations in both the mutual and combinatorial entailment levels (with several in the latter). In addition, the transformation of function test in this protocol is a simpler and logistically more practical assessment. In regard to the latter, in Moran et al., the transformation of function test was adapted from a task employed by Murphy, Barnes-Holmes, and Barnes-Holmes (2005), in which participants were assessed for transformation of the discriminative functions of stimuli that could be employed to mand for tokens in the context of a game. This task was relatively complex, and its implementation in the context of a computer-delivered protocol increased this complexity. In the current version of the protocol, a very different and considerably simpler test was employed, in which participants were initially required to perform particular actions in the presence of particular stimuli and were then tested for transformation of discriminative functions in the presence of stimuli in derived relations with the initial set.
A second way in which this study constitutes an extension of Moran et al. (2010) is that whereas the latter correlated the TARPA with the VABS, one subscale of which affords an indirect measure of communication ability, the current study correlated it with a widely accepted performance-based measure of language ability, namely the Preschool Language Scale. Fourth Edition (PLS-4; Zimmerman et al. 2002). Third, the current study employed the TARPA not just with children with ASD (Experiment 1) but also with typically developing children (Experiment 2). Testing was conducted on the latter group to acquire data from an alternative population of children than those with ASD, whose performance might be expected to allow more detailed analysis of the upper portion of the TARPA and whose performance level could also be compared with that of the children with ASD. Fourth, the current study employed a considerably larger number of participants (i.e., n=10 in Experiment 1 and n=13 in Experiment 2, giving a total of 23 for the study as a whole). Fifth, to investigate the appropriateness of the ordering of TARPA substages, the current study conducted a statistical hierarchical analysis of the protocol. Finally, whereas in Moran et al. (2010) a mouse was used as the input device, resulting in some procedural difficulties, in the current study, a touch-screen computer was employed, which eliminated these difficulties.
Participants Prior ethical approval for recruitment of participants for this study was obtained from the National University of Ireland, Galway, Research Ethics Committee. Ten children (7 male, 3 female; age range 3 years, 4 months-12 years, 5 months; see also Table 1), each of whom had an independent diagnosis of autism provided by a licensed psychologist, were recruited through a service provider for children with an intellectual disability (ID) at which the experimenter was employed. Two were attending a specialized school for children with an ID, one was being taught in the context of a home program, and the remaining seven were attending a mainstream school at which their learning was facilitated by a special needs assistant. None of the children had previously taken part in similar research. Consent for conducting the study was obtained from the director of the service provider. Parental consent was obtained for each child who participated and verbal consent was also obtained from each of the participants.
Table 1 Demographic information and TARPA performance parameters for Experiment 1 Pt. No Age(Y: M) Sex No. Sessions Mean Session Length 1 07:04 M 2 15 2 04:07 F 6 15 3 06:01 M 8 12 4 04:09 M 8 20 5 05:10 M 5 40 6 07:11 M 3 20-25 7 12:05 F 4 30 8 03:04 M 10 20 9 08:07 F 3 50 10 08:08 M 6 35 Pt. No Highest Level Passed Highest Level Tested 1 N/A SIL3(A&V) 2 SIL2(V) S2L3(A&V) 3 S2L1(V) S3L1(CD) 4 S3Li(CD) S3L2(ME) 5 S3L2(ME) S3L3(CE) 6 S3L4(TOF) S3L4(TOF) 7 S3L4(TOF) S3L4(TOF) 8 S2L2(V) S3LI(CD) 9 S3L4(TOF) S3L4(TOF) 10 S3L1(CD) S3L2(ME) pt. Participant, A Auditory, V Visual, CD Conditional Discrimination, ME Mutual Entailment, TOF Transformation of Functions
Materials The two measures employed were the PLS-4 and the TARPA.
PLS-4 This is a 130-item standardized test of language that consists of two subscales that assess auditory comprehension (AC) and expressive communication (EC) skills. The AC subscale assesses skills such as attention to speakers, object play, comprehension of basic vocabulary, response to grammatical markers, identification of rhyming words, and ability to make comparisons. The EC subscale assesses skills including object naming, object description, expression of quantity and the use of grammatical markers. There are a total of 62 AC and 68 EC items assessed. The PLS-4 yields norm-referenced test scores (standard scores, percentile ranks, and an age equivalent) for both subscales, as well as a total language (TL) score. These scores are available at 3-month intervals from birth to 11 months, and at 6-month intervals for ages 1 year to 6 years, 5 months. Test--retest reliability for the PLS-4 has been reported as ranging between .82 and .95 for the subscale (AC and EC) scores and between .90 and .97 for the total (TL) scale score, while internal reliability has been reported as ranging from .66 to .96 (Zimmermann et al. 2002). In addition, the test has been validated with typically developing children as well as children previously identified as having a variety of disorders, including language deficits, developmental delay, and autism.
TARPA The TARPA protocol (see Fig. 1 for screenshots of various elements) was programmed in Microsoft Visual Basic and was presented on a touch-screen Hewlett Packard ("Pavilion") laptop computer. The TARPA software presented all relevant stimuli and recorded all response data. The protocol consists of three stages as follows: (1) basic discrimination; (2) conditional discrimination (formally similar stimuli); and (3) conditional discrimination (formally dissimilar stimuli). The first two stages assess the participant's ability to learn skills that are precursors to relational framing, while the third stage, which is the most substantial, tests properties of emergent relational framing itself.
[FIGURE 1 OMITTED]
Stage (I) This initial stage tests the participant's ability to learn simple discriminations and has two tracks (visual and auditory), with three levels in each track. Level I of the visual track requires a visual discrimination between a blank box and an abstract picture (see Fig. 1, iii). In this, as in all other visual discrimination tasks on all three stages, a correct response involves touching the appropriate on-screen visual stimulus. Level 2 requires a discrimination between two abstract pictures, and Level 3 requires a discrimination between three abstract pictures. The other track features auditory stimuli. In the case of each of the auditory stimuli that is presented throughout the TARPA protocol, there is a corresponding on-screen "button." The auditory stimulus is produced when its corresponding button first appears on the screen as well as when its button is touched as a means of selecting that auditory stimulus. In Stage 1, Level 1 of the auditory track requires a discrimination between a spoken nonsense syllable and no sound. Level 2 requires a discrimination between two spoken nonsense syllables, and Level 3 requires a discrimination between three spoken nonsense syllables.
Stage (2) This stage tests the participant's ability to learn conditional discriminations, in which the sample and correct comparison are formally or physically similar, and has two tracks (visual and auditory) with three levels in each track. Level 1 of the visual track involves a two-choice conditional discrimination involving abstract pictures in which the designated correct comparison on each of the two task trials is identical to the sample. Level 2 is a three-choice conditional discrimination involving abstract pictures in which the designated correct comparison on each of the three task trials is identical to the sample. Level 3 involves the same task type as Level 2, but tests generalized identity matching by presenting a different set of abstract stimuli on each trial (see Fig. 1, iv). Level 1 of the auditory track (sec Fig. I, v) involves a two-choice conditional discrimination involving nonsense syllables in which the designated correct comparison on each of the two task trials is identical to the sample. Level 2 is a three-choice conditional discrimination involving nonsense syllables in which the designated correct comparison on each of the three task trials is identical to the sample. Level 3 involves the same task type as Level 2, but tests generalized identity matching by presenting a different set of nonsense syllables on each trial.
Stage (3) This stage involves conditional discriminations in which the sample and comparison stimuli are formally dissimilar (see Fig. 1, vi, for an example) referred to as arbitrary conditional discriminations) and includes four different stimulus modality tracks: Visual--Visual, Auditory--Visual 1, Au-ditory--Visual 2 and Auditory--Auditory. In addition, each track is further subdivided into levels as follows: (1) Conditional discrimination, (2) mutual entailment, (3) combinatorial entailment, and (4) transformation of function.
Stage 3, Level 1 involves a two-choice arbitrary conditional discrimination in which the designated correct comparison is physically dissimilar from the sample. In the Visual--Visual track, all stimuli are abstract pictures; in the Auditory--Audi-tory track, all stimuli are spoken nonsense syllables; in the Auditory--Visual 1 track, the samples are spoken nonsense syllables and the comparisons are abstract pictures. In the Auditory--Visual 2 track, the samples are abstract pictures, while the comparisons are spoken nonsense syllables.
Stage 3, Level 2 tests for mutually entailed arbitrary relational responding and involves three sections. Section 1 (A[right arrow] B) involves the same conditional discrimination as employed in Level 1. Section 2 (A [right arrow] B Maintenance) involves the same conditional discrimination as in Section 1 but in the absence of reinforcement. Section 3 (B [right arrow] A) tests for the mutually entailed counterpart relation to that probed in Sections 1 and 2. In the Visual--Visual and Auditory--Auditory tracks of Section 3, all stimuli are abstract pictures and spoken nonsense syllables, respectively; in the Auditory--Visual 1 track, the samples are abstract pictures while the comparisons are spoken nonsense syllables; while in the Auditory--Visual 2 track, the samples are spoken nonsense syllables while the comparisons are abstract pictures.
Stage 3, Level 3 tests for combinatorially entailed relational responding and involves six sections. Section 1 (A [right arrow] B) involves the same conditional discrimination as employed in Level 1. Section 2 (A [right arrow] B Maintenance) involves the same conditional discrimination as in Section 1, but in the absence of contingent reinforcement. Section 3 (C [right arrow] B) involves a new conditional discrimination in which the comparison stimulus is the same as in Section 1, but the sample is a novel stimulus. Section 4 (C [right arrow] B Maintenance) involves the same conditional discrimination as in Section 3, but in the absence of reinforcement. Section 5 (A [right arrow] B; C [right arrow] B) involves an assessment in the absence of reinforcement of both the previously trained and assessed conditional discriminations.Section 6 (A [right arrow] C; C [right arrow] A) tests for both the combinatorially entailed relations that can be derived from the previously trained unidirectional relations (i.e., A [right arrow] B and C [right arrow] B). In the Visual--Visual and Auditory--Auditory tracks of Section 6, all stimuli are abstract pictures and spoken nonsense syllables, respectively; in the Auditory--Visual I track, the A stimuli are spoken nonsense syllables while the B and C stimuli are abstract pictures; while in the Auditory--Visual 2 track, the A and C stimuli are spoken nonsense syllables while the B stimuli are abstract pictures.
Stage 3, Level 4 tests for transformation of functions via the combinatorially entailed relations (i.e., A [right arrow] C and C [right arrow] A) tested for in the previous level and involves three sections. Section 1 tests for maintenance of both the trained (i.e., A [right arrow]B and C [right arrow] B) and tested combinatorially entailed (i.e., A [right arrow] C and C [right arrow] A) relations from the previous stage. Section 2 trains a discriminative function for each of the A stimuli; namely, waving in the presence of Al and clapping in the presence of A2. Finally, Section 3 tests whether the C stimuli have acquired novel discriminative functions via derived relations (i.e., waving in the presence of C 1 and clapping in the presence of C2).
Procedure The first session involved PLS-4 assessment. TARPA assessment began in the next session and continued in subsequent sessions until complete.
PLS-4 This was administered to the child by the experimenter with the assistance of the child's caregiver or teacher (where required). The PLS-4 assessment was conducted in one sitting that lasted on average about 35 min per child.
TARPA Assessment using the TARPA protocol began in the next session. For this the child and the experimenter sat in front of a laptop computer on which TARPA tasks were presented. Each child required a number of assessment sessions. Session lengths were dependent on duration of on-task behavior, the length of time for which each child was available in that particular session, and progress on the TARPA. Specific information on number and approximate length of sessions is provided in Table 1.
Training, Maintenance, and Testing Some parts of the TARPA (e.g., Stage 1, Visual, Level 1) assessed the child's capacity to learn certain skills (i.e., to make a simple discrimination). On these training parts, a particular protocol was followed in which first the correct response was demonstrated by the experimenter, then the correct response on the part of the child was physically guided or prompted, and finally the child was assessed for an independent performance (this will hereafter be referred to as the DGI [Demonstration Guidance Independent] protocol). Training trials included a continuous schedule of reinforcement for correct responding involving computer-delivered reinforcement (see Fig. 1, vii) and verbal praise. Also, all participants were on individualized schedules of reinforcement. More specifically, in the case of each child, the schedule of reinforcement typically used in their normal educational setting was employed, potential reinforcers were chosen based on consultation with the child's caregiver/teacher, and participants were also given the opportunity to choose what they worked for throughout assessment (several participants--P4, P5, P6, P7, & P9--were asked, while in the case of others--P1, P2, P3, P8, & PLO choice boards were used). In addition to training parts, there were maintenance sections and derived relational testing sections. In the case of both of these, there were no demonstration or guided trials (i.e., the participant received independent trials only) and there was no contingent reinforcement; however, during these sections, children continued to be provided with noncontingent reinforcement, including praise.
Response Definitions Responding during each part (level/sec-tion) was scored as either "correct," "incorrect" or "no response." A "correct" response was defined as touching the correct on-screen stimulus while refraining from touching an incorrect stimulus. An "incorrect" response was defined as touching an incorrect stimulus or touching both correct and incorrect stimuli. Any other behavior was scored as "no response" and was counted as incorrect. If the participant performed correctly, then they were exposed to further similar trials in that level/section. If a participant failed to perform correctly on any trial, then they were reexposed to the demonstration and guided trials before continued exposure to independent trials. On a demonstration trial, the experimenter gave the vocal antecedent "look" before modeling touching the correct stimulus and saying, "This is the right one"; on a guided trial, the experimenter lightly touched participants' hand or elbow to guide them to the correct response; while on an independent trial, the participant was given a vocal antecedent (e.g., "Find the right one") and had to respond independently within 5 s.
Each level/section had a criterion for number of correct responses needed to pass as well as for cumulative errors designating a fail (see Table 9 in the Appendix). If a participant succeeded in demonstrating a predetermined number of consecutive correct responses, then they were deemed to have passed that level/section and were subsequently exposed to the next section.
Appendix Table 9 TARPA scoring system Stage Track Level Points Pass Fail Criterion Criterion 1 V I (I 0.33 5/6 CC 6 CI stimulus) V 2 (2 0.33 5/6 CC 6 CI stimuli) V 3 (3 0.33 8/9 CC 9 CI stimuli) A 1 (I 0.33 5/6 CC 6 CI stimulus) A 2 (2 0.33 5/6 CC 6 CI stimuli) A 3 (3 0.33 8/9 CC 9 a stimuli) 2 V t (2comps) 0.5 5/6 CC 6 CI V 2 (3comps) 0.5 5/6 CC 6 CI V 3 (GID) 1 5/5 CC 6 CI A I (2comps) 0.5 5/6 CC 6 CI A 2 (3cumps) 0.5 5/6 CC 6 CI A 3 (cumps) 1 5/5 CC 6 CI 3 VV CD 3 5/6 CC 6 CI VV ME 4 7/8 - Correct VV CE 4 7/8 - Correct VV TOF 3 7/8 - Correct AV1 CD 3 5/6 CC 6 CI AV 1 ME 4 7/8 - Correct AV 1 CE 4 7/8 - Correct AV 1 TOF 3 7/8 - Correct AV2 CD 3 5/6 CC 6 CI AV2 ME 4 7/8 - Correct AV2 CE 4 7/8 - Correct AV2 TOF 3 7/8 - Correct AA CD 3 5/6 CC 6 CI AA ME 4 7/8 - Correct AA CE 4 7/8 - Correct AA TOF 3 7/8 Correct Nok. GID Generalized Identity, CD Conditional Discrimination, ME Mutual Entailment, CE Combinatorial Entailment, TOF Transfer of Function, Wisual. A Auditory, WVisual Visual, AV / Auditory-Visual 1, ,4V2 Auditory--Visual 2, AA Auditory Auditory; CC Consecutively Correct, Cl Cumulatively Incorrect, comps comparisons
Criteria for Passing/Failing If a participant made a predetermined cumulative number of incorrect responses, then they were deemed to have failed that level/section. Failure at any particular point of the TARPA had consequences that depended on the part of the TARPA concerned (see Materials section). In Stage 1, participants were exposed to all levels of both the visual and auditory tracks. If a participant failed all levels in both tracks, then the assessment was finished at that point. However, if they passed even one level in either track, then they were also exposed to all levels of both tracks in Stage 2. With respect to the latter, there were similar guidelines. If a participant failed all levels, then the assessment was finished; however, if a participant passed even just one level in either track, then he or she was allowed to proceed to Stage 3. As described earlier, Stage 3 involved four tracks, each with a different stimulus modality, and the participant was exposed to each of the four tracks in turn. Each track involved four levels, each of which included a certain number of sections. Level 1 (A-B conditional discrimination training) involved only one Section. A participant was given two chances to pass. If the participant failed twice, then the assessment of that track came to a finish; if the participant passed, then he or she was exposed to Level 2 (mutual entailment training and testing). Level 2 involved three sections. The participant was given two chances to pass the first section (a review of A-B conditional discrimination training). If the participant failed, then the assessment finished for that track. If the participant passed, then he or she was exposed to the second section (unreinforced A-B conditional discrimination--"maintenance"). If the participant failed the latter, then he or she was reexposed to training in Section 1 (i.e., with reinforcement) and then given one further exposure to Section 2. If he or she failed again, then the assessment finished for that track. If the participant passed Section 2, then he or she was exposed to Section 3 (derived B-A mutual entailment testing). If the participant passed the test of mutual entailment, then he or she was moved on to Level 3 (combinatorial entailment training and testing), while if he or she failed, then the assessment for that track was finished. Level 3 involved six sections. If participants passed any section, then they moved on to the next one. If they failed Section 1 (A-B conditional discrimination training) twice, then the assessment for that track finished there. If they failed Section 2 (A-B maintenance), then they received one further exposure to both Sections 1 and 2, and if they failed the latter again, then the assessment of that track was finished. If they passed Section 2, then they were exposed to Section 3 (C-B conditional discrimination training). If they failed Section 3 twice, then the assessment for that track finished. If they passed Section 3, then they were exposed to Section 4 (C-B maintenance). If they failed the latter, then they received one further exposure to both Sections 3 and 4, and if they failed the latter again, then the assessment of that track was finished. If they passed Section 4, then they were exposed to Section 5 (A-B and CB conditional discrimination training, mixed maintenance). If they failed the latter, then they were exposed to the previous sections (i.e., 1 to 4) one further time. If they failed again, then the assessment for that track was finished. If they passed, then they were exposed to Section 6 (derived A-C and C-A combinatorial entailment testing). If they failed the latter, then their assessment in that track was finished, whereas if they passed, then they were exposed to Level 4 (transformation of function). In Level 4, the final level of assessment for each track, participants were first reexposed to sections that tested maintenance of the A-B and C-B conditional discriminations, as well as of the derived A-C and C-A relations. They were then exposed to discriminative function training. If they failed discriminative function training, then the assessment for that track was finished; if they passed, then they received transformation of function testing.
It should be noted that all data from participants in both Experiments 1 and 2 were included in a hierarchical analysis. To provide additional data, a subset of participants from both experiments was tested on all levels of the TARPA, meaning that they continued to be tested even after they had officially failed at a particular level. The participants exposed to this testing included P6 from Experiment 1. However, the results of the hierarchical analysis will not be discussed until the Results section for Experiment 2.
Results and Discussion
Scores for both the TARPA and the PLS-4 are shown (see Table I). Participants had to pass a level in order to receive all the points corresponding to that level. Scores for the TARPA were calculated (see Table 9, in the Appendix). Children who passed a level received all the points corresponding to that level. Scores (raw, standard, and age equivalent) for the PLS-4 were calculated based on criteria outlined in the PLS-4 manual. A Spearman's rank order correlation showed a strong and significant correlation between the TARPA scores and the PLS-4 total raw scores, p (8) = 0.99, (p = .002). Tests were also conducted to examine the relationship between TARPA performance and that on each of the two PLS-4 subscales (raw scores). These revealed significant correlations with both Auditory Comprehension, [rho] (8) = 0.94, ([rho] = .004), and Expressive Communication, rho (8) = 0.999, (rho = .002).
Individual TARPA Performance A detailed breakdown of individual performances on the TARPA is shown (see Table 2). There was wide variation in performance across participants. PI failed all levels in both tracks of Stage 1 (simple discrimination) and did not proceed any further. P2 passed the first two visual levels of Stage 1 and failed all subsequent levels to which he was exposed, including all levels of the auditory track in Stage 1 and all levels of both tracks in Stage 2. P3 and P8 each passed levels in both Stages 1 and 2, but failed all initial levels of Stage 3. P3 passed the first two visual levels of Stage 1 but failed the third level. The latter failure appeared to be an attentional issue, and he was therefore retested using new set of stimuli in the context of which he passed. He engaged in off-task responding while being tested for the first of the auditory levels of Stage I and was therefore again retested with a new set of stimuli and passed. He also passed the second of the auditory levels but then failed the third. He passed Level 1 of the visual track of Stage 2 but failed all subsequent levels of the visual track and all levels of the auditory track in this stage. He also failed the first level of each of the four tracks of Stage 3. P8 passed all levels in both tracks of Stage 1, with the exception of the final auditory level. In Stage 2, he passed the first two visual track levels and failed everything else, including all three auditory levels.
Table 2 TARPA and PLS4 scores for Experiment 1 Pt. TARPA PLS-4 PLS-4l PLS-4 PLS-4 PLS-4 LAE Number Score Raw AC Raw EC Tolal SS 1 0 19 11 30 N/A 00: 09 2 0.66 19 19 38 55 01: 01 3 2.16 25 23 48 55 01: 05 4 6.32 32 34 66 55 02: 01 5 22.65 52 54 10ft 55 04: 00 6 45 62 66 128 N/A 06: 04--06: 05 7 59 62 68 130 N/A >06: 05 8 2.65 32 25 57 55 01: 09 9 44 62 65 127 N/A 06: 01--06: 03 10 12.15 31 35 66 N/A 02: 01 Pt. Participant, AC Auditory Comprchension, EC Expressive Communication, SS Standardized Score, LAE Language Age Equivalent
All the remaining participants passed at least some levels of Stage 3, but there was also significant variation within this stage. P4 and PIO passed one or more tracks at Level 1 (arbitrary conditional discrimination) but failed to move beyond this level. 135 passed mutual entailment in two tracks (Visual--Visual and Auditory--Visual 2) but failed to show combinatorial entailment in any track. P6 and P9 both demonstrated at least combinatorial entailment in two tracks. P6 showed combinatorial entailment and transformation of function in the Visual--Visual track and combinatorial entailment alone in the Auditory--Visual 1 track, while P9 showed combinatorial entailment and transformation of function in the Visual--Visual and Auditory--Visual 2 tracks. Finally, P7 showed transformation of function for all tracks, except Au-ditory--Visual 2.
An additional analysis was performed with respect to the hierarchical ordering of TARPA substages. In a multistage hierarchically structured test, a participant should not be able to succeed at an ostensibly more difficult level after failing on an ostensibly less difficult one. The data reported by Moran et al. (2010) suggested that the initial TARPA protocol on which they conducted their analysis broadly conformed to this criterion. In order to perform a similar analysis on the data from the current TARPA, we converted the scores obtained by individual participants on each of the three stages into percentages of correct responding; for example, if a child was correct on all levels of a stage, then he or she scored 100 % correct, while if the child was correct on only half, then he or she scored 50 % correct. Comparison of these percentages showed that none of the 10 participants achieved a greater percentage on a more advanced than on a less advanced stage, thus supporting the hierarchical structuring of the TARPA.
Summaty Hence, the main pattern of results for Experiment I was the finding of a strong and significant correlation between the TARPA and the PLS-4, as well as between the TARPA and each of the two PLS-4 subscales (i.e., Auditory Comprehension and Expressive Communication) in children with ASD. Furthemore, as just indicated, this pattern of results was found with a more comprehensive protocol (involving most importantly a greater range of modalities in the tests for each of the derived relational properties) and with a larger sample of children. Finally, there was also evidence of the hierarchical structuring of the TARPA.
The aim of Experiment 2 was to extend the findings of prior TARPA research, including Experiment 1, in two potentially important respects. First, it employed typically developing children. Testing was conducted on the latter to acquire data from an alternative population of children than those with ASD. The performance of the former might be expected to allow more detailed analysis of the upper portion of the TARPA and would also allow for comparison with the ASD group. As suggested in the introduction, typically developing children readily learn derived relational responding through exposure to the natural language environment (see e.g., Lipkens et al. 1993; Luciano et al. 2007), while children with ASD, amongst other categories of developmental delay, do not easily learn these patterns (e.g., Rehfeldt et al. 2007). Hence, it might be expected that the former group would outperform the latter on the TARPA. The findings of Experiment 2 might also allow for comparison with respect to other dimensions, such as performance across different stimulus modalities.
This second experiment also added an additional 12 participants to the overall sample tested, giving a total of 22 for the study as a whole, thus increasing the power of a potential correlational analysis. This larger number also facilitated a more formal order analysis of the protocol (see, e.g., Krus et al. 1975) than had previously been possible. Order analyses have been used in the case of previous behavior analytic assessments. For example, Kerr et al. (1977) used an order analysis to demonstrate that levels employed in the Assessment of Basic Learning Abilities (ABLA) were hierarchically ordered. More specifically, they showed a general trend (111 out of a total of 117 participants) in which failure at any ABLA level predicted failure at higher levels, while passing a level only occurred if all lower levels had been passed. This finding has since been replicated in other studies (see Martin & Yu, 2000, for a review) and was also found to hold for typically developing children (Casey & Kerr, 1977). The order analysis conducted in the current study, which included a focus on the levels of Stage 3 in particular, was made possible by combining participants from both experiments. This afforded a sufficiently large sample size to allow a meaningful comparison between levels.
Participants Thirteen typically developing children (6 male, 7 female; age range 2 years, 11 months-6 years, 4 months; see also Table 2) were recruited through a primary school (St. John's National School, Breaffy, Castlebar Co. Mayo, Ireland) and a playschool attached to the school. Consent for conducting the study was obtained from the school principal for the primary school. Parental consent was obtained for each participant and verbal consent was obtained from each of the children. None of the children had taken part in similar research before. Prior ethical approval for recruitment of participants for this study was obtained from the National University of Ireland, Galway, Research Ethics Committee. One participant (P22) dropped out of the study before completing the TARPA assessment. Though the limited data provided by this participant could not be included in any of the correlational analyses, they were used in the order analysis.
Materials See Experiment 1.
Procedure The PLS-4 and the TARPA were administered in the same manner as in Experiment 1. Specific information on number and length of sessions is provided (see Table 3).
Table 3 Performances of individual participants for each TARPA stage and level in Experiment 1 Stage 1: Simple discrimination Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 Vis. L1 F P P P P P P P P P Vis. L2 F P P P FP P P P P P Vis. L3 F F FP P P P P P P P Aud. L1 F F FP P P P P P P P Aud. L2 F F P F P P P P P P Aud. L3 F F F F P P P F P F Stage 2: Nonarbitrary conditional discrimination (i.e., formally SIMILAR stimuli) Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 Vis. L1 n/a F P P P P P FP P P Vis. L2 n/a F F P P P P P P P Vis. L3 n/a F F P P P F F P F Aud. L1 n/a F F F P P P F P P Aud. L2 n/a F F F P P P F P F Aud. L3 n/a F F F P P P F P F Stage 3: Arbitrary conditional discrimination (i.e., formally DISSIMILAR stimuli) Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 Visual-Visual L1 CD n/a n/a F P P P P F P P L2 ME n/a n/a n/a F P P P n/a P F L3 CE n/a n/a n/a n/a F P P n/a P n/a L4 TOF n/a n/a n/a n/a n/a P P n/a p n/a Auditory-Visual 1 L1 CD n/a n/a F F P P P F P F L2 ME n/a n/a n/a n/a F P P n/a F n/a L3 CE n/a n/a n/a n/a n/a P P n/a n/a n/a L4TOF n/a n/a n/a n/a n/a F P n/a n/a n/a Stage 3: Arbitrary conditional discrimination (i,e., formally DISSIMILAR srimuli) Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 Auditory-Visual 2 L1 CD n/a n/a F F P P P F P P L2 ME n/a n/a n/a n/a P P P n/a P F L3 CE n/a n/a n/a n/a F F P n/a P n/a L4 TOF n/a n/a n/a n/a n/a n/a F n/a P n/a Auditory-Auditory L1 CD n/a n/a F F F P P F P P L2 ME n/a n/a n/a n/a n/a P P n/a P F L3 CE n/a n/a n/a n/a n/a F P n/a F n/a L4 TOF n/a n/a n/a n/a n a n/a P n/a n/a n/a F followed by bold P indicates that the participant showed seriousattention deficits during the first test and passed after retesting withnew stimuli CD Conditional Discrimination, ME Mutual Entailment; CE CombinatorialEntailment; TOF Transfer of Function
Results and Discussion
Table 3 shows each participant's score on the PLS-4 (raw auditory comprehension, raw expressive communication, total raw score, standard and age equivalent scores) and score on the TARPA, while Table 4 provides a detailed breakdown of individual TARPA performances. A Spearman's rank order correlation test was used to examine the relationship between performance on the TARPA and language. A statistically significant correlation was found (p=0.94, p=.0018) between total TARPA score and total raw PLS-4 score. Further tests were conducted to examine the relationship between TARPA performance and scoring on the subsections (raw scores) of the PLS-4. Findings showed significant correlations between the TARPA and auditory comprehension ([rho] = 0.98, [rho] = .0018) and between the TARPA and expressive communication ([rho] = 0.89, [rho] = .002). The auditory subsection correlated more strongly with the TARPA than the Expressive subsection.
Table 4 Demographic information and TARPA performance parameters for Experiment 2 Pt. Age (Y: Sex No. Mean Highest Highest No. M) Sessions Session Level Level Length Passed Tested II 04:00 M 8 25 S3L2 (ME) S3L3 (CE) 12 03:02 M 10 20 S3LI (CD) S3L2 (ME) 13 02:11 M 8 20 SIL3 (A&V) S2 (A&V) 14 04:11 F 4 25 S3L2 (ME) S3L3 (CE) 15 04:10 F 3 35 S3L3(CE) S3L4 (TOF) 16 06:01 F 4 35 S3L4 (TOF) S3L4 (TOF) 17 05:10 M 4 15-20 S3L4 (TOF) S3L4 (TOF) 18 05:01 F 2 30-45 S3L4 (TOF) S3L4 (TOF) 19 06:04 M 2 25-30 S3L4 (TOF) S3L4 (TOF) 20 05:08 M 3 30 S3L4 (TOF) S3L4 (TOF) 21 05:04 F 4 20-25 S3L2 (ME) S3L3 (CE) 22 04:10 F n/a n/a n/a n/a 23 05:02 F 5 20 S3L2 (ME) S3L3 (CE) Pr. Participant, A Auditory, V visual, CD Conditional Discilmination. ME Mutual Entailment. TOF Transformation of Functions
Individual TARPA Performance A detailed breakdown of individual performances on the TARPA is provided (see Table 4). All participants were typically developing children, but there was some variation in age, and the effect of this variation is visible in the table. The youngest participant (P13), who was just under 3 years of age, passed all levels (both visual and auditory) of Stage 1, but then failed all subsequent levels to which he was exposed, including all those in Stage 2 and the first (arbitrary conditional discrimination) level of each of the tracks on Stage 3. P12 was 3 years, 2 months old. Though failing the third level (three comparison discrimination of the auditory track of Stage 1; this was the only failure in Stage 1 of any child in Experiment 2), he passed Stage 2, Levels 1 and 2 (two and three comparison conditional discrimination with all visual stimuli) and also passed Level 1 (two comparison arbitrary conditional discrimination) of the Auditory--Visual and Auditory--Auditory tracks in Stage 3. P11, who was exactly 4 years old, passed all levels in both Stages 1 and 2, except for Level 3 (nonarbitrary relational responding or identity matching) of the visual track of the latter. He passed the first level on each of the tracks of Stage 3, except for the visual track, and also passed the test for mutual entailment in the Auditory--Visual track. All of the remaining participants passed all levels of Stages 1 and 2. Three of the participants (P15, P18, and P23) were approximately 5 years of age. These children all passed mutual entailment in at least one track, but none passed it in all four tracks. With regard to combinatorial entailment, there was some variation in that P23 failed to show this pattern of derivation at all, P15 showed it in just one track, and P18 showed it in two. The three oldest participants in the group were approximately 6 years of age. These children all passed mutual entailment in all cases except one (i.e., P17 on AV1), and each of them also passed at least two tests of combinatorial entailment. The last two children, P20 and P21, were between ages 5 and 6. P20 was 5 years, 8 months while P21 was 5 years, 4 months. Despite being only 4 months apart in age, however, the performance of these two children on both the TARPA and PLS-4 showed a much wider divergence. P20 showed a TARPA performance similar to that of the 6-year-olds (showing combinatorial entailment on all stages except Auditory--Auditory) while also achieving an age equivalent score of 6 on the PLS-4, while P21 showed a TARPA score similar to the lower scoring 5-year-olds, and achieved an age equivalence PLS-4 score of 4 years, nine months.
As in Experiment 1, an additional analysis was performed with respect to the hierarchical ordering of TARPA substages. Once again, we converted the scores obtained by individual participants on each of the three stages into percentages of correct responding. Comparison of these percentages showed that none of the 12 participants achieved a greater percentage on a more advanced than on a less advanced stage, thus once again supporting the hierarchical structuring of the TARPA.
Combined Correlational Analysis Data from Experiments 1 and 2 were combined for purposes of further correlational analysis. A Spearman's rank order test was used to examine the correlation between TARPA and PLS-4 performance for 22 participants across both experiments. A statistically significant correlation was found ([rho] = 0.96, [rho] < .00001) between total TARPA and total raw PLS-4 score. Further tests were conducted to examine the relationship between TARPA and PLS-4 subsections (raw score) performance. Findings showed significant correlations between the TARPA and both auditory comprehension ([rho] =0.95, [rho] < .00001) and expressive communication ([rho] = 0.95, [rho] < .00001).
Order Analysis Order analysis (Kerr et al. 1977; Krus, Bart & Airasian (1975); Marion et al. 2003) was used to evaluate the hierarchical structure of the TARPA. Table 5 compares each of the 10 TARPA levels with each of the other levels, and in the case of each comparison shows (a) number of confirmations [C] and disconfirmations [D] for those two levels, where a confirmation is a pattern in which the lower level was passed and the higher level was failed while a disconfirmation is a pattern in which the higher level was passed and the lower level was failed; (b) a z-score measurement for that pattern; and (c) a p value corresponding to that z score (see Table 5).
Table 5 TARPA and PLS4 scores for Experiment 2 Pt. TARPA PLS-4 PLS-4 PLS-4 PLS-4 PLS-4 LAE Number Score Raw AC Raw EC Total SS 11 18 49 52 101 90 03:08 12 11.65 39 42 81 87 02:08 13 2 35 40 75 91 02:11 14 22 55 57 112 96 04:05 15 30 59 63 122 116 05:04-05:05 16 45 61 66 127 99 06:01-06:03 17 44 60 66 126 107 05:11-06:00 18 44 58 62 120 94 05:02 19 48 61 66 127 99 06:01-06:03 20 48 61 65 126 106 05:11-6:00 21 22 55 61 116 86 04:09 22 * 55 57 112 96 04:05 23 26 57 65 122 100 05:04 05:05 Pt. Participant, AC Auditory Comprehension, EC Expressive Communication SS Standardized Score, LAE Language Age Equivalent
As can be seen, the majority of the cells of the table (i.e., 44/45) had more confirmations than disconfirmations of the predicted order. In addition, in most cells, there is a statistically significant ([rho] < .05) effect in favor of the predicted hierarchy for the comparison between number of confirmations and number of disconfirmations. In the case of all but one of the cells, nonsignificant effects trend in the predicted direction, but the number of recorded comparisons is low and this is likely the factor that is mainly responsible for the lack of significance. There are four cells in this category that include the comparisons between S1L1 and S1L2; S1L3 and S2L1; S2L1 and S2L2; and S2L3 and S3L1. For each of these, further data might indicate a reversal of the predicted hierarchical order. In the case of the first of these (S ILI vs. Si L2), it seems unlikely, because it would seem that two stimuli, even readily discriminable ones, are unlikely to be more discriminable than one stimulus versus the absence of a stimulus. Similarly in the case of the third comparison (S2L1 vs. S2L2), it would also seem unlikely, since the only difference in this case is that the higher level includes more comparisons. In the case of the second comparison (S1L3 vs. S2L1), it is arguable that further empirical work investigating relative difficulty level is warranted. The current analysis is based on the amalgamation in the cases of Stages 1 and 2 of visual- and auditory-level data. This particular disconfirmation involves one participant with ASD who failed the most difficult auditory level in Stage 1 and passed the least difficult visual level in Stage 2. A more in-depth empirical investigation might thus focus on the difference between auditory and visual stimuli. which may perhaps be most pronounced for those with ASD. We will return to this theme in the general discussion. In the fourth comparison (S2L3 vs. S3L1), it is arguable that further empirical work is also warranted. Despite S2L3 involves responding in accordance with nonarbitrary relations and S3L1 involves learning an arbitrary relation, the task in S2L3 involved three comparisons and required choosing correctly on five novel trial types, whereas the task in S3L1 involved two comparisons and the same trial type throughout.
One of the 45 data cells in Table 5 comparison S3L3 (combinatorial entailment) versus S3L4 (transformation of function) has more disconfinnations than confirmations, and thus the suggested hierarchy seems at odds with the found pattern. Of course, it should not be the case that participants can pass the latter after failing the former. Perhaps one reason that this test (which required the performance of particular actions by the participant dependent on the on-screen stimulus) might have been susceptible to a "false positive" (i.e., a participant passing for reasons other than showing genuine transformation of function) is that it involved more direct interaction with the experimenter than other tasks and might therefore have been more susceptible to inadvertent cuing by the latter. This is certainly an issue of which those testing at this level of the protocol should be aware.
Another possibility is that some participants simply responded consistently across the task, whether correct or incorrect, and that in the event that their initial answer was correct this consistency resulted in an ostensible pass. In order to minimize the possibility of this occurring in the future, further testing might expose participants to more than one transformation of function test. It should be noted that, even though the current analysis does not show a similar weakness with respect to order in other sections of the TARPA, the possibility that participants might have passed other levels in a similar way cannot be discounted. This possibility is made more probable by the fact that in almost all sections of the TARPA, the testing tasks involve just two comparisons. The decision to utilize two choice tests throughout the protocol was taken in order to minimize training times and thus make the TARPA assessment as brief and efficient as possible; nevertheless, this can increase the possibility of false-positive results achieved on the basis of "consistently maintained correct guessing."
Of course, given multiple stages and levels across the protocol, it is unlikely that the latter might allow participants to attain a much higher score than they would otherwise attain, but the possibility of "consistently maintained correct guessing" as the basis for one false positive is certainly something that needs to be addressed in the future development of the protocol. For example, requiring correct performance with more than one stimulus set might make such a possibility less likely.
A hierarchical analysis for each of the four tracks in Stage 3 is shown (see Table 6). Once again, it shows (a) number of confirmations [C] and disconfirmations [D]; (b) a z-score measurement for that pattern; and (c) a p value corresponding to that z score. A similar pattern as for the TARPA as a whole is apparent. More specifically, in most cells there are no disconfirmations and/or there is a statistically significant difference between the number of confirmations and disconfirmations, which supports the hierarchy. The significance pattern is weaker than for Stage 3 in Table 5, but this is because there are less exemplars per track than across tracks. Once again, the predicted hierarchical pattern appears to hold for most cells of the table with the main deviation in respect of this pattern appearing for the comparison between S3L3 (combinatorial entailment) and S3L4 (transformation of function) (Tables 7 and 8).
Table 6 Performances of Individual Partieipants for Each TARPA Stage and Level in Experiment 2 Stage 1: Simple discrimination Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 Vis. LI P P P P P P P P P P P Vis. L2 P P P P P P P P P P P Vis. L3 P P P P P P P P P P P Aud. LI P P P P P P P P P P P Aud. L2 P P P P P P P P P P P Aud. L3 P F P P P P P P P P P Stage 2: Nonarbitrary conditional discrimination (i,e., formally SIMILAR stimuli) participant P1 P2 P3 P4 P5 P6 P7 P8 P9 Pl0 11 Vis. LI P P F P P P P P P P P Vis. L2 P P F P P P P P P P P Vis. L3 F F F P P P P P P P P Aud. LI P F F P P P P P P P P Aud. L2 P F F P P P P P P P P Aud. L3 P F F P P P P P P P P Stage 3: Arbitrary conditional discrimination (i,e., formally DISSMILAR stimuli) Participant P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 Visual-Visual LI CD F P F* P P P P P P P P L2 ME F* n/a n/a F P P P P P P F L3 CE n/a n/a n/a F* F P P F P P n/a L4TOF n/a n/a n/a F* F* P P P* P F n/a Auditory-Visual 1 LI CD P F F* P P P P P P P P L2 ME F n/a n/a F P P F P P P F L3 CE n/a n/a n/a F* P F F* P P P F* L4 TOF n/a n/a n/a n/a F P* F* P P P F* Stage 3: Arbitrary conditional discrimination (i,e., formally DISSIMILAR stimuli) Participant P1 P2 P3 P4 P5 P7 P8 P9 P10 P11 Aud itory- 2 Visual LI CD P P F* P P P P P P P P L2 ME P F n/a P F P P P P P F L3 CE F n/a n/a F n/a F P P F P F* L4TOF n/a n/a n/a F* n/a n/a P P F* P F* LI CD P P F* P P P P P P P P L2 ME F F n/a F F P P F P F P L3 CE n/a n/a n/a F* n/a P F n/a F F* F L4TOF n/a n/a n/a F* n/a F F* n/a F* F* P* Participant P12 P13 Vis. LI P P Vis. L2 P P Vis. L3 P P Aud. LI P P Aud. L2 P P Aud. L3 P P participant P12 P13 Vis. LI P P Vis. L2 P P Vis. L3 P P Aud. LI P P Aud. L2 P P Aud. L3 P P Participant P12 P13 Visual-Visual LI CD P P L2 ME P P L3 CE F F L4TOF F* F* Auditory-Visual LI CD P P L2 ME F F L3 CE n/a F L4 TOF n/a P* Participant P12 P13 Aud itory- Visual LI CD DO P L2 ME n/a P L3 CE n/a F L4TOF n/a P* LI CD DO P L2 ME n/a F L3 CE n/a F* L4TOF n/a F* * indicates additional testing for the hierarchy. DO denotes dropout from testing. CD Conditional Diserimination, ME Mutual Entailment, TOF Transformation of Functions Table 7 Order analysis of the TARPA including all stages and levels based on the data provided by the 23 participants across Experiments 1 and 2 Stage/Level Stage 1 Stage 1 Stage 2 Stage 2 Stage Vis & Vis & Vis & Vis & Vis & 2 Aud L2 Aud L3 Aud L1 Aud L2 Aud L3 C D C D C D C D C D Combination Across Tracks Stage 1 L1 2 0 7 0 7 0 8 0 11 0 Vis & Aud z=1,4142 z= 2.6457* z=2.6457* z=2.82* z=3.316* P=0.0 P=0.0041 P=0.0041 P=0.0024 P=0.0005 787 Stage 1 L2 - 6 0 6 0 7 0 10 0 Vis & Aud z=2.449* z=2.449* Z=2.6457* Z=3.192* P=0.0072 P=0.0072 P=0,0041 P=0.0007 Stage 1 L3 - - 2 1 3 0 8 0 Vis & Aud Z=0.5773 Z=1.732* Z=2.82* P=0.2819 P=0.0416 P= 0.0024 Combination across Tracks Stage 2 L1 - - - 2 0 6 0 Vis & And Z=1.4142 Z=2.449* P=0.0787 P=0.0072 Stage 2 L2 - - - 4 0 Vis & Aud Z=2* p=0.0228 Stage 2 L3 - - - - - Vis & Aud Combination across Tracks Stage 3. - - - - LI. Cond. Dis (A-B) Stage 3. - - - - - L2. Mutual Entailment Stage 3. - - - - - L3. Stage 3. Stage 3. Stage 3. Stage 3. L1. L2 L3 L4 Cond. Mut. Comb. TOF Dis Entail. Entail. C D C D C D C D Comb. Entailment Stage 1 L1 13 0 11 0 12 0 12 0 Vis & Aud z=3.605* z=3.317* z=3.4* z=3.4* P=0.0002 P=0.0005 P=0.0003 P=0.0003 Stage 1 L2 13 0 9 0 12 0 12 0 Vis & Aud Z=3.605* z=3* z=3,4* z=3.4* P=0.0002 P=0.0013 P=0.0003 P=0.0003 Stage I L3 7 1 8 0 12 0 12 0 Vis & Aud Z=2.121* Z=2.82* Z=3.4* Z=3.4* P=0.017 P=0.0024 P=0.0003 P=0.0003 Stage 2 L1 8 1 8 0 12 0 12 0 Vis & And Z=2.333* Z=2.82* Z=3.4* Z=3.4* P=0.0009 P=0.0024 P=0.0003 P=0.0003 Stage 2 L2 6 0 8 0 12 0 12 0 Vis & Aud Z= 2.449* Z=2.82* Z=3.4* Z=3.4* p= 0.0072 p=0.0024 p=0.0003 p=0.0003 Stage 2 L3 1 0 6 0 12 0 12 0 Vis & Aud Z=l Z=2.45* Z=3.4* Z=3.4* P=0.1587 p=0.007 p=0.0003 p=0.0003 Stage 3. - 18 0 26 0 21 0 LI. Cond. Dis Z=4.238* Z=5.099 Z=4.5825* (A-B) p=0.0000 p=0.0000 p=0.0000 Stage 3. - - 17 0 14 2 L2. Mutual Z=4.123* Z=3.00* Entailment p=0.0000 p=0.0013 Stage 3. - - - 6 7 L3. Comb. Z=-0.27 p= Entailment 0.6064 A confirmation (C) is recorded when a participant passes a lower level and fails a higher level. A disconfirmation (D) is recorded when a participant fails a lower level and passes a higher level. An asterisk (*) denotes significance based on a (one-tailed) right-tailed p value (z > 1) Table 8 Order analysis of TARPA Stage 3 for each of the four tracks based on the data provided by the 23 participants across Experiments and 2 Levels Stage 3.L2 Stage 3.L3. Stage 3.L4. Mutual Comb. Transf Stim Entailment Entailment Function C D C D C D Visual-Visual Stage 3. L1. 3 0 6 0 6 0 Cond. Dis (A-B) z=1.732* z=2.449* z=2.449* p=0,0416 p=0.0072 p=0.0072 Stage 3. L2. - 5 0 4 0 Mutual z=2.236* z=2* Entailment p=O.O127 p=0.0228 Stage 3. L3. - - 2 1 Comb. Z=0.57736 Entailment p=0.2818 Auditory-Visual 1 Stage 3. L1. 7 0 5 1 4 0 Cond. Dis (A-B) z=2.6457* 7=2.236* z=2* p=0.0041 p=0.0127 p=0.0228 Stage 3.12. - 1 0 2 1 Mutual z=1 z=0.57736 Entailment p=0.1587 p=0.2818 Stage 3. L3. - - 2 2 Comb. z=0 Entailment p=0.50 Auditory-Visual 2 Stage 3. L1 2 0 8 0 5 B Cond. Dis (A-B) z= 1.4142 z=2.82* z=2.236* p= 0.0787 p=0.0024 p=0.0127 Stage 3. L2. - 7 0 4 0 Mutual z=2.6457* z=2* Entailment p=0.0041 p=0.0228 Stage 3. L3. - - 1 1 Comb. z=0 Entailment p=0.50 A uditory-Audi tory Stage 3. L1 6 0 7 0 6 0 Cond. Dis(A-B) z=2.449* z=2.6457* z=2.449* p=0.0072 p=0.0041 p=0.0072 Stage 3. L2. 4 0 4 1 Mutual z=2* z=1.345 Entailment p=0.0228 p=0.0893 Stage 3. L3. - - 1 2 Comb. z=-0.57 Entailment p=0.2843 A confimiation (C) is recorded when a participant passes a lower level and fails a higher level. A disconfirmation (D) is recorded when a participant fails a lower level and passes a higher level. An asterisk (*) denotes significance based on a (one-tailed) right- tailed p value (z>1)
General Discussion The TARPA protocol was developed to assess the emergence of derived relational responding or relational framing, which RFT theorists have argued to be the key process underlying language. Moran et al. (2010) showed a correlation between performance on an initial version of the TARPA and adaptive behavior as measured by the VABS in several children with ASD. The TARPA also showed a borderline correlation with the communication subscale of the VABS. This result supported the RFT approach to language as derived relational responding and the potential utility of this approach for predicting and remediating language for children with ASD, and, more specifically, suggested the potential of the TARPA as an RFT-based protocol for assessing derived relational responding.
The current study was similar to Moran et al.'s (2010) in that it correlated a version of the TARPA with a potentially important alternative measure of behavior. However, it extended Moran et al. in several key respects. First, this study used a more comprehensive and systematic TARPA protocol; second, it compared this protocol with a mainstream language assessment (i.e., the PLS-4); third, it employed the TARPA not just with children with ASD (Experiment 1) but also with typically developing children (Experiment 2); fourth, it used a substantially larger number of participants (i.e., n=10 in Experiment 1 and n=13 in Experiment 2, for a total of 23 overall); fifth, it conducted a statistical hierarchical analysis of the protocol; sixth, and finally, it used a touch screen rather than a mouse-based computer.
The main result of this study was the finding of a set of strong and significant correlations between the TARPA and the PLS-4 as well as between the TARPA and each of the two PLS-4 subscales (i.e., Auditory Comprehension and Expressive Communication). Furthermore, this pattern of results was found with a more comprehensive protocol (involving a greater range of modalities in the tests for each of the derived relational properties) and with a larger sample of children, including a subset of children with ASD (Experiment 1) as well as a subset of typically developing children (Experiment 2). This result thus constitutes further support for the TARPA as a methodology for assessment of derived relational responding and for RFT as a functional analytic approach to language.
The current study also advanced on Moran et al. (2010) in that it conducted a set of formal hierarchical analyses of the TARPA. In a multistage hierarchically structured test, a participant should not be able to succeed at an ostensibly more difficult level after failing on an ostensibly less difficult one. The data reported by Moran et al. suggested that the TARPA broadly conformed to this criterion, but the analysis used in that study was relatively rudimentary. The current study involved a set of more formal hierarchical analyses (see e.g., Kerr etal. 1977; Krus, Bart & Airasian (1975); Marion et al. 2003) that assessed the TARPA as a whole as well as focusing on the levels of Stage 3, which examined derived relational responding. Findings from both analyses indicated that the hierarchical structure of the TARPA is generally robust. The comparison between combinatorial entailment and transformation of function was one exception to this pattern. It was suggested that this potential weakness might be remedied in future versions of the protocol by exposing participants to more than one transformation of function test. It was also acknowledged that, even though the current analysis did not show a similar weakness with respect to order in other sections of the TARPA, the possibility that such weaknesses might exist cannot be discounted. It was argued that given multiple stages and levels across the protocol, it is unlikely that this type of pattern of performance might allow a participant to attain a much higher score than they would otherwise attain. Nevertheless, it can be argued that for important levels, such as those testing derivation, the TARPA might be improved by requiring correct performance with more than one stimulus set.
Moran et al. (2010) found that participants tended to show better performance with visual stimuli than with auditory stimuli. In the current study, a similar pattern is evident. To compare visual performance with auditory performance, we first compared participants' total scores on portions of the TARPA involving only visual stimuli (i.e., the total for scores on the visual tracks of Stages 1 and 2 plus the score for the Visual--Visual track of Stage 3) with their total scores on portions involving only auditory stimuli (i.e., the total for scores on the auditory tracks of Stages 1 and 2 plus the score for the Auditory-Auditory track of Stage 3). Eight out of the while P14 scored six on both) and one (P11) showed a higher score on auditory than on visual.
We also compared visual and auditory performance for Stage 3 alone by comparing aggregate performance on each of the four tracks in terms of the number of levels passed. For the children with ASD, the Visual--Visual track showed the highest number of passes (16), while the Auditory--Auditory and Auditory--Visual 1 track showed the lowest number (9). For the typically developing children, the Visual--Visual track also showed the highest number of passes (26), the Auditory--Auditory track showed the lowest number (16), while the two Auditory--Visual tracks showed intermediate numbers of passes (AV1=23; AV2=25). Finally, we also compared the tracks in terms of the number of participants who showed all three properties of derived relational responding (i.e., mutual entailment, combinatorial entailment, and transformation of function). On the Visual--Visual track, three children with ASD showed all three properties whereas in each of the other tracks which involved at least some auditory stimuli, only one did so. Out of the typically developing children, three participants showed all three properties on the Visual--Visual track, three showed all three on the Auditory--Visual 1 track, three showed all three on the Auditory--Visual 2 track, and none showed all three on the Auditory-Auditory track.
Hence, as suggested, in the case of both the children with ASD in Experiment 1 and the typically developing children in Experiment 2, participants tended to show better performance with visual than with auditory stimuli. Moran et al. (2010) suggested that this pattern of results might be explained on the basis of the difference in behavioral control obtained by simultaneous and successive discriminations. While this remains a very plausible explanation, another possible factor that might explain this pattern is that situations involving the types of auditory discrimination required in the TARPA protocol are not very common in the everyday socioverbal environment, at least not in the particular way in which they are presented in this context. It is possible that at least some of the trials involving auditory stimuli might require more training than other types of trials, because discrimination of auditory stimuli (i.e., discrimination of different sound patterns, identification of the context in which each is used) is essential in language learning, and yet it is not easily learned, requiring longer exposure in the everyday language environment.
Future Directions and Implications This study is the second in a series involving the use of the TARPA as a methodology for assessing derived relational responding. This work could be further extended in a number of possible directions in future studies. These might include (a) investigating the effect of the order of presentation of the Stage 3 tracks on scoring; for example, starting with the apparently least difficult track (e.g., Visual--Visual) and progressing through the other tracks to the apparently most difficult (e.g., Auditory--Auditory) might result in differences in scoring than the reverse order of presentation; (b) examining the test--retest reliability of the protocol; for example, will an individual tested on the TARPA using completely novel stimuli a week atter an initial test tend to get a similar score; (c) examining the performance of children with forms of developmental disability other than ASD on the current expanded version of the protocol; (d) investigating possible differences in the response patterns of various populations, including typically developing children as well as groups with distinct forms of developmental delay to various aspects of the protocol, including, for example, types of stimuli (e.g., abstract vs. real life) or forms of reinforcement (e.g., computer delivered vs. verbally delivered); (e) examining the effect of multiple exemplar training on performance in various stages of the TARPA; (t) expanding the TARPA protocol to examine forms of derived relational responding other than sameness (equivalence) relations (e.g., opposition, distinction, comparison).
Future research will further advance the TARPA as an efficient and effective assessment and training procedure targeting the core relational repertoire necessary for truly generalized language performance. Ultimately, the integration of computer-based assessment and training facilitated by protocols such as the TARPA will allow for increased efficiency in tracking students' performance and development over time, and increased efficiency and ease of data--based decision making as regards curricular sequencing and remediation. Thus far, the TARPA research has focused on assessment. However, versions of the protocol are currently being developed to directly facilitate training. Single--subject TARPA training is being conducted with individual children displaying particular deficits.
The TARPA protocol has the potential to significantly impact future research on the language development of both typically developing children and those with developmental delay. As a hierarchically sequenced measure of relational framing/language ability, the TARPA will provide an objective measure of progress and a unique measure of a core developmental skill. Assessment of the development of behavioral skills is important for both theoretical and practical reasons (Sidman 1986), and given the central importance of language in the human repertoire, this is particularly true with language development. A more fully completed version of the current protocol will contribute to theoretical knowledge by enabling researchers to track the sequential emergence and development of relational framing skills across a variety of research areas--from identifying more efficient remediation programs for combating early rote responding to tracking the development of perspective taking (deictic relational responding) or categorization (hierarchical relational responding) skills.
In summary, a more fully developed protocol would be a key instrument in facilitating the assessment and development of generative language and would thus constitute a significant addition to existing tools for early intensive behavioral intervention.
Conflict of Interest The authors hereby confirm the absence of any actual or potential conflict of interest regarding the present research.
Barnes-Holmes, D., & Murphy, C. (2007a). Addressing the generativity of language: A late reply to Chomslcy. In L. B. Zhao (Ed.), Autism research advances. New York, NY: Nova.
Barnes-Holmes, D., & Murphy, C. (2007b). Towards establishing generative language with children with autism. In B. S. Mesmere (Ed.), New autism research developments. New York, NY: Nova.
Barnes-Holmes, Y., Barnes-Holmes, D, Smeets, P. M., Strand, P., & Friman, P. (2004). Establishing relational responding in accordance with more-than and less-than as generalized operant behavior in young children. International Journal of Psychology and Psychological Therapy, 4, 531-558.
Barnes-Holmes, Y., Barnes-Holmes, D., & Smeets, P. M. (2004). Establishing relational responding in accordance with opposite as generalized operant behavior in young children. International Journal of Psychology and Psychological Therapy, 4, 559-586.
Berens, N. M., & Hayes, S. C. (2007). Arbitrarily applicable comparative relations: Experimental evidence for relational operants. Journal of Applied Behavior Analysis, 40, 45-71.
Carpentier, F., Smeets, P. M., & Barnes-Holmes, D. (2003). Equivalence-equivalence as a model of analogy: Further analyses. The Psychological Record. 53, 349-372.
Casey, L. & Kerr, N. (1977). Auditory-visual discrimination and language prediction [Monograph]. Rehabilitation Psycholoi., T. 24, 137-155.
Dymond, S., May, R. J., Munnelly, A., & Hoon, A. E. (2010). Evaluating the evidence base for relational frame theory: A citation analysis. The Behavior Analyst, 33, 97-117.
Giurfa, M. Zhang, S., Jenett, A., Menzel, R. & Srinivasan, M.V. (2001). The concepts of sameness and difference in an insect. Nature, 410, 930-932.
Greer, R. D., & Ross, D. E. (2008). Verbal behavior analysis: Inducing and expanding new verbal capabilities in children with language delays. New York, NY: Allyn & Bacon.
Harmon, K., Strong, R., & Pasnak, R. (1982). Relational responses in tests of transposition with rhesus monkeys. Learning and Motivation, 13 (4), 495-504.
Hayes, S. C., Barnes-Holmes, D., & Roche, B. (2001). Relational frame theory: A post-Skinnerian account of human language and cognition. New York, NY: Plenum Press.
Kerr, N., Meyerson, L., & Flora, J. A. (1977). The measurement of motor, visual, and auditory discrimination skills [Monograph]. Rehabilitation Psychology, 24, 95-112.
Krus, D. J., Bart, W. M., & Airasian, P. W. (1975). Ordering theory and methods. Los Angeles, CA: Theta Press.
Lipkens, R., Hayes, S. C., & Hayes, L. J. (1993). Longitudinal study of the development of derived relations in an infant. Journal of Experimental Child Psychologg 56(2), 201.
Lord; C., & McGee, J. P. (Eds.). (2001). Educating children with autism (Committee on Educational Interventions for Children with Autism, Division of Behavioral and Social Sciences and Education, National Research Council). Washington DC: National Academy Press.
Luciano, C., Gomez-Becerra, I., & Rodriguez-Valverde, M. (2007). The role of multiple-exemplar training and naming in establishing derived equivalence in an infant. Journal of the Experimental Analysis of Behavior 87(3), 349-365. doi:10.1901/jeab.2007. 08-06.
Malott, R. W. (2003). Behavior analysis and linguistic productivity. The Analysis of Verbal Behavior 19,11-18:
Marion, C., Vause, T., Harapiak, S., Martin, G. L., Yu, C. T., Sakko, G., & Walters, K. (2003). The hierarchical relationship between several visual and auditory discriminations and three verbal operants among individuals with developmental disabilities. The Analysis qf Verbal Behavior, 19, 91-105.
Martin, G.L., & Yu, D.C.T. (2000). Overview on research of the Assessment of Basic Learning Abilities test. Journal on Developmental Disabilities, 7(2), 10-36.
Moran, L., Stewart, 1., McElwee, J., & Ming, S. (2010). Brief report: the training and assessment of relational precursors and abilities (TARPA): a preliminary analysis. Journal of Autism and Developmental Disorders. 40(9), 1149-53.
Murphy, C., Barnes-Holmes, D., & Barnes-Holmes, Y. (2005). Derived manding in children with autism: synthesising Skinner's Verbal Behavior with relational frame theory. Journal of Applied Behavior Analysis, 38, 445-462.
Sidman, M. (1986). The measurement of behavioral development In N. A. Krasnegor, D. B. Gray, & T. Thompson (Us.), Developmental behavioral pharmacology (Vol. 5). Hillsdale, NJ: Erlbaum.
Stewart, I., & McElwee, J. (2009). Relational responding and conditional discrimination procedures: An apparent inconsistency and clarification. The Behavior Analyst, 32(2). 309-317.
Sundberg, M. L. (2008). VB-MAPP: Verbal behavior milestones assessment and placement program. Concord, CA: AVB Press.
Reese, H. W. (1961). Transposition in the intermediate size problem by pre-school children. Child Development, 32, 311-314.
Rehfeldt, R. A., .& Barnes-Holmes, Y. (2009). Derived relational responding: Applications for learners with autism and other developmental disabilities. Oakland, CA: New Harbinger.
Rehfeldt, R. A., Dillen, J. E., Ziomek, M. M., & Kowalchuk, R. K. (2007). Assessing relational learning deficits in perspective-taking in children with high functioning autism spectrum disorder. Psychological Record, 57(1), 23-47.
Roche, B., & Barnes, D. (1997). A transformation of respondently conditioned stimulus function in, accordance with arbitrarily applicable relations. Journal of the Experimental Analysis of Behavion 67, 275-300.
Wright, A.A. & Delius, J.D. (1994). Scratch and match: Pigeons learn matching and oddity with gravel stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 20, 108-112.
Zimmerman, I. L., Steiner, V. G., .& Pond, R. E. (2002). Preschool language scale (4th ed.). San Antonio, TX: Harcourt Assessment.
L. Moran * I. Stewart
School of Psychology, National University of Ireland
Galway, Galway City, Ireland
112 Havertbrd Drive, Wilkes-Barre, PA 18702, USA
218 S Castle St, Baltimore, MD 21231, USA
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||ORIGINAL ARTICLE; Training Assessement of Relational Precursors and Abilities|
|Author:||Moran, Laura; Stewart, Ian; McElwee, John; Ming, Siri|
|Publication:||The Psychological Record|
|Date:||Jun 1, 2014|
|Previous Article:||The impact of goals and pay on feedback-seeking behavior.|
|Next Article:||A first test of the implicit relational assessment procedure as a measure of forgiveness of self and others.|