Embedded learning strategy instruction: story-structure pedagogy in heterogeneous secondary literature classes.

Abstract. The effects of using the Embedded Story-Structure (ESS) Routine in a literature course were investigated. A heterogeneous group of 79 ninth graders, including 14 students with LD, were randomly assigned to one of two conditions, with instruction occurring in groups of 12 to 14 students in general education literature classes over a nine-day period. ESS instruction focused on three reading strategies: (a) student self-questioning, (b) story-structure analysis, and (c) summarizing. Instruction for the alternative condition, called comprehension skills instruction (CSI), was comprised of a package of research-based reading interventions. Statistically significant differences were found between groups in favor of the ESS Routine on measures of strategy use, story-structure knowledge, and unit reading comprehension. Moreover, results indicated equivalent gains for ESS students regardless of disability versus nondisability category.

**********

Meeting adequate-yearly-progress (AYP) goals in reading is a challenge for secondary school practitioners. Recently released National Assessment of Educational Progress (NAEP) data indicate that more than two-thirds of secondary students, including a high percentage of students with learning disabilities (LD), lack the reading skills necessary to succeed in school and the world of work (Perie, Grigg, & Donahue, 2005).

This record of student failure leaves educators in search of evidence-based practices that can be implemented across secondary settings in an attempt to close the performance gap that emerges between students' literacy skills that have plateaued at the fourth-grade level and the increasing academic grade-level demands (Deshler & Schumaker, 2006). Moreover, because many adolescents with LD are taught in general education settings (IDEA, 2004) so they can earn credits toward graduation, language arts teachers are increasingly being asked to shoulder a major part of the burden for comprehension instruction, especially as it relates to narrative texts. These teachers need evidence-based methods of delivering direct and explicit comprehension instruction that provides the necessary skill development for students with LD and other low-achieving students while simultaneously challenging high-achieving students in the same classroom.

One approach to defining instructional design and delivery to improve reading comprehension was suggested by Kintsch (2004), who argued that the goal of literacy instruction should be to get students engaged in processes equivalent to those that expert readers employ. One such process involves categorizing information in light of certain text structures. According to Kintsch, student knowledge and use of text structure favorably impacts comprehension, just as knowledge of syntax or vocabulary can. Text structure is believed to be most relevant to the reading process during encoding and during the reader's organization of the text into high-order units. While syntactic and semantic instruction foster sentence-level knowledge construction, discourse-level structure construction can be improved by teaching genre-specific text structures. Thus, according to this model, instruction should explicitly introduce students to the use of narrative text structure (or story structure) to aid in the conceptual understanding of narrative texts.

Indeed, awareness of underlying story structure has been shown to improve basic academic performance and lead to higher-order thinking, including causal reasoning (Gersten, Fuchs, Williams, & Baker, 2001). Narrative text structure has been investigated within three successive phases of research (Olson & Gee, 1988). Early research centered on developing empirical evidence to support a taxonomy of narrative elements that can be used to develop a basic understanding of story construction during encoding (e.g., Mandler & Johnson, 1977; Rumelhart, 1975; Stein & Glenn, 1978; Thorndyke, 1977). Overall, this research supported the existence of various theoretical models of story structure, surfacing universal components of a canonical structure for encoding (e.g., character, setting, conflict, resolution, and theme). Moreover, studies in this phase of research suggested that variations in reader knowledge of story structure are related to reader recall, which led to the second phase of story-structure research.

The second phase of research centered on developmental differences between types of readers with regard to the flexible use and complexity of story structure (e.g., Mandler &Johnson, 1977; Ouellette, Dagostino, & Carifio, 1998; Stein & Glenn, 1978; Whaley, 1981). A limited number of these studies explored developmental differences for individuals with LD (e.g., Montague, Maddux, & Dereshiwsky, 1990; Weaver & Dickinson, 1982). Results from this line of inquiry indicated that there were developmental differences related to story-structure knowledge, with story-structure knowledge appearing initially at the first-grade level and becoming well developed at the sixth-grade level for typical readers. In one study focusing on students with LD (Montague et al., 1990), no main effect was found for grade between 4th- through 11th-grade students with LD, indicating a story-structure plateau similar to the general reading achievement plateau reported by Warner, Schumaker, Alley, and Deshler (1980).

Due to the findings of developmental differences across groups of school-age readers, a third phase of research has focused on the instruction of the previously validated components of story structure to improve reading comprehension. While some studies have included elementary school students (e.g., Carnine & Kinder, 1985; Fitzgerald & Spiegel, 1983; Griffey, Zigmond, & Leinhardt, 1988; Idol & Croll, 1987; Short & Ryan, 1984), four relevant studies were conducted at the secondary level, including students in grades 6 through 11 in instruction of story-structure components.

In the first study, Singer and Donlan (1982) used a control-group design with 27 eleventh-grade students (no students with disabilities) randomly assigned to instructional groups of 15 over a six-day period. Story-grammar instruction was provided to the experimental group. Results on the lone measurement instrument, a 10-question multiple-choice quiz following each of six stories, showed that the only difference between groups was in favor of the experimental group on the quiz taken after the fifth story.

Gurney, Gersten, Dimino, and Carnine (1990) used a multiple-baseline design with seven high school students with LD in instructional groups of two or three in pullout classes over a nine-week period. The authors concluded that, on 7- to 10-question story-grammar and basal-type quizzes given every other day, students showed improvement on questions related to story grammar and no improvement on basal questions (no mean scores were provided on either question type). However, no individual student graphs were shown nor were group means reported, making it difficult to interpret the results.

In the third study, Dimino, Gersten, Carnine, and Blake (1990) conducted an experimental follow-up to Gurney et al.'s study (1990) with 32 low-performing ninth-grade students, including a subset of six students with LD and two students in Title I programs, over 19 days. Students were randomly assigned to the groups after being matched for reading achievement. Results on story-grammar and basal-type quizzes indicated statistically significant differences in favor of the story-grammar instruction group related to scores on story-grammar questions, basal questions, theme questions (a subset of story-grammar questions), and a summarizing task on the posttest and on the maintenance measure (mean raw score differences across all measures ranged from .3 [1.5% of total] to 2.5 [12.5% of total]). Results for the students with LD were not reported separately from group means.

Drawing from the work of Dimino et al. (1990) and Idol and Croll (1987), Gardill and Jitendra (1999) conducted a multiple-baseline study with six middle school students with LD in instructional groups of two in pullout classes. Results showed that (a) all students' scores on story grammar and basal comprehension tests improved concomitant with the onset of instruction, (b) all students' scores increased on generalization and maintenance story-grammar tests from baseline and for four students on basal tests, and (c) five of the six students demonstrated an increase in the number of story elements when they orally retold stories over baseline.

In summary, when instruction in story structure was implemented, results across the studies cited above indicated (a) differences within or between groups on criterion-referenced story-grammar measures (i.e., answering story-grammar-related questions about a given passage) when instruction was multi-phased and at least five hours in length; (b) inconsistent results on students' ability to answer basal (factual) questions; and (c) uncertain benefits for students with LD when taught in a heterogeneous group.

While Gersten and colleagues (2001) are accurate in their conclusion that instructing the awareness of underlying story structures has improved students' basic academic skills in some studies, the research is not complete. Several limitations are associated with the total body of research in this area, including both elementary and secondary studies.

First, there is a paucity of research on heterogeneous groups (or classes) of students that include students with LD, especially at the secondary level. Second, only a few studies focus on even moderate teacher-to-student ratios, and these studies typically fail to disaggregate data for students with LD. Hence, little is known about the application of this type of instruction in larger classroom settings and outcomes for students with LD. Third, control-group design studies have involved only limited components of research-based reading comprehension instruction for the control group. Thus, the value added by story-structure training cannot be determined in comparison. Fourth, no secondary study has considered the merits of packaging (or combining) elements of story structure to intensify instruction. Fifth, because the measures in previous studies have been criterion-based related to isolated passages, limited information has emerged regarding such variables as related knowledge (including knowledge of strategy components and literary terms), strategy use, written summaries of narrative passages, and student satisfaction with instruction.

Given the status of the research on story structure outlined above and the pressing need for secondary literacy instruction to bolster students' reading comprehension, an instructional routine, the Embedded Story Structure (ESS) Routine, was designed for use in general education classrooms serving heterogeneous populations of students. The routine is based on the findings of Swanson's (1999) meta-analysis on effective components of explicit cognitive learning strategy instruction (i.e., advance organizer, skill review, demonstration, modeling, guided practice, independent practice, and corrective feedback), as well as the validated components of story structure (e.g., Mandler & Johnson, 1977). This study investigated the effects of using the ESS Routine in an inclusive ninth-grade literature class relative to improving student use of reading comprehension strategies, knowledge of strategy components and literary terms, comprehension of stories, and satisfaction with the routine.

METHOD

Participants

Participants were 79 incoming ninth-grade students attending a summer school program for at-risk students at a private urban high school in a southeastern state. At-risk status was determined by school personnel in reading or mathematics based on test scores from EXPLORE (EXPLORE Test, 2005) taken during the spring of students' eighth-grade year. The student population reflected heterogeneous characteristics, as some students exhibited limited reading comprehension ability while others only exhibited limited abilities in mathematics (i.e., the latter group contained stronger readers).

The researcher met with all parents and students prior to the experiment to solicit written permission for participation. Students with permission were subsequently matched according to four variables. (1) Members of each matched pair were randomly assigned to one of two groups: (a) the embedded-story structure (ESS) group or (b) the comprehension skills instruction (CSI) group. Students in each group were also assigned randomly to one of three classes, for a total of six classes (three in each condition). Matching variables were (a) disability diagnosis versus no diagnosis; (b) EXPLORE reading comprehension national percentile score (within six percentile points) (EXPLORE Test, 2005); (c) gender; and (d) age in months (within six months). (See Table 1 for demographic data for the ESS and CSI groups.) Independent samples t-tests were employed to determine whether the groups were statistically different with regard to age and EXPLORE reading comprehension national percentile. Results showed that they were not, t(77) = .516, p = .608; t(77) = .609, p = .544, respectively.

An LD diagnosis was present for 14 subjects, based on the school's existing psychological profiles. (Table 2 provides demographic data related to these students in both groups.) Full-scale IQ scores were available for 13 of the 14 students on the Wechsler Intelligence Scale for Children (Wechsler, 1991, 2003). The seven students in the ESS group earned a mean full-scale IQ score of 103.7 (SD = 7.2), whereas the six CSI students earned a mean IQ score of 104.0 (SD = 2.5). Independent samples t-tests were employed to determine whether the groups were statistically different with regard to age, IQ score, and EXPLORE reading comprehension national percentile scores. Results showed that they were not, t(12) = -.050, p = .961; t(11) = .093, p = .928; t(12) = .315, p =. 758, respectively.

Setting

Instruction was provided in a typical classroom. The total school population was 1,480, drawing students from across the city. Classroom desks were arranged in four rows. An overhead projector was positioned on a table at the front of the room, along with a large presentation screen and two side-by-side whiteboards.

Instructional Materials

Eight short stories and a folktale were selected as passages to be used during instruction and student practice and as the basis for comprehension assessment for all students. The materials included renowned classics (e.g., Poe's The Tell-Tale Heart) as well as more contemporary stories that present accounts of dilemmas encountered by adolescents (e.g., Myers' The Treasure of Lemon Brown). Stories were 9-18 pages long, with lexile scores ranging from 600 (The Tell Tale Heart) to 1220 (The Monkey's Paw; Metametrics, 2000).

The Instructional Programs

Embedded story-structure routine. The Embedded Story-Structure (ESS) Routine targeted three strategies: (a) self-questioning (used during pre-reading), (b) story-structure analysis (used during reading), and (c) summary writing (used after reading). A graphic device, the ESS Organizer, was designed to facilitate the interactive construction of knowledge between students and teacher and the integration of the three strategies (see Figure 1).

[FIGURE 1 OMITTED]

The self-questioning strategy involves students asking and answering a series of seven questions initially introduced by the teacher that relate to eight critical components of story structure (e.g., main character, initiating event, time, place, central conflict, climax/turning point, resolution, and theme) (e.g., Dimino et al., 1990; Gardill & Jitendra, 1999). When implementing this strategy, students use seven individual question words (e.g., "who," "what," "when," "where," "which," "how," and "why") as a mnemonic device to help them remember the eight components and the related seven questions. (See Table 3 for the story components and questions.) Students record their answers to each self-question on the ESS Organizer.

The second strategy, story-structure analysis, involves filling in a Story-Structure Diagram on the back of the ESS Organizer by labeling specific events from a short story. To help them remember this strategy, students are provided with a second mnemonic device that connects each critical element of the Story Structure Diagram to a unique picture cue. Students draw these cues when labeling the Story-Structure Diagram.

Students use the third strategy, a summary writing strategy, to generate a written summary of the short story on the back of the ESS Organizer based on their answers to the self-questions using a four-sentence formula that includes each critical element of story structure.

Comprehension skill instruction. Comprehension Skill Instruction (CSI) was designed to parallel the ESS instruction by targeting three research-based strategies for instruction: (a) the LINCS Vocabulary Strategy (Ellis, 2000; used during prereading); (b) Question-Answer Relationships (QAR) (Raphael, 1982, 1986; used during reading); and (c) semantic summary mapping (Englert, Mariage, Garmon, & Tarrant, 1998; used after reading). The CSI strategies were chosen because they have previously been studied (separately) as mechanisms that might improve student reading comprehension. To parallel the ESS instruction, comparison students received a graphic device, called the CSI Organizer, to facilitate the interactive construction of knowledge between students and teacher. (2)

The LINCS Vocabulary Strategy involves the use of a set of mnemonic strategies, including a key word strategy, a visual imagery strategy, and a story strategy to link known information to new vocabulary words and their definitions (Ellis, 2000). Students record LINCS information related to a teacher-chosen vocabulary word on the first page of the CSI Organizer.

For the QAR strategy, students ask themselves and answer text-based and knowledge-based questions to develop story understanding. Text-based questions include (a) "Right There" questions, which require students to find answers explicitly stated right there in the text (e.g., "What is Johnny's father's job?"); and (b) "Think and Search" questions, which require locating answers stated explicitly in the text but require examination of more than a single location in the text (e.g., "What are some of the challenges facing the kidnappers?").

Students record QAR questions and answers on the first page of the CSI Organizer. Students use the third strategy, semantic summary mapping, to visually summarize stories by mapping self-identified critical components of the story. The strategy involves identifying and listing critical components of the story before organizing them into a connective semantic/concept map of related ideas on the back of the CSI Organizer.

Measurement Instruments

Strategy-use test. The Strategy-Use Test was administered to all students in both groups at pretest (Day 1), progress test (Day 5), posttest (Day 9), and maintenance (8 weeks after the posttest). The purpose of this measure was to provide evidence that the ESS students learned the ESS strategies that they were taught (and that the CSI students did not learn the ESS strategies) so that their learning could later be related to student performance on reading comprehension measures. Each of four forms of the test included three sections, each section corresponding to use of one of the three ESS strategies. The first provided seven lines on which students could write prereading questions related to novel passages. For the second section, students were asked to read a 260- to 300-word narrative passage (7.68.2 GE) from the Analytic Reading Inventory (ARI) (Woods & Moe, 2003) and correctly label a Story-Structure Diagram. (Each form of the test contained a different passage.) The third section on the test required students to write a summary of the passage that was approximately 60 words in length.

In order to control for passage difficulty, a counter-balanced design was used. That is, Form A was administered to half the class in the pretest and the other half of the class in the posttest, and vice versa for Form B. During the progress and maintenance tests, Form C was administered to half the class in the progress test and the other half of the class in the maintenance test, and vice versa for Form D. All sections of the test were scored using an answer key. The total number of points available was 24, with eight points each available for prereading questions, for correct diagram labels, and for a written summary.

Knowledge test. The Knowledge Test, administered to both groups, measured student knowledge of ESS strategy components and literary terms associated with the ESS strategies to provide evidence that students in the ESS group learned knowledge about the ESS strategies but students in the CSI group did not, so that this learning could later be related to students' performance on reading comprehension measures.

The knowledge test was divided into three sections. The first provided a blank Story-Structure Diagram and asked students to demonstrate knowledge of the location of story-structure elements by labeling each component (e.g., initiating event, climax, resolution) on the diagram. The second section asked students to demonstrate knowledge of the picture cues and their connections to the story-structure components by listing the appropriate elements of story structure along with the corresponding picture cue (e.g., initiating event--[??], time--[??]). Finally, the third section required students to answer four open-ended questions related to basic language arts knowledge (e.g., "What are the types of conflict?" and "What are the components of characterization?"). The total score for the Knowledge Test was 30 points, with 8 points each for the appropriate labels and correct picture cue answers and 14 points for the language arts knowledge questions. An answer key was used to score the test which was administered at pretest and posttest.

Unit comprehension test. The Unit Reading Comprehension Test measured student retention of information related to the eight stories read by both groups during instruction. The hypothesis was that students in the ESS group would retain more information than students in the CSI group. The test included 40 short-answer and fill-in-the-blank questions. One point was awarded for each correct answer for a total of 40 points.

To create two forms of the test, a test bank of 80 questions was created with eight sets of 10 questions per story. Each set contained six questions related to the story-structure elements of the story and four higher-order questions (two induction and two deduction questions). Question sets were paired for each story according to the difficulty level of story-structure element, and one question from each pair was randomly assigned to each form of the test. To ensure measurement reliability, a counterbalanced design was used during the pretest and posttest, with Form A administered to half the class in the pretest and the other half of the class in the posttest, and vice versa for Form B. Two certified secondary English teachers juried the measure and agreed that all questions were appropriate to the corresponding passage, all answers could be found in the passage, the answers in the answer key were correct, and that Forms A and B of the test were equivalent in difficulty (100% agreement in all areas).

Satisfaction surveys. To evaluate student satisfaction with the interventions, two social validity measures were also administered. The Reading Satisfaction Survey measured students' satisfaction with their own reading. Questions related to students' satisfaction with reading and remembering short stories or novels, writing summaries, and test performance on reading comprehension tests. The Satisfaction with Instruction Survey measured student satisfaction with the ESS Routine (or the CSI Routine) and its components. Questions here related to student satisfaction with how helpful instruction was with regard to improving story understanding, how easy it was to identify the specific elements of story structure, whether or not students could use what they had learned independently, and overall satisfaction with the routine. On both surveys, items were formatted on a 7-point Likert-type scale ranging from "Completely Dissatisfied" to "Completely Satisfied." The average rating was calculated for each item for each group.

Fidelity of treatment checklists. A fidelity checklist was used to assess the quality of teacher performance in implementing the specified instruction in each lesson. Each checklist was comprised of a list of 29-46 items, cataloging all the instructional practices associated with both routines and intervention-specific components for each routine appropriate to the given lesson, including (a) review of the previous lesson, (b) use of advance and post-organizers, (c) encouraging independent strategy use, (d) administering a quiz, (e) giving notes and defining terms related to ESS or CSI instruction, (f) reading the same stories, (g) periodically engaging students in dialogue, (h) demonstration and modeling of the ESS- or CSI-related strategies, and (i) teacher-led practice with ESS- or CSI-related strategies. An independent scorer and the researcher listened to randomly selected audio recordings of 22% of the instructional lessons for each group and completed the checklist. The fidelity score was calculated by totaling the number of observed instructional behaviors for the intervention, the number of unobserved instructional behaviors for the alternative intervention, and the number of observed instructional behaviors common to both interventions.

The total score was then divided by the total number of points possible, ranging from 29 to 46. Interscorer reliability. Interscorer reliability was determined by having two scorers independently score a random sample (20%) of each of the instruments at each point in the data-gathering plan for each group. An item-by-item analysis was used to compare observer recordings. For the Strategy-Use Test, scorers agreed on 1,443 out of 1,536 opportunities to agree (range from 83.3% to 100%) for 94% agreement. For the Knowledge Test, scorers agreed on 914 out of 960 opportunities to agree (range from 83.3% to 100%) for 95.2% agreement. For the Unit Reading Comprehension Test, scorers agreed on 1,233 out of 1,280 opportunities to agree (range from 83% to 100%) for 96% agreement. For fidelity of treatment, scorers agreed on 272 out of 282 opportunities to agree (range from 90% to 100%) for 96.5% agreement.

Procedures

Pretest and progress measure procedures. Pretesting occurred in two sessions. The first battery of measures was administered during a 90-minute block, and included the Strategy-Use Test (20 minutes), Knowledge Test (15 minutes), and the Unit Reading Comprehension Test (25 minutes). The Reading Satisfaction Survey was administered during the first 20 minutes of the first lesson. Additionally, the Strategy-Use Test was repeated during the last 20 minutes of the fifth lesson as a progress measure.

Instructional procedures. Teacher-to-student instructional ratios were 1:12, 1:13, or 1:14, depending on the class enrollment. The same teacher (and first author), a special education doctoral student with certification in secondary-level English, provided all instruction to both groups. Instruction for each class took place over a nine-day summer school session with 90 minutes of instruction on Days 1 and 9 and 120 minutes of instruction on Days 2 through 8, for a total of 17 hours of instruction. Instruction took place in four phases: (a) teacher demonstration and modeling of the targeted strategies, including think-aloud problem solving; (b) student-teacher collaboration and co-construction of knowledge and strategy use; (c) student peer collaboration (cooperative learning) and teacher-guided practice; and (d) independent student practice. The teacher used the cooperative learning and independent practice sessions as opportunities to circulate through the classroom to provide personal corrective feedback to individual students. The teacher also provided individual instruction at the teacher's desk when needed by a student. All lessons began with a review of the previous day's lesson and an advance organizer of the current day's instruction. All lessons ended with a brief review of the day's lesson.

Instruction was delivered with consideration given to two guiding principles: (a) the content of instruction (i.e., components of the three targeted strategies for each instructional condition); and (b) the four-phase pedagogy of instruction described above. (See Table 4 for a summary of instructional procedures.) The instructional sequence for the ESS and CSI groups was the same; only the content (the three strategies specified for each group) differed.

Day 1 instruction involved teacher introduction, demonstration, and modeling of all strategy components while using the appropriate organizer to provide students with an overall "big picture" example of how the three targeted strategies were to be used in combination in relation to the first story. At the end of Day 1 instruction, the teacher prompted the students to participate in using the strategies in relation to the end of the story in a whole-group teacher-guided practice activity.

The first half of Day 2 instruction focused on teacher demonstration and modeling of selected parts of two of the strategies in relation to Story #2. During the second half of the period, the students were prompted to use the strategies in a whole-group teacher-guided practice activity, also for Story #2.

On Day 3, students engaged in cooperative peer practice of the strategy components covered on Day 2 for the initial sections of Story #3 and independent practice for the remaining sections of the story.

Day 4 instruction mirrored Day 2 instruction using Story #4, but focused on the remaining strategy components that were not covered on Days 2 and 3.

On Day 5, students engaged in cooperative peer practice of all three targeted strategies for some sections of Story #5 and independent practice of all three strategies on other sections.

Days 6 through 8 again allowed for periods of peer collaboration and periods of independent practice of the combined strategies with sections of a new story each day and with students slowly transitioning to longer periods of independent work.

The final session (Day 9) concluded with a class-wide review of the complete unit of eight short stories covered. The teacher followed a written protocol for each lesson and concluded each class with a class-wide discussion of the day's short story. Students handed in their completed graphic organizers periodically to the teacher for corrective feedback. (For more details about the instructional procedures for each group, see Faggella-Luby, 2006.)

Posttest and maintenance procedures. Posttesting occurred in identical fashion to the pretesting with two group-administered sittings. During the first sitting, students completed the satisfaction surveys. In the second sitting, 90-minutes was provided for the battery of measures to be administered. For the maintenance probe, the Strategy-Use Test was administered eight weeks after the final lesson.

Design and Data Analyses

The study employed a control-group design with random assignment of members of matched pairs of students to two groups (ESS and CSI) to determine the effects of the ESS Routine. The same teacher conducted all instruction to control for teacher effects. Instructional time, stories, and the general instructional methodology were identical across the groups.

For the Strategy-Use measure, which was administered four times, a one-way analysis of variance (ANOVA) with one within-subjects factor and two between-subjects factors was conducted. Time with four levels (pretest, progress, posttest, and maintenance) was the within-subjects factor (independent variable). Instructional condition (ESS and CSI) and whether or not students had a diagnosed disability (students with disabilities and other peers) were the between-subjects factors.

For the Unit Reading Comprehension Tests and Knowledge Tests, which were each administered twice, a one-way ANOVA with one within-subjects factor and two between-subjects factors was conducted. Time with two levels (pretest and posttest) was the within-subjects factor (independent variable). Instructional condition (ESS and CSI) and whether or not students had a diagnosed disability (students with disabilities and other peers) were the between-subjects factors.

To determine between-group differences on the Unit Reading Comprehension Tests and Knowledge Tests, independent-samples t-tests were completed separately for each administration of a measure. A Bonferroni Correction was used to control for Type 1 error ([alpha] = .05/2 = .025). For the Strategy-Use Test, to determine within-group differences, paired-samples t-tests were completed separately for the ESS and CSI groups to compare the pretest to posttest scores, the pretest to maintenance scores, and the posttest to maintenance scores. Again, a Bonferroni Correction was used to control for Type I error for all pairwise comparisons ([alpha] = .05/(2x3) = .0083).

RESULTS

Strategy Use Test

Means and standard deviations for Strategy-Use Test scores are presented in Table 5 for students with disabilities and the remaining students in the ESS and CSI groups. The ANOVA results indicated no statistically significant time by condition by disability diagnosis interaction, Wilks' [LAMBDA] = .972, F(3,73) = .700, p = .555, multivariate [[eta].sup.2] = .028. However, a statistically significant time-by-condition interaction was found, Wilks' [LAMBDA] = .330, F(3,73) = 49.5, p <. 001, multivariate [[eta].sup.2] = .670, indicating a large effect size. Thus, regardless of whether or not students had a disability diagnosis, ESS subjects exhibited equivalent gains on this test.

Results of an independent samples t-test indicated no statistically significant difference between the ESS and CSI students' performance on the strategy-use pretest: t(77) =-.293, p = .770, d = .015. However, there were statistically significant differences between the groups, in favor of the ESS group, on all three remaining tests: the progress test, t(56.4) = -14.6, p < .001, d = .738; the posttest, t(60.5) = -15.9, p = <.001, d = .807, and the maintenance test, t(59.2) = -10.7, p = <.001, d = .542, indicating large and moderate effect sizes, respectively.

Results of the three paired-samples t-tests revealed significant differences on two comparisons for the ESS group: pretest to posttest, t(38) = -18.3, p < .001, [DELTA] = 4.27; and pretest to maintenance, t(38) = -10.84, p < .001, [DELTA] = 3.70, indicating large effect sizes. There was no significant difference between the posttest and the maintenance test, t(38) = 2.06, p = .046, [DELTA] = .41. No significant differences were found for all three pairwise comparisons for the CSI group: t(39) = -1.28, p = .210, [DELTA] = .21; t(39) = .110, p = .913, [DELTA] = .02; and t(39) = 1.38, p = .179, [DELTA] = .30, respectively.

Knowledge Test

Means and standard deviations for the Knowledge Test scores are presented in Table 6. The ANOVA results again indicated no statistically significant time by condition by disability diagnosis interaction, Wilks' [LAMBDA] = .99, F(1,75) = .138, p = .711, multivariate [[eta].sup.2] = .002. However, there was a statistically significant time-by-condition interaction, Wilks' [LAMBDA] = .376, F(1,75) = 124.4, p < .001, multivariate [[eta].sup.2] = .624, indicating a large effect size.

Results of an independent samples t-test indicated no statistically significant difference between the ESS and CSI groups on the pretest: t(77) = -.890, p = .376, d = .045. However, there was a statistically significant difference between the groups on the posttest: t(77) = -4.11, p < .001, d = .208, indicating a small effect size in favor of the ESS group.

Unit Reading Comprehension Test

Means and standard deviations for the Unit Reading Comprehension Test scores are presented in Table 7. ANOVA results indicated no statistically significant time by condition by disability diagnosis interaction, Wilks' [LAMBDA] = .992, F(1,75) = .621, p = .433, multivariate [[eta].sup.2] = .008. However, again there was a statistically significant time-by-condition interaction, Wilks' [LAMBDA] = .908, F(1,75) = 7.61, p = .007, multivariate [[eta].sup.2] = .092, representing a medium effect size. Thus, regardless of whether or not students had a disability diagnosis, the ESS subjects exhibited equivalent gains on the Unit Reading Comprehension Test.

Results of an independent-sample t-test comparing the pretest scores of the two groups indicated no difference between the groups: t(77) = -.104, p = .917, d = .005. However, there was a statistically significant difference between the groups on the posttest: t(54.4) = -15.3, p < .001, d = .776, in favor of the ESS group. This represents a large effect size.

Satisfaction

Reading satisfaction survey. Means and standard deviations for the Reading Satisfaction survey scores are presented in Table 8. Results from an independent samples t-test indicate no statistically significant differences between the groups on either the pretest or the posttest grand mean ratings: t(77) = -.516, p = .607, d = .03; and t(77) = .114, p = .909, d = .006, respectively. Two follow-up paired samples t-tests were conducted to examine within group differences from pretest to posttest. A Bonferroni Correction was used to control for Type I error for both pairwise comparisons ([alpha] = .025). Results indicated statistically significant differences and large effect sizes for both the ESS, t(38) = -8.62, p < .001, [DELTA] = 1.23, and the CSI, t(39) = -9.62, p < .001, [DELTA] = 1.23, groups between pretest and posttest ratings, with higher mean scores on the posttest indicating higher levels of reading satisfaction than at pretest.

Satisfaction with instruction survey. Means and standard deviations for ratings on the Satisfaction with Instruction Survey are presented in Table 9. A grand mean rating of 6.27 (SD = .54) was calculated for the ESS students, and a grand mean rating of 6.26 (SD = .54) was calculated for the CSI students. An independent samples t-test revealed no statistically significant difference between groups' satisfaction ratings, t(77) = -.053, p = .958, d = .003.

Fidelity of Treatment

The teacher completed a mean of 97% of the instructional steps across the analyzed lessons for the ESS Routine with a range from 90% to 100% for individual ESS lessons. A mean of 98% of the instructional steps across the analyzed lessons was completed for the CSI Intervention with a range from 94% to 100% for individual CSI lessons. No content components of the CSI intervention were taught during the ESS intervention. Likewise, no components of the ESS intervention were taught during the CSI intervention.

DISCUSSION

The methodological and theoretical presuppositions inherent in this study's design are directly related to the earliest work surrounding story-structure instruction. The study was designed to answer the question: Can story-structure components be taught to heterogeneous groups of learners, in general education settings, to improve reading comprehension without sacrificing the learning of the higher achieving peers?

The results are similar to those of previous studies regarding (a) positive effects of multi-phase instruction with moderate-sized groups including a variety of learner types and using authentic literature with secondary students (e.g., Dimino et al., 1990; Gardill & Jitendra, 1999; Idol, 1987); and (b) improved performance on criterion-referenced story grammar measures (e.g., Dimino et al., 1990; Gardill & Jitendra, 1999; Idol, 1987).

In addition, this study extends previous research and contributes to the field in several important ways. First, it supports Kintsch's (2004) suggestion to instruct components of narrative text structure to improve reading comprehension. Second, it furthers investigations of outcomes for heterogeneous groups of learners (including individuals with disabilities). Moreover, while relatively modest in scope, the study is the largest of its kind to date, involving 79 students, including 14 students with disabilities, in a control-group design. Third, the study extends the research into secondary or adolescent literacy, a growing but minimally explored area of study. Fourth, the study demonstrates that, unlike previous studies of story-structure instruction in which the inclusion of individuals with disabilities required instructional ratios of 3:1 (e.g., Gardill & Jitendra, 1999; Gurney et al., 1990) to produce gains, students can be taught successfully in instructional groupings of 14:1. Fifth, this is the first study related to teaching story structure that used research-based reading comprehension instruction in the comparison condition, providing substantial support to claims that differences were related to the benefits of the ESS Routine over another strong instructional model. Sixth, this study provides preliminary evidence about the effects of packaging elements of story structure to intensify instruction by combining student self-questioning, story mapping, and summarizing. Finally, the study adds three new assessment tools to the research on story structure, which enable measurement of intervention efficacy: (a) a strategy-use test, (b) a knowledge test, and (c) a unit test over all of the short stories covered during the unit. The strategy measures demonstrate whether or not students have learned and can use the strategies that have been taught. Such a demonstration helps to eliminate the alternative explanation that reading practice alone produced the differences in reading comprehension.

Implications for Practice

Study results support the use of the ESS Routine as an instructional intervention for improving reading comprehension of at-risk learners in heterogeneous inclusive secondary classrooms. Thus, the Strategy-Use Test and Knowledge Test results indicated that students receiving the ESS Routine instruction not only outperformed CSI students, but gains were equivalent regardless of the disability/no disability category. Moreover, participation in the routine led to significant improvements in strategy use from pretest to posttest, and from pretest to maintenance. No such trend existed for CSI students, whose Strategy-Use Test scores remained linear and low.

Results from the Unit Reading Comprehension Test indicated, once again, that students, including those with disabilities, benefited equally from use of the ESS Routine. Significant differences between students in the ESS group and the CSI group on the reading comprehension measure and the Knowledge Test are particularly significant, given their similar nature to criterion-referenced assessments common in language arts classrooms. These findings support the belief that instruction in the ESS Routine may bolster educational outcomes for individuals with disabilities in such classrooms.

The socially significant findings across all survey measures are positive indicators of the palatability of the ESS Routine. More specifically, differences within both groups from pretest to posttest scores for the Reading Satisfaction Survey indicate that, regardless of instruction type, students were more satisfied with their reading abilities after instruction. Further, student survey scores on the Satisfaction with Instruction Survey in both groups indicated not only overall satisfaction with the instruction, but also a feeling that this instruction: (a) helped them understand short stories, (b) was fun and interesting, and (c) would lead to likely independent use of the strategies. Again, these results are positive indicators to support the likelihood of instructional palatability.

Finally, an important aspect of this study design is that ESS condition scores on strategy-use and knowledge test measures when evaluated in light of scores on the unit reading comprehension test provide strong evidence to suggest that a relationship exists between student use and knowledge of ESS strategies and growth in reading comprehension.

Limitations

Several limitations apply to this study. First, the teacher for the study was also the researcher. While this arrangement controlled for teacher effects, the effects that may be produced by other teachers are unknown, as are other teachers' satisfaction with the routine and their ability to implement it.

Second, ESS Routine efficacy was measured in relation to heterogeneous groups of students, in part because federal law continues to advocate the need to educate individuals with disabilities in the least restrictive environment. The effects of the ESS Routine when implemented in supportive learning environments and under the optimal conditions of effective learning strategy instruction described by Swanson (2001) are unknown.

Third, the sample population included only 14 students with disabilities. While this percentage of the study population (18%) is larger than the reported prevalence rate of 6% of students with LD in reading found nationally in schools (National Research Center on Learning Disabilities, 2002), this small number of students with LD limits the generalizability of the results with regard to this type of student. Moreover, the subjects were all students referred to a summer school program.

Fourth, as a group, although the ESS student performance on the Unit Reading Comprehension Test was significantly higher than the performance of the CSI group, the level of their performance on the posttest may not be considered educationally significant (M = 24.0 or 60%, SD = 4.99). This is partially the result of rigorous application of a scoring rubric and the initial design of a difficult test to ensure against a ceiling effect. A second consideration is that the question type was short-answer, proving more challenging than the multiple-choice or fill-in-the-blank format commonly used in schools. Though the total number of hours of instruction (17) was typical for demonstrating effective learning strategy instruction (e.g., Deshler et al., 2001), the relatively short number of instructional days (9) may have produced an additional cognitive challenge for the specific student population. That is, complex packages of strategies (e.g., self-questioning, story mapping, summary writing) require sufficient time to master. Perhaps additional practice sessions would have yielded improved results.

Finally, a significant challenge to all the research to date on story-structure instruction is the lack of standardized reading comprehension measures. This study is no exception. In part, no such measure was used because of the relatively short amount of instructional time and the prediction that scores on a standardized reading test would not have a chance of changing after only nine sessions of instruction.

Future Research

Future research should include studies that involve implementing the ESS Routine by a teaching cadre other than the researcher across multiple classes and over a longer period of time. A second future direction is to implement the ESS Routine in a supportive learning environment for students with LD using the eight stages of instruction from the Strategic Instruction Model's Learning Strategies Curriculum (see Deshler & Schumaker, 2006) to allow maximum intervention effectiveness to be measured. Another study might focus on a component analysis of the routine.

All of these studies might involve standardized reading comprehension measures. Such research will be important to the field of special education and students with disabilities because it will lead to the development of successful pedagogy that is appropriate for all levels of learners in inclusive classrooms.

REFERENCES

Carnine, D., & Kinder, D. (1985). Teaching low-performing students to apply generative and schema strategies to narrative and expository material. RASE, 6(1), 20-30.

Deshler, D. D., Schumaker, J., Bulgren, J., Lenz, K., Jantzen, J. E., Adams, G., Carnine, D., Grossen, B., Davis, B., & Marquis, J. (2001). Making learning easier: Connecting new knowledge to things students already know. Teaching Exceptional Children, 33(4), 82-85.

Deshler, D. D., & Schumaker, J. (Eds.). (2006). Teaching adolescents with disabilities: Accessing the general education curriculum. Thousand Oaks, CA: Corwin Press.

Dimino, J., Gersten, R., Carnine, D., & Blake, G. (1990). Story grammar: An approach for promoting at-risk secondary students' comprehension of literature. The Elementary School Journal, 91(1), 19-32.

Ellis, E. S. (2000). The LINCS vocabulary strategy (2nd ed.). Lawrence, KS: Edge Enterprises.

Englert, C. S., Mariage, T. V., Garmon, M. A., & Tarrant, K. L. (1998). Accelerating reading progress in early literacy project classrooms: Three exploratory studies. Remedial and Special Education, 19(3), 142-159.

EXPLORE Test. (2005). Iowa City, IA: ACT, Inc.

Faggella-Luby, M. (2006). Embedded learning strategy instruction: Story-structure pedagogy in secondary classes for diverse learners. Unpublished doctoral dissertation, University of Kansas.

Fitzgerald, J., & Spiegel, D. L. (1983). Enhancing children's reading comprehension through instruction in narrative structure. Journal of Reading Behavior, 15, 1-17.

Gardill, M. C., & Jitendra, A. K. (1999). Advanced story map instruction: Effects on the reading comprehension of students with learning disabilities. Journall of Special Education, 33(1), 2-17.

Gersten, R., Fuchs, L. S., Williams, J. P., & Baker, S. (2001). Teaching reading comprehension strategies to students with learning disabilities: A review of research. Review of Educational Research, 71(2), 279-320.

Griffey, Q. L., Jr., Zigmond, N., & Leinhardt, G. (1988). The effects of self-questioning and story structure training on the reading comprehension of poor readers. Learning Disabilities Research, 4(1), 45-51.

Gurney, D., Gersten, R., Dimino, J., & Carnine, D. (1990). Story grammar: Effective literature instruction for high school students with learning disabilities. Journal of Learning Disabilities, 23(6), 33-343.

Individuals with Disabilities Education Improvement Act. (2004). P. L. 108-446.

Idol, L. (1987). Group story mapping: A comprehension strategy for both skilled and unskilled readers. Journal of Learning Disabilities, 20(4), 196-205.

Idol, L., & Croll, V. J. (1987). Story mapping training as a means of improving reading comprehension. Learning Disability Quarterly, 10, 214-229.

Kintsch, W. (2004). The construction-integration model of text comprehension and its implications for instruction. In R. B. Rudell & N. J. Unrau (Eds.), Theoretical models and processes of reading (5th ed., pp. 1270-1324). Newark, DE: International Reading Association.

Mandler, J. M., & Johnson, N. S. (1977). Remembrance of things parsed: Story structure and recall. Cognitive Psychology, 9(1), 111-151.

MetaMetrics. (2000). The Lexile framework for reading. Durham, NC: Author. Retrieved September 15, 2004, from http://lexile. com/faq.htm.

Montague, M., Maddux, C. D., & Dereshiwsky, M. I. (1990). Story grammar and comprehension and production of narrative prose by students with learning disabilities. Journal of Learning Disabilities, 23(3), 190-197.

National Research Center on Learning Disabilities (NRCLD). (2002). Researcher consensus statement. Information Digest Number 3. Retrieved November 19, 2005, from http://www. nrcld.org/html/information/articles/digests/digest3.html

Olson, M. W., & Gee, T. C. (1988). Understanding narrative: A review of story grammar research. Childhood Education, 64(5), 302-306.

Ouellette, G., Dagostino, L., & Carifio, J. (1998). The relationship of children's expectations for structure in story to their knowledge of literature and their reading ability. Reading Improvement, 35(3), 98-113.

Perie, M., Grigg, W., & Donahue, P. (2005). The nation's report card: Reading 2005 (NCES 2006-451). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Raphael, T. (1982). Question-answering strategies for children. The Reading Teacher, 36(2), 186-190.

Raphael, T. E. (1986). Teaching question-answer relationships, revisited. The Reading Teacher, 39(6), 516-522.

Rumelhart, D. E. (1975). Notes on a schema for stories. In D. G. Bobrow & A. M. Collins (Eds.), Representation and understanding (pp. 211-236). New York: Academic Press.

Short, E. J., & Ryan, E. B. (1984). Metacognitive differences between skilled and less skilled readers: Remediating deficits through story grammar and attribution training. Journal of Educational Psychology, 76(2), 225-235.

Singer, H., & Donlan, D. (1982). Active comprehension: Problem-solving schema with question generation for comprehension of complex short stories. Reading Research Quarterly, 166-185.

Stein, N. L., & Glenn, C. G. (1978). An analysis of story composition on elementary school children. In ?? Friedle (Ed.), New directions in discourse processing (Volume 2, pp. 53-120). Norwood, NJ: Ablex.

Swanson, H. L. (1999). Instructional components that predict treatment outcomes for students with learning disabilities: Support for a combined strategy and direct instructional method. Learning Disabilities Research & Practice, 14(3), 129-140.

Thorndyke, P. W. (1977). Cognitive structures in comprehension and memory of narrative discourse. Cognitive Psychology, 9, 77-110.

Warner, M. M., Schumaker, J. B., Alley, G. R., & Deshler, D. D. (1980). Learning disabled adolescents in public schools: Are they different from other low achievers? Exceptional Education Quarterly, 1(2), 27-35.

Weaver, P. A., & Dickinson, D. K. (1982). Scratching below the surface structure: Exploring the usefulness of story grammars. Discourse Processes, 5(3), 225-243.

Wechsler, D. (1991). Wechsler Intelligence Scale for Children (3rd ed.). New York: Psychological Corporation.

Wechsler, D. (2003). WISC-IV technical and interpretive manual. San Antonio, TX: Psychological Corporation.

Whaley, J. F. (1981). Readers' expectations for story structures. Reading Research Quarterly, 1(9), 107-111

Woods, M. L., & Moe, A. J. (2003). Reader's passages to accompany: Analytic Reading Inventory. Upper Saddle River, NJ: Merrill Prentice Hall.

MICHAEL FAGGELLA-LUBY, Ph.D., University of Connecticut, Center for Behavioral Education and Research.

JEAN S. SCHUMAKER, Ph.D., University of Kansas, Center for Research on Learning.

DONALD D. DESHLER, Ph.D., University of Kansas, Center for Research on Learning.

Correspondence should be addressed to: Michael Faggella-Luby, University of Connecticut, 249 Glenbrook Rd., Unit 2064, Storrs, CT 06269; e-mail: [e-mail]

NOTES

(1) Please note that there were 79 students who had permission to participate; thus, one student did not have a matched partner. After all the other matches were made, the 79th student was assigned to the CSI condition by a coin flip.

(2) Comprehension Skills Organizer available upon request.

Table 1
Demographic Data on All Subjects

                                         ESS             CSI

Category                              N       %       N       %

Total number of subjects                39              40

Gender
  Male                                  22   56.4       21   52.5
  Female                                17   43.6       19   47.5

Age (months)                        171.44          172.08

Standard deviation                    4.63            6.25

Ethnicity
  Caucasian                             30   77.0       29   72.5
  African American                       6   15.4        5   12.5
  Hispanic                               1    2.6        3    7.5
  Other                                  2    5.1        3    7.5

Avg. EXPLORE Percentile (reading)    47.97           50.70

Frequency EXPLORE Scores
  80th-89th Percentile                   1               1
  70th-79th Percentile                   6               6
  60th-69th Percentile                   7               8
  50th-59th Percentile                   7               8
  40th-49th Percentile                   9              12
  30th-39th Percentile                   0               0
  20th-29th Percentile                   4               2
  10th-19th Percentile                   2               0
     0-9th Percentile                    3               3

Table 2
Demographic Data on Students with Learning Disabilities

          Age                                            EXPLORE
       (months)     Gender    Ethnicity       IQ         Nat Per

ESS
S1        175         M           C           114          17
S2        174         M           O           101          29
S3        166         M           C           93           29
S4        165         F           C           101          51
S5        166         F           C           112          42
S6        177         M           C           102          42
S7        168         M           C           103          66
       M = 170.1                           M = 103.7    M = 39.4
       SD = 5.0                            SD = 7.2     SD = 16.2

CSI
S1        162         M           C           108          51
S2        167         M           O        mid range       42
S3        175         M          AA           105           7
S4        177         F       unmarked        104          51
S5        168         M           C           102          42
S6        166         M           C           101          51
S7        175         M       unmarked        104          51
       M = 170.1                           M = 104.0    M = 42.1
       SD = 5.0                            SD = 2.5     SD = 16.1

                       Disability *

          ADD         D           R           ULD

ESS
S1        **          **
S2        **          **         **
S3        **                     **
S4        **                     **
S5        **
S6                    **
S7                                            **

CSI
S1        **          **         **
S2        **
S3        **
S4        **
S5        **
S6                    **
S7                                            **

Note. Ethnicity: C=Caucasian; AA=African American; O=other.

* Disability: ADD=Attention Deficit Disorder; D=Dyslexia;
R=Reading Deficit; ULD=Unspecified Learning Disability.
IQ=Full Scale IQ score on the WISC-IV or WISC-III.

Table 3
Component, Question, and Picture Cue for ESS Instruction

Story-Structure Component   Story-Structure Question       Picture Cue

Main Character              Who is the main character?       [??]
(protagonist and
antagonist)

Central Conflict/           What is the central conflict     [??]
Initiating Event            and how does it begin?

Time                        When does the story take         [??]
                            place?

Place                       Where does the story take        [??]
                            place?

Climax                      Which decision or event is       [??]
                            the turning point?

Resolution                  How does the central             [??]
                            conflict end/resolve?

Theme                       Why did the author tell us       [??]
                            the story in this way?

Table 4
Summary of Instructional Procedures

                                   Your-Phase Pedagogy of
                                        Instruction

Content                        Demonstration   Teacher-Guided
of Instruction        Day(s)    & Modeling        Practice

* Whole strategy        1           **               **

1st half                2           **               **
of each substrategy     3
components

2nd half                4           **               **
of each substrategy
components              5

* Whole strategy       6-8
                        9

                                 Your-Phase Pedagogy of Instruction

Content                         Cooperative     Independent
of Instruction        Day(s)   Peer Practice      Practice      Review

* Whole strategy        1

1st half                2
of each substrategy     3           **               **
components

2nd half                4
of each substrategy
components              5           **               **

* Whole strategy       6-8          **               **
                        9                            **           **

* All components of specific instructional condition strategies
(i.e., self-questioning, story-structure analysis, and summarizing
for ESS; and QAR, LINCS, and semantic summary mapping for CSI).

Table 5
Strategy-Use Test Results

                      Pretest            Progress

            N      Mean       SD      Mean       SD

ESS
  SWD *      7      6.86     0.69     18.14     3.13
  Other     32      6.09     2.49     16.25     4.08

CSI
  SWD *      7      6.43     2.15      7.00     2.08
  Other     33      6.00     2.51      6.06     2.01

                      Posttest         Maintenance

            N      Mean       SD      Mean       SD

ESS
  SWD *      7     15.93     2.83     17.00     3.37
  Other     32     16.02     3.32     14.19     4.51

CSI
  SWD *      7      5.71     1.60      5.43     2.44
  Other     33      6.76     1.87      6.15     2.49

* SWD=Students with disabilities.

Table 6
Knowledge Test Results

                      Pretest           Posttest

            N      Mean      SD      Mean       SD

ESS
  SWD *      7     6.57     2.57     23.71     5.96
  Other     32     4.19     2.79     22.47     6.20

CSI
  SWD *      7     3.86     2.41      5.43     2.37
  Other     33     4.70     2.77      6.33     3.04

* SWD=Students with disabilities.

Table 7
Unit Reading Comprehension Test Results

                      Pretest           Posttest

            N      Mean      SD      Mean       SD

ESS
  SWD *      7     2.00     2.83     21.86     3.67
  Other     32     2.09     2.18     24.44     5.17

CSI
  SWD *      7     1.14     2.61     17.14     6.15
  Other     33     1.72     2.20     19.67     5.06

* SWD=Students with disabilities.

Table 8
Reading Satisfaction Survey Results

                     ESS                             CSI

           Pretest         Posttest        Pretest         Posttest

          M       SD      M       SD      M       SD      M       SD

Q1       4.64    1.56    5.90    0.99    4.65    1.27    6.15    0.83
Q2       4.59    1.50    6.15    0.90    4.28    1.40    6.12    0.94
Q3       4.05    1.64    6.13    1.11    4.18    1.80    5.79    1.20
Q4       4.72    1.40    6.15    1.11    5.15    1.23    6.08    0.96
Q5       5.58    1.27    6.18    0.97    5.68    1.19    6.30    0.79
Q6       5.79    1.26    5.77    1.14    5.25    1.41    5.77    0.99
Q7       5.17    1.21    5.85    0.90    4.75    1.52    6.03    0.92
Q8       4.68    1.34    6.05    0.94    4.30    1.22    6.07    0.86

Grand
Mean     4.89    0.92    6.02    0.73    4.78    0.99    6.04    0.65

Note. Item score can range from 1 to 7.

Table 9
Posttest Satisfaction with Instruction Survey Results

                   ESS                      CSI

           M        SD      N       M        SD      N

Q1        6.46     0.76     39     6.58     0.64     40
Q2        6.28     0.79     39     6.50     0.93     40
Q3        5.97     1.04     39     5.67     1.02     40
Q4        6.24     1.05     38     6.02     1.07     40
Q5        6.46     0.76     39     6.33     0.89     40
Q6        5.79     1.07     38     5.80     1.07     40
Q7        5.78     0.95     39     6.13     1.01     39
Q8        5.90     0.94     39     6.07     1.07     40
Q9        6.13     0.92     39     6.18     0.98     40
Q10       3.01     0.95     39     6.05     1.18     40
Q11       6.36     0.90     39     6.13     1.02     40
Q12       6.54     0.79     39     6.37     0.74     40
Q13       6.61     0.64     38     6.50     0.72     40
Q14       6.72     0.76     39     6.83     0.50     40
Q15       6.59     0.68     39     6.53     0.64     40
Q16       6.53     0.60     38     6.55     0.68     40

Grand
Mean      6.27     0.54     39     6.26     0.54     40

Note. Item scores can range from 1 to 7.