A longitudinal assessment of gifted students' learning using the Integrated Curriculum Model (ICM): impacts and perceptions of the William and Mary language arts and science curriculum.

With accountability being the latest watchword in education, it has become even more important for educators to produce evidence that curricular innovations do impact students and result in student achievement gains. If these innovations are to continue to compete successfully with other programs for limited dollars, there is a pressing need to demonstrate the effects of the intervention over time. Such analyses of student growth can also provide accurate quantitative measures of student learning, which can in turn be directly attributed to professional efforts of educators (Kupermintz, 2003). Sometimes, several years of exposure may be needed to achieve a reasonably large effect (Hamilton et al., 2003). Nowhere is there a greater need to show sustained student growth than in gifted education, where the viability of programs is constantly under siege. Studies that can contribute to our understanding of the value-added benefits of gifted programs are crucial to making arguments for continuity and new development initiatives.

How are curriculum effectiveness studies carried out? A review of the school effectiveness literature shows that researchers use a combination of methods to gather evidence. Comparisons usually focus on increase in achievement and/or on behaviors and attitudes of students who were exposed to an educational intervention and those who were not (Seltzer, Choi, & Thum, 2003). The most common measure of achievement is standardized tests, as these are usually mandated by school districts, are relatively economical in fiscal and man-hour terms, and tend to have good technical adequacy (Hamilton et al., 2003). However, such tests have severe limitations, especially if used as the sole measure to assess effectiveness of a gifted curriculum intervention. For one thing, these tests tend to have low ceilings, and it would be very difficult for gifted students who score high on the pre-tests to demonstrate growth, unless these tests are used off level (VanTassel-Baska, 2002). For another, these tests tend to utilize predominantly multiple-choice questions that tend to overemphasize basic skills (Resnick & Resnick, as cited in Linn, 2000) and have limited capacity for students to demonstrate more sophisticated skills such as literary analysis, scientific inquiry skills and mathematical problem-solving skills, a higher level of skill repertoire often emphasized in gifted curriculum. As a measure of curriculum effectiveness, standardized tests also fall short as they are not aligned to the content and types of learning emphasized in gifted programs.

Another way to assess curriculum effectiveness is to ascertain whether desired student behaviors and attitudes have been fostered as a result of the intervention. These types of data tend to be collected through perceptual surveys of major stakeholders: students themselves, their parents and teachers. These surveys seek to find out the extent to which stakeholders perceive the curriculum intervention's beneficial effects on the students. Some surveys also involve teachers' reporting on their use of innovative instructional strategies, and these self-reports are in turn analyzed to ascertain their relationship to student performance. Von Secker (as cited in Hamilton et al., 2003) used questionnaire data from the National Education Longitudinal Study of 1988 to examine the relationship between teacher-reported use of inquiry-based learning and student achievement and found a positive correlation. Wenglinsky (2000) also found a positive relationship between teacher-reported use of higher order thinking skills and student achievement scores. The use of survey questionnaires, however, also has an inherent weakness: responses tend to be subjective, and may reflect social desirability results rather than true results. (Hamilton et al.). For example, Gentry, Rizza and Owen (2002) showed that there was little relationship between student perceptions and teacher perceptions of classroom activities.

Classroom visits by independent observers provide a better measure of instructional practice and teacher and students' interaction with the curriculum, although they provide only a snapshot at a point in time. Schoenfeld (2002) in one study reported that implementation of innovative curriculum that involved inquiry-based practices and mathematics investigations was related to improved performance in mathematics.

Some evidence exists that open-ended, performance-based assessments are more suitable than traditional standardized multiple-choice tests to measure the range of competencies expected of students who have experienced a high-powered curriculum. Comparisons of the relationships between open-response scores and multiple-choice scores with reported use of instructional practices show that the former have a slightly larger relationship with reported teaching practices (especially in math) and are therefore better able to detect effects of curriculum intervention (Hamilton et al., 2003). One such indicator suggested by VanTassel-Baska (2002) is performance-based assessment that may combine the use of a long-term project with short-term products to enable one to find out "how the learner has progressed incrementally, as well as where he/she has ended up at the end of the ... intervention" (p.76). Pre-post tests keyed to learner outcomes in the curriculum and use of rubrics to judge the holistic quality of products provide quantitative evidence of student growth resulting from the intervention.

Contemporary studies of effective curriculum innovations in gifted education are scarce and tend to employ only one of the approaches found in the literature; yet some use multiple indicators as recommended by researchers to increase the validity of inferences based on observed gains in student achievement (Linn, 2000). A study that focused on gifted high school student learning as shown on different outcome measures was undertaken by Tyler-Wood, Mortenson, Putney, and Cass (2000). Participants in the program were 32 students identified on the basis of individual intelligence tests, group achievement test scores in math and science, and student interviews. The control group of 32 students in two other schools was matched with the experimental group on age, gender, grade, math and science scores on the Iowa Test of Basic Skills (ITBS), race, and home SES. All participants in the study were in the highest track for math and science in their schools. Using curriculum compacting and acceleration, teachers implemented the integrated math/science program over a period of 2 years. At the end of the intervention, all students took relevant subtests in the math and science sections of the American College Test. Experimental students scored significantly higher than control students on all the subtests. As these students exited high school, their Scholastic Aptitude Test (SAT) scores were compared to those of the control group, and the former scored significantly higher on the math section but not on the verbal test.

Grigorenko, Jarvin and Sternberg (2002) reported three studies that investigated the effectiveness of triarchically based instruction that emphasizes analytical, creative and practical thinking and learning skills. The studies involved 809 fifth grade students, 62 middle school students and 432 high school students. In the first study spanning 2 years, the purpose was to see if students" reading skills could be improved by enriched teaching strategies using an existing curriculum. The objective of the second study was to test whether a stand-alone triarchic reading curriculum relying solely on supplementary materials could significantly improve reading performance. The third study at the high school level involved instructional modifications to existing curriculum materials and blending triarchic instruction into different subject areas for a relatively short period of time. Grigorenko et al. reported that students in all three studies who received triarchic instruction showed greater gains in reading and comprehension on within-unit posttests than did students who received conventional instruction. These studies also showed that students benefited regardless of the grade level or subject matter tested.

Of the 21 research studies related to the Schoolwide Enrichment Model spanning 1983 to 1998 (Renzulli, 1999), 11 focused on affective gains ranging from improvement in student behaviors to better personal skills for project completion to improved teacher and parent attitude toward high ability students. Only two of the studies mentioned achievement scores, and both stressed that students receiving experimental treatment did as well as control group students and did not register any decline in achievement test scores. The effectiveness of the SEM appears to lie in the affective domain, and not in raising student achievement.

Some studies may have expected to find student gains but encountered a different finding. A study undertaken by Reis, Westberg, Kulikowich, and Purcell (1998) sought to examine whether students whose curriculum was compacted would tare differently from students whose curriculum was not compacted. The study involved 436 teachers teaching grades two through six in 27 school districts. Using the Classroom Practices Questionnaire and the compactor form to assess teachers' classroom practices, the researchers concluded that between 36% and 54% of content was eliminated. Using 336 students' pre- and post-intervention performance on the ITBS, the researchers concluded that results of experimental students whose curricula were eliminated by 40% to 50% were not different from those of control students whose curricula were not compacted. Although this study shows that compacted curriculum modifications will not hurt student performance in standardized tests, it did not show the effectiveness of the intervention and how it impacts student growth. It would be difficult to make any inference about student gains, since in an earlier study of replacement strategies teachers used, Reis and Purcell (1993: as cited in Johnsen, 1999) reported that while 60% of these activities reflected student interests, they lacked "appropriately challenging work for high ability students" (p. 110). A similar study to show that enriched instruction does not hurt traditional student achievement was undertaken by Gallagher and Stepien (1996). In their study, 167 high school students exposed to problem-based learning (PBL) instruction did not sacrifice content acquisition in American studies as assessed by a standardized multiple choice test.

Reis et al. (2003) conducted a study that investigated the effects of a reading program based on the SEM Reading Framework (SEM-R) on elementary students' reading achievement, fluency, and interest in reading. Two hundred and forty third- to sixth-grade students and their teachers in two low socio-economic urban schools were randomly assigned to the control and experimental groups. While the former continued with their existing reading (Success for All) and remedial reading programs, the experimental group utilized the SEM-R program for a period of 10 to 12 weeks. Based on results from one school (as one school's treatment fidelity was not maintained), Reis et al. reported that at the beginning of the intervention, 90% of the experimental students could not sustain independent reading for more than a few minutes, and only 10% could read independently for 5 minutes. The post-intervention results were "remarkable": 90% of the students in SEM-R classrooms could read for a period of 35 to 40 minutes. The researchers also found significant differences in the experimental groups' reading achievement (based on the Iowa subtest of reading comprehension), reading fluency (based on three passages that students read aloud to researchers for a period of one minute each), and reading attitude (as measured on the Elementary Reading Attitude Survey). Not only did these differences favor the experimental group, the researchers also found through subanalysis that the intervention benefited talented readers as well as average and below average readers.

A final set of research studies on curriculum effectiveness examined the effects of specific curriculum units designed specially for high ability learners. The first of these was a national study of science curriculum effectiveness based on a unit Acid, Acid Everywhere designed by the Center for Gifted Education at the College of William and Mary (VanTassel-Baska, Bass, Ries, Poland, & Avery, 1998). The goals of the unit were to develop student understanding of the concept of systems, to develop specific content learning, and to develop scientific research process skills. The effectiveness study involving 1,471 students in 62 classrooms in 15 school districts in 7 states focused explicitly on student application of the scientific process after being taught a 20- to 36-hour science unit. A pre-post test measure The Diet Cola Test--was used to assess students' ability to apply the scientific method and demonstrate scientific reasoning skills. Posttest means showed significant differences between the experimental students and comparison students of similar ability, with a large effect size of 1.3. Qualitative data collected from teachers suggested that the units produced high motivation among students and teachers and tapped into higher level skills of both.

In another study, VanTassel-Baska, Zuo, Avery and Little (2002) investigated the effectiveness of a language arts curriculum in terms of student gains in literary analysis and interpretation, and persuasive writing. Forty-six schools in diverse districts across 10 states participated in the study. Although students in grades two to eight, totaling 2,189, were pre-identified as gifted, there was a range of ability as different states had different identification criteria. Students were exposed to four curricular units organized around the Integrated Curriculum Model. Two pre/post performance-based assessments of literary analysis and interpretation and of persuasive writing were embedded in the units to assess student learning gains. Analysis of covariance of posttest means of treatment and control students indicated statistically significant differences favoring the treatment groups for both dimensions--literary analysis and interpretation and persuasive writing. Effect sizes using eta squared ranged from medium (.07) to large (.24). The treatment appeared to be similarly effective for male and female students, students across all SES groups, and grouping models. Teachers were positive about the units and perceived that students were enthusiastic and energized by the activities proposed.

Purpose

The purpose of this study was to assess the effects over time of implementing a differentiated curriculum for gifted learners designed around the Integrated Curriculum Model (ICM). There were two major research questions: 1 ) To what extent is there evidence of gifted students' growth as a result of the use of ICM-based curriculum? 2) To what extent is this curriculum meeting the needs of identified students as perceived by relevant stakeholders?

Methodology

Sample

A total of 973 students (grades three to nine) from one northeastern suburban school district comprised the overall student sample of this study, among which there were 116 third-graders, 106 fourthgraders, and 109 fifth-graders. Most of these students had been exposed to the William and Mary language arts and science units over a 3-year period and were tested by the corresponding performance-based assessments during the period of 1996 and 2002. In the perceptual part of the study, surveys were returned by 367 parents, 110 educators, and 732 students.

Instrumentation 

The instruments employed to assess student learning were performance based in orientation. Interrater reliability was .90, based on the use of outside trained observers over most years of data collection. The Diet Cola Test which assessed scientific experimental design facility has reported form reliability at .76 and interrater reliability from .90 to .95; it has been found to be a highly sensitive instrument to students' different responses and has been recommended for use with gifted students to assess high level science learning (Adams & Callahan, 1995). Researchers and project staff also found that it adequately assessed research skills in experimental design (VanTassel-Baska et al. 1998). The pre-post literary analysis assessment and pre-post writing assessment was developed and validated in earlier studies (VanTassel-Baska, Johnson, Hughes, & Boyce, 1996; VanTasselBaska, Zuo, Avery, & Little, 2002). The pre-post writing assessment rubric was also validated with other populations (Burkhalter, 1995).

The stakeholder survey instrument used in the study contained multiple-choice and Likert-type items, with two open-ended questions. It consisted of 28 to 31 items (depending on the stakeholder group) and was designed for electronic scanning. It included key sections derived from the "Best Practices" benchmarked by the National Association for Gifted Children (NAGC) standards in curriculum and instruction for the gifted. In addition, it probed staff development, personnel qualifications, and communication issues. Items regarding the utilization and impact of the curriculum were also included. Separate forms were tailored for parents and educators.

In the student survey, a section on curricular impact was designed to measure student perception and perceived cognitive and affective growth. Categories centered on mastery of advanced content, higher order thinking skills, creativity, research skills, and motivation to learn.

Procedures

The school district has an established data set recording gifted students' prepost assessment in key areas of language arts, specifically, literary analysis, persuasive writing, and grammar, and in the key areas of experimental design in science. The language arts units gifted students have been exposed to were Journeys and Destinations, Literary Reflections, and Autobiographies at grade three, four, and five, respectively. The corresponding science units that have been used at these grade levels were What a Find, Electricity City, and Acid, Acid Everywhere. In earlier studies, regardless of grade level or topical focus, the curricular units produced similar results (VanTassel-Baska et al., 2002; VanTassel-Baska et al., 1998). This performance-based assessment data set covering the 1996 to 2002 cohorts of elementary gifted students was analyzed to assess the effects of implementing the William and Mary language arts and science curriculum for gifted students.

Survey instruments were employed to probe stakeholder perceptions of curriculum implementation and efficacy, specifically, the level of challenge of the curriculum, stakeholders' satisfaction with the curriculum, as well as perceived benefits and liabilities. Focus groups were also conducted with students, teachers, and administrators to allow for a deeper understanding of stakeholders' perceptions of the curriculum.

Data Analysis

Both quantitative and qualitative techniques were used in data analysis. Descriptive statistics such as means, standard deviations, and percentages were used to present the survey data; inferential techniques such as paired samples t tests, and analysis of variance were used to investigate students' prepost achievement gains in targeted areas of language arts and science.

Limitations

There were several limitations to the study. First, there was a lack of a comparison group from the school district. All gifted students in the district have received the same treatment over the last 6 years, making an in-district comparison group difficult to establish. However, because of the multi-year track of students' performance on pre- and post-tests by multi-cohorts (1996-2002), an inference of the curricular impact might be made through the longitudinal pattern of performance.

Another limitation was the low rate of return of the parent survey (40%) and the teacher survey (48%), which might affect the overall generalizability of the survey findings. Finally, although the district has collected performance-based assessment data over a period of 6 years, the reported data in this study cover only grades three to five for any given cohort group. Current sixth- through eighthgraders' performance-based assessment data were not available.

Results

Paired-samples t tests were performed to examine students' academic growth in the program in the key areas of language arts and science over a period of 6 years (1996-2002). Table I shows that statistically significant gains from pre- to post-assessment were registered in literary analysis, persuasive writing, grammar, and scientific research skills, regardless of the grade level being tested. The effect sizes using Cohen's d index ranged from .52 to 1.38, suggesting that these academic gains were educationally important and statistically significant. Except for literary analysis, students' academic growth in persuasive writing, grammar, and scientific research skills have been large, an increase of more than one standard deviation above the pre-test means in corresponding areas. The magnitude of achievement growth in literary analysis was of medium size; the size of growth in persuasive writing appeared to slow down to the moderate level at fifth grade (d = 0.66), as compared to growth of 1.28, and 1.27 standard deviations above the pre-test means at the third and fourth grade. However, the overall academic growth steadily increased from lower to higher grade levels in all domains

Repeated Exposure Effect

Students in the study were using the William and Mary language arts and science units consecutively in their program. A total of 835 students had the experience of using the language arts units at the elementary school level, with three quarters of them having been exposed to the language arts units twice (21.3%) or three times (43.4%); 35.3% of them had only one year of exposure to the language arts units. A total of 660 students had used the William and Mary science units; the distribution of science curriculum exposure was similar to that of the language arts units.

Table 2 demonstrates the effect of repeated usage of the curriculum in language arts. The pre- and post-assessment scores at the fifth grade level were selected because there was a range of curriculum exposure from 1 to 3 years among the students, where repeated exposure effect could be calculated.

Using filth grade language arts assessment results for repeated exposure effect analysis, statistically significant mean differences were registered in all target areas of language arts from pre- to post-assessment, regardless of the frequency of curriculum exposure. However, there was a constant increase of mean scores on both pre- and post-tests along with the frequency of usage of the curriculum in literary analysis, persuasive writing, and grammar, suggesting a positive repeated curriculum exposure effect.

Table 3 shows pre- and post-assessment comparisons in scientific research skills among the students who received the science curriculum up to 3 years. Using students' fifth grade pre- and posttest scores in experimental design for comparison, statistically significant increases of achievement from pre- to post-assessment were registered regardless of the frequency of science curriculum exposure. The second time users of the curriculum appeared to have lower pre- and posttest scores than those of the first time and third time users of the science curriculum. Although the second time users of the science curriculum had the lowest pre- and post- mean scores, the magnitude of achievement of increase was the largest (d = 1.11). Students who received the curriculum three times achieved the highest pre- and posttest mean scores in experimental research design. Even though there was a dip in mean performance at fourth grade, achievement gains continued in fifth grade.

These results suggest that in both language arts and science there appears to be a repeated exposure effect of the curriculum. The increase of achievement in the assessment of student skills in literary analysis, persuasive writing, and grammar were constant along with the increasing usage of William and Mary language arts units; students' improvement in scientific research skills appeared to be also positively related to the value-added exposure to the science units.

Stakeholders' Perceptions of Curriculum Effectiveness

The stakeholder survey results showed that a majority of both parents and teachers perceived the William and Mary language arts (70% and above) and science curriculum (58% and above) as sufficiently challenging, while a slight majority of the student respondents cited both the language arts and science curriculum as challenging sometimes but not always.

A vast majority of both regular and teachers of the gifted (93.6% and 100.0% respectively) cited that they frequently employed a series of instructional approaches used in the units in their classes, including "higher order questions," "student discussions," and "individual and group project work." Parent survey results appeared to confirm teachers' self-report of their instructional behaviors.

A vast majority of parents (91.6%) expressed satisfaction with the overall program curriculum, which encompassed language arts, math, and science, in terms of its effectiveness in addressing their children's learning needs, although about 10% of parents noted opposite opinions. Over two-thirds of the educators echoed parents' views with the overall effectiveness of the curriculum.

A cross the stakeholder groups, the accelerated and challenging nature of the curriculum, its organization, scope and sequence, and peer interaction opportunities were perceived as the top benefits students received from the gifted program; moreover, the curriculum's emphasis on critical and complex thinking skills, discussion and debate were also cited as the positive sides of the curriculum. However, stakeholders also criticized the lack of flexibility in materials selection, the lack of variety of reading materials such as nonfiction as weaknesses of the language arts curriculum. Students commented in the survey that they wished a broader range of topics could be selected for their reading materials and that they could be part of the selection process. Some students noted that the curriculum could be delivered in a more lively way, such as engaging students in conversations, connecting the stories with movies, jokes, riddles, and field trips.

Teachers requested that more materials be supplied to help improve their background content knowledge of topics to be covered in the units. For language arts, the common request was for more nonfiction materials as state tests focused on this area. Teachers needed more materials for the science unit to help them fill in the gaps in content. Parents noted that the curriculum was dependent upon the teachers for delivery, with instructional quality of delivery somewhat inconsistent. Areas for improvement included more professional development to eliminate inconsistency within the program, increased number of writing assignments, and greater interdisciplinary integration.

Discussion

This study provides an important model for local school districts in tracking gifted student performance across years in relevant areas. Currently longitudinal data of this type is very limited, yet districts could begin to collect and aggregate student assessment results on appropriate measures as a routine path for evaluating the outcomes of their program in general or curricular intervention in particular. Both learning growth gains on relevant measures and perceptual data collected annually would provide a strong picture of program health. Use of the same performance-based measures over time also allows educators to assess the real learning of gifted students in higher level skills and task demands. By "taking the temperature" of stakeholder groups alongside quantitative performance data, a more holistic picture emerges. It is one thing for the medicine to be good for you; it is quite another to suggest that you perceive its beneficial effects and even like it. Moreover, such perceptual data also enhance our understanding of the teacher/student interactive motivational impact of high quality curriculum. Teachers were challenged by the curriculum and passed their interest and excitement along to their students, who in turn ignited teachers even more.

Another theme of interest in this study relates to the issue of novelty in comparison to consistency in curriculum use and in related teacher training. Research studies in science, math, and language arts have suggested that the consistent use of research-based instructional practices, such as higher-level thinking, metacognition, and problem-solving, enhance student learning over time (Wenglinsky, 2000; Hillocks, 1986). Teacher training studies have stressed the need for training to be based within content areas, emphasizing the ways particular strategies enhance learning (Kennedy, 1999). Guskey (2000) has even suggested that teacher change is highly related to the extent to which teachers believe the use of particular strategies will improve learning for their students. While consistency in using research-based curriculum and accompanying strategies has been supported in the literature, common practice tends to favor novelty, the search for new and different ways to deliver different topics.

Clearly, the William and Mary curriculum in its careful design and guidelines for implementation suggest that consistency is both desirable and necessary to impact gifted student learning at a group level. Newer curriculum models (e.g., Tomlinson, et al., 2002) may offer novelty but lack evidence of effectiveness. Just as consistency in curriculum implementation and teacher training is necessary to avoid curriculum drift, so too is teacher stability in program assignment. For teachers working with gifted learners to implement curriculum effectively, they must implement the same curriculum for 3 years. Such stability over time enhances their skills to teach as effectively as possible. Yet the reality in many school districts is just the opposite. Teachers are assigned to teach gifted classes for less than 3 years, thus jeopardizing the extent of student learning.

In addition to the issue of novelty versus consistency, there is also the issue of breadth versus depth. Novelty represents breadth, moving on from one topic to the next and from one skill set to the next. Yet depth requires "staying the course," acknowledging that mastery of higher level skills requires consistent application over time. The William and Mary curriculum units that employ the ICM model offer a clear choice of depth of learning in higher level skills within the requisite subject area.

Conclusion

Gifted student learning at grades three to five has been enhanced at significant and important levels through the use of the William and Mary curriculum in language arts and science across individual academic years. Moreover, student learning over time (1996-2002) has been enhanced by the use of these curricula as well. Repeated exposure over 2 to 3 year periods across cohort groups demonstrates strong increasing achievement patterns.

The majority of all stakeholder groups found the curriculum to be effective and beneficial in very specific ways. Top rated benefits across groups of parents, students, and educators in both content areas stressed the accelerated challenge, critical thinking, problem solving, and peer interaction.

Implications for Practice

The evidence base for the use of the William and Mary language arts and science curricula continues to grow. Yet, the curriculum may need to expand in respect to options available to students, especially in selected areas. This expansion could be a natural progression from the units currently being employed. The gifted program could also benefit from engaging teachers in ongoing curriculum development activities to ensure dynamism and teacher involvement in shaping curriculum implementation.

Gifted programs need to provide more targeted professional development activities for teachers in gifted education. for all teachers in relevant gifted topics, and for administrators and other professional staff. Staff development sessions for gifted teachers need to be targeted on curriculum implementation issues such as strategies and lesson plan translation. As was seen in this study, teachers were uneven in their capacity to deliver a high quality curriculum, a situation that could be ameliorated with more training and oversight at the implementation stage. Curriculum implementation needs to be monitored to ensure adequate attention to curricular objectives. A monitoring system would ensure that teachers are adhering to and deviating from curriculum usage in appropriate ways.

More curriculum development work also needs to be done to align curriculum and state or district objectives more closely in relevant areas and to extend curricular opportunities into integrated areas of learning like the arts, history and technology. Alignment of the curriculum with state standards needs to occur in order to ensure that the gifted program represents a "value-added" approach, moving students through the standards but providing work and challenge beyond them as well.

Implications for research include a need for replication studies in other districts that can document the 3-year growth patterns found in this study for science and language arts learning at higher levels. Other studies, using appropriate level and type of standardized assessments, would further validate these findings, as would a study examining long term outcome differences on high stakes measures such as Advanced Placement tests and the Scholastic Aptitude Test. Studies that analyze gifted student performance from underrepresented groups would also contribute to our understanding of the curriculum's flexibility of use.

Table 1

Paired-Samples t Tests (1996-2002)

Testing Assessment           N        Pretest        Posttest
Grade                                 Mean (SD)      Mean (SD)

3       Literary Analysis    468       6.87 (2.41)    9.12 (2.85)
        Persuasive Writing   461       7.93 (2.96)   14.82 (4.73)
        Science Research     252       3.34 (1.61)    6.63 (2.51)
4       Literary Analysis    413       8.18 (2.50)   10.52 (3.19)
        Persuasive Writing   433      11.17 (4.15)   17.94 (5.04)
        Grammar              275      12.09 (4.94)   21.67 (3.42)
        Science Research     276       4.83 (1.81)    7.83 (2.77)
5       Literary Analysis    426       9.42 (3.61)   10.93 (3.43)
        Persuasive Writing   450      15.05 (4.40)   18.45 (5.17)
        Grammar              167      17.40 (4.14)   22.29 (3.47)
        Science Research     280       6.31 (2.44)    9.06 (2.68)

Testing Assessment           t         d
Grade

3       Literary Analysis    17.13 *    0.79
        Persuasive Writing   27.54 *    1.28
        Science Research     21.70 *    1.37
4       Literary Analysis    14.64 *    0.72
        Persuasive Writing   26.34 *    1.27
        Grammar              29.23 *    1.76
        Science Research     18.19 *    1.09
5       Literary Analysis    10.77 *    0.52
        Persuasive Writing   14.07 *    0.66
        Grammar              17.83 *    1.38
        Science Research     16.70 *    1.00

* p < .001

Table 2

Repeated Exposure Growth--Language Arts

Testing Assessment   Usage      N        Pretest        Posttest
Grade                Frequency         Mean (SD)        Mean (SD)

5       Literary       1        70        8.56 (3.56)   10.31 (3.39)
        Analysis       2        42        8.83 (2.71)   10.04 (2.08)
                       3        30       19.76 (3.75)   11.26 (3.61)
        Persuasive     1        71       12.46 (4.11)   16.59 (4.97)
        Writing        2        44       14.45 (3.97)   17.86 (4.47)
                       3       328       15.74 (4.31)   19.00 (5.21)
        Grammar        1        23       12.78 (3.29)   18.91 (4.95)
                       2        14       17.36 (3.37)   22.79 (2.00)
                       3       129       18.20 (3.82)   22.83 (2.93)

Testing Assessment   t           d
Grade

5       Literary     4.88 **    0.58
        Analysis     3.15 *     0.49
                     8.79 **    0.51
        Persuasive   6.57 **    0.78
        Writing      5.23 **    0.79
                    11.33 **    0.63
        Grammar      5.96 **    1.24
                     6.88 **    1.83
                    15.96 **    1.41

** p < .001

Table 3

Repeated Exposure Growth--Experimental Research Design

Testing     Usage             Pretest      Posttest
Grade     Frequency    N      Mean SD      Mean (SD)       t        d

5             1        43   5.81 (2.02)   8.63 (3.16)    5.76 **   0.85
              2        45   5.20 (2.61)   7.76 (2.72)    7.42 **   1.11
              3       186   6.72 (2.40)   9.52 (2.43)   13.75 **   1.01

** p < .001

REFERENCES

Adams, C. M., & Callahan. C. M. (1995). The reliability and validity of a performance task for evaluating science process skills. Gifted Child Quarterly, 39, 14-20.

Burkhalter, N. (1995). A Vygotsky-based curriculum for leaching persuasive writing in the elementary grades. Language Arts, 72, 192-196.

Gallagher, S. A., & Stepien, W. (1996). Content acquisition in problem-based learning: Depth versus breadth in American studies. Journal for the Education of the Gifted. 19, 257-275.

Gentry, M., Rizza, M. G., & Owen, S. V. (2002). Examining perceptions of challenge and choice in classrooms: The relationship between teachers and their students and comparisons between gifted students and other students. Gifted Child Quarterly, 46, 145-155.

Grigorenko, E. L., Jarvin. L., & Sternberg, R. J. (2002). School-based tests of the triarchic theory of intelligence: Three settings, three samples, three syllabi. Contemporary Educational Psychology, 27, 167-208.

Guskey. T. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin.

Hamilton. L. S., McCaffrey. D. F., Stecher, B. M., Klein, S. P., Robyn, A., & Bugliari, D. (2003). Studying large-scale reforms of instructional practice: An example from mathematics and science. Educational Evaluation and Policy Analysis, 25(1), 1-30.

Hillocks, G., Jr. (1986). Research on written composition: New directions for teaching. Urbana, IL: ERIC Clearinghouse on Reading and Communication Skills and National Conference on Research in English.

Johnsen, S. (1999). Renzulli's model: Needed research. Journal for the education of the gifted. 23, 102-116.

Kennedy, M. (1999). Form and substance in mathematics and science professional development. NISE Brief 3 (2), 1-7.

Kupermintz, H. (2003). Teacher effects and teacher effectiveness: A validity investigation of the Tennessee Value Added Assessment System. Educational Evaluation and Policy Analysis. 25, 287-298.

Linn, R. L. (2000). Assessments and accountability. Educational Researcher, 29(2), 4-16.

Reis, S. M., Gubbins. E. J., Richards, S., Schrieber, R., Briggs, C., Jacobs, J., & Eckert, B. (November, 2003). Talented readers. Presentation at the National Association for Gifted Children 50th Annual Convention, Indianapolis, IN.

Reis. S. M., Westberg, K. L., Kulikowich, J. M., & Purcell, J. H. (1998). Curriculum compacting and achievement test scores: What does the research say? Gifted Child Quarterly, 42, 123-129.

Renzulli, J. (1999). Twenty-five year perspective. Journal for the Education of the Gifted. 23, 1-54.

Schoenfeld. A. H. (2002). Making mathematics work for all children. Issues of standards, testing and equity. Educational Researcher, 31(1), 13-25.

Seltzer, M., Choi, K., & Thum. Y. M. (2003). Examining relationships between where students start and how rapidly they progress. Using new developments in growth modeling to gain insight into the distribution of achievement within schools. Educational Evaluation and Policy Analysis. 25, 263-286.

Tomlinson, C. A., Kaplan. S. N., Renzulli, J. S., Purcell, J., Leppien, J., & Burns, D. (2002). The parallel curriculum: A design to develop high potential and challenge high ability learners. Thousand Oaks, CA: Corwin Press.

Tyler-Wood, T. L., Mortenson, M., Putney, D., & Cass, M. A. (2000). An effective mathematics and science curriculum option for secondary gifted education. Roeper Review, 22, 266-269.

VanTassel-Baska, J. (2002). Assessment of gifted student learning in the language arts. Journal of Secondary Gifted Education, XIII, 72-76.

VanTassel-Baska, J., Bass, G., Ries, R. Poland, D., & Avery, L. (1998). A national study of science curriculum effectiveness with high ability students. Gifted Child Quarterly, 42, 200-211.

VanTassel-Baska, J., Johnson, D., Hughes, C., & Boyce, L. N. (1996). A study of language arts curriculum effectiveness with gifted learners. Journal for the Education of the Gifted, 19, 461-468.

VanTassel-Baska, J., Zuo, L., Avery, L. D., & Little, C. A. (2002). A curriculum study of gifted student learning in the language arts. Gifted Child Quarterly, 46, 30-44.

Wenglinsky, H. (2000). How schools matter. The link between teacher classroom practices and student academic achievement. Education Policy Analysis Archives, 10. Retrieved November 20, 2003 from http://epaa.asu.edu/epaa/v10n12/

Manuscript submitted February 11, 2004.

Revision accepted April 12, 2004.

Annie Xuemei Feng is the research and evaluation coordinator of the Center for Gifted Education at the College of William & Mary. Her research interests include curriculum effectiveness studies, program evaluation research, gender-related studies, and cross-cultural research. E-mail: [e-mail]

Joyce VanTassel-Baska is the Jody and Layton Smith Professor of Education at the College of William and Mary in Virginia, where she has developed a graduate program and a research and development center in gifted education. Dr. VanTassel-Baska has published widely, including five recent books and over 300 monographs, book chapters, and articles in referred journals. She also serves as the editor of Gifted and Talented International E-mail: [e-mail]

Chwee Quek is a doctoral student at the College of William and Mary and the assistant editor of Gifted and Talented International E-mail: [e-mail]

Wenyu Bai is a doctoral student at the College of William and Mary. His research interests include philosophical precocity and developmental strategies. E-mail: [e-mail]

Barbara O'Neill is the coordinator of gifted programs in Greenwich Public Schools. E-mail: Barbara_O'[e-mail]