The impact of engagement in large-scale assessment on teachers' professional development: the Emergent Literacy Baseline Assessment Project.

Abstract. This paper presents findings of an investigation into the impact of teachers' engagement in the Emergent Literacy Baseline Assessment (ELBA) project upon their professional development. The ELBA project was designed to identify Cypriot students' literacy skills upon entry to primary school. A secondary purpose was to enable pre-primary teachers to assess students using effective and developmentally appropriate methods. Findings support the importance of establishing links between teachers' professional development and assessment reform policy by providing teachers the opportunity to be actively involved in the development, implementation, and evaluation of system-wide assessment initiatives. For the purposes of this study, 132 teachers received training and then they administered the ELBA performance test to their students. Self-assessment questionnaires and a focus-group interview provided data about the perceived impact of ELBA upon teachers' professional development. Arguments in favor of engaging teachers in large-scale tests in order to improve their professional skills in assessment and suggestions for further research are provided. ********** Large-scale assessment flourished during the 1990s (Kifer, 2001). Standards, assessment, accountability, and grading dominated education discussions. These issues are at the center of every modern education reform effort. Where national testing programs previously were expected mainly to monitor student achievements, new and different demands are now in place. Internationally, we are confronted with a marked growth in the need for assessment to fulfill a variety of purposes: for policy decision-making and accountability (evaluative purposes of assessment), for reporting students' achievements and awarding qualifications (summative purposes of assessment), and for informing and optimizing students' learning within the classroom (formative purposes of assessment). Typically, these moves have been driven by legislation to raise standards (Postlethwaite, 1994). However, the different purposes for which teachers and schools use information about the achievement of their students has led to an "assessment burden," as reflected in an overload of assessment and assessment role conflict (Sebatane, 1998). Kyriakides (1998) reports that Cypriot primary teachers lack both assessment knowledge and confidence in assessment, a finding reinforced by a number of researchers in other countries. Gilmore (2001) identified "an urgent need to address the assessment literacy of New Zealand teachers" (p. 3), and Black and Wiliam (1998) refer to a "poverty in assessment practice" (p. 5) in the United Kingdom. Thus, building teacher capacity in assessment through systematic engagement of teachers in assessment reform programs needs to be a priority throughout all career stages of teaching.

In this context, this article presents results of a study to identify what benefits pre-primary teachers gained, personally and professionally, from their engagement in the Emergent Literacy Baseline Assessment (ELBA) project. The decision to engage teachers in large-scale assessment was based on the authors' acknowledgment of the classroom teacher's central role in assessment. The best person to judge students' work and monitor their progress is the teacher closest to them (Association for Childhood Education International/ Perrone, 1991). For example, when a teacher reads a student's piece of writing, he/she can refer to previous writing efforts of the student or to a book that the class is currently reading, a trip recently taken, etc. By thoughtfully responding to the surrounding context, which is never really separate from the text, the teacher can better interpret the student's writing. The engagement of the teacher in large-scale tests may help her to better understand students' responses to a task and identify their learning needs. Thus, classroom environment and the teacher were acknowledged as central to the ELBA assessment program.

Teachers' engagement in the ELBA project was expected to not only provide information about children's needs but also inform teachers' ongoing instructional practice and thereby contribute to their professional development. In addition, the conditions under which the ELBA project may help shift teaching literacy toward a more intellectually challenging direction are examined. The researchers recognize the importance of developing a policy on baseline assessment in literacy for formative reasons (Kyriakides, 1999), as well as the fact that there is no guarantee that providing resources in baseline assessment will, in fact, result in more effective teaching and learning in primary schools and, hence, higher education standards.

Empowering Teachers' Assessment Through the Engagement of Teachers in Large-Scale Assessments

Professionalization of teaching is often perceived as an important way to enhance the status and the quality of teaching (Cheng, 1996; Huberman, 1990). Professionalization of teaching may involve systematization of education knowledge, long-term professional training, and professional autonomy. During the last decade, a considerable body of literature has emerged on professional development (e.g., Garet, Porter, Desimone, Birman, & Yoon, 2001). The research literature contains a mix of large- and small-scale studies, including intensive case studies of classroom teaching, evaluations of specific approaches to improving teaching and learning, and surveys of teachers about their preservice preparation and inservice professional development experiences. In addition, extensive literature exists that describes best practices in professional development.

Relatively little systematic research has been conducted, however, on the effects of professional development on actual and perceived changes in assessment practice (Kyriakides, 1998). This lack can be attributed to the fact that prevailing practice in student assessment seems to undermine teacher professionalism (Flockton & Crooks, 1997; Stake, 1991). It seems odd that in the discussions in many countries about "empowering" teachers' professionalism and restructuring schools (e.g., the United States, the United Kingdom, Australia, Cyprus), few reform proposals address the issue of how to shift the balance of status and influence in favor of teacher assessment over external modes of assessment. Furthermore, large-scale assessment seems to be unconcerned about the central mission of education--the instruction of children (Popham, 1999). Information collected from 31 countries to investigate trends in large-scale testing revealed a clear trend toward adding, not dropping, testing programs (Phelps, 2000). However, the investigation did not result in the identification of systematic attempts in any of these countries to develop a policy on teachers' professional development that acknowledges the ability to devise appropriate methods for gathering information about students' learning as a central skill of the professional teacher. This is despite the fact that the quality of teachers determines to some extent the implementation of education policy (Fullan, 1991). Assessments are expected to produce major changes in schools and provide a means for establishing that the changes occurred. As educators turn to the task of implementing these reforms, however, they face a complex array of questions and concerns that they are not prepared to address. Thus, there is a need for a strong link between assessment reform programs and teacher development, and this link could be achieved by providing teachers the opportunity to be actively involved in the development, implementation, and evaluation of system-wide assessment initiatives such as the ELBA project.

Currently, conversation within the educational assessment community about the impact of large-scale assessment is being directed increasingly towards the consequential aspects of validity (Kane, 2001), rather than its impact upon teachers' professional development. Perspectives range from fears about "teaching to the test" to confidence in the capacity of these assessments to model, support, and positively shape curricular and instructional reform (Cizek, 2001). Such conflicting perspectives, combined with growing recognition that changes in practice are not easy to bring about, have created particular interest in investigating the extent to which large-scale assessment programs might benefit instruction (Firestone, Mayrowetz, & Fairman, 1998; Goldberg & Rowsell, 2000). These studies reveal that involvement of teachers in large-scale assessment can support the kinds of changes in instructional practice that are necessary to meet new performance standards. For instance, Sheingold, Heller, and Storms (1997) showed that teachers' involvement in scoring enhanced teachers' understanding and application of the course outcomes and gave them an expanded picture of students' outcomes. Similarly, Falk and Ort (1998) concluded that teachers' involvement in scoring standards-based performance assessments deepened their understanding of standards, and offered new perspectives on what students know and can do. However, these studies have not attempted to examine the extent to which teachers changed their practice. In this context, this study is an attempt to extend the conversation about consequential validity by exploring the impact on teachers' instructional practice of their engagement in not only scoring but also administering a performance test. Responding to injunctions that research on the consequences of testing needs to be intensive and contextualized (Goldberg & Roswell, 2000), the authors examined the extent to which teachers' engagement in the ELBA project had an impact upon their own knowledge and beliefs about teaching and assessment of emergent literacy. They also investigated the extent to which teachers' classroom practices and special school policies were changed.

The Emergent Literacy Baseline Assessment Project

In the present study, the professional impact of pre-primary teachers' engagement in a large-scale assessment project on emergent literacy, namely ELBA, is investigated. The ELBA project was designed to identify Cypriot students' emergent literacy skills upon entry to primary school. It was expected that the importance of developing a national policy on emergent literacy baseline assessment would emerge. A secondary purpose was to enable pre-primary teachers to assess students in literacy using effective and developmentally appropriate methods (Kelly, 2000).

The formative purpose of baseline assessment is based on the fact that children of a similar age are not at the same level and do not progress at the same rate. Thus, information gathered from each child can be used to match the skills or content of a task to the level of the child. Ultimately, these findings should have implications for how teachers organize their classes. Moreover, baseline assessment, as its name implies, provides the base against which students' subsequent education progress can be measured. Measures of students' educational progress in a particular school, relative to that made by similar students in other schools, have come to be called "value-added" assessment. Thus, baseline assessment may contribute to the attempts of central governments to evaluate school units by adopting the technique of value-added assessment (Department for Education and Employment, 1995), which is more valid in exploring the effectiveness of a school unit than using outcome data only (Fitz-Gibbon, 1995).

With this in mind, the authors designed a performance test to assess the knowledge and skills in emergent literacy identified in the Cyprus Curriculum (Ministry of Education, 1994), and that are in line with the main findings of research relating to early language and literacy development (Bishop & Adams, 1990; Catts, 1989, 1997; Carts, Fey, Zhang, & Tomblin, 1999, 2001; Harste, Woodward, & Burke, 1984; Holdaway, 1979; Whitehurst & Lonigan, 1998). For the purposes of this project, literacy is defined as an emerging process that begins very early in life as parents and young children respond to each other using nonverbal, preverbal, and oral communication (Strickland & Morrow, 1989). Literacy development is an ongoing, lifelong process that is considered a cultural, social, and cognitive achievement and is best mastered by children as they are actively engaged in interpreting and making sense of the complex system of signs and symbols in their world (Harste, Woodward, & Burke, 1984). The development of literacy includes the acquisition of oral language, which serves first as a precursor and then as a companion to reading and writing. To become literate, children also must develop an awareness of the structure and function of print, as literacy develops in and out of the school setting. Educators and policymakers alike have reached a consensus that children must develop phonological awareness, or the knowledge that language is made up of a system of sounds, and that unsuccessful readers are those who fail to recognize the sounds of language in processing oral and written information. In reviewing multidisciplinary research evidence accumulated over the past 40 years, the National Reading Panel (2000) concluded that children who fail to develop phonological awareness at an early age, and most critically by grade 2, have little chance of being academically successful. Longitudinal research also indicates that students who are disadvantaged in acquiring this early reading skill have an extremely difficult time catching up to their peers, and that this striking gap in achievement only increases over time (Felton & Pepper, 1995; Juel, 1988; Snow, Burns, & Griffin, 1998; Stanovich, Cunningham, & Cramer, 1984).

The considerations noted above were instrumental in designing the literacy components to be assessed by ELBA. As indicated by research, the areas most predictive of literacy success or failure in the elementary years included print awareness, phonological awareness, and oral language (Adams, 1990; Blachman, 1991; Catts et al., 1999; Morrow, 1993). It should be noted that these three areas of literacy competency are also defined by the Cyprus Curriculum (Ministry of Education, 1994), the English national curriculum (DfEE, 1999), and the National Reading Panel (2000).

ELBA was designed from a developmentally appropriate practices (DAP) framework, and all 10 assessment tasks were administered by classroom teachers in the pre-primary classroom setting, thus utilizing natural environments and familiar adults. Specific literacy concepts and behaviors related to the three predictive components of literacy (print awareness, phonological awareness, and oral language) were selected, defined, and embedded within the 10 tasks that students completed either individually or in small groups. These teacher-directed tasks were selected, piloted, and revised, based on several critical qualities of developmentally appropriate assessment practices, including utility, acceptability, authenticity, equity, and sensitivity (Bagnato & Neisworth, 1999). A sequenced format was recommended to teachers; however, tasks were designed to be administered over several days and in a variety of formats that allowed for flexibility.

The content of the activities and the procedures for administering and scoring ELBA were developed with the assistance of two pre-primary consultants at the Ministry of Education and Culture in Cyprus. The 10 activities chosen included tasks familiar to Cypriot children and consisted of drawing and writing (self-portraits and family portraits), storytelling (using wordless picture books and open-ended toys and manipulatives), listening for details (auditory memory), and matching and sorting letters and pictures, as well as some activities that were not familiar (phonemic awareness, sound blending, segmenting). All oral language activities were audiotaped and transcribed for data analysis. Further details of the assessment development, the items, and the psychometric properties of the test may be found in a technical report of the test (Kyriakides & Kelly, 2003).

In May 1999, a sample of Cypriot students who were expected to attend primary school in the next school year (average age = 5.32 [decimal years]) was assessed. The stratified technique was used to select 92 out of the 223 public Cypriot pre-primary schools. The schools were divided into four groups. The variables taken into account for dividing the schools into these groups were the place of the school (rural or urban) and the size of the school (small schools with fewer than three teachers or big schools). The choice of the school sample in each group was random. The proportion of rural schools in the sample was the same as the proportion of rural schools in the population. Moreover, the percentage of small schools in the sample was the same as the percentage of the small primary schools in the population. All of the teachers of the school sample (n = 132) were asked to administer the test only to those students who were expected to attend year one during the next school year (n = 788). In this sense, this study is different from many other baseline studies in which students have been assessed during the second term of their first year at primary school (Blatchford & Cline, 1992). Since Cypriot pre-primary classrooms are relatively small (average size = 17) and consist of different age groups of children, each teacher administered the test, on average, to seven students.

For reasons mentioned previously, ELBA was designed to be administered by classroom teachers. To address training needs and facilitate standardization of test administration, a seminar was held in early April 2000, when topics related to the importance and the purposes of baseline assessment in general, and to the importance of the project in particular, were presented. In addition, the literacy components of ELBA and the standards for administration of the assessment were covered. Participation in the seminar and test administration was entirely voluntary. All teachers in our sample attended the seminar (n = 132), and they subsequently administered the test to their students entering primary school. Teachers were asked to observe how each student responded to the tasks and to provide feedback based on specific criteria for the structured observations that they completed. Flexible interviews were used for the purpose of gaining insight into each child's thoughts (Ginsburg, Jacobs, & Lopez, 1998). Each teacher was given standardized instructions for administering the test and was invited to contact the researchers if she! he encountered any problems.

Methods

In early June, after the administration of ELBA, teachers were asked to complete a self-assessment questionnaire, which explored the following issues: 1) the impact of ELBA upon teachers' perceptions of baseline assessment; 2) teachers' attitudes about the appropriateness of test activities; 3) teachers' attitudes about the quality of the data collected through the test; 4) teachers' understanding and application of key relations among outcomes, activities, and evaluative criteria; 5) teachers' use of test results; and 6) the effect of the ELBA project on his/her teaching and assessment practice. Of the 132 teachers approached, 124 responded--a response rate of 94 percent. The response rate implies that the findings are generalizable to the group of teachers who administered the ELBA test. The reliability of the questionnaire findings was measured by calculating the values of Cronbach Alpha for the four scales used to measure teachers' attitudes towards the ELBA test. The values of Cronbach Alpha were higher than 0.85, implying that the reliability of the measures used was satisfactory (Cronbach, 1990).

Two weeks after the teacher's self-assessment, a group interview was conducted with 34 randomly selected teachers (out of the 132 teachers who administered the ELBA test). A group interview, rather than individual semi-structured interviews, allowed the teachers to express their views about the project's impact and to interact with each other (Robson, 1998). The group interview included an initial conversation about the impact of ELBA, specific queries about the effects of administering ELBA on each respondent's day-to-day classroom instructional and assessment practices, and reflection on the factors mediating those effects. During the group interview, the researchers guided the discussion by posing clarifying questions and recorded the teachers' main points regarding the impact of ELBA.

The constant comparative method (Maykut & Morehouse, 1994) was used in order to conduct "within case analysis" (Denzin & Lincoln, 1998) of each teacher's attitudes towards the impact of ELBA. Transcripts were read with the intent of identifying integrating themes, foci, frequently used metaphors, and possible incongruities. The data were summarized through the identification of key descriptors of the various ways ELBA affected the teachers' professional development and through relevant concepts and categories, such as "perceived importance of formative purposes of baseline assessment" and "characteristics of appropriate evaluative criteria." With this method, the researchers tried to examine the suggestions made by each individual during the group interview. Descriptors across interviews became redundant and clear patterns began to emerge, which the researchers used to examine the validity of the questionnaire findings by matching the qualitative data derived from the group interview with the quantitative data gathered by teachers' responses to the questionnaire. A measure of match was derived by comparing relevant parts of the questionnaire with the interview data gathered in this study. Although this measure does not necessarily imply high validity, since it is possible that they are both invalid, the use of both questionnaire and interview methods provides a basis for triangulation of data (Cohen, Manion, & Morrison, 2000).

Results

Impact of ELBA Upon Teachers' Perceptions of Baseline Assessment The literature (Fullan, 1991; Nisbet, 1973) long has supported the view that teachers' perceptions are one of the most critical factors for the effectiveness of introducing any curriculum reform. Calderhead (1987) argues that research into teachers' thinking shows "how unrealistic it is to conceive of innovation as a set of pre-formulated ideas or principles to be implemented by teachers" (p. 17). New forms of assessment are more likely to be implemented if teachers' perceptions are compatible with the underlying philosophy of the assessment reform (Kyriakides, 1997). Thus, the investigation of the impact of ELBA on teachers' perceptions of baseline assessment could contribute to the development of a national policy on baseline assessment in Cyprus.

The figures in Table 1 are based on the information derived from responses to questionnaire items concerned with teachers' perceptions of baseline assessment, and with the extent to which their engagement in the ELBA project influenced their perceptions about baseline assessment. Percentages of teachers agreeing and disagreeing with each item concerning the main aspects of a proposed policy on baseline assessment are shown in columns 3 and 4. Teachers were also asked to indicate whether they had noticed any change in their perceptions about baseline assessment after administering ELBA. Thus, the percentages of teachers who believed that their engagement in ELBA made them consider each item as less important are shown in column 5; whereas those who believed that their engagement in ELBA made them consider each item as more important are shown in the last column. The following observations arise from this table.

First, more than 85 percent of teachers agreed with two of the purposes of baseline assessment--namely, the formative and the early identification of students with special educational needs (SEN). Moreover, the majority of them acknowledged that their engagement in the ELBA project made them realize how important these two purposes are. Second, more than 70 percent of teachers agreed that information gathered from baseline assessment should be used for the development of curriculum policy, for the integration of primary and pre-primary education, and for measuring students' educational progress (items 5, 6, and 10). This implies that Cypriot teachers, as a group, believed that the development of a national policy on baseline assessment may have significant implications for the development and evaluation of education policy. Third, the majority of Cypriot teachers agreed that information gathered from baseline assessment could be used for evaluative reasons (item 3), and more than 3 out of 10 of them mentioned that they had developed more positive attitudes towards this purpose. It cannot be claimed, however, that Cypriot teachers, as a group, considered this purpose important. Fourth, different opinions about the use of baseline assessment for summative reasons have been expressed. Moreover, the figures of column 5 reveal that this is the only opinion for which a number of teachers have developed negative attitudes as a result of their engagement in the ELBA project. Fifth, 9 out of 10 teachers believed that baseline assessment should form a natural part of teaching and that the results of baseline assessment should be kept in students' records. Moreover, almost four out of five teachers believed that results of baseline assessment should be reported to teachers of primary school, and three out of four of them supported the idea that these results also should be reported to parents. Finally, the majority of teachers not only agreed that the Ministry of Education should provide them performance tests on baseline assessment and guidelines on how to conduct baseline assessment, but also acknowledged that their engagement in the ELBA project made them realize how important is for the Ministry of Education to provide this kind of support to pre-primary teachers.

Teachers' Attitudes About the Appropriateness of Test Activities, Evaluative Criteria, and Scoring Guidelines

The figures in Table 2 are based on the information derived from teachers' responses to items of the questionnaire concerned with the appropriateness of each test activity. Teachers rated each item according to a scale of 1 (not appropriate or useful at all) to 7 (very appropriate and useful). Columns 3 and 4 refer to the extent to which each activity provided useful information regarding students' skills in emergent literacy; the columns also show relevant mean values and standard deviations. Means and standard deviations of teachers' responses regarding the appropriateness of the evaluative criteria for interpreting student responses are shown in the next two columns. Finally, the last two columns refer to the mean values and the standard deviations from responses to the items asking teachers to express their views regarding the explicitness of the scoring guidelines of each task.

The following observations arise from Table 2. First, all the mean values of column 3 are very high; the majority of them (8 out of 11) are higher than 6.00. Moreover, the standard deviations are relatively low. This implies that Cypriot teachers, as a group, considered all the activities of the ELBA test useful, and believed that each of them provided rich information about their students' literacy skills. Second, the figures of column five reveal that the mean values showing the extent to which teachers considered appropriate the evaluative criteria given to them range from 5.95 up to 6.21. Finally, 9 out of the 11 means of column seven are higher than 5.80, and the relevant standard deviations are relatively low. Thus, Cypriot teachers considered the scoring guidelines of these tasks to be very helpful. However, scoring guidelines in relation to activities five and nine were not found to be very helpful. This might be due to the fact that these two tasks were used to evaluate students' oral language skills and that the tasks were open-ended. These findings are in line with the fact that, although the inter-rater reliability of each activity was satisfactory, the correlation coefficients that emerged from teachers' scoring of these two activities were not as high as for all the other test tasks (Kyriakides & Kelly, 2003).

Teachers' Attitudes About the Data Collected Through the Test

Teachers were asked to indicate the extent to which ELBA provides information regarding each of the literacy skills that test developers intended to assess through the test. Table 3 shows the means, standard deviations, and percentages of teachers who believed that either no information or only limited information about each of the skills was provided by the test, and percentages of teachers who thought that all the information needed or a lot of information was provided. The following observations arise from Table 3.

First, the mean values of skills concerning print awareness are high, ranging from 3.96 up to 4.16. Moreover, six out of seven of them are higher than 4.00. This implies that ELBA provided rich information to teachers regarding students' print awareness. Second, students' phonological awareness are assessed at a very satisfactory level, since all of the relevant mean values are higher than 4.00. This argument is supported by the fact that less than 10 percent of Cypriot teachers believed that either no information, or only limited information, on students' phonological awareness was provided by ELBA. Third, means concerning oral skills are relatively low, but none of them is lower than 3.50, and three out of eight of them are higher than 3.75. Fourth, although standard deviations of the concepts and behaviors assessed are relatively low, four of them are higher than 1.00, and all of them refer to oral skills. Fifth, only six out of the 19 percentages shown in column 3 of Table 2 are higher than 15 percent. These six percentages refer to teachers' opinions regarding the information about students' oral skills that are provided by ELBA. It therefore can be claimed that the figures in Table 3 reveal that although teachers considered the content validity of ELBA to be satisfactory, a significant percentage of teachers believed that more emphasis should be placed on teacher assessment and analysis of oral skills.

Teachers' Use of Test Results

Teachers were asked to report how they used the results of ELBA. Graph 1 illustrates the percentage of teachers who utilized the test for each of the seven suggested uses. The following observations arise from Graph 1.

[GRAPHIC 1 OMITTED]

First, the great majority of Cypriot teachers (more than 85 percent) supported the idea that students' ELBA scores helped identify which purposes of the literacy program of the pre-primary curriculum each student achieved. More than 80 percent of teachers also believed that ELBA test scores helped them decide how to identify and meet each student's learning needs and how to use their teaching time and their resources in order to meet their needs. This result implies not only that the formative purpose of baseline assessment was considered to be very important (see Table 1), but also that pre-primary teachers attempted to use the information that they gathered for formative reasons. Second, the majority of teachers (53 percent) stated that the ELBA test scores helped them to identify students with learning difficulties. However, only 35 percent of teachers indicated that they reported to primary teachers results of the ELBA test concerning the students who were found to be potentially "at risk" of academic failure. This implies that despite the intention of the research team, ELBA was not used by Cypriot teachers as a screening instrument in the early identification of students with SEN in literacy. Third, a relatively high percentage of teachers (45 percent) concentrated on the over all levels of achievement of their students and used ELBA scores for summative reasons. More specifically, they decided to report the total test scores to primary teachers in order to help them put the grade 1 students in different classes, and with an equal percentage of able and less able students with respect to literacy performance. Fourth, more than 80 percent of them did not discuss with parents the results of their children's test scores. In addition, the great majority of pre-primary teachers (88 percent) did not report the results of their students' performance at the staff meeting in order to use them as sources for evaluating the effectiveness of their school. This implies that despite the fact that the majority of Cypriot teachers agreed with the evaluative purpose of baseline assessment and mentioned that information gathered from baseline assessment should be reported to parents (see Table 1), ELBA did not appear to affect this aspect of their teaching practice.

The Effect of the ELBA Project Upon Teaching Practice

The figures in Table 4 are based on the information derived from teachers' responses to items of the questionnaire concerned with the impact of the ELBA project upon teaching practice. Percentages of teachers agreeing and disagreeing with the ways that the ELBA project influenced their teaching practice are noted in Table 4, as well as means and standard deviations. The following observations arise from Table 4.

First, more than 75 percent of teachers agreed that their engagement in the ELBA project gave them new ideas for teaching and assessing literacy skills. Moreover, their engagement in the ELBA project made them feel more confident in assessing students' literacy skills. Furthermore, the relevant means are higher than 4.00 and their standard deviations are smaller than 1.00. This finding implies that the ELBA project had a significant impact upon teaching and assessment of literacy for the group of teachers who were engaged in administering the ELBA test. Second, more than 70 percent of teachers claimed that their engagement in the ELBA project helped them change their assessment practice not only in literacy, but also in the other curriculum subjects, and that it made them feel confident in assessing their students. Third, more than 60 percent of teachers believed that their expectations and their teaching aims in literacy changed due to their engagement in the ELBA project. Finally, more than 50 percent of the teachers did not identify any impact of their engagement in the ELBA project upon the development of their own school policy. This finding is in line with those mentioned in the previous section, which indicated that their students' responses to the ELBA test were not discussed at the staff meeting, and thereby the project did not have any impact on the development of school policy (Graph 1). Therefore, it can be claimed that the figures in Table 4 reveal that teachers believed that their engagement in the ELBA project had a significant impact upon their own teaching practice, but not on the teaching practice of their colleagues or on general school policies regarding the literacy curriculum.

Analysis of Qualitative Data That Emerged From the Group Interview

A general consensus emerged among the teachers that their engagement in the ELBA project made them more confident in assessing their students. It also was acknowledged that they realized how important it is to conduct systematic assessment of their students' skills. In this context, the majority of teachers suggested that their engagement in the ELBA project was "such a valuable experience that it would be ideal if every pre-primary teacher could have the opportunity to engage in administering the test and then be involved in scoring students' responses" (teacher two). Moreover, five teachers supported the notion that they learned more from scoring students' responses to the ELBA test than from most courses they had taken.

As a consequence, the discussion among the teachers who took part in the ELBA project was then concentrated on the scoring procedure. All of the teachers acknowledged the value of stepping beyond the isolation and idiosyncrasies of their own classroom and seeing the range of possibilities of students' work. As a result, they were "galvanized" by what they saw in scoring to raise expectations, especially in relation to their students' print awareness, because they saw what some students could achieve. Nothing can illustrate better their appreciation of the scoring procedure than their identification of scoring as a "window into classroom practice" (teacher four) and as a "stimulus for change" (teacher ten). Moreover, almost all of them agreed with the opinion expressed by teacher twelve, who claimed that scoring made them more critical and deliberate in their work by inviting them to more carefully scrutinize tasks, students' responses, and the criteria by which responses are evaluated. Furthermore, teacher five claimed that scoring "makes you think," and this point led the teachers to conclude that their engagement in the scoring procedure made them more thoughtful and more focused on determining the goals of their teaching and assessment, and to consider how these goals aligned with the specification table of ELBA.

Their discussion then focused on the actual results that emerged from their students' responses to the ELBA test. It was made explicit that teachers realized how important it is to develop a baseline assessment policy for formative reasons. The great majority of them indicated that they were surprised by the fact that there were "so many differences" among their students' skills in emergent literacy. It was also pointed out that they realized that a relatively high percentage of their students responded correctly to most of the tasks measuring print awareness. For example, teacher twenty mentioned that "since we are not supposed to teach children to correspond letters with their sounds, I did not expect my students to answer correctly the tasks dealing with this skill." Almost all of the teachers who participated in the group interview agreed with this comment. A consensus also emerged that a major consequence of the teachers' engagement in the process of scoring students' responses was that they realized that they had to raise their expectations, especially in relation to print awareness, either because they saw what their students could achieve, or because they realized the danger of failing to expect the most from students. One teacher, concluding that her engagement in ELBA made her "more ruthless, but more liberal," summarized well many teachers' perception that scoring students' responses to the ELBA test led them simultaneously to raise standards and to be more flexible in allowing for different ways to meet these standards. This perception seems to sup port the importance of conducting baseline assessment for formative reasons.

As far as the purpose of baseline assessment concerning early identification of students with SEN, four teachers expressed their concerns about the dangers of labeling 5-year-old children. However, the same teachers did eventually decide that this purpose had a positive aim and could, if implemented sensitively, lead to beneficial outcomes for the children. Moreover, teacher eighteen stated that she did not feel confident "to claim that a student at such an early age is a student with special needs" and suggested that "if the Ministry of Education is interested in early identification of students with special needs, a battery of tests should be developed and educational psychologists should visit the school, and both the teacher and the educational psychologist should make a decision by taking into account the responses of the child in these tests." However, three other teachers claimed that although they believe that teachers should ask for support from professionals, it is the responsibility of teachers to decide whether a child needs a special treatment. It also was supported that "it is useless to identify early a child who needs a special treatment unless we find ways to provide them a special treatment" (teacher nine). This view was supported by all the teachers, and teacher twenty-one reminded everybody that "the impact of this special treatment upon the development of students' skills also should be investigated in order to evaluate the appropriateness of this special treatment, and then the teacher could make decisions on what kind of support should be provided to these children." It therefore can be claimed that teachers' views about the early identification of students with SEN seems to be in line with the argument made in the literature that it is necessary to develop a sequential model of baseline assessment, combining a screening phase for all students entering primary school, followed by an assessment phase for those identifying as "at-risk" on the screening measure (Lindsay & Desforges, 1998).

According to most teachers, the ELBA project did not have an impact upon the development of their own school policy. This was attributed to the fact that each Cypriot teacher is isolated in her own classroom and there is no priority at the school level For any attempt to develop a school-based curriculum policy. More specifically, teacher eleven pointed out that "it is very difficult for us to persuade our colleagues that we could develop a policy on emergent literacy based on the results of ELBA, since neither me nor my colleagues believe that the inspector would be happy to see us take initiatives that we are not supposed to take." The idea that the officers of the Ministry of Education can be the sole definers, arbiters, and guardians of curriculum policy encourages professional dependency, and is the main obstacle for this project to have an impact on changing the management of curriculum improvement at the school level. It also was acknowledged that the engagement of teachers in ELBA did not have an impact upon the establishment of better relations between teachers and parents.

However, the teachers agreed among themselves that their engagement in ELBA had an impact upon their own individual professional practice. Their expectations for changed instruction as a result of their engagement with the ELBA test included, in order of frequency, the following:

* Assigning more tasks dealing with print awareness and reading for a variety of purposes and across different content areas

* Using more well-defined rubrics in classroom assessment

* Incorporating more literacy performance activities in their classrooms and creating hands-on and group activities similar to those included in ELBA

* Encouraging students to explain and elaborate orally.

Teachers indicated that their expectations for changed instruction as a result of their engagement in ELBA were not the same as they were at the initial stage of their involvement in the project. By the end of their engagement, teachers tended to identify ways of reconceptualizing their teaching that were less focused on the test itself and more concerned with global issues (e.g., using print awareness and reading across the curriculum, and helping students to explain their thinking processes in order to identify their learning needs). Thus, there was a recognizable movement away from a narrow concern with teaching to the test toward using what they had learned from their engagement in the ELBA test to inform and improve teaching more generally.

Overall, the group interview confirmed and added emphasis to the questionnaire responses. Although the teachers were not without their criticisms of the test, their engagement in the ELBA project led every teacher to support the idea that ELBA had improved, or had the potential to improve, teaching and learning, largely by encouraging teachers to include more print awareness and phonological tasks of various types and purposes, and by raising expectations of student achievement.

Discussion

The evidence presented above can be discussed in terms of its implications for the establishment of a systematic way to improve teachers' professional skills in assessment by asking them to take part in large-scale assessment projects. Black and Wiliam (1998) argue that, in England, "the sum of all policy initiatives doesn't add up to an effective policy because something is missing." It can be claimed that the "something" missing is any direct help (or professional development) for teachers to achieve better assessment practices within classrooms.

Although the last decade has witnessed a growing recognition of the need for significant changes in educational assessment practices (Shepard, 1989), research on test use in the schools has been sporadic and limited in scope. The existing research that specifically addresses the impact of national assessment programs focuses on two major categories of effects: 1) the narrowing of the curriculum and the loss of local input into curriculum planning and instructional design, and 2) the diminishing morale of teachers and students (Bond & Cohen, 1991). Clearly, studies investigating the impact of large-scale tests upon teachers' professional development should be conducted.

The present study has shown that teachers who were engaged in the administration of ELBA reported a favorable impact on their instruction and on their perceptions of baseline assessment. It therefore can be assumed that the involvement of teachers in large-scale assessment may support the kinds of changes in instructional practice that are necessary to meet new performance standards. It is important to note, however, that the findings of this study are based primarily on teacher self-reports, and that even though both quantitative and qualitative methods were used to collect data about attitudes towards engagement in the ELBA project, the validity of the data can be questioned (Cohen, Manion, & Morrison, 2000). This limitation indicates that further research is needed to test the validity of this study. Examples of further study might include direct systematic observation of assessment practices, which could be triangulated with questionnaire and interview data. Further study also is needed to identify the extent to which the effects of participating in the ELBA study are dependent on teachers' current professional developmental level. In this study, such a research question could not be answered, since none of the Cypriot pre-primary teachers were familiar with the use of any baseline assessment procedures, nor with the use of rubrics and performance assessment. However, a follow-up study is currently being conducted that investigates the impact of the ELBA study upon a group of teachers who had been already involved in a project on baseline assessment in mathematics, and upon a group of teachers who had no experience using rubrics and performance assessment. This study may help us examine whether teachers who started with an approach that shared a common basis with the content of a large-scale assessment may get fewer benefits from their engagement in this type of assessment.

Even if we are not in a position to claim that the ELBA project may have an equally significant impact on the professional development of teachers with much more experience in using rubrics and conducting performance assessment, it is important to acknowledge the fact that those individual teachers who took part in the ELBA project clearly indicated that this project had a significant effect upon their professional development. More specifically, the teachers stated that their participation in the scoring procedure was a valuable experience and that they were surprised by the range of possibilities of their students' work. This outcome led them to raise their expectations, especially in relation to their students' print awareness, which is an area not given much attention in the public pre-primary schools of Cyprus. It can be argued that teachers' engagement in ELBA, and especially in scoring their students' work, deepened their understanding of standards, offered new perspectives of what their students know and can do, and may serve as a powerful catalyst for raising education standards. But what is probably much more important is the fact that teachers accepted the idea that their engagement in the scoring procedure made them more thoughtful and more focused on determining the goals of their teaching and assessment. It can be concluded that engagement in large-scale assessment projects can serve to make teachers more reflective, critical, and deliberate--all characteristics of effective teaching (Helsby, 1995).

As far as the impact of ELBA upon teachers' own individual professional practice, it was reported that their teaching and assessment practice was significantly influenced by their engagement in the ELBA project. For example, it was evident that the teachers began assigning more tasks dealing with print awareness and reading for a variety of purposes and across different content areas. It is also important to note that teachers acknowledged in the group interview that by the end of their engagement, they tended to identify ways of reconceptualizing their teaching that were less focused on the test itself. This implies that there was a recognizable movement from teachers showing a narrow concern with teaching to the test to using what they had learned from their engagement in ELBA to inform and improve teaching more generally. Although this shift could be viewed as the most important finding of this study, as long as professional development resources are directed primarily toward helping teachers align their instruction with particular large-scale tests, we may expect to see teachers only mimic the superficial characteristics of the tasks of the test. When valuable staff development is allocated to task development, without guided support in ensuring that tasks embody features of effective performance-based instruction and classroom assessment, we may see teachers change their practice by following a narrow teaching-to-the-test approach. The findings of this study revealed that it is possible to build teacher capacity in assessment through their systematic engagement in assessment reform programs. However, further research is needed in order to identify the type of support that teachers need in order to apply what they learn from their experience with large-scale tests to improve their teaching and assessment practice. Although engagement in large-scale assessment often challenges and energizes teachers, it does not necessarily provide them with a comprehensive and well-grounded understanding of effective assessment practice and performance-based instruction. This implies that researchers must shift their attention from identifying shortcomings in teachers' applications of what they learned through their engagement in large-scale tests to questions of precisely how, and how well, teachers apply what they learned from that experience.

Table 1

Percentages of teachers who agreed or disagreed with each aspect of
baseline assessment policy, and percentages of teachers who indicated
that their engagement in the ELBA project made them consider each item
as less important or more important

No.   Attitudes towards baseline assessment          teachers who
                                                 disagree *    agree **

1.    Information gathered from baseline            02.3         90.9
        assessment should be used for
        formative reasons.

2.    Information gathered from baseline            03.0         85.8
        assessment should be used to identify
        individual children with special
        education needs.

3.    Information gathered from baseline            15.3         59.1
        assessment should contribute to the
        evaluation of pre-primary education.

4.    Information gathered from baseline            33.3         53.0
        assessment should be used for
        summative reasons.

5.    Information gathered from baseline            07.6         75.8
        assessment should contribute to the
        development of curriculum policy.

6.    Information gathered from baseline            12.9         71.4
        assessment should be used for
        measuring students' education
        progress.

7.    It is essential to keep records of the        01.5         89.1
        results of baseline assessment.

8.    Baseline assessment should form a             02.3         91.1
        natural part of teaching activities.

9.    Results of baseline assessment should
        be reported to:
      a) parents                                    11.7         74.2
      b) teachers of primary school                 09.9         82.5

10.   The development of a national policy on       08.3         71.9
        baseline assessment will contribute
        to the integration of primary and
        pre-primary education.

11.   The Ministry of Education should              05.3         78.1
        provide pre-primary teachers with
        performance tests in order to conduct
        baseline assessment.

12.   The Ministry of Education should              02.3         74.2
        publish non-statutory guidelines on
        baseline assessment.

No.   Attitudes towards baseline assessment       teachers who consider
                                                    less        more
                                                    imp (+)    imp (++)
1.    Information gathered from baseline            00.0         59.8
        assessment should be used for
        formative reasons.

2.    Information gathered from baseline            00.0         48.5
        assessment should be used to identify
        individual children with special
        education needs.

3.    Information gathered from baseline            00.0         33.2
        assessment should contribute to the
        evaluation of pre-primary education.

4.    Information gathered from baseline            10.6         23.5
        assessment should be used for
        summative reasons.

5.    Information gathered from baseline            00.0         50.8
        assessment should contribute to the
        development of curriculum policy.

6.    Information gathered from baseline            00.0         47.0
        assessment should be used for
        measuring students' education
        progress.

7.    It is essential to keep records of the        00.0         58.3
        results of baseline assessment.

8.    Baseline assessment should form a             00.0         64.3
        natural part of teaching activities.

9.    Results of baseline assessment should
        be reported to:
      a) parents                                    00.0         40.1
      b) teachers of primary school                 00.0         42.1

10.   The development of a national policy on       00.0         44.7
        baseline assessment will contribute
        to the integration of primary and
        pre-primary education.

11.   The Ministry of Education should              00.0         60.6
        provide pre-primary teachers with
        performance tests in order to conduct
        baseline assessment.

12.   The Ministry of Education should              00.0         58.3
        publish non-statutory guidelines on
        baseline assessment.

* This group of teachers either disagreed or absolutely disagreed.

** This group of teachers either agreed or absolutely agreed.

(+) This group of teachers stated that their engagement in the ELBA
project made them consider each item as less important.

(++) This group of teachers stated that their engagement in ELBA made
them consider each item as more important.

Rating scales:

A) 1: I absolutely agree; 2: I disagree; 3: I do not know/I cannot say;
4: I agree; 5: I absolutely agree

B) 1: Less important; 2: The same as before; 3: More important


Table 2
Means and standard deviations according to teachers' attitudes
about the usefulness of information provided by each task, the
appropriateness of evaluative criteria used to interpret students'
responses to each task, and the explicitness of scoring guidelines
given for each task.

No.   Test Activities                  Usefulness      Evaluative
                                         of data        criteria
                                      Mean    S.D.    Mean    S.D.

1.    "Self Portrait"                 5.88    0.93    6.12    0.76
2.    "Shared Book Reading"           6.21    0.72    5.95    0.89
3.    "Read the Sign"                 6.38    0.53    6.19    0.72
4.    "Family Portrait"               6.15    0.66    6.02    0.76
5.    "Storyboard Activity"           6.49    0.60    6.01    0.76
6.    "Mary's Walk"                   6.05    0.93    6.21    0.70
7.    "First Sound Bingo"             5.95    0.96    6.02    0.82
8.    "Alphabet Activity Cards"       6.01    0.89    6.07    0.76
9.    "Storytelling"                  6.45    0.83    6.14    0.86
10.   A) Phonological activities      6.15    0.93    6.05    0.83
      B) Drawing geometrical shapes   5.99    1.04    6.01    0.75

No.   Test Activities                   Scoring
                                      Mean    S.D.

1.    "Self Portrait"                 5.98    0.73
2.    "Shared Book Reading"           5.81    0.92
3.    "Read the Sign"                 6.10    0.63
4.    "Family Portrait"               6.05    0.76
5.    "Storyboard Activity"           5.39    1.21
6.    "Mary's Walk"                   6.05    0.83
7.    "First Sound Bingo"             6.01    0.76
8.    "Alphabet Activity Cards"       6.03    0.79
9.    "Storytelling"                  5.25    1.33
10.   A) Phonological activities      6.05    0.83
      B) Drawing geometrical shapes   6.19    0.74

Rating Scales 1-7

A) 1 = No useful information was provided ... 7 = All the useful
information I needed

B) 1 = The evaluative criteria were not appropriate at all ... 7 = The
evaluative criteria were very appropriate

C) 1 = Scoring guidelines were not helpful at all ... 7 = Scoring
guidelines were very helpful


Table 3
Means and standard deviations of the extent to which information
regarding each literacy skill was provided by the ELBA test, the
percentage of teachers who believed that not enough information about
each skill was provided, and those who believed that a lot of
information was provided.

No.   Literacy concepts and behaviors   % of teachers who   Mean   S.D.
      assessed by the ELBA test           not        a
                                        enough *   lot **

      PRINT AWARENESS
1.    Book conventions                    09.1      81.8    4.16   0.81
        (left-to-right,
        beginning-end, top to bottom)
2.    Graphic symbol awareness            07.6      77.3    4.02   0.90
        (letter, word, sentence)
3.    Letter identification               09.1      78.8    4.04   0.78
        (same-different)
4.    Writing (scribbling, drawing,       10.6      80.3    4.08   0.95
        inventive spelling, name)
5.    Letter-sound correspondence         07.6      81.8    4.26   0.86
6.    Visual discrimination               13.6      74.2    3.96   0.98
        (matching, same-different)
7.    Environmental print                 12.1      75.8    4.03   0.99
        (first name, common place
        names)

      PHONOLOGICAL AWARENESS
1.    Auditory perception                 09.1      75.8    4.06   0.92
        (environmental)
2.    Auditory memory (words,             07.6      81.8    4.14   0.85
        phrases, phonemes)
3.    Auditory discrimination and         06.1      83.3    4.12   0.81
        phonemic awareness (rhyming,
        blending)
4.    Auditory comprehension (word        07.6      85.8    4.15   0.89
        classification,
        categorization)

      ORAL LANGUAGE
1.    Complete sentences                  13.6      66.7    3.82   0.92
2.    Descriptive vocabulary              18.2      63.6    3.59   1.05
3.    Questioning strategies              22.7      65.2    3.55   1.12
4.    Conversational skills               24.2      60.6    3.53   1.05
        (pragmatics, discourse)
5.    Narration of fictional story        16.7      65.2    3.67    0.98
        (storytelling)
6.    Categorical organization and        15.1      63.6    3.61    0.99
        sequencing
7.    Use of temporal markers             21.2      65.2    3.76    1.09
8.    Imaginative skills                  13.6      75.8    3.92    0.91
        (creativity, expressiveness)

* This group of teachers believed that either no or only limited
information about students' skills was provided.

** This group of teachers believed that either a lot or all the
information needed was provided.

Rating scale: 1: Not at all; 2: Limited; 3: Neither few nor many;
4: A lot; 5: All the information needed


Table 4
Percentages of Cypriot teachers who agreed and those who disagreed with
the following items concerning the impact of the ELBA project upon
teaching practice, and their means and standard deviations.

No.   Impact of engagement          % of teachers who       Mean   S.D.
      in the ELBA project upon    Disagreed *    Agreed**
      teaching practice

1.    Due to my engagement in         16.6         68.8     3.62   1.05
        the ELBA project, I
        redefined my
        expectations for my
        students in literacy.

2.    My engagement in the            21.2         61.2     3.57   1.08
        ELBA project made me
        change my teaching
        aims in literacy.

3.    My engagement in the            12.1         75.8     4.06   0.89
        ELBA project gave me
        ideas about using new
        teaching activities
        for language
        development purposes.

4.    My engagement in the            15.1         72.8     4.01   0.95
        ELBA project helped me
        change my assessment
        practice in literacy.

5.    My engagement in the            28.8         56.1     3.52   1.16
        ELBA project helped me
        change my assessment
        practice generally.

6.    My engagement in the            09.1         78.8     4.05   0.92
        ELBA project made me
        feel more confident in
        assessing students'
        skills in literacy.

7.    My engagement in the            19.7         71.2     3.82   1.01
        ELBA project made me
        feel more confident in
        assessing students'
        skills in any field.

8.    Due to my engagement in         37.9         43.9     3.28   0.94
        the ELBA project, our
        school changed its
        policy in teaching
        literacy.

9.    Due to my engagement in         30.3         54.6     3.59   0.96
        the ELBA project, our
        school changed its
        assessment policy in
        literacy.

10.   Due to my engagement in         33.3         49.9     3.31   1.04
        the ELBA project, our
        school changed its
        assessment policy.

This group of teachers either disagreed or absolutely disagreed.

** This group of teachers either agreed or absolutely agreed.

Rating scale

1: I absolutely disagree; 2: I disagree; 3: I do not know/I cannot say;
4: I agree; 5: I absolutely agree

References

Adams, M.J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press.

Association for Childhood Education International/Perrone, V. (1991). Standardized testing. An ACEI position paper. Childhood Education, 67, 131-142.

Bagnato, S. J., & Neisworth, J. T. (1999). Collaboration and teamwork in assessment for early intervention. Child and Adolescent Psychiatric Clinics of North America, 8(2), 347-363.

Bishop, D., &Adams, C. (1990). A prospective study of the relationship between specific language impairment, phonological disorders and reading retardation. Journal of Child Psychology and Psychiatry, 31(7), 1027-1050.

Blachman, B.A. (1991). Early intervention for children's reading problems: Clinical applications of research in phonological awareness. Topics in Language Disorders, 12, 51-65.

Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. London: King's College London School of Education.

Blatchford, P., & Cline, T. (1992). Baseline assessment for school entrants. Research Papers in Education, 7(3), 247-269.

Bond, L.A. & Cohen, D.A. (1991). The early impact of Indiana statewide testing for educational progress on local education agencies: Administrators' perceptions. In R. Stake & G. O'Sullivan (Eds.), Advances in program evaluation: Vol. 1B. Effects of mandated assessment on teaching (pp. 75-100). Greenwich, CT: JAI Press.

Calderhead, J. (Ed.). (1987). Exploring teachers' thinking. London: Cassell.

Catts, H. W. (1989). Phonological processing deficits and reading disabilities. In A. Kamhi & H W. Catts (Eds.), Reading disabilities: A developmental language perspective (pp. 101-132). Needham Heights, MA: Allyn & Bacon.

Catts, H. W. (1997). The early identification of language-based impairments and reading disabilities. Language, Speech and Hearing Services in the Schools, 28, 86-89.

Catts, H. W., Fey M. E., Zhang, X., & Tomblin, J. B. (1999). Language basis of reading and reading disabilities: Evidence from a longitudinal investigation. Scientific Studies of Reading, 3, 331-361.

Catts, H., Fey, M. E., Zhang, X., & Tomblin, J. B. (2001). Estimating the risk of future reading difficulties in kindergarten children: A research-based model and its clinical implementation. Language, Speech, and Hearing Services in the Schools, 32(1), 38-50.

Cheng, Y. C. (1996). Relation between teachers' professionalism and job attitudes, educational outcomes, and organizational factors. Journal of Educational Research, 89(3), 163-171.

Cizek, G. J. (2001). More unintended consequences of high-stakes testing. Educational Measurement: Issues and Practice, 20(4), 19-27.

Cohen, D., Manion, L., & Morrison, K. (2000). Research methods in education (5th ed.). London: Routledge/Falmer.

Cronbach, L. J. (1990). Essentials of psychological testing (3rd ed.). New York: Harper & Row.

Denzin, N. K., & Lincoln, Y. S. (Eds.). (1998). Collecting and interpreting qualitative materials. Thousand Oaks, CA: Sage.

Department for Education and Employment. (1995). Value added in education: A briefing paper from the Department for Education. London: Author.

Department for Education and Employment. (1999). The national curriculum for England. London: Author.

Falk, B., & Ort, S. (1998). Sitting down to score: Teacher learning through assessment. Phi Delta Kappan, 80(1), 59-64.

Felton, R. H., & Pepper, P. P. (1995). Early identification and intervention of phonological deficits in kindergarten and early elementary children at risk for reading disability. School Psychology Review, 24, 405-414.

Firestone, W., Mayrowetz, D., & Fairman, J. (1998). Performance-based assessment and instructional change: The effects of testing in Maine and Maryland. Educational Evaluation and Policy Analysis, 20, 95-113.

Fitz-Gibbon, C. T. (1995). The value added national project: General report. London: School Curriculum and Assessment Authority.

Flockton, L., & Crooks, T. (1997). Social studies assessment results 1997: National education monitoring report 8. Dunedin, New Zealand: University of Otago, Educational Assessment Research Unit.

Fullan, M. G. (1991). The new meaning of educational change. London: Cassell.

Garet, M., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-946.

Gilmore, A. (2001, May). Bringing national assessment into the classroom: Learning opportunities and challenges for the national education monitoring project in New Zealand. Paper presented at the 27th IAEA conference, Brazil.

Ginsburg, H. P., Jacobs, S. F., & Lopez, L. S. (1998). The teacher's guide to flexible interviewing in the classroom: Learning what children know about mathematics. Boston: Allyn and Bacon.

Goldberg, G. L., & Rowsell, B. S. (2000). From perception to practice: The impact of teachers' scoring experience on performance based instruction and classroom assessment. Educational Assessment, 6(4), 257-290.

Harste, J., Woodward, V., & Burke, C. (1984). Language stories and literacy lessons. Portsmouth, NH: Heinemann.

Helsby, G. (1995). Teachers' construction of professionalism in England in the 1990s. Journal of Education for Teaching, 21(3), 317-332.

Holdaway, D. (1979). The foundations of literacy. Sydney, Australia: Ashton Scholastic.

Huberman, M. (1990). Research on teachers' professional lives. International Journal of Educational Research, 13, 34-46.

Juel, C. (1988). Learning to read and write: A longitudinal study of fifty-four children from first through fourth grade. Journal of Educational Psychology, 80, 437-447.

Kane, M. T. (2001). Current concerns in validity theory. Journal of Educational Measurement, 38(4), 319-342.

Kelly, K. (2000). Developmentally appropriate assessment of emergent literacy in kindergarten. In A. Gagatsis, C. Constantinou, & L. Kyriakides (Eds.), Learning and assessment in mathematics and science (pp. 377-390). Nicosia, Cyprus: Department of Education, University of Cyprus.

Kifer, E. (2001). Large-scale assessment: Dimensions, dilemmas and policy. Thousand Oaks, CA: Corwin Press.

Kyriakides, L. (1997). Influences on primary teachers' practice: Some problems for curriculum change theory. British Educational Research Journal, 23(1), 39-46.

Kyriakides, L. (1998). Professional influences on teachers' perceptions of teaching and assessment in Mathematics. Scientia Paedagogica Experimentalis, 35(1), 263-276.

Kyriakides, L. (1999). Research on baseline assessment in mathematics at school entry. Assessment in Education: Principles, Policy and Practice, 6(3), 357-375.

Kyriakides, L., & Kelly, K. (2003). Score validity of the ELBA test. Nicosia, Cyprus: University of Cyprus.

Lindsay, G., & Desforges, M. (1998). Baseline assessment: Practice, problems and possibilities. London: David Fulton Publishers.

Maykut, P., & Morehouse, R. (1994). Beginning qualitative research: A philosophical and practical guide. London: The Falmer Press.

Ministry of Education. (1994). The new curriculum. Nicosia, Cyprus: Author.

Morrow, L. (1993). Literacy development in the early years (2nd ed.). Boston: Allyn & Bacon.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of scientific research literature on reading and its implications for reading instruction [Online]. Available www.nichd.nih.gov/publications/nrp/smallbook.htm

Nisbet, J. (1973). The school council. Case studies of educational innovations: 1. At the central level. Paris: Centre for Educational Research and Innovation/Organisation for Economic Cooperation and Development.

Phelps, R. P. (2000). Trends in Large-Scale Testing outside the United States. Educational Measurement: Issues and Practice, 19 (1), 1121.

Popham, W. J. (1999). Where large scale educational assessment is heading and why it shouldn't. Educational Measurement: Issues and Practice, 18(3), 13-17.

Postlethwaite, T. N. (1994). Monitoring and evaluation in different education systems. In A. C. & T. N. Postlethwaite (Eds.), Monitoring the standards of education (pp. 23-46). Oxford: Pergamon.

Robson, C. (1998). Real world research. Oxford: Blackwell.

Sebatane, E. M. (1998). Assessment and classroom learning: a response to Black and Wiliam. Assessment in Education, 5(1), 123-130.

Sheingold, K., Heller, J., & Storms, B. (1997). On the mutual influence of teachers' professional development and assessment quality in curricular reform. Paper presented at the annual meeting of American Educational Research Association, Chicago.

Shepard, L.A. (1989). Why we need better assessment. Educational Leadership, 46(2), 4-8.

Snow, C., Burns, M., & Griffin, P. (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

Stake, R. (Ed.). (1991). Advances in program evaluation: Vol. 1A. Effects of mandated assessment on teaching. Greenwich, CT: JAI Press.

Stanovich, K., Cunningham, A., & Cramer, B. (1984). Assessing phonological awareness in kindergarten children: Issues of task comparability. Journal of Experimental Child Psychology, 38, 175-180.

Strickland, D., & Morrow, S. (Eds.). (1989). Emerging literacy: Young children learn to read and write. Newark, DE: International Reading Association.

Whitehurst, G., & Lonigan, C. (1998). Child development and emergent literacy. Child Development, 69(3), 848-872.

L. Kyriakides University of Cyprus

K. L. Kelly Colorado Department of Education