Examining the Reflective Outcomes of Asynchronous Computer-Mediated Communication on Inservice Teacher Development.

This study explored the professional development experiences of 28 practicing teachers in 10 Chicago suburban schools involved in a two-year technology supported Problem-Based Learning curriculum development effort. Asynchronous computer-mediated communications (CMC) were featured as teacher communication tools of the project. The computer-mediated discourse produced by the teachers was compared with the discourse produced by teachers in face-to-face meetings. Research methods including discourse analysis and archival data analysis were applied to determine the nature of the teacher discourse and its reflective content. The results show that while the computer-mediated teacher dialogue was less interactive, it was significantly more reflective (t=4.14, p=.001) than face-to-face discourse. The study findings suggest that the value of CMC lies in its ability to facilitate professional collaboration between teachers and encourage critical reflection on educational policy and practice.

Computer-mediated communication (CMC) presents teachers with new opportunities for communication. Though the use of CMC suggests more convenient access to professional colleagues, it does not ensure professional growth and learning experiences. The purpose of this study was to determine if and how critical reflection--as a meaningful professional development objective--arises from computer-mediated collaborative dialogue. This research begins by examining the role of collaboration and dialogue in teacher learning. Applications of CMC for teacher development are reviewed followed by a description of the study context. The results of the research are presented followed by a discussion of network technology's capacity for facilitating new conceptions of inservice teacher development, and engaging professionals in an analysis of practice that is both contextually relevant and informed by the experiences of peers.

TEACHER PROFESSIONAL DEVELOPMENT

Calls for educational reform in our nation's schools are driven by the desire to achieve high standards in student performance. These standards are aimed at preparing students for a technology-dominated work place where teamwork, complex and meaningful problem solving, and critical thinking are necessary skills (McKinsey & Co., 1995; The Secretary's Commission on Necessary Skills, 1991). Many reform efforts targeting these outcomes, however, have been unable to produce the kind of learning demanded by reformers. Reaching school improvement objectives initiated by the school reform movement, Darling-Hammond (1998) indicated, is "likely to be dependent on dramatically increased support for teachers and learning" (p. 6). Failures of past reform efforts, Darling-Hammond continued, has been due to "the lack of sustained, serious, systemic investments in the knowledge base of individual educators" (p. 11).

Professional development for teachers constitutes formal and informal processes of knowledge and skill building. Types of professional development over the past several decades have included the pursuit of advanced degrees, school- and district wide meetings, conferences and workshops, and personal study on selected topics. Though it had taken a permanent place in school culture, professional development lacked, for the most part, the ability to improve student learning and teacher practice (Rallis, Ross-man, Phelgar, & Abeille, 1995). Lieberman (1995) characterized such professional development activities as nothing more than the delivery of an assortment of relatively abstract ideas providing little support to the practice of continuous learning. Little (1993) added that development activities were episodic and training oriented, placing teachers in passive roles as consumers of knowledge produced elsewhere.

Achieving a clearer vision of improvement in the current of school reform requires an expanded recognition of teacher development needs. Teachers should have opportunities to examine their beliefs about teaching and learning and to construct their own knowledge in a supportive environment that encourages risk taking and reflection. Development activities of this type are necessary for teachers to meet the increasingly complex school roles they are asked to play (Borko & Putnam, 1995).

In response to the call to improve the quality of inservice teacher development, the National Staff Development Council (NSDC) compiled a set of development guidelines from a comprehensive study of the lessons learned about staff development over the past 20 years. The guidelines contain ideas for involving teachers in decisions about student learning, teacher research, and school-shared decision-making. The most emphasized component of the NSDC guidelines is the need to provide opportunities for teachers to collaborate with peers to make sense of the teaching and learning process (Sparks, 1994).

COLLABORATION IN TEACHING

Collaboration is generally described as a process of willing cooperation with peers and colleagues to reach educational objectives. In schools, however, teachers often work more in isolation from--than in collaboration with--each other. In a study of teachers' collegial relations, Rosenholtz (1988), using case study methods and repeated measures, arrived at some conclusions about the effects on teachers working in isolation. In interviews with 55 teachers from schools classified as having isolating characteristics, Rosenholtz found that collaboration included little if any sharing of existing materials and ideas; that planning and problem solving with colleagues rarely happened at all; and that teachers prefered to keep discipline problems to themselves.

There are few reasons why collaboration should not be the teaching norm in contrast to frequently disabling isolation. Collaboration can have high situational relevance like, for example, when shared knowledge about a single student's needs, abilities, and progress results in a more individualized approach to that student's development. Mobilizing a critical mass of voices to cause change to happen quickly and uniformly in a school or district is also viewed as an advantage of local collaborations (Darling-Hammond, 1996). A second set of outcomes attributed to collaborations and group work is not so confined to the locality of participation. Outcomes of collaborations, whether on a local or distant network, might include the development of stronger teacher voices to represent their perspectives (Jervis, 1996); the emergence of new teaching roles and leadership opportunities (Hammerman, 1997); the development of general problem-solving skills (Damon, 1984); and the motivational value of being a member of a he althy group (Rogers, 1970). These outcomes are a process of reframing and redescribing the everyday activities of teaching in ways that promote new insights. Lord (1994) describes this process as one of critical colleagueship where an attempt at "productive disequilibrium" is made though informed debate, honest disagreement, and constructive conflict. A similar process labeled "critical dissonance" by Cochran-Smith (1991) is described as "serious talk about teaching and reforming teaching in the large collaborative community" (p. 305).

Dialogue Between Teachers

An important element of collaboration is conversation. Giving utterance to an idea or summarizing a thought forces a cohesive explanation of interrelating ideas. The process of articulating ideas, beliefs, or reservations, as Koschman, Kelson, Feltovich and Barrows (1997) suggest, enhances retention, clarifies one's position on an issue, forces the learner to take a stand on an issue in the presence of peers, and commits the conversant to evaluate and assess that knowledge in light of new information (p. 93).

The importance of collaborative conversation is illustrated by Feldman (1997), who studied the outcomes of teacher discourse of a small group of high school physics teachers in northern California. Feldman depicted collaboration among these teachers, who met consistently over a period of three years, as taking the form of "long and serious conversations" (p. 5). Conversations of the type Feldman describes run a spectrum from anecdote telling to the trying out of ideas to systematic inquiry all of which lead to the effect of "enhanced normal practice" (p. 18). Feldman suggested that teachers engage in a process of critical inquiry and arrive at a shared understanding of their teaching as they talk, listen, reflect, and respond to each other.

Research by Feldman and others (Hollingsworth, 1994; Cochran-Smith & Lytle, 1993) illustrated the importance of dialogue in the development of teacher communities. Collaborative conversation encourages relational knowledge that links what teachers learn and understand about their practice to other conditions that impact student learning such as family influences and the educational setting. Perhaps most important, collaborative settings are the likely contexts in which critically reflective exchanges about learning and instruction can take place between teachers.

COLLABORATIVE REFLECTION

Because reflection has generated a good deal of interest among teacher education reformers, theorists, and researchers, a number of meanings of the term have emerged. Rather than consensus, these ideas have generated very different and even opposing ways of thinking about reflection in teacher development (Placier, 1996). Unfortunately, the lack of agreement of what is meant by reflection puts the idea on the brink of becoming a mere slogan for educational reform (Hatton & Smith, 1995; Zeichner, 1996).

According to John Dewey (1910), reflection is the "active, persistent, and careful consideration of any belief or supposed form of knowledge in light of the grounds that support it and the further conclusions to which it tends" (p. 6). Dewey's rationale for reflection is simple: to examine facts in such a way as to respond to the question at hand. Reflection, Dewey said is a three-fold process of problem definition, means-end analysis, and generalization.

While Dewey's and others' (Hullfish & Smith, 1961; Wildman, Niles, Migliaro, & McLaughlin, 1990) interpretations of reflection are categorized as a means for learning, this research applies Schon's (1987, 1991) depiction of reflection as a social-professional activity in which teachers adapt knowledge to specific situations. As the very essence of professional activity, reflection is a continual process that engages teachers in framing and re-framing problems while designing and evaluating solutions. In describing the role of reflection in developing new knowledge and understanding among teachers, Feldman (1997) illustrated how reflection is a collaborative activity:

They [teachers] begin with a cooperative process in which one of the teachers starts to talk and the others listen. As they listen, they think about what is being said and relate it to their own histories, their intentions, and their relations to others. Reflection occurs, and the ones who have listened, respond. The responses are answers to questions, related anecdotes or bits of narrative, or questions, which act in the evolution of the conversants' direction. (p. 11)

While reflection is expected to lead to outcomes addressing the day-today issues of teaching, many also regard it as a strategy for attempting to understand "the role that schools actually play within a race, class, and gender divided society" (McLaren, 1989, p. 163). When teachers examine the issues of ethics, morals, and justice in education, they are opening up discourse about the role of schools in a democratic society (Sparks-Langer & Colton, 1991). This kind of discourse frames reflection in terms of critical theory where teachers raise questions about such things as student assessment, ability tracking, and classroom management in an effort to address issues of equity and power.

Newer visions of professional development emphasize critical reflection on teaching practice through collaboration and collegial dialogue. Research on approaches bearing these qualities indicate that by using them, teachers are better able to make and sustain improved instructional practices with greater consistency than when attempting to make these improvements alone or when supported by traditional professional development approaches (Corcoran, 1995; Darling-Hammond, 1996; Lichtenstein, McLaughlin, & Knudsen, 1992; Lieberman & McLaughlin, 1993). Unfortunately, the research also indicates that due to time, cost, and lack of will and vision, opportunities to engage in professional development experiences that are collaborative, collegial, and reflective are limited (Lichtenstein, McLaughlin, & Knudsen, 1992; Little, 1993, Lieberman, 1995).

COMPUTER-MEDIATED REFLECTIVE DIALOGUE

CMC facilitates person-to-person or person-to-group contact by means of computer networks. Examples of CMC include electronic mail, listservs, threaded forums, electronic bulletin boards, network videoconferencing, conferencing software, and multi-user domains (Romiszowski & Mason, 1994, p. 439). CMC can be synchronous, where participants interact in real time, or asynchronous, where communication turnaround may be delayed for hours or days. In its role of bringing together diverse voices, CMC is thought to be especially suited to the task of linking teachers together in experiences that may be both professionally and personally rewarding (Honey, 1995; Ringstaff, Sandholtz, & Dwyer, 1994; Kimball, 1995).

Several characteristics of computer-mediated communication suggest possible potential for reflective discourse. Perhaps the most notable characteristic of network-based communication is the speed with which it sends and retrieves messages to support participant interaction. The time and place independence of the medium allows messages to be sent from any computer equipped with network capability at any time, yet retrieved only when a participant logs onto a system. The flexible time control that network-based communication provides for engaging in discourse; investigating related information; and constructing, communicating, and refining ideas may present what Moller (1998) suggests is the best opportunity to "fully maximize the thinking aspect of knowledge building" (p. 7). This technology enables many participants to interact on multiple conversational topics. The storage capacity of the technology allows users to retrieve segments of a previous discussion, to focus ongoing dialogue, to challenge the accur acy of documented messages, and to eliminate the pressure and tedium of note taking. Finally, because of its text orientation, network-based "communication conversants tend to heavily omit unnecessary linguistic material," which subsequently better orients and organizes the structure and sequence of decision making (Condon & Cech, 1996, p. 79).

Despite CMC's ability to connect teachers, little is known about the technology's ability to facilitate teacher collaborative reflective processes. Studies that do address reflection are usually done in the highly controlled context of preservice teachers development (Colton & Sparks-Langer, 1993; Kenny, Andrews, Vignola, Schilz, & Covert, 1999; Mickelson & Paulin, 1997; Ropp, 1998). Only a few studies address the reflective quality of computer-mediated discourse for practicing teachers. Of those studies, little description of the reflective processes or outcomes of collaborative teacher discourse is offered.

One of the earliest efforts offering an insight into the application of network-based communications is the LabNet project. In 1989 the Technical Education Research Center (TERC) launched the LabNet project as a technology-supported teacher-enhancement program aimed at high school physics teachers. LabNet organized 99 physical science teachers from across the county into clusters of 6 to 10 teachers in a summer workshop experience. Teachers used the asynchronous network to communicate with peers both in and out of their clusters. An analysis of the conversation of these teachers showed discourse outcomes of growing teacher confidence for teaching physics, increased enthusiasm for teaching, and a sense of belonging to the physics teaching community (Spitzer, Wedding, & Dimuaro, 1995). These outcomes are attributed in part to the reflective nature of the teacher discourse. Unfortunately, the study does not treat reflection as a systematic variable, and no discussion on the nature of the reflection or the proce ss used to examine the reflective content is made.

Like LabNet, the Mathematics Learning Forums--a collaborative effort of the Bank Street College and the Center for Children and Technology--facilitated teacher intercommunication through participant mailing lists (listervs). The project was designed to help teachers use and integrate new mathematics curriculum standards into their teaching. Researchers studying the project found that telecommunications helped teachers overcome a number of obstacles to professional development, most notably that of time for development activities (Honey, Bennet, Hupert, Kanze, Meade, Panush, Powell, Spielvogel, Dubitsky, Cohen, Melnick, & Petersen, 1994). Researchers in the Mathematics Learning Forums Project extol CMC's "capacity to bring together new learners in a reflective and constructive fashion" (p. 172). Though the Mathematics Learning Forums were a revolutionary effort to develop teacher capacity to engage students in learning, the claim that reflective outcomes occurred is made only by way of post-project observatio n. Furthermore, the study fails to elaborate on the reflective content of the posts or determine how the reflective content of forum discourse might be different from other discourse modes that were part of the project.

The most informative study of reflective outcomes of CMC is McMahon's (1996) research on the PBS Mathline project This project brought together middle school teachers using a wide range of technologies--video, computers, satellite, and closed circuit broadcast television--to deliver and discuss material aligned with National Council of Teachers of Mathematics (NCTM) standards in curriculum, teaching, and assessment. The online electronic support system linked 25 to 30 teachers at a time. McMahon studied the flow, frequency, and volume of the 393 messages posted to the listserv over the eight weeks of the course. Using a four-point reflection rubric to determine the reflective nature of electronic messages in the listserv, McMahon discovered that 29 % of the participants posted at least one critically reflective message. A message was critically reflective when it "raised issues exploring underlying beliefs, motivations, and implications related to teaching and learning" (p. 91).

McMahon's is the first study to address reflection in the context of computer-mediated communication. The study is limited because reflection scores were based on the highest capacity of reflection participants achieved rather than the mean reflective content of the dialogue of individuals involved. The result is over-estimated reflective scores. Furthermore, no indication is provided in this study of what percentage of the 343 posted themselves was critically reflective, what processes were at work to achieve reflective outcomes, or what contextual factors best facilitated teacher reflection.

The lack of computer-mediated reflective processes as a focus of study leaves many questions about the capacity of computer networks to host critically reflective communication. The excitement that computer networks have generated around communication coupled with the role of critical reflection in improving teacher practice suggest that more inquiry is necessary to illuminate the potential that computer networks have for hosting collaborative, collegial reflection. This study attempts to address that gap in our knowledge base by addressing the following question: Can asynchronous computer networks host critically reflective dialogue? Secondary questions ask: What level of reflection does the discourse achieve and how does it compare to face-to-face discourse?

THE STUDY SETTING AND METHODS

This study is part of a North Central Regional Educational Laboratory project on Technology Supported Interdisciplinary Problem-Based Learning (PBL). The project originated from frequent requests by schools for frameworks and professional development services to support the increasing number of teachers involved in problem-based curriculum development. The project attempts not only to create a seamless connection between content areas but also to incorporate the use of networked technologies both for student learning and teacher development.

Twenty-eight teachers from ten elementary schools in a Chicago metropolitan suburb participated in the two-year project. The teachers were generally novice technology users but worked in schools with substantial technological capacity. Each teacher's classroom was connected to the district network where the Internet, e-mail, and curriculum production tools could be accessed. Most of the schools were connected through a 64 kilobyte line, but two of the ten schools had a high-speed T1 connection. Approximately one networked computer was available for every ten students.

The primary goal at the outset of the program involved building teacher capacity for developing PBL curricula. Teacher teams completed and delivered their first PBL unit in the spring of the first project year. Teachers provided written critiques on their units shortly after, and planned for refinements to the first PBL units and the development of a second unit through the summer.

The focus of the second year of the initiative was to use new technology tools to expand teacher instructional practices and skills in PBL curricular development. Project teachers provided input on the development of an electronic toolkit located in a district server file folder where electronic tools could be retrieved by teachers to develop and refine their PBL units. The kit included database, communication, graphics, word processing, and program/multimedia authoring tools. About halfway through the second year of the project, the electronic toolkit became fully functional and was introduced to project teachers.

These network communication tools became available just as the district implemented a policy which restricted teacher availability for organized professional development events. Because of substitute shortages, district administrators limited the time teachers could spend away from classrooms for personal and professional development events. Personal leave time was also reviewed and reduced. These policies severely limited the face-to-face contact between teachers from various schools participating in the project. The network communication channels established in this project met their test in attempting to host some of the ongoing communication among teachers that new district policies had curbed.

This study takes a multimethod, quasi-experimental approach to data collection and analysis. Comparative content and statistical analysis of computer-mediated and face-to-face discourse comprise the bulk of inquiry into reflective quality of the two mediums. The following sections describe the data collection and analysis techniques used.

Data collection. To determine what levels of collaborative reflection are present when teachers interact under normal circumstances, researchers recorded face-to-face work meetings of school teams consisting of two to five teachers. Teachers were told that their meetings were being audiotaped as a part of the data collection for project evaluation. Teachers were unaware that the level of reflective discourse was the variable of interest in this study. Researchers recorded six teacher meetings from February to May of the final project year. This schedule closely corresponded to the timeline that teachers had for developing their second integrated PBL unit, delivering it to their classes, and documenting the unit for archival purposes. The recording of the face-to-face meetings also ran concurrently with the collection of group computer-mediated communication. Of the six team meetings recorded, two each took place at the beginning, middle, and end of the timeline. Audiotapes were transcribed in preparation for analysis.

The collection of computer-mediated communication commenced through the same four month period that face-to-face data were gathered. A specific focus of communication tool use was for completing PBL curricular units. However, the specific use of CMC to the curriculum development end was unstructured and left to the design of the teacher teams. The unstructured nature of the CMC use (no moderator, no mandatory use required) provided an opportunity to determine how and for what purposes network use naturally evolved.

Collection and storage of CMC discourse between members of the group was ongoing. Researchers categorized messages posted to the common project forums as they were produced. Reading the posts as they appeared provided an indication of the pace of online activity and the topics that were addressed.

Analysis. The focus of analysis was the computer-mediated and face-to-face discourse produced by PBL project participants. This total does not include electronic messages posted by project staff, misposted messages (resent, misdirected), cross-posted messages from other forums, or messages authored by nonproject personnel. While analysis identifies the flow, frequency, and volume of communication activity and the nature of the dialogue, it centers on the reflective attributes of the discourse.

All computer-mediated and face-to-face communications between project participants were scored on a seven-point reflection rubric. The rubric is based on Simmons, Sparks, Starko, Pasc, Colton, & Grinberg's, (1989) taxonomy for assessing reflective thinking. This framework for analyzing the reflective discourse embraces a model of teacher development in which teachers acquire new information that helps them reach "new and creative solutions" to decision-making through collaborative dialogue leading to reflection (Colton & Sparks-Langer, 1993; p. 49). The taxonomy of reflective levels for group discourse is identified and illustrated in Table 1. Low-level reflective responses are those which merely describe events and appear disconnected from the observer. More reflective responses richly describe events and attempt to explain them in light of theory or principle.

To prepare the face-to-face discourse for analysis it was "chunked" into frames comparable to that of the electronic discourse. The chunking of face-to-face discourse was guided by principles of distributional accountability (Schiffrin, 1987), which suggested that utterances are related to each other by their relationship to a common theme. Chunking resulted in 222 distributionally accountable portions of face-to-face discourse ranging in size from 1 to 12 occasionally very short exchanges between teachers (Table 2).

A team of three independent raters with doctoral degrees in education and a combined 40 years of experience in educational research participated in message rating training and calibration to ensure the reliability of the results. After all identifying information (school and individual) was removed from electronic messages and transcripts, raters judged each of the chunked exchanges in the face-to-face (n=222) and computer-mediated (n=179) messages using the seven-level rubric previously described.

Data analysis proceeds by comparing the presence of reflective overtures, comments, and observations of the face-to-face dialogue against reflective qualifies of the CMC dialogue. Interrater correlates are determined to note the consistency of ratings, and statistical analysis includes a t-test for dependent means. Time is also examined as a function of reflectiveness.

RESULTS: COMPUTER-MEDIATED COMMUNICATION VOLUME AND PARTICIPATION

Assessing participant interaction. The comparison of face-to-face and computer-mediated discourse began with the analysis of the conversation on selected dimensions of interaction. As a basic component of conversation, interaction involves the selection and delivery of words that unite the speaker, listener(s), and content. Specific discourse strategies can be employed to heighten the interaction of participants in the dialogue. Variables of discourse interaction examined here include the use of conversational involvement strategies ("wh" clauses: who, what, when, when, where), conversational cooperation (ratio of answered to unanswered questions), and sequential accountability (coherence between utterances in discourse). Table 3 presents a comparison of all the face-to-face and computer-mediated discourse using the chi-square test of association between the observed proportions in each interaction variable.

In all cases, the number of observations made of the selected interaction variables in face-to-face discourse exceeds the interaction of CMC. In three of the five variables: "wh" clauses, amplifiers, and conversational cooperation, results are significant at the p [less than].05 level. The results show that face-to-face discourse generally rates higher in interactivity than does CMC.

Assessing reflective content. Like variables of interaction, (i.e., involvement, conversational cooperation, and sequential accountability) reflection, defined in a social collaborative way, is also an indicator of the medium's ability to promote interactivity between discourse participants. Talking, sharing, exploring, and analyzing are important interactions in sense making and, by themselves, constitute key components in the critical reflection process. Reflection is distinct from interaction, however, in that it requires a certain amount of self-disclosure about professional beliefs and practice.

A comparison of the ratings (independent t-test for equality of means) between the reflective levels of the two communication mediums shows that computer-meditated communication has significantly higher ratings on the seven-point reflective scale than does face-to-face communication (t = 4.14, p = .001). A test of rater consistency produced an inter-rater reliability statistic of .87 on the face-to-face discourse achieved and an item alpha level of .80 on the GMC discourse for the three raters involved.

A breakdown of the ratings of reflectiveness in Figure 1 shows the percentages of ratings assigned to each of the seven reflective levels. The majority of messages (70% for the face-to-face, 63% for CMC) were rated at the first and second levels of reflection and generally show that neither CMC nor the face-to-face discourse is abundantly reflective. These findings are not like those of Hamann, Rubenstein, & Georgi (1999) who found that the reflections of their student teachers were more superficial than strategic.

The fact that different distributions appear for each discourse mode with positive skews of very different strengths, however, suggests that critical reflection may have a more purposeful role in CMC facilitated discourse than in face-to-face discourse.

In response to the questions this study poses, the data show that asynchronous computer networks are capable of facilitating reflective discourse at a level which encourages teachers to collaboratively examine their practice in light of instructional theory. In comparing the computer and face-to-face mediums, computer-mediated discourse achieves a significantly higher reflective level than does face-to-face discourse.

To determine if technology comfortability is related to the level of reflection a teacher exhibits on the medium, comfortability and reflection scores are correlated. Comfortability scores were determined on a four-point self-report scale of competence using telecommunications technologies and their functions. Reflection scores are derived from rater observations made of the computer-mediated discourse. A mean reflective score was calculated for each of the 28 teachers participating in the computer-mediated discourse. A bivariate correlation comparing telecommunications technology comfortability and observed reflection shows a fairly strong positive correlation (r = .63, p[less than].0l) between the two variables.

Reflection over time. A final analysis activity further defines the relationship between the face-to-face and CMC discourse by exploring the possibility of any increase in discourse reflectiveness over the duration of the project timeline. Face-to-face teacher meetings recorded at the beginning, middle, and end of the project period were compared with each other. A similar comparison was made of the CMC discourse. Each of the 179 CMC messages were chronologically ordered and then divided into thirds to roughly match the meeting intervals. Comparisons of the first third of the messages were made against the second and final thirds. Figure 2 illustrates the change in reflection over time between the two mediums.

The face-to-face discourse shows gradually increasing reflectiveness over time from each of the three observation periods to the next. When the means scores for the observations at the three periods were tested for statistical significance, one-way analysis of variance shows no significant difference at the p [greater than].05 level (Table 4).

CMC discourse shows a curious pattern over the three observation periods. Between the first and second periods reflective scores decreased, while the second and third periods show a substantial increase. When the differences between the reflectiveness at these three time periods were tested, a one-way analysis of variance indicates a significant difference (Table 5). Post hoc tests [1] on the means indicate differences between each of the three means observed.

The increase in CMC reflective scores occurs only after a significant drop in the second of the three observations over time. Differences over time are possibly explained by the early initial use of CMC by participants who were experienced and skilled at using the medium. As less confident and skilled teachers forayed into the discourse, their contributions reduced the overall reflection scores produced. More comfort with CMC possibly increased the reflective scores at the time of the third observation. The increase in reflective scores for both the face-to-face and CMC discourse at the final observation may also be explained by the end-of-the-year "ritual" of reviewing the year's work, summarizing it, and considering revisions for the next year, which in and of itself constitutes reflective activity.

Examining the Reflective Outcomes of Asynchronous Communication 303

SUMMARY

Independent rater assessments show that computer-mediated discourse achieves a higher overall reflective level than reflections generated by teachers in face-to-face discourse. Although more reflective, CMC proved not to be as interactive as face-to-face discourse. When time is a factor in the analysis, there is no significant difference in three time-differentiated observations made of reflectivity in the face-to-face discourse. In CMC the difference is characterized by a dip in the reflectivity scores from the first to the second observation. Reflectiveness at the third and final observation then rises to levels significantly above scores at the previous two observations.

DISCUSSION AND IMPLICATIONS

While this study provides some insights on how teachers react to educational telecommunications technologies and use them to build their professional knowledge, there are several limitations to consider. Foremost, teachers included in this study came with interests and attitudes toward learning with and about computer-mediated technologies that were likely different from many of their peers. Most of the participants were also white middle-class women whose approach to talking, thinking, and reflecting is likely to be different from teachers of another class, race, or gender. The school where these teachers work is a technology rich environment in which teachers are provided extensive opportunities to participate in technology-integrated professional development events. Contextual constraints were also placed on the study including limited face-to-face contact and narrow curriculum development time constraints. Also, this study packaged discourse in very small chunks and by doing so limited both the scope and extent to which reflection was analyzed. Results could have been different if entire strands of discourse on individual topics were the analyzed.

Despite the limitations of the study, teachers found that the convenience, quality, breadth, and volume of peer-provided information facilitated by network technology improved their knowledge of educational theory, policy, and the educational community. Still some teachers in this study remained hesitant about the use of technology for an intimate level of discussion. Follow-up interviews revealed that nearly half the teachers participating in this study firmly believe that CMC cannot a replace face-to-face conversation; that the disjointed presentation of information on the medium is difficult to understand; and that disclosure on a public forum brings professional risks. These and other reservations remind us that network technology is not an answer to every teacher's professional development needs.

Teacher collaboration. Still this study suggests that the use of computer networks to share experiences and examine beliefs, creates a new interactive dynamic for teachers that requires trial and testing, as the data on reflection over time show, before any productive relationship between teacher and technology can be continued. This study suggests there is an evolution of teacher CMC use that begins with observation; advances to application of the medium for low-risk and low-importance uses; and progresses to more substantial, revealing, and reflective uses.

A probable key to moving CMC participants through this continuum of use, beginning at caution and eventually arriving at confidence, is the "climate" of interaction. Climate is defined here as the extent to which teachers in the project feel comfortable in the open exchange of ideas, opinions, and feelings over the medium. Our experience with developing an open climate of exchange on the network medium begins with a teacher's simple observation of the medium in action. The next stage includes the contribution of a few innocuous posts to the dialogue. When the teacher feels confident that online peers will both accept and treat their contributions respectfully discourse emerges that is more descriptive and self-revealing. In this research, self-disclosure appeared to be the catalyst for many reflective contributions to on-line discourse, embedding teacher learning in real experiences. At times, although not frequently in this study, this discourse linked what teachers already know to new conditions of educati onal practice to produce relational knowledge.

Teacher reflection. When teachers in this study physically convened in their teams to work on their PBL curricula, the transcripts of a selected number of those meetings show that their work was highly task focused. Though these face-to-face discussions were dotted with what independent raters determined were reflective exchanges, face-to-face discourse failed in several dimensions of analysis to reach equivalent breadth and depth of teacher reflection on CMC. This study finding suggests that when there is specified work to be done, such as identifying instructional strategies for a particular learning outcome, the task drives the agenda. Unless critical reflection is planned as a part of the process or becomes an objective of a face-to-face meeting itself, reflective activities will be sporadic at best. In the CMC discourse examined in this study, task driven or not, the time independence of the medium gave participants a greater chance to step outside of the minutiae of curriculum development tasks and ask themselves: Why are we doing this? What are the long-range consequences of this decision? What meaningful relationship does this activity have with key learning outcomes?

In conclusion, a study of a schoolwide collaboration focused on shared reflection as a response to elementary school reform, Markholt (1998) declares, "... in school, it was easier to talk about procedures than to engage in an in-depth examination of teaching and learning" (p. 13-14). This study confirms what Markholt suggested is a very difficult proposition--to encourage teachers to substantively address teaching issues when they are face-to-face. A feasible alternative for such discussion may be through network technology mediums.

Note

(1.) Student-Newman-Keuls (SNK) post hoc analysis for homogeneous subsets.

References

Borko, H., & Putnam, R. (1995). Expanding a teacher's knowledge base: A cognitive psychological perspective on professional development. In T.R. Guskey & M. Huberman (Eds.), Professional development in education: New paradigms and practices, (pp. 35-65). New York: Teachers College.

Cochran-Smith, M. (1991). Learning to teach against the grain. Harvard Educational Review, 61(3). 279-310.

Cochran-Smith, M., & Lytle, S. (1993). Inside/outside: Teacher research and knowledge. New York: Teachers College Press.

Colton, A.B., & Sparks-Langer, G.M. (1993). A conceptual framework to guide the development of teacher reflection and decision making. Journal of Teacher Education. 44(1), 45-54.

Condon, S.L., & Cech, C.G. (1996). Functional comparison of face-to-face and computer-mediated decision making interactions. In S.C. Herring (Ed.), Computer-mediated communication: Linguistic, social and cross-cultural perspectives (pp. 65-80). Philadelphia: John Benjamins.

Corcoran, T.C. (1995). Transforming professional development for teachers: A guide for state policymakers. Washington, DC: National Governors Association.

Darling-Hammond, L. (1996). The right to learn and the advancement of teaching: Research, policy, and practice for democratic education. Educational Researcher, 25(6), 5-17.

Darling-Hammond, L. (1998). Teachers and teaching: Testing policy hypothesis from a national commission report. Educational Researcher, 27(1), 5-15.

Damon, W. (1984). Peer education: The untapped potential. Journal of Applied Developmental Psychology, 5(1), 331-334.

Dewey, J. (1910). How we think. Boston: D. C. Heath and Co.

Feldman, A. (1997). The role of conversation in collaborative action research. Paper presented at the annual conference of the American Educational Research Association, Chicago, IL.

Hamann, J., Rubenstein, D., & Georgi, D. (1999). Confluent educational perspective and reflective practice: Summary of an exploratory study. Paper presented at the annual conference of the American Educational Research Association, Montreal, Quebec.

Hammerman, J.K. (1997, March). Leadership in collaborative teacher inquiry groups. Paper presented at the annual conference of the American Educational Research Association, Chicago, IL.

Hatton, N., & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, 11(1), 33-49.

Hollingsworth, S. (1994). Teacher research and urban literacy education: Lessons and conversations in a feminist key. New York: Teachers College.

Honey, M., Bennett, D., Hupert, N., Kanze, B., Meade, T., Panush, E.M.S., Powell, K., Spielvogel, R., Dubitsky, B., Cohen, M., Melnick, H., & Petersen, L. (1994). The mathematics learning forums online: Using telecommunications as a tool for reflective practice. Machine Mediated Learning, 4(2&3), 163-176.

Honey, M. (1995). Online communities: They can't happen without thought and hard work. Electronic Learning 14(4), 12-13.

Hullfish, H.G., & Smith, P.G. (1961). Reflective thinking: The method of education. New York: Dodd, Mead.

Jervis, K. (1996). Eyes on the child: Three portfolio stories. New York: National Center for Restructuring Education, Schools and Teaching.

Kenny, R.F., Andrews, B.W., Vignola, M., Schilz, A., & Covert, J. (1999). Toward guidelines for the design of interactive multimedia instruction: Fostering the reflective decision-making of preservice teachers. Journal of Technology and Teacher Education, (7)1, 13-31.

Kimball, L. (1995). Ten ways to make online learning groups work. Educational Leadership, 53(2), 54-56.

Koschman, T.D., Kelson, A.C., Feltovich, P.J., & Barrows, H.S. (1997). Computer-supported problem-based learning: Principled approach to the use of computers in collaborative learning. In T.D. Koschmann (Ed.), CSCL, theory and practice of an emerging paradigm (pp. 110-131).

Lichtenstein, G., McLaughlin, M., & Knudsen, J. (1992). Teacher empowerment and professional knowledge. In A. Lieberman (Ed.), The National Society for Studies in Education 91st yearbook (Part II). Chicago: University of Chicago.

Lieberman, A., & McLaughlin, M.W. (1993). Networks for educational change: Powerful and problematic. Phi Delta Kappan, 73(9), 673-677.

Lieberman, A. (1995). Practices that support teacher development: Transforming conceptions of professional learning. Phi Delta Kappan 76(8), 591-596.

Little, J.W. (1993). Teachers' professional development in a climate of educational reform. Educational Evaluation and Policy Analysis, 15(2), 129-152.

Lord, B. (1994). Teacher professional development: Critical colleagueship and the role of professional communities. In N. Cobb (Ed.), The future of education: Perspectives on national standards in America (pp. 188-210). New York: College Entrance Examination Board.

Markholt, A. (1998). Developing a reflective, collaborative response to reform: Barriers and possibilities in an urban elementary school. Paper presented at the annual meeting of the American Educational Research Association. San Diego, CA.

McKinsey & Company (1995). Connecting K-12 schools to the information highway. Palo Alto, CA.

McLaren, P. (1989). Life in schools. New York: Longman.

McMahon, T.A. (1996). From isolation to interaction? Computer-mediated communications and teacher professional development. Unpublished doctoral dissertation, Indiana University, Bloomington, IN.

Mickelson, K.M., & Paulin, R.S. (1997, March). Beyond technical reflection: The possibilities of classroom drama in early preservice teacher education. Paper presented at the annual meeting of the American Educational Research Association. Chicago, IL.

Moller, L. (1998, March). Designing communities of learners for asynchronous distance education. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Placier, M. (1996). The cycle of student labels in education: The cases of "culturally deprives" and "at risk." Educational Administration Quarterly, 32, 236-270.

Rallis, S.F., Rossman, G.B., Phelgar, J.M., & Abeille, A. (1995). Dynamic teachers: Leaders of change. Thousand Oaks, CA: Corwin Press.

Ringstaff, C., Sandholtz, J.H., & Dwyer, D. (1994). Trading places: When teachers use student expertise in technology intensive classrooms. People and Education, 2(4), 479-505.

Rogers, C. (1970). Encounter groups. London: Allen Lane, The Penguin Press.

Ropp, M.M. (1998). Exploring individual characteristics associated with learning to use computers in preservice teacher preparation. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Romiszowski, A.J., & Mason, R. (1996). Computer-mediated communication. In D. Jonasson (Ed.), Handbook of research for educational communications and technology (pp. 438-456). New York: Simon & Schuster Macmillan.

Rosenholtz, S. (1988). Teachers' workplace. New York: Longman.

Schiffrin, D. (1987). Discourse markers. Cambridge: Cambridge University.

Schon, D.A. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass.

Schon, D.A. (1991). The reflective turn: Case studies in and on educational practice. New York: Teachers College.

Simmons, J.M., Sparks, G.M., Starko, A., Pasch, M., Colton, A., & Grinberg, J. (1989). Exploring the structure of reflective pedagogical thinking in novice and expert teachers: The birth of a developmental taxonomy. Paper presented at the annual conference of the American Educational Research Association, San Francisco, CA.

Sparks, D. (1994). A paradigm shift in staff development. Journal of Staff Development 15(4), 26-29.

Sparks-Langer, G.M., & Colton, A.B. (1991). Synthesis of research on teachers' reflective thinking. Educational Leadership, 48(6), 37-44.

Spitzer, W., Wedding, K., & DiMauro, V. (1995). Strategies for the purposeful uses of the network for professional development. Cambridge, MA: Technical Education Research Centers. [Online]. Available: http://hub.terc.edu/terc/LabNet/Guide/00-Pubinfor.htm

The Secretary's Commission on Necessary Skills. (1991). Skills and tasks for jobs: A SCANS report for America 2000. Washington DC: Department of Labor.

Wildman, T.M., Niles, J.A., Migliaro, S.G., & McLaughlin, R.A. (1990). Promoting reflective practice among beginning and experienced teachers. In R.T. Clift, W.R. Houston, & M.C. Pugach (Eds.), Encouraging reflective practice in education: An analysis of issues and programs (pp. 139-162). New York: Teachers College.

Zeichner, K.M. (1996). Teachers as reflective practitioners and the democratization of school reform. In K. Zeichner, S. Melnick, & M.L. Gomez (Eds.), Currents of reform in preservice teacher education (pp. 199-214). New York: Teachers College.

Table 1
Taxonomy of TeacherReflective Thinking
         Description
Level 1  No description of event.
         Message unrelated to practice.
Level 2  Events and experiences,
         described in simple, layperson
         terms, generally unattached
         to classroom activities.
Level 3  Descriptions of events and
         experiences employ peda-
         gogical terms.
Level 4  Explanation of events or
         experiences is accompanied
         by rationale of tradition or
         personal preference.
Level 5  Explanation of an event or
         experience using cause/effect
         principle.
         Illustration
Level 1  "I've lost the address to the PBL
         file folder, can anyone tell me
         where to find it?"
Level 2  "I knew I was taking a big chance
         leaving my Toshiba laptop on a
         worktable for students to use.
         When I saw the thing fall to the
         ground I said to myself, 'I told you
         so.' Although none of the kids
         intended for the event to happen.
         they have a hard time with the give
         and take of teaming."
Level 3  "When we began our PBL unit on
         survival, we were unsure as to what
         to expect from the students, what
         to ask, and how to evaluate the
         process. We found the hardest part
         was developing the question. Once
         we found an open-ended, clear,
         solvable question (which took a lot
         of refining), the rest of the
         answers fell into place."
Level 4  "She used simulations because
         research on multiple intelligences
         says it works. Our principal pushes
         the multiple intellegences agenda
         too so it was worth it to give it a
         try."
Level 5  "I am a little mixed up. I said I
         would be happy to post what we
         have written as a guide to anyone
         else who was interested. I have
         posted our unit description on the
         network neighborhood. I was not
         aware of having to use any other
         template. I hope what we have
         posted is in order and will be
         beneficial to others as they develop
         their units."
Level 6  Explanation provided that
         identifies cause and effect
         factors while also considering
         contextual factors,
Level 7  Explanation of events,
         experiences, or opinions that
         cites guiding principle and
         current context, while
         referencing moral and
         ethical issues,
Level 6  One of the concerns I have is that
         made-up problems are seen as
         (and, in fact, are) both pretty
         thin, pretty obviously incomplete,
         and not the students' (in terms of
         owning the problem). By 'thin' I
         mean the difference between a
         made-up problem and a real one is
         that the real one is endlessly rich
         in complexity and detail in a way
         that a made-up one just can't be.
         At some point a student pursuing
         some idea or implication will come
         up against something that the
         problem-setter just didn't have
         time to imagine."
Level 7  "The students need to be taught
         some problem-solving skills. One
         philosophy holds that there is a
         group of skills (Independent Study
         Skills) that need to be taught
         through direct instruction. This
         means instruction in the process
         itself, not merely as it relates
         to the problem at hand. The
         students need to acquire these
         skills, and practice them in order
         to be able to do PBL successfully
         (or at least to do it in the
         manner in which we expect them
         to). The skills are a lot like
         the Big Six and other models,
         but the emphasis is on direct
         instruction of the skills."
Table 2
Face-to-face and CMC Dialogue
Content Comparison
                                           Number    Utterances/
                                           of Words  Messages
Face-to-Face Discourse (teacher meetings)  19,000    846
Computer-Mediated Discourse (listserv)     19,303    179
                                           Chunks/    Words
                                           Meassages  Per
                                                      Chunk/
                                                      Post
Face-to-Face Discourse (teacher meetings)  222         86
Computer-Mediated Discourse (listserv)     179        108
Table 3
Chi-Square Values on Discourse Interaction Variables
                                   CMC  Face-to-
                       Parameter   (n)  Face (n)  [[X.sup.2].sub.1]
Involvement Strategie
 "Wh" clauses            Words     220    326           22.99
 Indefinite pronouns     Words     435    474            2.404
 Amplifiers              Words     220    288           10.35
Conversational
Cooperation            Questions    25     95           10.01
Sequential
Accountability         Utterances   15     43            3.009
                        p
Involvement Strategie
 "Wh" clauses          .001
 Indefinite pronouns   .065
 Amplifiers            .001
Conversational
Cooperation            .002
Sequential
Accountability         .064
Figure 2.
Changes in mean reflective score over three observation periods
Beginning  Middle   End
  2.751    2.183   3.289
  2.152    2.243   2.529
Table made from line graph
Table 4
Comparison of Face-to-Face Discourse Over Time
Source          Sum of   df   Mean Square   F    Sig.
                Squares
Between Groups    5.35     2     2.67      2.94  .055
Within Groups   199.11   219      .91
Total           204.47   221
Table 5
Comparison of CMC Discourse Over Time
Source          Sum of   df   Mean Square    F    Sig.
                Squares
Between Groups   36.681    2     18.34     20.71  .000
Within Groups   155.883  176       .89
Total           192.564  178

[Graph omitted]