Online project-based learning: how collaborative strategies and problem solving processes impact performance.

The goal of this study was to gain insights into the interactions that occur in online communications in a project-based learning activity implemented in an undergraduate course. A multi-case study was conducted of six collaborative groups, focusing on the types and frequencies of interactions that occurred within each group and the perceptions that students had of their experiences in this type of learning environment. It was found that the interactions within each group closely followed established steps in the problem solving process. The findings of this study go further in explaining specific indicators that may determine how well a group performs when using CMC as a support mechanism for project-based learning. High achievers tend to start early, are consistent in the frequency and extent to which they post messages, develop a sense of camaraderie online, are effective organizers and coordinators within the online environment, and engage in a deep, rich thought provoking dialog with a high degree of idea exchange. Low achievers on the other hand are slow starters, are erratic and inconsistent in posting messages, do not form bonds online, are not effective in organizing and accomplishing tasks online, and engage in shallow, directive dialog with little questioning and exchange of ideas. Students also differentiated between asynchronous and synchronous systems as to the type of tasks that are best suited for each. There was a general consensus that asynchronous system are best for tasks that require reflection, time, and deeper thought and synchronous systems are better for brainstorming, as a forum for the free flow of ideas, and for building group solidarity and social connection.

**********

INTRODUCTION

Curriculum development in higher education has been influenced in recent years by reform efforts that emphasize changes in how learning environments and processes are viewed, on how learning is affected by student traits, and on how new technologies can effectively be utilized to enhance learning (Chickering & Gamson, 1987; Magolda, 1992; Pascarella & Terenzini, 1998). A major challenge is making a shift from traditional pedagogies that are instructor-centered, to a social constructivist paradigm where students are encouraged to not only work individually to solve relevant problems in the academic disciplines, but also to work collaboratively in their endeavors (National Research Council, 1996; National Science Foundation, 1996). Instructional practices encouraging student-centered learning and in which students take a more active role in the creation of their own knowledge are emerging in postsecondary classrooms (Bruffee, 1999; Lattuca & Stark, 1995; Magolda, 1992; Spear, 1989). Social constructivist viewpoints, socio-cultural theories, and principles of situated cognition have all contributed to a greater interest in how the social aspects of learning are reshaped and enhanced by the technological tools that are used to support instructional methodologies (Koschmann, 1996).

Because communication can be facilitated through the Internet it is important to increase understanding of how both asynchronous and synchronous computer-mediated communications (CMC) can be used to carry out tasks in a naturally collaborative network. An emerging body of research suggests that collaborative learning environments at the college level are effective in that they increase student achievement, promote favorable attitudes toward learning, and decrease student attrition when compared to courses that are taught using more traditional pedagogies (Springer, Stanne, & Donovan, 1999). However, it has been noted that in order for these positive effects to occur students must possess the necessary skills of collaboration (Bosworth, 1994). These include, but are not limited to interpersonal or social skills (openness and solidarity), group management/development skills (involvement and control), and inquiry skills (clarification, inference, judgment, and strategies) (Henri, 1991; Lundgren, 1977; McDonald & Gibson, 1998).

When collaborative learning is implemented in college classrooms gains in student achievement, the development of metacognitive strategies, improved class attendance, and increased student satisfaction have been found (Alexander & DeAlba, 1997; Dansereau, 1983; Groccia & Miller, 1996; Michaelsen, 1983; Smith, 1987). Caplow & Kardash (1995) suggest that instructors provide a clear rationale for using this form of activity, there be a balance between the process and the product of the collaborative activity, and the instructor monitor the progress of the project, and be flexible enough to make changes if deemed necessary, and specify evaluation procedures. This is contradictory to the idea that students in a highly collaborative model have total control of their learning. An instructional technique that incorporates collaborative learning as a component and that has garnered recent attention in the literature is project-based learning (PBL). Student projects offer an ideal situation to provide problem-solving opportunities that present real world problems that are scaled back so that they are doable in the confines of the classroom. Project-based learning can be thought of as learning through a series of theme-related activities that are based in authentic, real world problems in which the learner has a certain amount of control over the learning environment and the design of the learning activities (Morgan, 1987; Slavin, 1995).

There is a substantial body of research on how individuals and groups solve problems in a variety of educational settings. From the pioneering work of Polya (1957), Wertheimer (1959), and Newel & Simon (1972), models have been developed that attempt to explain how individuals solve problems. The IDEAL model (Bransford & Stein, 1993) of the problem solving process identifies five primary components: Identify problems and opportunities; define goals; explore possible strategies; anticipate outcomes and act; and look back and learn. There have been numerous models of problem solving for various types of group settings that utilize the scientific method of problem solving to approach the development of ideas, methods, or products (Hare, 1992). The IDEAL model is specific to individual problem solving activities, but can also be applied to problem solving in the context of groups. Several principles related to the processes that are used by groups to solve problems have been identified. In general, groups go through phases of development in which they define a situation, acquire resources, develop roles of individual group members, coordinate tasks, and finally come to some type of closure on their activities. The general consensus concerning most tasks related to group problem solving is that individuals first act in isolation and then come together to share or combine their products with other members of the group to arrive at a solution. When comparing groups in their performance in problem solving activities, several factors contribute to the superiority of one over the other. Bransford and Stein (1993) offer several possible instructional applications of their IDEAL model, one being project-based instruction.

Project-based learning and collaborative learning are highly compatible and in a way are essential to each other for effective implementation into the university classroom. Incorporating these types of learning activities into everyday teaching strategies poses challenges to both instructors and students. Increased time demands placed on both teachers and students may be confounded if projects are collaborative in nature and are introduced in addition to regularly scheduled classroom tasks. Meeting outside of class to work on these projects and instructor facilitation and encouragement of student involvement requires additional time. To make it easier for students to collaborate and for instructors to monitor collaborative projects it may be beneficial to incorporate new technologies such as computer-mediated communications (CMC).

Several advantages have been suggested for using CMC as a supplement to traditional lecture-based instructional techniques (McComb, 1994). CMC extends learning beyond the four walls of the classroom, allowing dialog to continue beyond normally scheduled class times. Secondly, CMC bridges barriers that are often created because of inequities in the balance of power that normally exists in the typical college classroom. Instructors and learners can be engaged in the learning process as partners such that the focus is no longer limited to the knowledge of the instructor delivered via lecture. Instead, emphasis is on the process of mutually shared learning. A third and final advantage is that CMC is efficient in providing easy access to resources, facilitating assignment turnaround, simplifying record keeping, and increasing the ability to focus participation. A multi-study research project addressed several propositions and hypotheses related to collaboration and asynchronous learning networks (ALNs) (Hiltz, Coppola, Rotter, & Turoff, 2000). Three longitudinal studies (a field study of ALN, a field experiment on collaborative learning, and semi-structured interviews with faculty) focused on 26 courses offered as part of an undergraduate degree program in Information Systems. They found that the quality of learning is dependent on active student participation, properly implemented pedagogical strategies, and that collaborative assignments effect student motivation in a positive way.

To fully understand the nature of discussions that evolve in an online environment it is important to gather pertinent information concerning the interactions that occur within the CMC systems. Various approaches have been utilized to accomplish this. Henri (1991) suggests that interaction can be determined by observing the types of statements (a statement being defined as the "unit of meaning") and the relationships that exist between those statements. In particular, she has derived a categorical typology of these statements to be used in coding textual transcripts of these interactive statements. These include explicit interaction (direct responses and comments), implicit interaction (indirect responses and comments), and independent statements. Others have suggested that this method does not give a clear interpretation of the complex interactions that occur in online discussions and suggest that message maps (graphical representations of interactions) provide a better representation of the dialog that occurs between conference participants (Hara, Bonk, & Angeli, 2000; Howell-Richardson & Mellar, 1996). Lee, Liang, & Chan (1999) devised an extensive interaction analysis table, which categorizes interactions into categories (domain-specific, general problem solving, and emotional) and then labels a statement under these categories according to its type (suggestions, questions, instructions, strategies, comments, and others) and its usage within each type.

RESEARCH QUESTIONS

The research project described below looks at the online discourse of collaborative groups engaged in project-based learning from the combined perspectives of interaction analysis and collaborative processes. The purpose of this mixed-method study was twofold. One objective was to understand the strategies that students use to share knowledge and collaborate in the completion of a group task. It is important to understand the processes that are involved in the negotiation, implementation, and completion of tasks in both synchronous and asynchronous systems. The underlying questions, some of which were established prior to conducting the research and some which emerged were:

* How did the online interactions evolve over the stages of planning, design, and development of their projects?

* How did the online interactions differ across the asynchronous and synchronous systems?

* How were the interactions different between groups with high quality and those with low quality projects?

* In what ways did individuals of varying temperament types differ in the dialogic contributions they made towards task completion?

A second objective was to gain insight into student perceptions and attitudes about their participation in an online learning community. Research shows that student satisfaction with the learning process may be an important determinant in increased achievement and learner outcomes. The specific questions formulated to explore this were:

* What benefits/limitations did students perceive in the synchronous versus asynchronous system?

* What differences existed in the perception of students about leadership and task distribution between groups with high and low quality projects?

METHODOLOGY

A multi-case study of six collaborative groups was conducted in which qualitative and quantitative research methodologies were utilized to gain an understanding of the processes that occur within groups during the completion of a project that was supported by an online learning environment. The course selected for this study was composed of twenty upper level undergraduate students at a major state university. The main objective of the course was to teach students basic instructional design principles and to introduce them to the use of technology in learning environments. This study focused on a required Website design project that was assigned over a 6-week period and culminated with students presenting their final project to the class.

The project-based learning activity required each group to plan, design, and implement a Website. The purpose of the Website was to provide information about a group-selected topic to a targeted audience. Students had access to an instructional technology lab that had some of the software necessary for completion of project modules. Other labs were also available on campus where the students had access to software not available in the class lab. In addition to these labs, a Web-based learning environment, Blackboard, Inc.'s CourseInfo 4.0 (2001), offered a Web space for each group to collaborate via CMC, specifically through a discussion board (asynchronous) and a virtual classroom (synchronous).

DATA COLLECTION PROCEDURES

Several sources of data were available and consisted of transcripts of all online dialogs in both asynchronous and synchronous environments, an open-ended questionnaire administered at project completion, online usage statistics, and a personality inventory. The CMC data were analyzed using the techniques of a manifest content analysis (Berelson, 1952) based on a model developed by the researchers.

The 15-item questionnaire was developed by the primary researchers to gather information about student perceptions and attitudes about the collaborative process and how the CMC systems contributed to the completion of project tasks. A constant-comparative analysis (Glaser & Strauss, 1967) of the open-ended survey that all participants provided at the end of the study was conducted after the completion of the analysis of the CMC data.

Students were administered the Keirsey Temperament Scale to determine individual personality types and their scores were used in the formation of project groups of three to four members that were heterogeneous in terms of these types. Keirsey (1998) places personality types into four primary categories with each containing four subtypes: Guardian (Supervisor, Inspector, Provider, Protector); Artisans (Promoter, Crafter, Performer, Composer); Idealists (Teacher, Counselor, Champion, Healer); and Rationals (Fieldmarshal, Mastermind, Inventor, Architect). This instrument was used because of the short amount of time that is needed to administer it to students. Although there are no published validity and reliability statistics on the Keirey Temperament Sorter, it has been determined that initial reliability correlations have been acceptable and it also correlated well with the statistically sound and widely used Myers-Briggs Type Indicator (Berens, 1996).

In conjunction with the analysis of the discourse, the group projects were evaluated with a rubric, which targeted the criteria of content, design, and creativity of the group's final Website. Each criterion consisted of four elements that were scored on a 3-point Likert scale such that a minimum score of 12 and a maximum score of 36 were possible. Three raters scored the projects and the results were combined to determine the quality of each project with 36 being the lowest possible score and 108 the highest. For this study one high performance group (94/108) and one low performance group (64/108) were chosen to compare the interactions that occurred within the online dialog.

DEVELOPMENT OF THE MODEL AND ANALYSIS PROCEDURES FOR THE CMC DIALOG

Several models of interaction were considered to guide the analysis of the dialog that occurred within the context of the CMC systems that study participants utilized to complete their project tasks (Bosworth, 1994; Gunawardena, Anderson, & Lowe, 1996; Henri, 1991; Lee, Liang, & Chang, 1999). A model suggested by Henri (1991) has been used in several studies that look at interaction patterns in collaborative online learning environments and was used to do a preliminary analysis of a sample of the data. After some preliminary coding and analysis of the results it was determined that, although often used by others, this model posed significant problems in interpreting the interactions that occurred. Not wishing to totally abandon this model, an attempt was made to combine Henri's coding model with the suggestions given in Bosworth's (1994) taxonomy of skills necessary for effective learning in collaborative environments. The Henri model and Bosworth's taxonomy paralleled each other in several aspects, in particular those concerning the interactions that were social in nature, those dealing with group building and management, and those interactions that involved inquiry and conflict resolution. A model was developed that incorporated both Henri and Bosworth's suggestions and a sample of dialog was again coded and analyzed.

Still not fully satisfied with the results the researchers examined several studies that used all or parts of Henri's model and began to note that the categories of interaction lacked operational definitions that were appropriate for the learning environment of this study. Lee, Liang, and Chan (1999) developed an interaction analysis table for a study of synchronous learning situations. This model was considered and again a sample of dialog was coded and analyzed. Although promising at first, as the analysis evolved weaknesses in the model became apparent. First, the model was loosely based on some of Henri's suggestions, which as mentioned above, are not well suited for this study. Secondly, as the researchers got deeper into the coding, they realized that there were overlaps between the categories and types of interactions that posed serious problems in determining how certain passages should be coded. A comparison of the researcher's coding and another coder's interpretations yielded serious inconsistencies. Agreement could not be reached on several types of the interactions proposed in this model.

After some consideration of the research literature pertaining to group processes and the interactions that occur within face-to-face groups, as well as that of problem based learning and the use of projects in learning, a model was developed by the researchers that provided a better fit with the purposes of this study. Three categories of interactions were established: Task Activity, SocioEmotional, and Other. Bransford & Stein's (1993) IDEAL problem solving model was adapted for the analysis of the interactions that occurred during the tasks necessary for the completion of the project. It was also determined that some socio-emotional aspects that occur during the process of completing tasks in an online environment might be important in answering the research questions concerning task completion. Several theories of small group development and the social interactions occurring within these groups were considered and it was determined that the model developed by Hare (1976, 1992), as described in Davies (1996), was best suited to the interactions, which were observed. The other category included interactions of the off-task, organizational, and technical types. Table 1 provides operational definitions of the codes that were utilized in the analysis of this data.

RESULTS

The data were coded, organized, and quantified using Atlas.ti, a qualitative analysis software application. Each posted message was first coded according to the categories of the model (problem-solving, socio-emotional, and other). Filters were then applied to isolate messages for each category and the subtypes of each category were then coded. Once all categories and subcategories were coded further queries were run and super-codes were created to extract the frequency for each subcategory. These quantified results were then used to create frequency tables.

EVOLVING INTERACTIONS WITHIN THE ASYNCHRONOUS SYSTEM

A cross-case analysis of the online interactions of all six groups provided insights into how interactions evolved over the three stages of the project (planning, design, and development). When looking at the total interactions of all groups within the asynchronous system there was evidence to suggest that the process of completing the required tasks for the project followed the standard stages of problem solving. The project modules, organized by the instructor, required planning activities, followed by design activities, and culminated in development of the Website. Problem solving interactions were present in 59% of the total interactions (408) within the online dialog across all stages of the project.

Figure 1 displays the percentages of the types of problem solving interactions as they occurred across the stages of the project. Of the 241 problem solving interactions, 83 occurred during planning, 52 during design, and 106 during development. During the planning stage the interactions focused predominately on identifying the problem. The data also indicated that the students were simultaneously defining goals and exploring strategies. The presence of the latter two interaction types indicates that while formulating goals they were discussing the feasibility of strategies for achieving those goals.

As students interacted during the design stage, while there was still some goal definition taking place, there was an increase in exploring and acting on strategies. It was evident that there was a dramatic increase in acting on strategies during the development stage with some exploration of strategies still occurring. These results revealed that the nature of the tasks during the design and development stages required that participants interact in the online environment in ways that supported the implementation of the strategies necessary for completing the project. Overall, the interaction analysis indicated that the steps and sequence of the model of problem solving used for this study were present in the online communications within the groups. It also illustrated the recursive nature of the problem solving process.

Socio-emotional interactions were present in 18% of the total interactions (408) within the online dialog across all stages of the project. This was much lower than that of problem solving, which comprised 58% of the interactions. Almost half of socio-emotional interactions (45%) occurred during the development stage of the project. This may indicate that the group members had established rapport with each other and were more willing to post messages of this type.

Within this category's subtypes (agreement, antagonism, disagreement, and solidarity) certain trends were observed (See Figure 2). During the planning stage the student interactions were primarily positive, showing agreement and building solidarity. It was likely that participants were attempting to seek approval of the other group members and were trying to establish amiable relationships within their group through acceptance, understanding, concurrence, and compliance. There were also indications that they were coming together as groups using humor, showing appreciation, and demonstrating commitment within each group. In conjunction with these positive interactions there was evidence that some antagonistic interactions occurred.

While students were engaged in the design stage, an increase in interactions reflecting solidarity was observed. They praised each other's contributions, were empathetic toward each other, and offered help in completing difficult tasks. There were some antagonistic statements, mostly apologetic in nature. A balance between agreement and disagreement with suggestions offered by group members was somewhat evident.

There was a high occurrence of antagonistic interactions during the development stage. Some of these statements were apologetic in nature, referring to an inability to complete a task in a timely manner. Others reflected alternative actions taken by a group member to compensate for another member's inability to complete a task.

The other category, which included off-task, organizational, and technical issues, comprised 23% of the 408 total interactions observed. Distinctions among these types were observed across the stages of the project (See Figure 3). It was clear that students utilized the system to organize group tasks required for completing project tasks. During the planning stage more concerns related to technical issues were raised. Students were seeking solutions to problems they had with accessing Blackboard from their home computer.

During the design stage the proportion of organizational interactions increased indicating that it was necessary for completing design activities. Fewer technical interactions indicated that the problems with the systems may have been overcome and resolved. Technical issues decreased during the development stage and were no longer related to problems with using the asynchronous system, but referred to using the Web design software. Throughout all three stages, off-task interactions were present and it was obvious that students were utilizing the online forum for personal communications.

DIFFERENCES BETWEEN HIGH AND LOW PERFORMING GROUPS

Two groups that utilized only the asynchronous system were compared (high quality project and low quality project) and differences in their online interactions were observed. When comparing the two groups it was noted that the low achieving group had fewer interactions (38) than the high achieving group (80).

The high group identified their problem during the planning stage, whereas the lower group started identifying the problem during the planning phase and was still doing so during the developmental stage (See Figure 4). The high group was making apparent progress early in the process. They were not only identifying the problem and clarifying goals, but were also beginning to explore and act on strategies. Within the low group most of their interactions occurred during the development stage and still were related to exploring strategies. The members of the high group were exploring and acting on strategies in the design phase and their interactions mainly were directed toward acting on strategies during the development stage. These patterns of interaction may explain why one group outperformed the other in that the high achieving group was more actively engaged in the problem solving process throughout the project timeline. It may also be an indication that they developed effective strategies for completing tasks within the appropriate stages of the project.

The high group posted about the same number of socio-emotional statements as the lower group; however, their postings were dispersed across all stages of the project (See Figure 5). The low group did not interact at a socio-emotional level until the design phase. The members of the low group did not disagree with other members of their groups indicating that they accepted task-related suggestions without question. Within the other category, there were observed differences in the number of organizational statements. The low group did not post any organizational messages until the development stage and then they only posted two of this type. In comparison, the high group posted organizational statements across all stages. This indicates that the higher performing group utilized the online system to manage and allocate tasks necessary for project completion more effectively than the lower performing group (See Figure 6).

[FIGURE 4 OMITTED]

Additional analysis of the content of the messages revealed differences between the two groups with respect to the quality of the interactions that occurred. The high performing groups posted thought provoking questions and there was evidence of a free flowing exchange of ideas. To the contrary, the low performing groups posted statements that were more directive, with very few questions posed and a very restrictive exchange of ideas. The high groups had discussions that were more in depth, whereas the low group discussion was more inclined to be surface level in nature. Several days would often elapse before a member of the low achieving group would post a message to the discussion board. In contrast, the high achieving group posted frequently and in most cases at least one message was posted and responded to every day (See Table 2).

DIFFERENCES ACROSS ASYNCHRONOUS AND SYNCHRONOUS SYSTEMS

Students were not required to use the synchronous system to complete the project tasks, but three groups voluntarily chose to use it. There were 726 total interactions occurring within the synchronous system. Of these, 44% were categorized as problem solving, 21% were socio-emotional, and 34% were other. This pattern was similar to the interactions in the asynchronous system: (problem solving -- 59%; socio-emotional -- 18%; other -- 23%). There were fewer interactions related to problem solving.

[FIGURE 5 OMITTED]

[FIGURE 6 OMITTED]

During the planning stage of the project there were no interactions of the problem solving type and only one interaction of the socio-emotional type. The interactions that did occur during this stage were primarily off-task socio-emotional. During the design stage, interactions of the problem solving type (55% of 463 messages posted) dominated the conversation. In the development stage of the project there was a significant shift to interactions in the other category (54% of 242) and these were primarily off-task socioemotional and organizational. These shifts may indicate that this system was used as an extension of the asynchronous system in completing necessary problem solving tasks during the design stage of the project with more off-task and organizational interactions occurring during the development stage when groups were wrapping up their projects.

The variations in the kinds of interactions that occurred within both the asynchronous and synchronous systems were analyzed and several trends were apparent (See Figure 7). Within the asynchronous system the interactions during the planning stage of the project were more task-oriented and primarily of the problem solving type. The interactions that were observed in the synchronous system were off-task socio-emotional and indicated that the students were trying to get to know each other. There were no observable differences between the two systems during the design stage of the project. During the development stage, the interactions within the asynchronous system were of the problem-solving type and were task-oriented; whereas, the interactions in the synchronous system were primarily off-task socio-emotional. This data indicated that the students used the synchronous system during the planning and development stages to interact with each other socially rather than to accomplish project tasks. It was also noted that of the three high performing groups two of them utilized both the asynchronous and synchronous systems in the process of working on the project. The high performing group that did not use the synchronous system was the least technology proficient group of the six and met face-to-face outside of class. Although this was a small sample, it appears that the use of both systems may allow for the variations in the interactions necessary for groups to collaborate effectively in completing project tasks.

TEMPERAMENT AND THE TYPES OF INTERACTIONS

It was found that individuals of varying temperament types differed in their dialogic contributions. Of the 16 temperament types suggested by Keirsey, 7 were represented among the students participating in the study: Artisan Performer (n = 1), Artisan Promoter (n = 1), Guardian Protector (n = 3), Guardian Provider (n = 5), Guardian Supervisor (n = 8), Idealist Champion (n = 2), and Idealist Teacher (n = 1). An analysis of the data indicated a possible relationship between temperament and how group members contributed to the completion of tasks in the CMC systems. Of particular interest was that artisan promoters appeared to be dominant in their contributions to the online dialog during the design stage reflecting their propensity to be go getters and their ability to manipulate those around them. Another type that was of interest to the researchers in this study was the idealist teacher. It was found that their postings in the early stages of the project reflected statements of solidarity that reflects their ability to guide others in a fair and even handed way. Both of these types made more solidarity statements leading the researchers to believe that they were active in promoting group development. This fits well with their people-oriented personality types as described in the Keirsey Temperament Scale--II. Further analysis that concentrates on the effects of temperament type in collaborative online learning environments may provide additional insights into the role that individuals of varying temperaments play.

STUDENT PERCEPTIONS

The consensus of all participants was that they preferred face-to-face meetings, using the asynchronous and synchronous systems in support of their offline activities and meetings. Their preferences for online systems was primarily for using asynchronous CMC although those that used the synchronous system thought that it had some utility as well.

Among the benefits identified for the asynchronous system was its anytime, anywhere aspect. Students liked the discussion board in that messages could be posted and left for others to review and add comments at a time that was convenient. However, this aspect seen as positive by some was viewed as a negative by others in that asynchronous communication created problems because delay time often occurred when other students did not check the board often enough. Several participants noted that they liked the fact that this system is a good place to pull things together that individual group members are working on. Several students noted that while their group found asynchronous communication very useful in the planning and design stages of the project, they found it somewhat cumbersome during the development stage. There were several comments that indicated that students found that asynchronous CMC was better suited for organizational tasks and for presenting "ideas for thought" for which the group needed some time to consider. Overall most groups felt that asynchronous system was the most useful in the completion of project tasks.

Of the students that used the synchronous system most responses were positive and they found several advantages over the asynchronous CMC for certain kinds of tasks. They preferred using synchronous CMC for brainstorming and later using the asynchronous system to follow up what was discussed there. They thought that this system was better suited for the free flow of ideas. Most liked the near face-to-face feel of chatting. Several commented that this system was more efficient when fast decisions were needed. Those that used both felt that each had its place, depending on the task to be completed. Several individuals who were in groups that did not use the synchronous system at all or used it later in the semester for another class assignment felt that this system might have benefited their groups in completing tasks, particularly tasks that are not deemed as being efficiently accomplished using the asynchronous system.

In high performing groups, there was a perceived leader; however, this person often shared the leadership role with other group members. Their primary leadership role was to organize tasks and delegate responsibilities. Many tasks involved shared leadership such as getting everyone on the "same page," planning fact-to-face meetings, and moderating online communications. Leaders played the role of facilitator, not dictating what everyone needed to do. Leadership roles were enhanced by the CMC in that it was found to be an effective way of "tying up loose ends" and bringing things together for online submission of required components.

In the effective groups, tasks were perceived to be fairly distributed with some members taking on smaller and more frequent tasks and others performing more difficult tasks that were more time consuming. These groups felt that the CMC systems that they used were effective in managing group tasks and in getting tasks accomplished. They felt that as they progressed through the process of designing a Website that they became more proficient at utilizing online communications. One participant noted that as the project progressed he became more proficient at knowing when group members would post messages to the discussion area. He began, "to get a feel for when others posted messages" making the exchange of information more efficient. Both groups found that they were not managing their time effectively, particularly online, early in the project and made adjustments to correct this.

The low performing groups had relatively no leadership. One group had a self-perceived leader who claimed to assign tasks to other group members and would "suggest meeting times to work on the project and tell each person what they needed to have completed for the next meeting." Other group members saw this individual not as a leader but as more of a dictator. The other group had a dominant leader who assumed responsibilities for many of the project tasks. This leader utilized the asynchronous system to set up face-to-face meetings and to coordinate activities in the discussion board. He also was the group member to seek out help from the instructors when he or the other members had technical difficulties. The other members in this group recognized the contributions that their leader made to the project, but for the most part thought that tasks were fairly distributed. The leader did not perceive this to be true and felt that his burden of responsibility for getting tasks completed was unfairly placed on his shoulders. In general, there was a sense that each member did more than others and that other group members were not sufficiently dedicated to the project. One person felt that one of her primary responsibilities was to play negotiator to other bickering group members. Other complaints were that members were slow to respond to online messages, requiring someone else to assume responsibilities, causing duplication of tasks, and wasting time.

Although a few students indicated that they preferred working independently, most students enjoyed the experience, found the project challenging, and felt that working with others was beneficial to their learning. For most, this was their first experience in collaborative learning of this type. Several were very proud of their final product and felt a "sense of accomplishment." They indicated that the collaborative experience was "fun" and it gave them an opportunity to meet new people and to establish new friendships.

DISCUSSION

The results of this study showed that the online dialog that occurs when a group of students is effectively engaged in a project-based learning activity corresponds to the IDEAL model for solving a problem (Bransford & Stein, 1993). Task distribution and completion in the online communications systems are observable and can be monitored through an analysis of the online dialog. There may be benefits that dictate which system, synchronous or asynchronous, to use in diverse learning environments. It has been suggested that asynchronous systems are more effective for tasks that require reflection, time, and deeper thought beyond what can be accomplished in the classroom. In contrast, tasks that require spontaneous interactions are best suited for face-to-face discussions (Meyer, 2003). The findings of this study support the notion that synchronous systems may afford opportunities for face-to-face type conversations, for brainstorming, and as a forum for the free flow of ideas. It also reinforces the assumption that the synchronous systems may be more conducive to situations that require solidarity building, group social connection, and a sense of community. How these systems are utilized to support various learning objectives needs further research.

Differences between the online interactions of high and low performing groups were found in this investigation. Although there is evidence that the interaction level of groups is not indicative of how well students perform on a course examination, students in highly interactive asynchronous groups perform better on written assignments that require the exchange and sharing of ideas (Picciano, 2002). It has also been determined that students who utilized a CMC system for solving a case study produced better reports than those that did not have access to this tool (Hiltz, Coppola, Rotter, & Turoff, 2000). The findings of this study go further in explaining specific indicators that may determine how well a group performs when using CMC as a support for project-based learning. High achievers tend to start early, are consistent in the frequency and extent to which they post messages, use both systems effectively, develop a sense of camaraderie online, are effective organizers and coordinators within the online environment, and engage in a deep, rich thought provoking dialog with a high degree of idea exchange. Low achievers, on the other hand, are slow starters, are erratic and inconsistent in posting messages, primarily use only the asynchronous system, do not form bonds online, are not effective in organizing and accomplishing tasks online, and engage in shallow, directive dialog with little questioning and exchange of ideas.

Indicators of social networks such as friendship, advice, and adversarial exchanges contribute to student performance (Yang & Tang, 2003). There is evidence to suggest that the higher achieving groups of this study were more socially engaged and by the end of the project had established friendships. There is research to support the theory that social presence is a contributing factor in students' perceived learning in online environments (Richardson & Swan, 2003). It has also been found that students who do not prefer online communications for problem-based learning have indicated that group dynamics may be a contributing factor to their preference for face-to-face interactions (Hong, Lai, & Holton, 2003). There were indications that the higher performing groups, investigated in this study, exhibited group dynamics that encouraged positive social interactions within the dialog of their online discussions. These groups exhibited higher levels of camaraderie and their interactions indicated that a social presence existed online and was an extension of their face-to-face meetings.

It is important to understand the differences that exist between high and low achieving groups in their perceptions of their learning experience when online communication is used. Students from high performing groups indicated that they shared leadership responsibilities, distributed tasks evenly and according to their individual talents, and were able to define and understand task responsibilities in an online environment. It has been found that these are contributing factors for productive groups (Hare, 1992). Student perceptions of the process of using online communications to complete a learning project indicates that there are preferences for the types of systems (asynchronous versus synchronous) used and that this depends on the type of task, the expediency necessary for completing the task, and the characteristics of individual group members. There is little research that looks at which of these communications systems are best suited for specific types of group processes. Practitioners who wish to use online communications as a part of their teaching strategies should consider these factors when designing instructional units. Empirical studies that further analyze these factors are necessary to understand their influence on the learning process.

It was found that norms were established within the high performing group in their use of the CMC system of choice. Group members established rules for when to check messages, who took on specific tasks, and for making organizational decisions. Groups tend to establish these norms early and to use the CMC system to establish, organize, and implement tasks based on these rules (Biesenbach-Lucas, 2003). The lower achieving group in this study did not demonstrate this norm setting within the dialog of their postings. Not only is it important that instructors be aware of the formation of these norms, they also must be aware that not all students are prepared to work in collaborative environments. It is important for instructors to introduce skills that are important for effective collaboration (Bosworth, 1994). This may be especially important in an online learning environment. Factors that contribute to successful online collaboration need to be identified and categorized so that practitioners have a better understanding of the knowledge, skills, and dispositions needed by students for effective online collaborative interaction. Identification of these factors will also benefit researchers as they investigate online learning environments and further develop online learning models that are supported by empirical evidence.

Although the primary focus of this research was not on temperament type as a factor in the types of interactions that occur online, it was found that this might be a barometer of which group members initiate and carry out specific types of tasks. The utilization of the Keirsey Temperament Scale student pro-files was considered in the formation of the groups for this study to guarantee that the groups were as heterogeneous as possible. Student characteristics, particularly personality and learning styles, may indicate how well group members interact within, and utilize the CMC systems. It has been suggested that interpersonal interactions are factors that contribute to successful online learning (Rafaeli & Sudweeks, 1997). Student personality types and learning styles may be important considerations in the establishment and development of these interpersonal interactions and attention to these characteristics should be a part of the formation of online collaborative groups. The character makeup of a group is important in how members perform tasks both online and in face-to-face settings. It is, therefore, important that when implementing CMC into the curriculum to consider student characteristics and learning styles (Meyer, 2003). Further research that focuses on these issues and how they influence student perceptions and performance is needed and may very well substantiate some of the preliminary findings of this study.

This study provides implications for practice by suggesting possible methods for the evaluation of online activities by establishing a model for analysis of the interactions that occur online. It also raises an awareness of the potential problems that may impede successful completion of assignments when utilizing online communication. Instructors may need to intervene and facilitate group involvement if it becomes apparent that a group is not actively involved early in an activity. To the instructor, this study pro-vides an understanding that asynchronous and synchronous systems may be better suited for specific learning goals and objectives. To administrators and policy makers this study provides insights into the intricate planning that is necessary for effective online learning. Guidelines for developing strategies for implementing online learning are necessary and an awareness of the difficulties with which instructors are faced when integrating online learning into the curriculum should be acknowledged.

Future research should include empirical studies that focus on the effectiveness of specific CMC systems in the context of specific learning goals and objectives. Comparisons must be made between the type of system (asynchronous vs. synchronous), student performance within these systems, and the usage patterns that are apparent within these systems. Qualitative studies that explore how students function in various CMC systems can pro-vide a richer understanding of the cognitive and affective factors that influence learning in an online environment. Further refinement of interaction analysis models that fit specific learning environments is also needed. It is important to develop a better understanding of how instructors facilitate and involve themselves in the online interactions with their students in ways that are effective in the learning process.

By observing students' online interactions, a better understanding of the strategies used to complete tasks and the collaborative skills necessary to be effective participants in online group work will be gained. As courses are offered via distance technologies, it is important to implement effective collaborative learning activities via online systems. Our research contributes to the existing literature on the use of online computer systems for implementing instructional activities in university courses. It provides insights to the importance of and the differences in how students utilize synchronous and asynchronous systems and to the strategies needed by students to complete course projects successfully. The processes that take place in collaborative learning are complex and this medium provides a vehicle for capturing and analyzing the discourse that takes place in the process of collaboration. In this way it provides researchers interested in collaboration another tool for analyzing the progression and development of collaborative skills.

Table 1 Interaction Analysis Model

Interactions                     Types

Problem Solving                  Identify Problem
Statements that refer to course  Students identify problems and
content, project modules, or     opportunities in the context of the
knowledge/information for tasks  tasks that they might encounter in
to be completed.                 the project.
                                 Define Goals
                                 Students make decisions as to the goals
                                 needed for completion of project tasks.
                                 Explore Strategies
                                 Students determine the strategies that
                                 are needed to complete tasks for
                                 project completion.
                                 Act on Strategies
                                 Students perform the strategies that
                                 are necessary to complete the project
                                 task(s)
                                 Look Back
                                 Students reflect on completed tasks to
                                 determine if their goals are being met.
Socio-Emotional                  Shows Solidarity
Statements that refer to task-   Jokes, raises other's status, gives
specific attributes that         help or reward, shows sympathy/empathy,
reflect personal feelings or     gives praise, shows appreciation,
affective support.               demonstrates commitment to the group.
                                 Shows Antagonism
                                 Deflates other's status or defends/
                                 asserts self. Gives explanation of
                                 reasons a task is not completed on
                                 time. Apologizes for failure to
                                 complete task on time. Takes
                                 alternative action when another member
                                 is unable to complete tasks.
                                 Shows Agreement
                                 Passive acceptance, understands,
                                 concurs, or, complies. Seeks approval.
                                 Shows Disagreement
                                 Passive rejection, formality, or
                                 withholds help.
Other                            Off-task Socio-Emotional
Statements that refer to non     Statements that are socio-emotional and
task-specific attributes.        do not refer to task specific
                                 activities
                                 Organizational
                                 Statements that are related to
                                 facilitating online discussion or group
                                 tasks. May include references to fTf
                                 meetings.
                                 Technical
                                 Statements that refer to technical
                                 difficulties experienced or directly to
                                 issues that concern the system that the
                                 participant is using.

Asynchronous System

             Identify                                     Act
             Problem   Define Goals   Explore Strategies  on Strategies

Planning     33%       23%            27%                 18%
Design        8%       25%            37%                 31%
Development   6%        2%            27%                 65%

Figure 1. Types of Problem-Solving Interactions Across Project Stages

Note: Table made from bar graph.

Asychronous System

             Shows                        Shows
             Agreement  Shows Antagonism  Disagreement  Shows Solidarity

Planning     43%        30%                0%           26%
Design       12%        24%               12%           53%
Development  21%        42%               12%           24%

Figure 2. Types of Socio-emotional Interactions Across Project Stages

Note: Table made from bar graph.

Asychronous System

             Off Task Socio-Emotional  Organizational  Technical

Planning     22%                       48%             30%
Design       12%                       76%             12%
Development  26%                       65%              9%

Figure 3. Types of Other Interactions Across Project Stages

Note: Table made from bar graph.

Table 2 Comparing the Interactions of High and Low Achieving Groups

High                               Low

High frequencies of interactions   Low frequencies of interactions
posting messages almost daily      posting messages intermittently with
                                   several days between each
Identification of problems occurs  Most identification of problems
during the planning stage of the   occurs during the planning stage,
project                            however is still present during the
                                   development phase
Exploration of strategies begins   Exploration of strategies begins in
in the planning phase              the design phase
Acting on strategies evidenced     Acting on strategies evenly
across all stages, but primarily   distributed during the design and
in the development stage           development stages; no evidence
                                   during the planning phase
Socio-emotional interactions       No socio-emotional interactions in
present throughout the project     the planning stage
Organizational interactions are    No organizational interactions until
present throughout the project     the development stage
Thought provoking questions posed  More directive statements rather than
and exchange of ideas flow freely  questions and exchange of ideas is
                                   restricted
More in depth discussions          More surface level discussions

               Total Problem Solving  Total Socio-Emotional  Total Other

Asynchronous   62%                    17%                    17%
  Planning
Synchronous     0%                     5%                    95%
  Planning
Asynchronous   60%                    20%                    20%
  Design
Synchronous    53%                    23%                    21%
  Design
Asynchronous   55%                    17%                    28%
  Development
Synchronous    27%                    18%                    53%
  Development

Figure 7. Comparison of Interactions Between Communication Systems

Note: Table made from bar graph.

References

Alexander, D.S., & DeAlba, L.M. (1997). Groups for proofs: Collaborative learning in a mathematics reasoning course. PRIMUS, 7(3), 193-207.

Berelson, B. (1952). Content analysis in communications research. Glencoe, IL: Free Press.

Berens, L., (1996). Type & temperament. Bulletin of Psychological Type, 19(2), 8-9.

Biesenbach-Lucas, S. (2003). Asynchronous discussion groups in teacher training classes: Perceptions of native and non-native students. Journal of Asynchronous Learning Networks, 7(3). Retrieved November 20, 2003 from http://www.aln.org/publications/jaln/v7n3/v7n3_biesenbach-lucas.asp

Bosworth, K. (1994). Developing collaborative skills in college students. In K. Bosworth & S. Hamilton (Eds.), Collaborative learning: Underlying processes and effective techniques. Number 59, 25-31, San Francisco: Jossey-Bass

Bransford, J. D., & Stein, B. S. (1993). The IDEAL problem solver (3rd ed.). New York: W. H. Freeman and Company.

Bruffee, K. A. (1999). Collaborative learning: Higher education, interdependence, and the authority of knowledge (2nd ed.). Baltimore, MD: Johns Hopkins University Press.

Caplow, J. A. H., & Kardash, C. A. M. (1995). Collaborative learning activities in graduate courses. Innovative Higher Education, 19(8), 207-220.

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-7.

Dansereau, D. F. (1983). Cooperative learning: Impact on acquisition of knowledge and skills. (Report No. 341). Abilene, Texas: U.S. Army Research for the Behavioral and Social Sciences (ERIC Document Reproduction Service No. ED 243 008).

Davies, M. F. (1996). Social interaction. In Hare, A. P. Blumberg, H. H., Davies, M. F., & Kent, V. K. (Eds.). Small groups: An introduction. Westport, CT: Praeger.

Glasser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.

Groccia, J. E., & Miller, J. E. (1996). Collegiality in the classroom: The use of peer learning assistants in cooperative learning in introductory biology. Innovative Higher Education, 21(2), 87-100.

Gunawardena, C. N., Anderson, T., & Lowe, C. A. (1996). Interaction analysis of a global on-line debate and the development of a constructivist interaction analysis model for computer conferencing. Paper presented at The Annual Conference of the American Educational Research Association, New York.

Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology class. Instructional Science. 28(2), 115-152.

Hare, A. P. (1976). Handbook of small group research (2nd ed.). New York: The Free Press.

Hare, A. P. (1992). Groups, teams, and social interaction: Theories and applications. New York: Praeger.

Henri, F. (1991). Computer conferencing and content analysis. In Collaborative learning through computer conferencing: The Najaden Papers. New York: Springer-Verlag.

Hiltz, S. R., Coppola, N., Rotter, N., & Turoff, M. (2000). Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, multi-method approach. Journal of Asynchronous Learning Networks, 4(2). Retrieved July 18, 2002 from http://www.aln.org/alnweb/journal/jaln-vol4issue2.htm

Hong, K., Lai, K., & Holton, D. (2003). Student's satisfaction and perceived learning with a web-based course. Educational Technology & Society, 6(1). Retrieved November 20, 2003 from http://ifets.ieee.org/periodical/vol_1_2003/hong.html

Howell-Richardson, C., & Mellar, H. (1996). A methodology for the analysis of patterns of participation within computer mediated communication courses. Instructional Science. 24(1), 47-69.

Keirsey, D. W. (1998). Please understand me II: Temperament, character, intelligence. Del Mar, CA: Prometheus Nemesis.

Koschmann, T. (1996). Paradigm shifts and instructional technology: An introduction. In T Koschman (Ed.), CSCL: Theory and practice of an emerging paradigm, 1-24, Mahwah, NJ: Lawrence Erlbaum.

Lattuca, L. R., & Stark, J. S. (1995). Modifying the major: Discretionary thoughts from ten disciplines. The Review of Higher Education, 18(3), 315-344.

Lee, F., Liang, S., & Chan T. (1999). An attempt to design synchronous collaborative learning environments for peer dyads on the World Wide Web. Journal of Educational Computing Research, 21(2), 221-253.

Lundgren, C. C (1977). Trainer style and member attitudes toward trainer and group in t-groups. Small Group Behavior, 8(1), 47-62.

Magolda, M. B. M. (1992). Students' epistemologies and academic experiences: Implications for pedagogy. The Review of Higher Education, 15(3), 265-87.

McComb, M. (1994). Benefits of computer-mediated communication in college courses. Communication Education, 43, 159-183.

McDonald, J., & Gibson, C. C. (1998). Interpersonal dynamics and group development in computer conferencing. The American Journal of Distance Education, 12(1), 7-25.

Meyer, K. A. (2003). Face-to-face versus threaded discussions: The role of time and higher-ordered thinking. Journal of Asynchronous Learning Networks, 7(3). Retrieved November 20, 2003 from http://www.aln.org/publications/jaln/v7n3/v7n3_meyer.asp

Michaelsen, L. K. (1983). Team learning in large classes. In C. Bouton and R. Y. Garth, (Eds.), Learning in Groups, (pp. 13-22). San Francisco: Jossey-Bass.

Morgan, A. R. (1987). Project work in open learning. In M. Thorpe and D. Grugeon (Eds.), Open learning for adults, (pp. 245-251). Harlow, London: Longman.

National Research Council. (1996). From analysis to action: Undergraduate education in science, mathematics, engineering, and technology. Washington, DC: National Academy Press.

National Science Foundation (1996). Shaping the future: New expectations for undergraduate education in science, mathematics, engineering, and technology. Washington, DC: Report by the Advisory Committee to the National Science Foundation Directorate for Education and Human Resources.

Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

Pascarella, E., & Terenzini, P. (1998). Studying college students in the 21st century: Meeting newchallenges. Review of Higher Education, 21, 151-165.

Picciano, A. (2002). Beyond student perceptions: Issues of interaction, presence, and performance in an online course. Journal of Asynchronous Learning Networks, 6(1). Retrieved November 20, 2003 from http://www.aln.org/publications/jaln/v6n1/v6n1_picciano.asp

Polya, G. (1957). How to solve it. Garden City, NY: Doubleday Anchor.

Rafaeli, S., & Sudweeks, F. (1997). Networked interactivity. Journal of Computer-Mediated Communications, 2(4). Retrieved July 18, 2002 from http://www.ascusc.org/jcmc/vol2/issue4/rafaeli.sudweeks.html

Richardson, J. C., & Swan, K. (2003). Examining social presence in online courses in relation to students' perceived learning and satisfaction. Journal of Asynchronous Learning Networks, 7(1). Retrieved November 20, 2003 from http://www.aln.org/publications/jaln/v7n1/v7n1_richardson.asp

Slavin, R. E. (1995). Cooperative Learning (2nd ed.). Needham Heights, MA: Simon and Schuster.

Smith, K. A. (1987). Educational engineering: Heuristics for improving learning effectiveness and efficiency. Engineering Education, 74, 274-279.

Spear, K. (1989). Sources of strain in liberal education. The Review of Higher Education, 12(4), 389-401.

Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69(1), 21-51.

Wertheimer, M. (1959). Productive thinking. New York: Harper and Row.

Yang, H., & Tang, J. (2003). Effects of social network on students' performance: A web-based forum study in Taiwan. Journal of Asynchronous Learning Networks, 7(3). Retrieved November 20, 2003 from http://www.aln.org/publications/jaln/v7n3/v7n3_yang.asp

W. RANDALL THOMAS

East Baton Rouge Arts and Technology School, LA, USA

[e-mail]

S. KIM MACGREGOR

Louisiana State University, Baton Rouge, LA, USA

[e-mail]