A comparative study of computer conferencing and face-to-face communications in systems design.

The purpose of this study was to investigate the use of computer conferencing and face-to-face communications to support the work of student teams in a systems analysis project. The study compares the differences in decision quality associated with the projects accomplished by these teams, each team's confidence in the group decision, and each team's satisfaction with the overall decision process.

The research questions were: (a) will the computer conferencing groups produce solutions of higher decision quality, as measured by group scores for decision quality, analytical strength, technical strength, research base for drawing conclusions, and the effectiveness of the group presentation; and (b) will the face-to-face groups feel greater confidence in the group decision and greater satisfaction with the group decision process, because of the personal relationships that have been developed, as measured by confidence in the group decision, satisfaction with the group decision process, and peer evaluations of participation of their teammates and effectiveness of their teammates?

Based upon the findings, computer conferencing afforded opportunities for handling evaluative tasks more effectively than face-to-face meetings. Computer conferencing offers the benefits of producing better decisions because of broader participation, expression of a wider range of opinions, and greater analysis. Computer conferencing can lower time pressures and create the psychological distance among participants needed to engage in more open, candid exchanges of opinions. Its shortcomings are the time it takes to exchange messages, the increased difficulty of coordinating and clarifying ideas, and the increased time it takes to develop a consensus and to arrive at a decision.

**********

The use of computer conferencing and online instructional materials is becoming increasingly popular within institutions of higher education. Many colleges and universities are developing and offering online courses and are supplementing traditional courses with online instructional materials.

The impacts of using electronic communications and online computer conferencing are still unclear. Some of the research on the impacts of new communications technology is based upon the perceptions of the participants, and there are mixed results regarding the impact of electronic communications on decision quality, consensus-building, team participation, and team effectiveness. In addition, some of the research in this field is based upon experiments and tasks which are not integrated into the actual assignments and performance expectations of a course or academic program.

The purpose of this study is to compare the progress and performance of teams using computer conferencing versus face-to-face communications in accomplishing a systems design task in a senior level information systems design project class. The activities, perceptions, and results are based upon class assignments which were accomplished as a part of course requirements, and which are consistent with the actual design tasks performed by business systems analysts.

REVIEW OF RELATED LITERATURE

A review of the literature can help to provide background into what have we learned with respect to group processes in computer conferencing versus face-to-face communications. In an experiment to compare effectiveness in requirements determination using computer conferencing versus face-to-face communication, Couger, Higgins, and McIntyre (1993) found that that computer conferencing promoted equality of participation, greater expression of ideas, and development of a "group" memory. In addition, electronic communications provided more effective, interactive communications, better coordination, and extended the definition of a meeting by increasing the time for accomplishing work.

In another study comparing face-to-face and computer-mediated communication, Adrianson and Hjelmquist (1991) studied the degree of consensus, communication pattern, attitudes to media, and personality. They found no differences in problem-solving efficiency, equality, or dominance, but concluded that face-to-face communications induced more conformity and opinion change as compared to computer-mediated.

Another study by Gallupe and McKeen (1990) determined the effects of the use of a Group Decision Support System (GDSS) on face-to-face versus remote meetings. Use of the GDSS increased the time it took groups to reach a decision, yet had no impact on the decision quality of either the face-to-face or the remote groups. However, the GDSS groups supported by computer conferencing had greater confidence in the decision. In their study of GDSS groups, George, Easton, Nunamaker, and Northcraft (1990) found that these groups were less likely to reach consensus, took more time to reach a decision, and had more equal levels of member participation than manual groups.

In their studies of groups using face-to-face communications and computer conferencing, Hiltz, Johnson, and Turoff (1986) found that groups were less likely to reach agreement in the computerized conferencing mode, and that there were proportionately more of the types of task-oriented communication associated with decision quality in the computerized conferences. However, group decisions were equally good in both modes.

In their examination of group processes involving student teams, Siegel, Dubrovsky, Kiesler, and McGuire (1986) found that social equalization was higher in computer conferencing groups, in that group members participated more equally in discussions, and also that computer-mediated groups exhibited more uninhibited behavior. Computer conferencing groups made fewer remarks than face-to-face and took longer to make decisions. Decisions of computer-mediated groups shifted further away from members' initial individual choices than group decisions which involved face-to-face discussions. In another study, Weisband (1992) found that the content and tone of electronic discussions was quite different from face-to-face discussions, but the process of group decision-making was about the same.

While many of the studies involved experimental situations, a study by Wilson, Morrison, and Napier (1997-1998) was based upon the use of computer conferencing in the work of student project teams in a software engineering course. Students were randomly assigned to three-member project teams, and each team conducted generation, execution, and choice tasks. The findings indicated that computer conferencing supported generation tasks better than execution and choice tasks, and that face-to-face communications supported execution tasks better than generation tasks. Face-to-face communications supported socialization the most effectively.

Finally, a more recent study by Ocker, Hiltz, Turoff, and Fjermestad (1995-1996) found that computer conferencing had a marginally higher effect on solution quality in systems design teams consisting of graduate students in business and computer science. In addition, the computer conferencing groups were judged to be considerably more creative.

In conclusion, some of these overall findings can be summarized. In most of the studies to date, the use of computer conferencing versus face-to-face communications doesn't have a measurable impact on decision quality. Computer conferencing may promote greater equality of participation, more extensive opinion-giving, greater expression of ideas, and greater creativity, while face-to-face groups may build closer social relationships and experience greater conformity.

In most of these studies, the findings are based upon the perception of the participants, rather than objective performance measures. The task situations are often experimental, rather than being an integral part of class assignments and course expectations. In the following study, the findings are based upon actual performance measures acquired as a part of an actual information systems design project assignment. The results may help to identify some of the experiences and results in an online versus a traditional class environment.

Objective

The objective of this study is to investigate the use of computer conferencing and face-to-face communications to support the work of student teams in a systems analysis project. In particular, the study compares the differences in decision quality associated with the projects accomplished by these teams, each team's confidence in the group decision, and each team's satisfaction with the overall decision process.

Research Questions

Research Question 1. Will the computer conferencing groups produce solutions of higher decision quality, as measured by group scores for decision quality, analytical strength, technical strength, research base for drawing conclusions, and the effectiveness of the group presentation?

Research Question 2. Will the face-to-face groups feel greater confidence in the group decision and greater satisfaction with the group decision process, because of the personal relationships that have been developed, as measured by confidence in the group decision, satisfaction with the group decision process, and peer evaluations of participation of their teammates and effectiveness of their teammates?

RESEARCH METHOD

Setting and Subjects

The class project was conducted in a senior-level information systems design project class. Students were randomly assigned to 16 teams, with three members per team. Eight of the teams conducted the project activities using face-to-face (FF) communications, and eight of the teams conducted the project activities using computer conferencing (CC).

The project involved the evaluation of alternative software supporting a mapping application designed for marketing analysis purposes. The project was conducted for a large consumer goods company which was interested in having an external evaluation of alternative mapping packages. As can be seen from the detailed project assignments, each group was assigned a mapping package and conducted their investigation in four parts, including (a) establishment of selection parameters for their package; (b) establishment of weights for each of these parameters; (c) use of the weighted factor method of evaluation to assess their package; and (d) team selection of the best three packages to support the user's application requirements.

THE TASK

Each of the teams was required to reach consensus on the four parts of the investigation of alternative design options within specific time frames. The CC groups were given time parameters for completing each phase of the project and could exchange e-mail throughout this time frame. The FF groups participated in scheduled meetings, spaced one week apart (30-40 minutes for phase one, phase two, etc.). The CC groups kept a log of their e-mail messages and outcomes for each phase of the project, while the FF groups submitted tapes of each of the group meetings and the outcomes of each phase.

Detailed Assignments

Directions for groups using traditional meetings. You will set up four 45-minute to one-hour meetings to accomplish the four parts of the project. Each meeting should be tape recorded to maintain an ongoing record. A recorder will be responsible for taping each session and for submitting the tapes at the end of the project. At the beginning of each session, each participant should introduce/identify themselves.

Directions for groups using electronic mail. You will communicate with your team members using e-mail to accomplish these project objectives. The phases of the project must be accomplished within these time frames. When each team member sends an e-mail message, he/she must copy it to the other two team members. One team member will serve as a "recorder" and keep a file of all of the e-mail messages to be submitted along with the projects described.

The first assignment. Between April 6 and April 8:

Objectives: Establish Selection Parameters, which are specific criteria to use in determining the value of the various mapping software packages. You can use the list of advantages and disadvantages compiled by the various groups as a starting point 

Output: List of Selection Parameters

The second assignment. Between April 8 and April 13:

Objectives: Establish Weights for the Selection Parameters

Output: Weights for the Top-Level Parameters

The third assignment. Between April 13 and April 15:

Objectives: Use the weighted factor approach to score your mapping package (Multiply the weights times the scores for your package to develop the overall weighted average for your package)

Output: Weighted factor summary for your package (spreadsheet)

The fourth assignment. Between April 20 and April 22:

Objective: Use the team presentations for each mapping package to determine the best packages.

Output: Identify your first, second, and third choice and provide reasons for your selections.

Team reports. Tuesday, April 27 (in-class)

INDEPENDENT AND DEPENDENT VARIABLES AND MEASURES

The independent variable was the mode of communication (CC vs. FF) being used by the teams. The dependent variables involved decision process and decision perceptions. Decision process was measured by group scores, as shown:

Group scores:

* Decision Quality: 1 to 5 scale (5 = high quality)

* Analytical Strength: 1 to 5 scale (5 = high analytical score)

* Technical Strength: 1 to 5 scale (5 = high technical score)

* Research Base: 1 to 5 scale (5 = high/well-researched)

* Group presentation: 1 to 5 scale (5 = excellent)

Decision perceptions were measured by team member scores for confidence in the group decision, satisfaction with the group decision process, and peer evaluations of teammates' participation and effectiveness. These measures are described below. In addition, team members were asked to give their views of the advantages and disadvantages of the media, either FF or CC, which they used to conduct the activities of their group.

Individual measures:

* Confidence: 1 to 7 scale (7 = high confidence in group decision; 1 = low)

* Satisfaction: 1 to 7 scale (7 = high satisfaction with group decision process; 1 = low)

* Teammate 1 participation (5 = excellent; 1 = limited)

* Teammate 1 effectiveness (5 = high effectiveness of contribution; 1 = low)

* Teammate 2 participation (5 = excellent; 1 = limited)

* Teammate 2 effectiveness (5 = high effectiveness of contribution; 1 = low)

Other factors which were measured for each student included GPA, the grade point average, the grade earned in CMIS 470, and the individual grade earned on the project report.

Results

The results revealed some new findings with respect to the impact of computer conferencing upon group decision-making, team effectiveness, and decision quality. First, the decision-making mode did not seem to influence team members' assessment of their confidence in the decision. Both the members of the CC teams and the members of the FF teams expressed relatively high confidence in their group decisions, as shown in Table 1.

With respect to satisfaction with the decision-making process, a higher percentage of the members of the face-to-face groups reported high satisfaction, as compared with the computer conferencing groups (Table 2).

This might be explained by the relationship-building opportunity created by the face-to-face meetings. In our analysis of the team members' perceptions of the advantages and disadvantages of face-to-face versus computer conferencing (see Tables 3 and 4), a number of members of computer conferencing groups commented about the lag times in exchanging messages, server downtime, and difficulties with computer access. These factors may have accounted for some of the dissatisfaction expressed by CC team members. Table 3 gives further insight into the FF team members' perceptions of the benefits of face-to-face communications.

Most of the advantages pertain to having an opportunity to clarify ideas, to establish personal contact, to gain immediate feedback, and to benefit from convenient scheduling. Table 4 summarizes the perceived advantages and disadvantages of computer conferencing.

While CC team members commented on convenience and ease of use of e-mail, they also mentioned delayed response time, technical bottlenecks, and lack of personal interaction as drawbacks.

Performance Measures

The performance measures of each team's accomplishments included decision quality, analytical strength, technical strength, research base for drawing conclusions, and the effectiveness of the team presentation. As you can see from Table 5, the teams using computer conferencing to accomplish project activities performed significantly better on all of these dimensions of performance.

This could be explained in a number of ways. First, electronic communications provided an ongoing "group memory" which was shared among the participants. Given the electronic record they shared, the CC groups may have been able to coordinate and analyze the technical details more effectively. Second, there is some indication from the electronic transcripts of computer conferences that the members spent more time developing and analyzing factors and alternatives as a part of the decision-making process, and this may have contributed to the overall effectiveness and comprehensiveness of their results. Third, since the team members were randomly assigned to groups, these results seem to be related to the process of decision-making afforded through the use of computer conferencing.

PEER EVALUATIONS OF TEAM MEMBERS

Team members evaluated their peers on two dimensions, including "participation" on the team and "overall effectiveness of the contribution." The findings show that the team members in the computer conferencing groups evaluated their teammates slightly higher in terms of participation, but that both groups evaluated their peers in a similar fashion with respect to effectiveness of the contribution.

The interesting aspect of this finding is that the team members in both the computer conferencing and face-to-face groups evaluated each other highly with respect to the effectiveness of their contributions, even though the actual results (decision quality, analytical strength, technical strength, research base, and group presentation) indicate that the actual performance results of the computer conferencing teams was superior on all dimensions (Table 6).

One explanation for this finding is that perceived quality of the output of the face-to-face teams was greater than the actual quality of the output of these teams. This may be related to the fact that face-to-face team participants enjoy personal interactions which may yield higher perceived confidence levels in decision-making, even though these confidence levels may not be well-grounded in the actual results. This finding could also be related to evidence that the computer-conferencing members could organize and manage a great deal of technical detail in the process of sharing their results with each other, and perhaps this systematic approach and shared record helped them to develop a more thorough analysis and evaluation of the alternative design options.

DISCUSSION AND CONCLUSIONS

The first research question asked: Will the computer conferencing groups produce solutions of higher decision quality, as measured by group scores for decision quality, analytical strength, technical strength, research base for drawing conclusions, and the effectiveness of the group presentation?

With respect to this question, the researchers found that the teams using computer conferencing achieved higher performance results on all of these dimensions. One reason may be that computer conferencing creates sufficient "psychological" distance among participants, so that they express their opinions in greater detail and are less likely to be party to the "group think" phenomenon that often emerges in group face-to-face meetings and can be used to reinforce a consensus prior to in-depth analysis, discussion, and evaluation.

Participants in the computer conferencing mode also have opportunities for greater depth of expression, detailed data collection, an expression of alternative views, and an ongoing recording of discussions-all of which might be conducive to a more rigorous evaluation of the problem and a more complete recording of the results.

The second research question was: Will the face-to-face groups feel greater confidence in the group decision and greater satisfaction with the group decision process, because of the personal relationships that have been developed?

The members of the computer conferencing and face-to-face teams evaluated their confidence in the decision similarly, even though the face-to-face members reported greater satisfaction with the decision-making process. The higher level of satisfaction may be attributed to the personal relationships developed through the meetings, and opportunity to more quickly bring the process to closure. The lesser degree of satisfaction reported by the participants on the computer conferencing teams was largely related to the time lag between communications and the technical complications. In addition, some team members found it difficult to bring discussions to closure. These findings are consistent with the findings of similar studies.

There are some unique contributions of this study. Based upon these findings, computer conferencing afforded opportunities for handling evaluative tasks more effectively than face-to-face meetings. Computer conferencing offers the benefits of producing better decisions because of broader participation, expression of a wider range of opinions, and greater analysis. Computer conferencing can lower time pressures and create the psychological distance among participants needed to engage in more open, candid exchanges of opinions. Its shortcomings are the time it takes to exchange messages, the increased difficulty of coordinating and clarifying ideas, and the increased time it takes to develop a consensus and to arrive at a decision.

In addition, face-to-face meetings may be less effective in performing the complex tasks of evaluation, because of the pressures of time and the participants' psychological resistance to expressing opinions which were contrary to those of their teammates. Getting everyone together at a fixed time and place can be restrictive and does not leave open the possibility of new ideas which members wish to share with other team members outside of meeting times. Sometimes, time constraints place pressure on team participants to go along with the majority opinion, rather than raising potentially time-consuming and controversial issues.

These findings provide some basis for the increased relevance of using online computer conferencing to enable remote work groups to accomplish tasks requiring ongoing analysis, discussion, and evaluation. This helps to reinforce the trend toward using online resources to augment instruction, and makes an effective argument that excellent learning results can be achieved in the process of student collaboration in online environments.

Table 1

Decision-Making Mode and Confidence in the Decision

                       Low Confidence  High Confidence

Face-to-Face                29%              71%
Computer conferencing       30%              70%
Table 2

Types of Decision-Making Mode and Satisfaction with the Process

                       Low Satisfaction  High Satisfaction

Face-to-face                 25%                75%
Computer conferencing        60%                40%
Table 3

Face-to-Face Decision-Making: Perceived Advantages and Disadvantages

                                Count  Percent

Perceived Advantages

Opportunity to clarify ideas
     "talk out details"
     "clarify ideas"
     "reduce confusion"
     "develop common focus"      14      35%
Personal contact
     "personal contact"
     "give and get"
     "get acquainted"
     "positive interaction"       9      22%
Speed of decision-making
     "quick response"
     "immediate information"
     "faster"                     9      22%
Convenience
     "scheduled"
     "easy prep"
     "convenience"                8      20%
Total                            40     100%

Perceived Disadvantages

Inconveniences associated with
fixed meeting times
     "meeting times"
     "schedule conflicts"
     "missing members"           24      74%
Miscellaneous responses           9      27%
Total                            33     100%

Note. There are 24 participants and they had the option of listing up to
three advantages and three disadvantages. The totals are based on the
total number of advantages and disadvantages offered by these
twenty-four participants.
Table 4

E-mail Conferencing: Perceived Advantages and Disadvantages

                                    Count  Percent

Perceived Advantages

Convenience/ Ease of Use
  "easy access"
  "user friendly"
  "no time wasted"
  "work at home"
  "no schedule"
  "24 hour a day access"             29      76%
Miscellaneous responses               9      24%
Total                                38     100%

Perceived Disadvantages

Response time + technical
 complications
  "waiting for reply"
  "communication slow"
  "slow response"
  "response time"
  "mail in and out"
  "servers down"
  "busy lines"
  "computer access"                  21      55%
Barriers to clarification of ideas
  "bad discussion"
   hard to "express ideas"
  "hard to make decisions"            7      18%
Lack of personal interaction
  "no face to face"
   no "getting in touch"
  "no group talk"                     4      11%
Miscellaneous responses               6      16%
Total                                38     100%
Table 5

Face-to-Face versus E-mail Conferencing: Means on Five Performance
Measures

Dimensions of Performance  Face to Face  E-conference  Difference

Decision Quality (1-5)         2.67          3.85         1.18
Analytical Strength (1-5)      2.33          3.65         1.32
Technical Strength (1-5)       2.42          3.30         0.88
Research Base (1-5)            2.37          3.40         1.03
Group Presentation (1-5)       2.75          3.85         1.10

Composite Score               12.54         18.05         5.51

Note. Mean scores are calculated for 24 individuals in the face-to-face
groups and 20 individuals in the e-mail conferencing groups.
Table 6

Teammates' Scores Below the Top Score

Decision Mode        Participation Scores        Effectiveness Scores

                     Count         Percent     Count        Percent

Face-to-Face          7(48)          14%       10(48)         21%
E-mail Conferencing  12(40)          30%       11(40)         27%

Note. Participants scored each of their teammates on two dimensions:
"participation on the team" (1-5) and "overall effectiveness of
contribution" (1-5). Consequently, there are 88 participation scores and
88 contribution scores.

References

Adrianson, L., & Hjelmquist, E. (1991). Group processes in face-to-face and computer-mediated communication. Behavior and Information Technology, 10(4), 281-296.

Couger, J.D., Higgins, L.F., & McIntyre, S.C. (1993). Unstructured creativity in information systems organizations, MIS Quarterly. 17(4), 375-397.

Gallupe, R.B., & McKeen, J.D. (1990). Enhancing computer-mediated communication: An experimental investigation into the use of a group decision support system for face-to-face versus remote meetings. Information and Management, 18(1), 1-13.

George, J.F., Easton, G.K., Nunamaker, J.F., & Northcraft, G.B. (1990). A study of collaborative group work with and without computer-based support. Information Systems Research, 1(4), 394-415.

Hiltz, S., Johnson, K., & Turoff, M. (1986). Experiments in group decision-making, I: Communications process and outcome in face-to-face computer conferences. Human Communication Research. 13(2), 225-252.

Ocker, R., Hiltz, S.R., Turoff, M., & Fjermestad, J. (1995-96). The effects of distributed group support and process structuring on software requirements development teams: Results on creativity and quality. Journal of Management Information Systems, 12(3), 127-153.

Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T. (1986). Group processes in computer-mediated communication. Organizational Behavior and Human Decision Processes, 37(2), 157-187.

Weisband, S. (1992). Group discussion and first advocacy effects in computer-mediated and face-to-face decision-making groups. Organizational Behavior and Human Decision Processes, 53(3), 352-380.

Wilson, E.V., Morrison, J.P., & Napier, A.M. (1997-1998). Perceived effectiveness of computer-mediated communications and face-to-face communications in student software development teams. Journal of Computer Information Systems, 39(2), 2-7.