Piloting participatory designing within a collaborative learning environment.
The present study focuses on analyzing university students' collaborative designing may be supported by an environment for computer-supported collaborative learning (CSCL) and how participatory designing may be applied in an educational context. The study examines university students' collaborative design process as it occurred in a complex and authentic design task--designing EuroCSCL 2001 conference bags--and explores how students worked in the virtual design environment to jointly advance their design knowledge, obtaining and applying knowledge provided by avid conference-goers about their previous experiences of the conference bags.
The research and development of networked design environments and computer support for collaborative design have become major areas of design research (Gabriel & Maher, 2000; Hennessy & Murphy, 1999; Kvan, Yip, & Vera, 1999; Maher, Simoff, & Cicognani, 2000; Seitamaa-Hakkarainen, Raunio, Raami, Muukkonen, & Hakkarainen 2001). The emergence of computer-supported environments for collaborative learning and working encourages one to explore possibilities of teamwork in designing and learning in virtual design studio settings. Collaboration and teamwork are essential aspects of professionals' practical activity in the field of modern design (Ferguson, 1992; Nonaka & Takeuchi, 1995). In an earlier study, the investigators analyzed knowledge exchange between students of design teams and described students' roles and contributions to knowledge-building discourse in the design process (Seitamaa-Hakkarainen, et al., 2001). The authors of the present study understand collaborative designing as a process of actively communicating and working together on a design task, jointly determine design constraints, and coordinate efforts to create a shared design product (see Dillenbourg, Baker, Blaye, & O'Malley, 1996; Lahti, Seitamaa-Hakkarainen, & Hakkarainen, 2001).
The core curriculum in any design field consists of working in design studios. Studio-based instruction is considered to be the axis of architectural education (Akin, 2002; Kvan, 2001; Schon, 1987). Design studio courses provide ample opportunities to develop higher-level design competencies through conducting projects that simulate "real world" design scenarios. An effective design program should educate students to work in teams that consist of participants representing multiple professions and domains. A program should be based on practical and workable pedagogical models adequately embedded in practical educational contexts that help teachers to find meaningful ways of using the new technology to support collaborative designing (Kvan, 2001; Mailer et al., 2000).
According to Kvan (2001), from the virtual design studios, as compared with in-the-flesh ones, there emerge much higher demands on students to take responsibility for planning, self-regulating, and self-organizing their design activities. It is important not to leave students working without guidance in the design process. A teacher is needed to structure the collaborative efforts and provide advice for the students in virtual design studios, just as in traditional ones. Modeling and demonstration by the teacher are known to be crucial in facilitating the development of students' design expertise.
The idea of giving an expert or knowledgeable user an important role in designing follows from the notion of participatory designing. As the complexity of proposed artifacts increases, involvement of the users becomes more and more important to ensure usability of the product. The design of a successful product is dependent upon incorporating information from a number of knowledge domains; it must perform at a functional level, meet aesthetic criteria, be economically produced, and moreover, it should be safe. Participatory design emphasizes the importance of acquiring relevant information from the end user of the product and ensuring that the product manifests the end user's viewpoint. The designer has to obtain knowledge of users and their activity, and thus the design project should try to include users as a part of the project team from the very beginning (Popovic, 1999).
Besides users' roles, teachers' roles have not been analyzed closely. According to Wells (2000), teachers--in general--may be either coinquirers or organizers of a learning community's activities. Accordingly, one may recognize corresponding roles of design teachers participating in designing, and talk about (a) teachers as codesigners or (b) teachers as leaders and organizers of the process. We maintain that the same two roles may be played by users.
The emergence of computer-supported learning environments promises to provide new resources for participatory design. A fundamental aspect of these environments is to provide tools for progressive discourse interaction between the designers and users. The new collaborative technology allows design thinking to be more explicit and easier to access by one's fellow designers, and it also enables participating designers to share their ideas and to construct a joint understanding of the design problem and its possible solutions. However, it is not easy to make thinking visible, share, and build ideas with others. The use of collaborative technology may fail due to pedagogical, technological, or organizational obstacles (Lipponen, 1999; Sikkel, Gommer, & van der Veen, 2001). Collaborative technology enables the users to rely on socially distributed intellectual resources embedded in the learning community while conducting various projects (Maher et al., 2000; Seitamaa-Hakkarainen, Lahti, Muukkonen, & Hakkarainen, 2000). A central aspect of collaborative knowledge building is to create shared knowledge objects, such as ideas or product designs, to engage in an intensive synchronous and asynchronous interaction among the participants for further developing and building on those objects, and to reuse the emerging knowledge later on for solving new problems or reflecting on the process (Bereiter, 2002).
To participate in expert-like processes of collaborative designing, the participants' activities need to be structured and scaffolded in many ways. Figure 1 schematically depicts various aspects of collaborative designing and provides a model of distributed expertise in the collaborative design process (Seitamaa-Hakkarainen, et al., 2001). The purpose of the model is more to illustrate essential aspects of collaborative designing than to provide a fixed or inflexible design framework. However, this model has been our framework, as educators, for providing cognitive scaffolding that encourages students to engage in expert-like designing in computer-supported collaborative work. By categorizing and labeling design notes posted to virtual design environment, the designer is guided to specify the design idea, generate and articulate multiple working ideas, evaluate those critically, search for new information, comment on fellow designers' design process, and share the whole process with the other members of the design team including the user of the product in question.
[FIGURE 1 OMITTED]
The present study relies on a web-based environment, the Future Learning Environment (FLE2, an open source version of a new generation FLE software is available at http://fle3.uiah.fi), which is a networked learning environment providing a distributed database designed to support collaborative designing (Leinonen, Mielonen, Seitamaa-Hakkarainen, Muukkonen, & Hakkarainen, 1999). The FLE system was developed by the Media Laboratory, University of Art and Design Helsinki in collaboration with the Department of Psychology, University of Helsinki. The FLE2 provides tools for collaborative designing, so that multiple actors can asynchronously work and communicate through discourse and visual representations in the virtual learning environment. The users are able to access the FLE2-environment from any Internet-linked computer, and they are able to work with the FLE2 database using their standard office applications, to produce documents in various formats, such as text, graphics, video, or WWW-links. The FLE2-environment consists of several modules that facilitate collaborative building of knowledge and aid the design process, and it provides "thinking tools" for organizing and structuring these processes. The system provides tools for discussing ongoing design processes as well as sharing concepts and visual design ideas. The main modules of the FLE2 include the Virtual WebTop, the Knowledge Building Module and the Library. Figure 2 presents the Virtual Web Top.
[FIGURE 2 OMITTED]
The Virtual WebTop refers to a personal adjustable display window, which is automatically opened as the user logs onto the system. The Virtual WebTop contains graphical links, represented by folders, which give information about the course attended, and provide links to the Virtual WebTops of the other participants of the group. The Virtual WebTop is a place for the user to store the documents created by standard office applications in various formats. In the background of the Virtual WebTop is a metaphor of an open office in which one can go to work at any desk, observe what other people are working on, and leave your own notes. Accordingly, the Virtual WebTop allows a student to visit the WebTops of other students or teachers in the same course.
The Knowledge Building (KB) Module provides a shared space for working together to solve problems and develop ideas and thoughts generated by the users. All KB messages within a project are posted to the shared space, and are visible as lists of messages; the messages produced to the database can include attached files such as a text, graphics, video or WWW-links. In the conceptual phase of designing, the module supports generation, externalization, and explication the design ideas and determination of the design constraints. A user of FLE2 preparing a note for posting is asked to label that note by choosing a Thinking-Tool label (i.e., design problem, design idea, comment, deepening knowledge, metacomment, and summary). By categorizing design notes posted to the KB-module according to the model of collaborative designing (see Figure 1), the user is guided to specify the design ideas, generate, and articulate multiple ideas, and evaluate them critically, search for new information, comment on fellow designers' process and share the whole process with the other members of the team. Figure 3 presents the Knowledge Building (KB) Module.
[FIGURE 3 OMITTED]
In this study, the role of collaborative technology in supporting user participation in the design process is examined by reporting results from an experiment in using FLE2 in a university-level design course. The problem addressed was to analyze how the teacher's and users' participation supports the collaborative design process undertaken by students working in the networked learning environment.
Participants and Data Collection
The design project took place at the University of Joensuu, Savonlinna, Department of Teacher Education. The design task was quite challenging (i.e., authentic and complex); the students were asked to design and produce functionally and aesthetically delightful conference bags for the EuroCSCL conference. The participants of the study were students in the first year of a training program for textile teachers at the university level. They worked in six design teams, consisting of four students in each team. Each team had its own named "user" (i.e., an avid conference goer). The students were instructed to intensively collaborate within their own design team and to search for deepening knowledge from the users of the product. There were three tutors who helped students become accustomed to using the FLE2-environment, and one of them sent messages to the FLE2. One and the same teacher (the second author) gave all lectures and participated in the FLE2-environment. During the project, the students received some background information, such as a design brief, EuroCSCL Conference WWW pages and research articles concerning participatory design in general. Moreover, pictures and examples products of the various kinds of conference bags were included in the FLE2-course information context.
The present study followed a method of design-studio teaching that had the following structure: a statement of the design problem and definition of the design context in the form of a brief, the understanding of the brief, and explorations of the users' needs, presentation of the design ideas verbally or graphically and reviews of proposals. The course consisted of weekly lectures, face-to-face design sessions during the lectures, and individual/ team designing in the FLE2-environment between the lectures (Table 1). The teacher expected the virtual design studio to support teamwork outside the lectures and increase users' possibilities to actively participate in the students' design processes. All users lived far away from the course setting (Savonlinna). This study was based on an analysis of the participants' productions posted to FLE2's database. The face-to-face videotaped interactions from two design teams and the questionnaires about students' attitudes and use of the FLE2-environment will be reported elsewhere.
The students received the design task in the second lecture, and at that time they were introduced to using the FLE2-environment. They were informed that the productions of each whole group, rather than the contributions of individual students would be assessed. To provide feedback about functional aspects of the proposed conference bags, the students' designs were evaluated twice: (a) the students' visual ideas were evaluated by the team of users and the teacher, and (b) the prototypes were evaluated by the jury. The students also engaged in technical designing and tested their technical solutions in the textile laboratory.
Methods of Data Analysis
The present study investigates aspects of teacher's and users' roles in a participatory design process. We were interested in exploring how teacher and users worked in the virtual design environment to jointly advance students' design process. The following results are based on a qualitative analysis of the participants' design discourse as recorded in the database of the virtual design environment. The methods of social network analysis were applied to study interaction between the students, teacher, and the users in the FLE2 database. A qualitative content analysis was carried out by analyzing the design thinking, activities, and interaction between the students', teacher's, and the users' statements posted to that database.
Analysis 1: Social Interaction in Networked Designing
Methods based on social network analysis (Scott, 1991) were used to study participants' social position in the collaborative design process. The same methods have been used in other studies to analyze patterns of social interaction in CSCL (e.g., Palonen & Hakkarainen, 2000). The participants' positions in the networked discussions were analyzed using Freeman's degree as a centrality measure. Cognitive centrality describes the importance or isolation of a member in the communication network. The degrees were counted from the number of messages that the participants sent to others (outdegree) or received from others (indegree) in FLE2. Cognitive centrality of the participants was further assessed by using Freeman's "betweenness" value that indicates the participant's position in mediating knowledge flow within the community (Borgatti, Everett, & Freeman, 1999). An actor has a high betweenness value if he or she interacts with actors of the network who do not have direct network connection with each other. To have a large "betweenness" centrality, the actor has to lie in the shortest path between several, otherwise separated actors in the network.
The measure of density was used to evaluate the general level of interaction in the design teams. Density is a simple way to measure a network: the more actors that have relationships with one another, the denser will be the network (Scott, 1991, p. 74); hence it indicates, here, a measure of the intensity of interaction among the participants. Density was computed from a dichotomized matrix of KB messages (the participants had or had not sent or received messages to each other, the frequency of messages did not matter) and, thus, it could vary from 0 to 1. The calculation of density of the design teams took account of all participants' contributions. All social networked analyses were performed using the UCINET program (Borgatti, et al., 1999).
Analysis 2: Participants' Role in Design Process
To analyze the design process in FLE2, the investigators considered students', teachers's, and users' design thinking, design activities, and their exchange of information. The knowledge-building messages posted to the database were segmented into propositions representing one main idea (Chi, 1997). All text in each message was first divided into segments, each representing a separate idea. Two independent researchers were used to assess inter-coder reliability of the segmentation. To analyze the reliability of segmentation, an independent coder classified approximately 15% of messages. The inter-coder reliability of segmentation was .85, indicating that the reliability of segmentation was satisfactory.
The qualitative content analysis was based on the classification that was developed in our earlier study, which analyzed university students' collaborative design process (Seitamaa-Hakkarainen, et al., 2000). The classification schema comprises the following categories: (a) design thinking, (b) design activity, (c) co-authoring, and (d) information exchange. A summary of our classification schemata is presented in Figure 4.
[FIGURE 4 OMITTED]
The design-thinking category involves specifying (a) design context, (b) working idea, (c) new information, (d) comment, and (e) metacomment. Each statement was considered to represent just one of these subcategories in terms of its dominant content. How these components succeed and relate to each other may be explained as follows: By specifying design context the design problem was determined (for example: Does anyone know about research conferences and conference bags?). Design progressed by articulating and elaborating working ideas through which the final design emerged (I have been sketching a green conference bag. You find it in the attachment). The participants also searched for and provided new information concerning design context or design elements (I interviewed one researcher who has participated many conferences and here comes what he was thinking about conference bags). During the design process, participants commented on each other's ideas and information. Through metacomments, participants assessed whether the design process was progressing in the desired direction, and appropriate methods were being used; they considered how the design task was shared, and accomplishments of members of the design team (Hi, we have to now discuss what kind of bags we are going to design. Please make your comment as soon as possible so that we can go on).
The present investigators used the second category, design activity, to assess the modes of participating in problem solving during the design process. These subcategories were easily identified in the design statements. The variable had four discrete values according to whether there was (a) analyzing, (b) proposing, (c) evaluating, and/or (d) organizing in the process. Each statement was considered to represent just one of these subcategories. Analyzing statements generated information (e.g., design constraint or design elements) that refined the design problem. Proposing refers to an active process of finding a solution for the design or stating new ideas. Evaluating statements refers to assessment of a design situation or solutions made with respect to particular design elements or sketches. Organizing the process refers to statements that helped the team to regulate the socially shared process of designing. In previous studies, the present investigators were particularly interested in how the students organized their collaborative activities during the design process. The process organization variable represented statements that helped the team to regulate the socially shared process of designing. These included, for instance, statements that referred to organizing of design teams' meetings, sharing of tasks between team members, making joint decisions, and practices of working within FLE2. The previous study (Lahti, et al., 2001) illustrated that the relative proportion of statements that focused on organizing the team members' joint activities and efforts of collaborative designing (process organization) made sense as an indicator of the intensity of collaboration.
The third category, coauthoring, covered only KB messages produced by the students. In analyzing students' notes, we recognized that sometimes they produced coauthored notes. The subcategories were identified in the design statements as follows: (a) one student produced idea, (b) part-of-group co-produced idea, and (c) group-together produced idea.
The present investigators used the fourth category, information exchange, to assess the content of interaction between teacher, users, and students. This category became very finely graded and comprised the following aspects: (a) a student asks user's and teacher's experience, (b) a student asks user's and teacher's feedback, (c) a student asks team members' feedback, (d) a user or teacher provides his or her own experience, (e) a user or teacher provides feedback, (f) a student refers to outsider user information, (g) a student refers to general information, and (h) students' own opinion.
Each segment representing the main idea was coded and analyzed by using SPSSWIN program. To analyze the reliability of classification, an independent coder classified approximately 13% of all statements; the coefficient for rater agreement was .84, which was considered satisfactory.
Analysis 1: Social Interaction in Networked Designing
The entire database consisted of 211 knowledge-building messages. The students posted 149 messages, on average 6.2 messages per student (minimum was 0, maximum 26 messages) to FLE2's database during the course. The teacher posted 35 messages, and users posted 27 messages. Team members' activities and users' participation varied considerably from one team to another (Table 2). The teacher and users' centrality in the design process was analyzed as previously described. The analysis indicated that the participants' social network had a relatively centralized structure (92% in the case of sent, and 82% in the case of received messages). The teacher's extremely high betweenness value indicates that she was mediating information exchange between the students, their teams, and the users. A main part of virtual design activity took place within the design teams, and students generally did not actively comment on design process across the teams.
Table 2 reveals the teams' differing patterns of participating in virtual designing. Team 4 appeared to be the most productive in posting KB messages, and the number of their sent and received messages was higher than those of other teams. Team 3 did not participate in virtual designing as actively as the other teams. An examination of the data indicated, further, that only some of the students and users participated intensively in the design process.
The analysis indicated that the students' network of interaction was not very dense; specifically, 0.22 (SD--0.86) for symmetrized data (direction of commenting ignored). In the case of the asymmetric graph based both on sent and received comments, the measure was 0.16 (SD=0.74) of the possible ties. Detailed analyses indicated that the teacher distributed her coaching efforts equally across design teams.
The figures might give an impression of a generally shallow level of participation. Yet one should take into consideration that much activity could have taken place outside of the FLE2-environment. In many cases, a few students on the team appeared to take responsibility for organizing the virtual design process and interacting with the teacher and the users, whereas the others were more active in design processes that took place outside of the FLE2-environment.
Analysis 2: Participants' Role in Design Process
To analyze the design process in FLE2 the present investigators analyzed the teacher's, users', and students' design thinking, design activities, coauthoring, and exchange of information. Teams posted 211 knowledge-building messages in all, consisting of 762 design statements (both the teacher's and users' statements were included this analysis). All the messages were task-oriented and relatively easily analyzable according to the current classification schema.
Table 3 presents the general frequencies and proportions of students', the teacher's and users' design-thinking statements. There appeared to be significant differences in such statements between students, teacher, and users. Table 3 shows how each group emphasized different subcategories during the design course. The frequency distributions of design-thinking statements across students, teacher and users differed from each other in a statistically significant way (df=8; [chi square]=127.5; p<.001). To examine whether or not the observed frequencies in each cell deviated from what might be expected by chance alone, the present investigators carried out cell-specific exact tests (Bergman & El-Khouri, 1987). The results indicated that the students provided most of the working ideas (f=231; 42%). This was in accordance with expectations, since the students were requested to engage in self-directed design process. The users emphasized new information (f=55; 50%) considerably more often than the others. The teacher appeared to provide more comments (f=35; 33%) and metacomments (f=25; 24%) than did the other participants. In general, through metacomments the participants assessed whether the design process was progressing in the desired direction and how people should proceed in the task.
Table 4 presents the general frequency distribution of students', teacher's, and users' design-activity statements. Design-activity statements appeared to differ significantly between the groups (df=6; [chi square]=85.7, p < .001). The users appeared to emphasize the analyses of design context (f=45; 41%), whereas the teacher did not focus on the analyzing stage (f=13; 12%). Instead, the teacher appeared to emphasis of the organizing the students' design process (f=50; 47%). In students' design activity statements, approximately 21% (f=113) were process organizing statements. The students proposed numerous design ideas (f=211; 39%), but they did not often evaluate these ideas (f=72; 13%).
The further analysis indicated that the students co-authored several KB messages. Approximately 71% of the students' ideas were produced by one student, 9% of the ideas were written in pairs, and 20% of ideas were posted on behalf of all team members. In order to analyze more closely the interaction patterns between the students, teacher, and the users, the present authors analyzed the categories of information exchange. Table 5 presents frequencies and proportions of the information exchange.
Table 5 illustrates that there appeared to be significant differences between the teams' designing in respect to how they used information and acquired knowledge and feedback from the users. Teams 1, 2, and 6 received most of the statements representing users' experience during their designing. In the case of Team 2, the user or teacher gave also feedback about the students' ideas more often (f=31; 18%) than in the other cases. Team 1 acquired much more information from the users outside the present network environment by interviewing some other conference goers (f=24; 20%). Team 1 produced design ideas, but they did not ask for any direct feedback from the user or fellow team members.
These series of excerpts illustrate how the students in Team 1 acquired outsider user information and how their user provided new facts about this particular conference. In assessing designs of the participating students, many users emphasized how a good conference bag could be used for many purposes afterwards. It could, for instance, not only function in other academic situations, but also serve a participant who is going on a picnic or to beach for swimming. The latter series of excerpts shows clearly how the students of Team 2 actively produced sketches and asked for feedback, which they also got from both the teacher and the user. An evaluation of the quality of students' design took place at the end of the project; the jury found Team 2's conference bag to be the best, whereas Team 1's conference bag was considered to be the least satisfactory.
The present investigators examined how the FLE2-environment supported collaborative designing through organizing a participatory design course in which teams of first-year textile-teaching students solved an authentic and complex design task--designing a conference bag for those attending the EuroCSCL conference. The students collaborated in sharing their design knowledge and searching for new information provided by the users, who were active conference goers and who volunteered for this study. In general, the qualitative content analysis of the students' productions posted to FLE2's database indicated that the knowledge produced by the students represented the prototypical elements of design process revealed in previous studies (e.g., Seitamaa-Hakkarainen, et al., 2000). Accordingly, the students analyzed the design context; they generated their solutions through a sustained process of developing and testing design ideas, and evaluated them.
However, some crucial limitations of the study should be taken into consideration in assessing the role of FLE2. First, FLE2 was used most effectively during weeks 2 to 9, but not any further during the stage of actually manufacturing the prototype. Second, working within the FLE2-environment appeared to involve the main part of the reflective aspects of designing whereas work time outside the system was dominated by practical activities needed for doing the designing. Third, the students also had the possibility to monitor the FLE2 without actively contributing to the database. It was not, however, possible to analyze the intensity of students' reading of KB messages because FLE2 did not provide any tracking data.
Previous studies (Kvan, et al., 1999; Seitamaa-Hakkarainen, et al. 2001) have indicated that there are two important aspects of designing that virtual design environments may scaffold: defining the design context and acquiring new information. In the same way, in this study the students defined the design context actively. Moreover, the users' important contributions to the scaffolds of students' design process became apparent; the users provided a lot of new information concerning the design context. Especially, in the case of Team 2's design process, the user directly provided her own experiences and feedback to the students about the latter's solutions. Students of Team 1 also acquired outsider user experiences but did not rely on interaction with their named user while testing their design ideas.
The teacher's role became that of organizer of the collaborative design process. It is evident that the teacher is needed to structure the collaborative efforts and provide advice for the students in virtual design studios, just as in traditional ones. Some of the teams took more responsibility of their design process whereas some of them organized their process only partially in FLE2. In Team 2's design process, the teacher and user were more involved in the students' designing since they were actively invited to the discussion of the relative merits of solutions. The user and teacher became design partners with the students even if they did not directly propose new solutions or sketches. Our conclusion is that there is some evidence of participatory designing in Team 2's activities.
These results indicate that the virtual design environment may support the design process, especially during the stages of problem structuring and visual designing. It appears to us that the cognitive scaffolding of expert-like designing encouraged and provided conceptual tools for the students to reflect on their own design thinking, and that participatory designing (i.e., including the user as a designer's partner) is possible to arrange in the FLE2-environment. There are limitations to these findings. Since the study involved intensive investigation of a small number of participants, the data do not support the drawing of comparative conclusions; in particular there is no direct evidence permitting the assessment of the degree to which students' performance in this design task may have been better than in some alternative context. We do not know about the representativeness of this sample of participants, and our provisional conclusions need to be refined with other samples and other settings.
The present investigators have found that it takes significant time and effort to develop pedagogically sound computer-supported design courses. We believe, however, that there is a long-term return on this investment, which renders these efforts worthwhile. The use of collaborative environments appears to be most productive in courses that require collaboration, knowledge sharing, or exchange of visual materials. In particular, the present authors found that the FLE2 facilitated teamwork and provided a convenient means for user participation. The FLE2--environment, does not displace traditional instruction, but provides tools that can be used in intimate connection with traditional instructional methods. Kvan (2001) claimed that in the virtual design studio, a teacher's facilitation role takes on a much larger importance than in the traditional setting, and this is consistent with our data.
These results suggest that collaborative technology may provide new tools for students to share important aspects of their design thinking. An essential aim of the present study was to facilitate direct student-expert partnership, that is, provide the students with access to authentic users' knowledge so that the student designers might apply it. In the present case, however, the activity of the user also varied; either the students did not ask the active user's contribution, or the volunteer users did not have enough time to participate in the virtual design process. As the present investigators continue with the FLE-project, they will explore possibilities of facilitating more intensive and online interaction between students and experts, so that the students learn to work productively with customers from the very beginning of their projects. It might be important to improve the participants' awareness of what is going on within a networked learning environment (e.g., setting up a notification system that transmits information about students' activities to the experts), so that very busy experts could follow what students are doing and provide timely feedback for them (Jermann, Soller, & Muehlenbrock, 2001). To conclude, it appears to be profitable to engage experts more closely in knowledge-building discourse with the students.
Table 1 Structure of the EuroCSCL 2001 Conference-Bag Design Project Subject of the Lecture and Week Type of Activity Design Task 1 Introductory Lecture Introduction to individual and (2 h) collaborative design process Setting up design teams 2 Lecture User-Centered Design: Design (2 h) Task & Design Brief User- Activity-Context-analysis: A team's mind maps Introduction to the function of FLE2 and help for FLE-tutors 3 Lecture (2 h) Product analysis of the Participating in knowledge conference bags. Bags in FLE2 building in FLE2 Users start participating in FLE2-environment 4 Lecture (2 h) Visual designing: Brain Visual designing sketching-method Users provide their experiences in FLE2-environment 5 Independent Teamwork (2 h) Visual designing: Sketches Textile laboratory (3h) posted to the FLE2-database Testing of materials and technical solutions 6 Lecture (2 h) Sketches sent to the jury (six members) 7 Independent Teamwork (2 h) Pattern making: Exploring technical solutions and selecting materials Feedback from six users in FLE 8 Lecture (4h) Scale Drawing: Detailed Sewing Technology Lecture & designing & First prototype Teamwork (4 h) Sewing Technology: Exploring technical solutions by sewing and experimenting 9-12 Sewing Technology Acquiring materials and Independent Teamwork (4 h) developing the prototype Prototype making 13 Lecture (2h) Students' Exhibition of the Conference Bags and Evaluation of the Project Prototypes sent to the jury 14-15 Jury's (i.e., five users and teacher) Evaluation Table 2 The Centralization of Participation in Virtual Designing Groups of Sent messages Received messages Measure participants of betweenness n M SD M SD M SD Team 1 4 4.0 4.1 4.5 3.7 6.5 7.5 Team 2 4 5.8 5.4 6.3 6.1 13.9 19.1 Team 3 4 2.8 1.9 3.5 3.1 8.2 7.6 Team 4 4 8.3 4.3 8.8 3.4 12.8 10.6 Team 5 4 3.8 3.5 3.8 6.8 16.1 32.3 Team 6 4 5.8 6.8 5.3 7.9 23.7 35.1 Teacher 1 34.0 0 31.0 0 272.0 0 Users 9 3.1 2.9 2.7 2.4 1.0 2.8 TOTAL 34 5.4 6.5 5.4 6.5 17.8 48.3 Table 3 Frequency Distribution of Design Thinking Students Teacher & Tutor Design thinking f % f % Context 92 16.8 10 9.4 Working idea 231 ([dagger]) 42.3 13 * 12.3 New information 101 * 18.5 23 21.7 Comment 54 * 9.9 35 ([dagger]) 33.0 Metacomment 68 12.5 25 23.6 TOTAL 546 100.0 106 100.0 Users Design thinking f % Context 12 10.9 Working idea 14 * 12.7 New information 55 ([dagger]) 50.0 Comment 13 11.8 Metacomment 16 14.5 TOTAL 110 100.0 Note. Significance tests are based on hyper-geometric probability estimates (see Bergman & El-Khouri, 1987); * = Observed frequency smaller than expected by chance alone (p < .001); ([dagger]) = Observed frequency larger than expected by chance alone (p < .001). Table 4 Frequency Distribution of Design Activity Students Teacher & Tutor Design activity f % f % Analyzing 150 27.5 13 * 12.3 Proposing 211 ([dagger]) 38.6 14 * 13.2 Evaluating 72 * 13.2 29 27.4 Process organizing 113 * 20.7 50 ([dagger]) 47.2 TOTAL 546 100.0 106 100.0 Users Design activity t % Analyzing 45 ([dagger]) 40.9 Proposing 16 * 14.5 Evaluating 25 22.7 Process organizing 24 21.8 TOTAL 110 100.0 Note. Significance tests are based on hyper-geometric probability estimates (see Bergman & El-Khouri, 1987); * = Observed frequency smaller than expected by chance alone (p < .001); ([dagger]) = Observed frequency larger than expected by chance alone (p < .001). Table 5 Frequency Distribution of Type of Information Exchange Team 1 Information exchange f % Ask user's experience 7 5.9 Ask user's feedback 0 0 Ask team members' feedback 4 * 3.4 User provides own experience 16 13.6 User provides feedback 9 7.6 Refers to outsider user information 24 ([dagger]) 20.3 Refers to general information 25 ([dagger]) 21.2 Students' own opinion 33 28.0 TOTAL 118 100.0 Team 2 Information exchange f % Ask user's experience 11 6.3 Ask user's feedback 11 6.3 Ask team members' feedback 14 8.0 User provides own experience 21 12.0 User provides feedback 31 ([dagger]) 17.7 Refers to outsider user information 5 2.9 Refers to general information 16 9.1 Students' own opinion 66 37.7 TOTAL 175 100.0 Team 3 Team 4 Information exchange f % f % Ask user's experience 1 1.4 9 6.2 Ask user's feedback 1 1.4 1 0.7 Ask team members' feedback 11 15.9 27 18.5 User provides own experience 8 11.6 5 * 3.4 User provides feedback 2 2.9 9 6.2 Refers to outsider user information 2 2.9 4 2.7 Refers to general information 11 15.9 16 11.0 Students' own opinion 33 47.8 75 51.4 TOTAL 69 100.0 146 100.0 Team 5 Team 6 Information exchange f % f % Ask user's experience 8 8.1 11 7.1 Ask user's feedback 8 8.1 1 0.6 Ask team members' feedback 25 25.3 26 16.8 User provides own experience 3 3.0 27 17.4 User provides feedback 12 12.1 13 8.4 Refers to outsider user information 2 2.0 1 0.6 Refers to general information 11 11.1 5 * 3.2 Students' own opinion 30 30.3 71 45.8 TOTAL 99 100.0 155 100.0 Note. Significance tests are based on hyper-geometric probability estimates (see Bergman & El-Khouri, 1987); * = Observed frequency smaller than expected by chance alone (p < .001); ([dagger]) = Observed frequency larger than expected by chance alone (p < .001). Table 6 Design Interaction Between the Students, Teacher, and the Users Student Elaine "I was asking something" Deepening Knowledge 2000-10-10 Hi! I had an "interview" with one conference visitor, who gave me some ideas and requests. He thought that briefcase shaped bags that had a shoulder strap in addition to handles were among the best ones he had been using. He had also been satisfied with their subdued colors. Anyway, he was not that sure if we should also have subdued colors after I told him something about this particular conference. I think that we should emphasize it a bit, so that even an outsider can recognize what it was meant for. As an improvement, he suggested carefully chosen closing mechanisms. The cover is usually fastened with a clip with opens easily by itself. The inside pockets lose their contents easily because they cannot be closed at all. He thought that a zipper would be the best solution. (A costly one, maybe). He had one more idea, a pocket for passport. One has to carry it quite often. Student Theresa "I was asking ..." Comment 2000-10-29/ was asking a friend of mine what kind of bag it should be. The answer was that there must be no pockets inside. Putting folders into the bag is difficult if there are extra pockets. The bag must be big enough for a big folder. There is usually a separate briefcase for the laptop. Pockets outside are good (for passport, mobile phone etc.). Student Elaine "Note" Problem 2000-11-01 In the briefing, it was said the \out\look of the bag should reveal the subject of the conference. How should we do that? User Mina "The subject of the conference" Problem 2000-11-09. Here we have an interesting question. The subject is, after all, computer supported collaborative learning, which combines modern techniques and educational theories. The participants are not tech freaks but researchers in the field of learning who use the new technologies. Did you know that the conference is also the first European Computer Supported Collaborative Learning conference? There have been other ones in the United States earlier. What means of expressing the nature of the conference are there in the look of the bag? Color? Logo? I have never thought about that. The bag should not be too streamlined and at least not blue, as it is in traditional engineers' style. Student Ann "Designing the bag" Starting Problem 2000-10-18 Comments and sketches of bag design phases are placed here The first sketch can be found from the thread "Possible proper ties of the bag / a sketch" Attachments: bag1.jpg Student Ann "Bag sketch "Working Theory 2000-10-18 Here is an idea about the shape of the bag. Actually we thought that the bag would be more round, it is the way things are today (modern cars etc.). Joining the handles together looks like a good idea. For sure it is something new. There should be a cushion in it [joined handles] to make carrying the bag more comfortable. Later it would be suitable for a shopping bag. There would not be a partition in the bag, but large pockets and a few smaller ones on both sides of the bag. Attachments: bagsketch.jpg Teacher "Nice shape" Comment 2000-10-18 I found the shape pretty good, but does the way it is carried give too feminine an impression? At least I think so. The large pockets and some small pockets to the both sides of the bag are a good decision. Student Ann "Commenting on the comments "Working Theory 2000-10-20. Lots of thanks for the comments! The latest sketches seemed to give you a lot of thoughts. Something worth refining, being too feminine seemed to be a problem. Cute; we wondered, before publishing the picture. Does the round shape that we eventually gave to the bag make it look too feminine? Apparently not. Well, there is again an attachment with sketch and possible patterns. If there are any men reading this please comment on the topic of the day, would a man also like to carry this one on his shoulder? In my opinion the original sketch had a touch of youth and "worth keeping" in it, if I say so myself. We thought that a good choice of colors would compensate for the feminine feel of the bag, if it is still here in the new proposal. Attachments: patterns.jpg User Lisa "Handbag or bag?" Comment 2000-10-20 Hi, Of the two sketches I liked the one on left, the other one was too handbag-like. Cheers, Lisa the user
Henna Lahti and Professor Pirita Seitamaa-Hakkarainen designed the study, collected the data, and developed methods used in analysis. Lahti was responsible for analysis of the data, interpretation of the results, and writing the manuscript. Seitamaa-Hakkarainen and Dr. Kai Hakkarainen provided theoretical and methodological guidance during the process. Lahti is a doctoral student in the Graduate School of Multidisciplinary Research on Learning Environments. The Academy of Finland and Finnish Cultural Foundation have supported this research work.
Akin, O. (2002). Case-based instruction strategies in architecture. Design Studies, 23(4), 407-431.
Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum.
Bergman, L., & El-Khouri, B. (1987). Exacon: A Fortran 77 program for the exact analysis of single case cells in a contingency table. Educational and Psychological Measurement, 47, 155-161.
Borgatti, S. P., Everett, M. G., & Freeman, L. C. (1999). UCINET 5.0 Version 1.00. Natick: Analytic Technologies.
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide, Journal of the Learning Sciences, 6, 271-315.
Dillenbourg, P., Baker, M., Blaye, A., & O'Malley, C. (1996). The evolution of research on collaborative learning. In P. Reimann & H. Spada (Eds.) Learning in humans and machines: Towards an interdisciplinary learning science (pp. 189-211). Oxford: Elsevier.
Ferguson, E. S. (1992). Engineering and the mind's eye. Cambridge, MA: The MIT Press.
Gabriel, G., & Maher, M.L. (2000). An analysis of design communication with and without computer mediation. In S. A. R. Scrivener, L.J. Ball, & A. Woodcock (Eds.) Collaborative design (pp. 329-337). London: Springer.
Hennessy, S., & Murphy, P. (1999). The potential for collaborative problem solving in design and technology. International Journal of Technology and Design Education, 9, 1-36.
Jermann, P., Soller, A., & Muehlenbrock, M. (2001). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.) Proceedings of EuroCSCL 2001 European Perspectives on Computer-Supported Collaborative Learning (pp. 324-331). Universiteit Maastricht, the Netherlands.
Kvan, T. (2001). The pedagogy of virtual design studios. Automation in Construction, 10, 345-353.
Kvan, T., Yip, W., & Vera, A., (1999). Supporting design studio learning: An investigation into design communication in computer-supported collaboration. In C. M. Hoadley & J. Roschelle (Eds.) Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference. Mahwah, NJ: Lawrence Erlbaum.
Lahti, H., Seitamaa-Hakkarainen, P., & Hakkarainen, K (2001). The nature of collaboration in computer supported designing. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.) Proceedings of EuroCSCL 2001 European Perspectives on Computer-Supported Collaborative Learning (pp. 381-388). Universiteit Maastricht, the Netherlands.
Leinonen, T., Mielonen, S., Seitamaa-Hakkarainen, P., Muukkonen, H., & Hakkarainen, K. (1999). FLE-Tools Prototype: A www-based learning environment for collaborative knowledge building. Proceedings of ENABLE99 (Enabling Network-based Learning), June 2-5, 1999, Espoo, Finland.
Lipponen, L. (1999). Challenges for computer supported collaborative learning in elementary and secondary level: Finnish perspective. In C. M. Hoadley & J. Roschelle (Eds.) Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 368-375). Mahwah, NJ: Lawrence Erlbaum.
Maher, M. L., Simoff, S. J., & Cicognani, A. (2000). Understanding virtual design studios. Berlin: Springer.
Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company: How Japanese companies create the dynamics of innovation. New York: Oxford University Press.
Palonen, T., & Hakkarainen, K. (2000). Patterns of interaction in computer-supported learning: A social network analysis. In B. Fishman & S. O'Connor-Divelbiss (Eds.) Proceedings of the Fourth International Conference of the Learning Sciences (pp. 334-339). Mahwah, NJ: Lawrence Erlbaum.
Popovic, V. (1999). Product evaluation methods and their importance in designing interactive artifacts. In W.S. Green & P.W. Jordan (Eds.) Human factors in product design. Current practice and future trends (pp. 26-36). London: Taylor & Francis.
Schon, D. A. (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass.
Scott, J. (1991). Social network analysis. A handbook. London: Sage.
Seitamaa-Hakkarainen, P., Lahti, H., Muukkonen, H., & Hakkarainen, K. (2000). Collaborative designing in a networked learning environment. In S. A. R. Scrivener, L.J. Ball, & A. Woodcock (Eds.) Collaborative design (pp. 411-420). London: Springer.
Seitamaa-Hakkarainen, P., Raunio, A. M., Raami, A., Muukkonen, H., & Hakkarainen, K. (2001). Computer support for collaborative designing. International Journal of Technology and Design Education, 11(2), 181-202.
Sikkel, K., Gommer, L., & van der Veen, J. (2001). A cross-case comparison of BSCW in different educational settings. In P. Dillenbourg, A. Eurelings, & K. Hakkarainen (Eds.) Proceeding of EuroCSCL 2001 European Perspectives on Computer-Supported Collaborative Learning (pp. 553-560). Universiteit Maastricht, the Netherlands.
Wells, G. (2000). Dialogic inquiry in education. In C.D. Lee & P. Smagorinsky (Eds.) Vygotskian perspectives on literacy research. New York: Cambridge University Press.
University of Helsinki
University of Joensuu
University of Helsinki