Course Management Systems as tools for the creation of online learning environments: evaluation from a social constructivist perspective and implications for their design.

The Internet and the Web offer academic institutions solutions for covering the massive demand for education and transition towards student-centered, social constructivist educational models, in accordance with the demands of the knowledge-based society. This article reports on an investigation aimed at presenting a synthesis of recent research on the applications of Course Management Systems (CMS) in academic institutions, and evaluating the research directions and findings from a social constructivist perspective. The investigation explored current uses of CMS in academia, the impact of CMS on the educational processes, faculty and students, the differences among various modes of CMS-based online learning, and the extent to which CMS support social constructivist approaches to learning. It was found that CMS are currently used in a variety of disciplines for on campus, mixed-mode, and complete online courses, yielding positive student attitudes and enabling faculty to create online learning environments of social constructivist inspiration. However, creating such environments by means of CMS still poses significant workload on faculty for structuring online interactions, and monitoring and supporting students. Furthermore, the facilities currently offered by CMS still refrain from effectively supporting alternative forms of assessment and collaborative knowledge building activities. Implications for the design of CMS are considered.

**********

In recent years, there has been a growing demand for Higher Education (HE) expressed by adults with varying ages and needs (Duderstadt, 1998). At the same time, the Internet and the Web are becoming increasingly accessible from HE institutions, the workplace, and the home, enabling new, online forms of learning for both offcampus and oncampus students, and offering HE institutions solutions for covering the afore-mentioned demand across barriers of space and time.

The exigencies of our contemporary knowledge-based society call for a redefinition of the generic skills that students should develop within HE (Oliver & McLoughlin, 2000). Furthermore, coupled with the diverse needs of adult learners, they question the traditional faculty-centered educational models used in HE institutions calling for a transition to learner-centered models, where students are seen as active, lifelong members of learning communities (Duderstadt, 1998). Contrary to the industrial society where knowledge was static, centralized, and not easily accessible, in today's knowledge-based society, knowledge is dynamic, widespread, and directly accessible (Gros, 2002). Consequently, it is essential that students develop skills for life-long and independent learning, such as information literacy and metacognitive skills, problem solving skills, critical thinking skills, time and task management skills (Oliver & McLoughlin, 2000), as well as communication and teamwork skills, which are essential in the contemporary workplace (Dede, 1996; Harasim, 1999; Oliver & McLoughlin, 2000). In addition, adult learners need flexible, self-paced learning that takes into account their individual characteristics and learning styles. However, the traditional behaviourist model of instruction, which is usually followed in lectures and tutorials in HE, cannot favour the development of the afore-mentioned skills or meet the needs of adult learners. This model emphasizes instruction-centered strategies and knowledge transmission, content fragmentation, and sequential delivery, and assessment methods focused on recognition and recall of isolated, out of context knowledge (Fetherston, 2001; Heinecke, Dawson & Willis, 2001). The student's conceptions are disregarded, and he/she is viewed as a passive receiver of knowledge (Fetherston, 2001) with no control over the objectives, content and pace of learning (Hazari, 1998). It should, thus, be avoided that the Internet and the Web be merely used for a replication of this delivery model online (Dede, 1996; Fetherston, 2001; Heinecke et al., 2001).

On the contrary, the constructivist educational model provides rich, active learning environments, which are likely to favour the acquisition of higher order skills, and cover the needs of adult learners. Constructivism emphasizes the active and personal construction of knowledge by the student through his/her experiences and social interaction within a learning environment (Bostock, 1998; Heinecke et al., 2001). Constructivist instructional principles place importance on: (a) negotiation of learning objectives, (b) student control over his/her learning, (c) authentic, purposeful and contextual learning, (d) problem solving, (e) collaborative learning, (f) multiple, alternative perspectives, (g) knowledge construction and validation through action and discourse, (h) authentic, contextual assessment, and (i) development of metacognitive skills (Heinecke et al., 2001). The instructor is viewed as a facilitator and coach of the student activity. The goal is the development of deep understanding, and the student is given opportunities to express his/her conceptions and interpretations (Fetherston, 2001; Heinecke et al., 2001). Piaget and Vygotsky pioneered research into constructivist learning in the 20th century (Gros, 2002), introducing the cognitive constructivism and social constructivism schools of thought respectively. Although Vygotsky had emphasized the role of the social context and learner's social interactions in the construction of knowledge from the 1930s, expanding the Piagetian view of learning as a result of interaction between the learner and the object of knowledge, his theory has been rediscovered in recent years, as developments in communications technologies have provided interesting tools for collaborative knowledge construction (Gros, 2002). Social constructivism emphasizes learning in context and through collaboration within a learning community (Gros, 2002). Knowledge is individually constructed, but constantly negotiated through interactions within the learning community, which leads to evolving communal and individual knowledge (Angelo, 2000). This learning approach particularly suits HE because it reflects the processes through which knowledge is built in academic disciplines. These disciplines, each characterized by shared concepts, dialect, and culture, constitute examples of socially constructed and dynamically evolving knowledge (Angelo, 2000). According to the social constructivist approach, faculty and students form learning communities, within which, they collaborate to achieve shared learning goals (Angelo, 2000). Instructional strategies deriving from this approach include: project-based learning, case-based learning, problem solving, collaborative learning, cognitive mentoring, and situated learning (Heinecke et al., 2001; Gros. 2002).

Many authors argue that the information and communication facilities offered by Internet and Web technologies can support innovative, constructivist pedagogy (e.g., Hazari, 1998; Oliver, 2000). In particular, the unprecedented opportunities for resource sharing and Computer-Mediated Communication (CMC) offered by these technologies empower social constructivist approaches to learning, enabling the creation of online learning communities for construction of shared knowledge across barriers of space and time (Scardamalia & Bereiter, 1994, 1996; Dede, 1996; Pall off & Pratt, 1999; Harasim, 2000; Fetherston, 2001). The Internet and the Web could, thus, be used for a re-engineering of the educational processes in HE, introducing constructivist approaches that would be difficult, if not impossible, to apply in face-to-face instruction (Bostock, 1998).

In the mid-1990s, faculty who pioneered online courses used various, general purpose tools to create and run their courses, such as HTML editors for the creation of course web pages, e-mail, bulletin boards, and chatrooms for course communication (Hazari, 1998). However, in recent years, as academic institutions increasingly adopt Internet and Web technologies in their educational processes, integrated software solutions, known as Course Management Systems (CMS), have been developed to help faculty implement and manage web-based courses. A CMS is an application intended to reduce the effort and technical skill necessary to build and run web-based courses, providing support for the following major tasks (McCormack & Jones, 1998; Hayes, 2000): (a) organization and distribution of digital learning materials, (b) communication and collaboration, (c) student assessment and self-assessment, and (d) class management. A CMS typically offers the instructor and students tools for (Edutools, 2003): (a) curriculum design (e.g., course templates, curriculum sequencing, customized look and feel), (b) course delivery (e.g., course management, automated testing and scoring, grading, student tracking), (c) communication (e.g., e-mail, chat, discussion forums, whiteboard, file exchange), (d) productivity and student involvement (e.g., course calendar, personal home pages, self-assessment, groupwork). Within a CMS-created course, a student can interact with the course content and other course users, such as the instructor, teaching assistants, and peers.

Taking into account the afore-analysed need for innovative, social constructivist approaches to teaching and learning in HE, and given that CMS are the primary vehicles for the creation of online learning environments in HE institutions, a question that is posed is whether and to what extent CMS can support such approaches, which are potentially enabled by Internet and Web technologies. The answer to this question would provide useful insight into the true educational value of CMS to both faculty considering to use CMS in their courses and, most importantly, to the designers of these software applications who should constantly aim at improving them in alignment with the requirements of the teaching and learning processes in HE.

AIM

This article reports on an investigation, which focuses on a twofold aim: (a) to present a synthesis of recent research studies on the applications of CMS in HE institutions, and (b) to evaluate the research directions and findings of these studies from a social constructivist perspective. In particular, the research questions of the investigation were as follows:

1. How are CMS currently used in HE institutions and what is their impact on the teaching and learning processes and their main users, namely the instructors and students?

2. Do CMS currently support the creation of online learning environments inspired by the social constructivist approach to learning, which is potentially promoted through the use of Internet and Web technologies?

3. Which are the differences among the following modes of CMS-based online learning: (i) technology-enhanced oncampus learning, (ii) mixed-mode oncampus and offcampus learning, and (iii) complete online learning?

4. What are the implications of the answers to these questions for the design of CMS?

METHOD

The investigation was conducted through keyword searches in all journals and proceedings indexed in the Digital Library (DL) of the Association for the Advancement of Computing in Education (AACE), a major source of information on e-learning research, and through subsequent study, analysis, and synthesis of the detected articles. The investigation was carried out in two phases, as follows.

The first phase took place in August 2003 and consisted of keyword searches in the AACE DL (http://dl.aace.org). The default option of searching for keyword occurrences within the title and abstract of an article was maintained. The searches were limited to most recent articles, namely articles that had been published during the last five years, from 1999 to 2003. The keywords used referred to popular, at the time of the searches, CMS and where "webct," "blackboard," "learningspace," "topclass," and "virtual-u." They yielded 107, 72, 6, 6, and 5 articles respectively. After eliminating multiple instances of the same article, as well as articles that were either irrelevant (e.g., did not refer to teaching/learning in HE) or technical (e.g., evaluations of software and technical features of CMS), a total of 146 articles resulted.

During the subsequent second phase, these articles were studied with a view to making the main themes of research on the applications of CMS in HE institutions emerge. As a result, five main themes emerged: (a) adoption of CMS by faculty members, (b) evaluation of the use of CMS in the teaching and learning processes independently of discipline, (c) evaluation of the use of CMS in the teaching and learning processes in specific disciplines, (d) evaluation of the use of CMS for collaborative learning activities, and (e) examination of CMS as to their potential to support innovative instructional approaches. Each of the articles was assigned to one of these themes, according to its predominant focus. The list of the 146 articles and their grouping into the five themes is given in the Appendix. Within each theme, the research directions and findings of the articles were summarized and synthesized. The research questions of the present study served as a guide for this process. The conclusions drawn from the five themes were synthesized to produce the overall conclusions of this study.

RESULTS

In the detected articles, CMS were referred to by a variety of terms, such as "e-learning systems," "e-learning platforms," "online course delivery systems," "courseware management tools," "virtual learning environments," "networked learning environments". In the rest of the article, the term "Course Management System" (CMS) will be used. The detected articles related to the use of CMS for three modes of online learning: (a) technology-enhanced oncampus learning, where CMS-based online learning activities are devised as complements to regular oncampus instruction, (b) mixed-mode oncampus and offcampus learning, where occasional face-to-face oncampus instruction is complemented by CMS-based online learning activities, and (c) complete online learning, that relies exclusively on CMS-based online learning activities.

In what follows, the five themes that emerged from the investigation are presented and discussed with respect to the findings of the broader online learning literature. The last theme (examination of CMS as to their potential to support innovative instructional approaches) is examined in more detail with a view to providing insight into the interplay between the design of CMS and the diverse modes of learning that CMS potentially support or preclude, and to identifying implications for the design of CMS so that these systems empower social constructivist approaches to learning. Finally, the differences among the three afore-mentioned modes of CMS-based online learning, which arose from the five themes, and their implications for the design of CMS are examined.

Adoption of CMS by Faculty Members

A part of the detected studies addresses issues such as: (a) the training and support of faculty in the use of CMS, (b) the actual processes of adoption of CMS by faculty, (c) the attitudes of faculty towards the use of CMS, and (d) the new demands that are posed for faculty with the introduction of CMS.

The synthesis of the findings of the studies indicates that although faculty initially feel hesitant towards using CMS or question the pedagogical benefits of these systems, they finally realize their value when using them for online instructional activities (e.g., Freeman, Brimley & Rosen, 1999; Serrano, Santiago & Medina, 2000; Boer & Fisser, 2002). Furthermore, they recognize the benefits offered to their students in terms of flexibility and accessibility (e.g., Xu, Sloan & Novikova, 2002). Crucial factors for the adoption of CMS by faculty are the training and support that faculty receive, with hands-on training, in both using a CMS and developing online instructional strategies, being more motivational (e.g., Nealy, Williams & Autrey, 2002; Payeras, 2003). There is diversity in how faculty members use CMS in their courses ranging from exploitation of certain facilities to full exploitation of the CMS's potential (e.g., Bai, Chuvessiriporn & Lehman. 2002; Boer & Fisser, 2002), although most faculty members prefer an asynchronous rather than a synchronous mode of interaction with their students (e.g., Zhang, 2002). Despite the generally positive attitudes of faculty, the CMS adoption rate is not satisfactory (e.g., Caris, 2003) mainly due to the increased workload that is demanded from faculty in order to learn to use a CMS, prepare online learning materials, and also assess students' postings and products, and provide timely and quality feedback to students (e.g., Huang, 2002; Xu et al., 2002; Gonzales, Bussmann & Bovard, 2003). Finally, with the transition towards teaching through CMS, new roles are posed for faculty who become more the facilitators and moderators of students' online activity, and, hence, must develop effective student-centered instructional strategies (e.g., Mills, 2001; Solberg, 2001; Bermant & Knight, 2002).

These findings are consistent with the online learning literature, which indicates that instructor workload increases with online courses (Harasim, Hiltz, Teles & Turoff, 1997; Harasim, 2000; Reeves, Baxter & Jordan, 2002; Linge, 2003). This workload is due to online course development and especially to facilitation of CMC activities (Woltz et al., 1997). It is also agreed that faculty should be provided with opportunities to develop student-centered online instructional activities (Chiang, 2002; Reeves et al., 2002; Bennett & Bennett, 2003).

The studies summarized under this heading indicate that faculty members are becoming aware of the advantages of CMS-based online learning. However, they need support in developing student-centered instructional strategies by means of these systems. Furthermore, they use CMS in diverse ways, which indicates that CMS should enable a variety of instructional approaches. Finally, CMS impose a significant workload on faculty, particularly when student-centered, participatory approaches to learning are adopted. The of CMS should, thus, aim at minimizing that workload through providing better support for such approaches.

Evaluation of the Use of CMS in the Teaching and Learning Processes Independently of Discipline

This strand is comprised of studies that assess the impact of the use of CMS on the teaching and learning processes from the instructor and the student perspectives. The studies report mainly on experiences from institution-wide or cross-departmental use of CMS, examining issues such as: (a) faculty and students' perceptions of and attitudes towards teaching/learning through CMS, (b) students' perceptions of the effectiveness of CMS. (c) students' satisfaction with their online experience, and (d) students' preferred CMS facilities (e.g., discussion board, chat tool).

The synthesis of the findings indicates that CMS are generally viewed to be user-friendly, effective, and useful by faculty and students (e.g., McCourt & Widmayer, 2000; Widmayer & McCourt, 2000; McClelland, 2002; Dean, 2003). Regarding faculty, the findings agree with these presented earlier in this article, with the major problem reported being that of the increased workload (e.g., Lazenby, 1999; Widmayer & McCourt, 2000). Regarding students, they are satisfied with their online experiences, and positive towards CMS-based learning (e.g., McKenzie, Bennett, Mims & Davidson, 2001; Magoun, Owens, Gillan & Stout, 2003), considering it effective (e.g., Bohley, 2002; Newland, 2003). Students find CMS helpful and beneficial, and believe that, through using them, their learning and achievement are enhanced in terms of both subject matter knowledge and information technology skills (e.g., Alexander, 1999; Marghitu, Sankar & Raju, 2003). They also particularly appreciate the flexibility of studying anyplace/anytime and the opportunity to easily access course material and participate in online discussions (e.g., McKenzie et al., 2001; McClelland, 2002). However, certain studies also revealed some problems that students had faced, and which had caused partial dissatisfaction to them, namely difficulties in using CMS and limited interactivity with the instructor and peers (e.g., Lazenby, 1999; Belvin, 2003).

These findings meet similar outcomes of the online learning literature (Morss, 1999; Harasim, 2000; Heeler & Hardy, 2002; Linge, 2003), which indicate that students show positive attitudes towards the use of CMS and satisfaction with their online learning experiences, considering that CMS offer them increased flexibility and increased opportunities for learning, peer interaction, and exposure to information technology.

The studies summarized in this section show that faculty and students generally have positive attitudes towards CMS-based online learning and consider it effective. For students, in particular, CMS offer helpful and flexible alternatives to learning. However, the findings also indicate that students should be offered opportunities for increased participation and interaction by means of CMS. CMS designers should take this into account, so that these systems promote true student-centered learning.

Evaluation of the Use of CMS in the Teaching and Learning Processes in Specific Disciplines

A significant part of the detected studies addresses the use of CMS within specific disciplines, examining issues such as: (a) the strengths and weaknesses of CMS regarding the needs of these disciplines, (b) the faculty and students' responses to the use of CMS, and (c) the impact of CMS on students' learning. The investigation showed that CMS have been used in various disciplines, namely computer science, mathematics, foreign languages, business, engineering, medicine, nursing, physics, chemistry, history, and sociology. In addition, within teacher education, they have been used in educational technology, literacy, mathematics, science, and even music.

The findings of the studies indicate that CMS have been used successfully in subjects that lend themselves to collaborative, experiential learning through CMC interactions, within authentic problem-solving tasks. Examples comprise: (a) medicine, where students have resolved case-based medical problems (Keppell, Kennedy & Harris, 2000). (b) engineering, where students have formed virtual teams (Koosha, Madadnia & Anderson, 2002), (c) foreign languages, where students have collaborated in the target language (Masuyama & Shea, 2003), (d) distance learning, where students have analyzed real distance learning cases (Davis, Li & Nilakanta, 2001), and (e) history, where students have conducted historical research using authentic sources (Stapleton & Newman, 2002). CMS have also successfully supported many online communities of learners in teacher education, mainly in the area of educational technology (e.g., Fisher, 1999; Parker, 2002). For instance, they have linked student teachers conducting fieldwork at schools with inservice teachers and faculty, within the context of authentic educational activities (e.g., Beckett et al., 2002).

In the studies grouped under this heading, CMS were found to be useful, mainly for resource sharing and CMC. Students' learning and motivation were enhanced (e.g., Weston, 2000; Stapleton & Newman, 2002; Bohlin, 2003; Bolek, Forsythe, Pinhey & Murray, 2003), and instructors found the experience interesting (e.g., Newcombe, Kinslow & Goss, 2001), although they had to cope with the facilitation and moderation workload (e.g., Fisher. 1999; Bao, 2000), and with new, alternative forms of student assessment, such as assessment on the basis of students' postings on the discussion board or assignments published on the CMS's presentation areas (e.g., Kunkel, 2002; Legare, 2002; Arquette, 2003; Cheney, 2003). However, the investigation showed that CMS do not lend themselves easily to subjects, such as computer programming or computer applications, that involve considerable hands-on practice, require the demonstration of certain skills by students, and demand specific instructional designs that impose even more workload on the instructor (e.g., Croop, 1999; Thompson-Sellers, 2001).

These findings support similar outcomes of the online learning literature, which show that the nature of the discipline poses specific requirements regarding the online learning environment (Kurtz, Parks & Nicholson, 2002; Reeves et al., 2002; Kandzia, 2003). For example, the intensive student-instructor and peer interactions involved in the teaching of programming are proven to be difficult to transfer to a CMS-based environment (Reeves et al., 2002).

The studies summarized in this section indicate that CMS are currently used in a wide variety of academic disciplines. CMS use has been successful in subjects that offer themselves to discussion-oriented approaches that can be supported through the CMC and resource sharing facilities of CMS. Within such contexts, CMS have indeed enabled the creation of learning environments of social constructivist inspiration, imparting to faculty and students satisfactory online experiences. However, faculty members were charged with increased workload for students' facilitation, moderation, and assessment. Furthermore, CMS were found to not entirely support the intensive laboratory-like activities and interactions demanded by certain subjects. Appropriate CMS facilities should, thus, be designed to remedy these problems.

Evaluation of the Use of CMS for Collaborative Learning Activities

This strand of studies focuses on the investigation of the interactions that take place through the CMC tools of CMS, within collaborative inquiry, knowledge-building, and problem-solving learning activities. These interactions are instructor-student, mentor-student, or student-student. The discussion boards of CMS are used as the primary medium for interactions. Specifically, discussion boards are used for: (a) seminars, (b) class discussions, (c) help from instructors, mentors and peers, and (d) communication within and among groups of students. The studies examine the online discourse communities that are formed.

The findings of the studies generally agree in that, through structured and purposeful online dialogue, students become more engaged in their learning, articulate ideas, negotiate meaning and collaboratively construct shared knowledge, which leads to their reflective discourse and critical thinking being promoted and their learning experiences becoming enhanced (e.g., Carey, 2000; Loving, 2000; Leh & Winograd, 2002; Vellom & Mascazine, 2002; Garvin, 2003). Furthermore, students appreciate the flexibility of studying anyplace/anytime (e.g., Maeers, 2000), they consider that they learn as much as they do in face-to-face instructional situations, and some believe they learn more effectively and in greater depth online (e.g., Korhonen, 2001). It also seems that there is a direct relationship between, on the one hand, the students' level of interaction and the sense of community developed, and, on the other hand, the students' perceived cognitive learning and satisfaction with the online course (e.g., Worrall, 2000; Topper, 2003). However, although the discussion boards of CMS do support student collaboration, they still lack the flexibility to make this collaboration practical and straightforward so that students easily share their ideas and digital products (e.g., Haack, Alagic, Gibson, Watters & Rogers, 2003).

Regarding the instructor, the studies stress that for the success of such social constructivist approaches to learning, his/her role is critical in facilitating, scaffolding, monitoring, and assessing students' interactions, and that a great part of his/her workload derives exactly from these tasks (e.g., Loving, 2000; Korhonen, 2001; Lucking, Rovai & Ireland, 2001; Aschermann, Buckley & Harrison, 2003). Nevertheless, they also reveal that CMS do not yet fully support the instructor, by providing appropriate built-in tools, in the afore-mentioned tasks, namely to easily: (a) structure interactions so that they are educationally profitable for students (e.g., Maeers, 2000; Young, 2001), (b) monitor students' interactions with a view to scaffolding students (e.g., Topper, 2003), and (c) assess students' learning based not only on quantitative, but also qualitative data drawn from their interactions (e.g., Garvin, 2003; Topper, 2003). It is therefore difficult and time-consuming for the instructor to organize and follow the interactions, especially when these become intensive (e.g., Verkler, 2001; Haack et al., 2003), and additional tools, beyond these currently offered by CMS, are needed for evaluating the interactions (e.g., Garvin, 2003; Giza, 2003).

These findings are consistent with the online learning literature, which reports that online interactions contribute significantly to the development of students' thinking skills (Harasim, 2000; Linge, 2003), although their management poses significant difficulties to the instructor (Woltz et al., 1997; Linge, 2003).

The studies summarized in this section show that CMS support collaborative learning activities in HE. Currently, the main vehicles for such activities are CMS discussion boards. The effects of these activities on students are enhanced learning experiences and development of reflective skills. However, for faculty, successfully putting to work such social constructivist approaches translates into a high workload, because of the lack of adequate CMS facilities to structure and manage students' interactions. Such facilities, together with more sophisticated and functional facilities for student collaboration, need to be incorporated into the design of CMS.

Examination of CMS as to Their Potential to Support Innovative Instructional Approaches

The studies presented in the two previous sections referred to CMS-created courses that incorporated social constructivist instructional principles, namely learning in real-world contexts, case-based learning, learning though participation in communities of learners, and collaborative construction of meaning. This section comprises studies that examine CMS' capability to support innovative instructional approaches as well as issues related to the design of CMS. Specifically, the studies address the following issues: (a) flexibility of CMS to support various instructional approaches, (b) potential of CMS to support alternative assessment, (c) comparisons between CMS-created and custom-built courses regarding support of innovative instructional approaches, (d) extensions to CMS for supporting innovative instructional approaches, (e) design of CMS on the basis of social constructivist theory, and (f) learning style, accessibility and usability issues in the design of CMS. Given that these studies open up interesting research perspectives and have particular implications for the design of CMS, they are presented in greater detail.

A question that is posed is whether CMS induce specific instructional models, incorporating in their own design facilities that obstruct or promote certain type of pedagogy, or are malleable to instructional design. There are researchers who assert that current CMS facilities cannot fully support innovative pedagogy, whereas others maintain that CMS are malleable (Oliver, Harper, Reeves, Strijker & Westhuizen, 2002). Specifically, certain researchers argue that CMS are designed to better support a traditional instructional model based on information transmission and students' testing, thus limiting faculty to that model, whereas pedagogy based on discursive interactions with embedded assessment is less well supported by these systems (Cronje, 2002; Ingraham, Watson, McDowell, Brockett & Fitzpatrick, 2002). However, it was found that although the availability of certain facilities in a CMS does suggests certain instructional strategies, faculty tailored the use of a CMS to their instructional needs (Pountney & Aspden, 2002) and to the needs of their students (Hanafiah, 2002), implementing both traditional, instructor-led as well as constructivist, student-centered courses through the same CMS. CMS have also supported online courses informed by specific, nontraditional, discursive, and collaborative instructional approaches, namely Laurillard's conversational framework (Carmichael, 2001) and Schmidt's investigative learning theory (Laaksonen & Front, 2002), although the interaction aspects of these approaches were not supported entirely successfully through the CMS discussion boards (Carmichael, 2001). It, thus, appears that CMS are malleable tools, provided that they offer facilities varied and sophisticated enough to support equally well diverse instructional models.

Regarding assessment, although CMS are useful for traditional types of assessment (e.g., multiple-choice questions), they are still found lacking regarding alternative types of assessment. Certain such types (e.g., portfolio assessment) are possible (Musgrove, Knee, Rodney & Musgrove, 2001), but it is still difficult and time-consuming for the instructor to apply other assessment methods compliant with the social constructivist philosophy (e.g., peer assessment), as it also is to effectively monitor and scaffold students (Korpi, 2000; McMahon & Luca, 2000), given that CMS provide only quantitative measures of student participation.

Certain studies refer to comparisons between CMS-created and custom-built courses regarding support of social constructivist instructional strategies, namely collaborative knowledge building and problem solving. Although both kinds of courses supported such strategies, custom-built courses were found to offer better collaborative facilities, compared to CMS-created courses, where the discussion board was the main collaboration medium (Bullen, 1999; McMahon & Luca, 2000). Furthermore, an attempt to implement social constructivist strategies based on creation and sharing of digital artefacts through a CMS revealed weaknesses (namely inadequate CMS facilities for content creation by students, and for sharing, peer- review, and discussion of this content), which were resolved by adding custom-built modules to the CMS (Nelson, Palumbo, Wellings & Smith, 2003). CMS are, thus, still lacking compared to sophisticated custom-built instructional tools (Bailin, 2002). These weaknesses of CMS are attributed to the fact that these systems were initially developed to support the teacher-centered model of instruction and have not yet fully escaped it (Bullen, 1999; Nelson et al., 2003). In an effort to remedy these weaknesses, tools or whole environments to be used in conjunction with CMS have been developed, with a view to enhancing students' opportunities for collaboration with faculty and peers (Danchak & Pedersen, 2002; Mackie & Ziemer, 2003).

Other studies address the design processes of CMS, and the impact of design choices on the type of instructional approaches that can be implemented through a CMS, making a distinction between commercial, mainstream CMS and CMS designed on the basis of sound learning theory. Contrary to commercial CMS that evolved out of the instructionist model, Virtual-U is an example of a CMS designed from the outset with a focus on social constructivist learning theory, comprising a variety of collaborative facilities (Laferriere, Breuleux & Bracewell, 2000). Virtual-U has been successfully used in HE for the creation of online learning communities for collaborative inquiry, knowledge-building, and creation of artefacts (Laferriere et al., 2000). A related issue is that of the expandability of CMS with new features deriving from the teaching and learning needs of faculty and students. The "commercial" software philosophy constrains CMS users to the features currently available in a CMS until its developers provide new features (Brown & Byrne. 1999). On the contrary, the "participatory design" philosophy, which maintains that the users of a product participate in its design process (Schuler & Namioka, 1993), is by nature close to social constructivist philosophy, and could contribute towards the design of CMS that truly evolve out of faculty and students' needs and the requirements for innovative pedagogy. Applications of this approach were found in a study on a collaboration between Virtual-U designers and users (Currie & Campos, 1999) and in a more recent study on one of the emerging "open source" CMS (Dougiamas & Taylor, 2003), namely CMS, the software source code of which, is publicly available and evolving out of the efforts and needs of their community of users.

Finally, learning style, accessibility and usability issues enter into the design of CMS. The extent to which CMS can effectively support diverse learning styles needs to be further researched with a view to creating more student-centered online courses (McCartney, 1999). Accessibility and usability issues are also important (Pearson & Koppi, 2001: Tselios, Avouris, Dimitracopoulou & Daskalaki, 2001; Zaharias, 2002). In particular, the usability of a CMS influences considerably the educational process conducted through it (Tselios et al., 2001), which underlines the need to design CMS with effective student-centered user interfaces (Zaharias, 2002). Specifically, to support social constructivist approaches, the design of a CMS user interface should aim at facilitating intuitive, straightforward, student-centered collaboration, and eliminating usability problems that affect students' learning (e.g., Bullen, 1999).

The studies summarized in this section indicate that CMS have so far been used for the support of constructivist, collaborative learning environments in HE, and that they are technological tools potentially malleable to instructional design, on condition that they incorporate a variety of facilities to effectively support different instructional models. This corroborates the assertion that CMS can support learning environments of social constructivist inspiration. However, at the same time, it reinforces the need that the design of CMS encompasses adequate facilities to better support such enviroments. Specifically, in accordance with the conclusions drawn from the previous sections, this section also indicates that better collaborative facilities as well as student scaffolding, monitoring, and assessment facilities are needed. Furthermore, it becomes clear that the instructional and design philosophies that inform the design process of a CMS have an impact on the instructional approaches that can be implemented through it as well as on its expandability. CMS designed on the basis of the social constructivist and participatory design philosophies are more likely to succesfully support the current needs of online learning communities, and to evolve in accordance with these communities' needs.

Differences Among Various Modes of CMS-Based Online Learning

The focus of this final section is to examine the differences among: (a) technology-enhanced oncampus learning, (b) mixed-mode oncampus and offcampus learning, and (c) complete online learning.

CMS have been successfully used for technology-enhanced oncampus learning (e.g., Richardson & Felkel, 2001; Sosin, 2003). Faculty opt for complementing their oncampus courses with CMS-based learning activities in an effort to render them more student-centered, and to encourage experiential, problem-based, and collaborative learning (e.g., Porter & Corderoy, 1999; Kunkel, 2002). In particular, the CMC tools of CMS are used to this end and have enabled the formation of online learning communities, parallel to oncampus courses (e.g., Worrall, 2000). Students show positive attitudes towards this mode of online learning, considering it flexible and conducive to enhanced learning (e.g., Singh, 2002; Watson & Boggs, 2003).

CMS have also served the needs of mixed-mode (e.g., Akanbi, 2001; Arquette, 2003) and complete online courses (e.g., Fisher, 1999; Strickland & Strickland, 2001). Courses of these types are found to be particularly useful to nontraditional students (e.g., students in employment), and in institutions with large numbers of such students, intense and continuous use of CMS is made (e.g., Alexander, 1999). Both types of courses are found to yield students' satisfaction and positive attitudes towards these modes of online learning, which students consider convenient, suitable to their particular needs and effective in terms of learning outcomes (e.g., Heaton, Washington & Pauley, 2002; Moore, Moore, Henderson & Diack. 2002; Tucker, Armfield & Blocher, 2003). However, contrary to oncampus students who can easily refer to campus infrastructures and technical support services (e.g., Belvin, 2003), students enrolled in mixed-mode or complete online courses, often have to deal with technical problems during CMS use alone, which may cause them some anxiety (e.g., Arquette, 2003). In fact, as possibilities for face-to-face contact decrease, the demand for students' information technology skills increases, with complete online courses being better suited to students who have adequate computing skills and are also capable of studying independently (e.g., Wickham, 2001; Bolek et al., 2003).

The review of the studies also showed that students particularly value physical contact and personalized attention (e.g., Arquette, 2003). In fact, the occasional face-to-face meetings among the instructor or mentor and students, that take place in mixed-mode courses, have proven to foster a sense of community among course participants, to increase student motivation, and to help students resolve course-related problems, and familiarize themselves with the CMS (e.g., Korhonen, 2001; Johansson & Ruokamo, 2002; Tucker et al., 2003). These elements have proven more difficult to attain in complete online courses, and in certain cases, such courses switched to mixed-mode because students felt more comfortable with this format (Dyches, Smith & Syal, 2003). This may also be related to the nature of the subject studied, which may comprise learning activities that rely on physical interpersonal interactions (e.g., Jackson, 2002; Dyches et al., 2003). Furthermore, in mixed-mode courses, collaborative interactions through the CMC tools of the CMS have proven very helpful in maintaining the learning community between the occasional meetings (e.g., Loving, 2000; Akanbi, 2001), although they were found more functional when complemented by face-to-face student collaboration (Johansson & Ruokamo, 2002). It appears that as physical contact diminishes, online collaboration becomes more essential, and the demand for effective collaborative facilities, that simulate the intuitiveness of face-to-face interaction, increases.

As deduced from the review of the studies, although all modes of CMS-based online learning impose new demands and a higher workload for faculty compared to traditional instruction (e.g., Kunkel, 2002), faculty workload and time commitment for developing and facilitating courses through a CMS increase as possibility for face-to-face contact decreases, and hence, are higher for mixed-mode and all the more so for complete online courses compared to technology-enhanced oncampus courses. Specifically, the scarcity or absence of physical contact calls for (Strickland & Strickland, 2001; Wickham, 2001; Roberts, 2002): (a) a more detailed approach to instructional design that may compensate for the lack of opportunities to provide verbal explanations and face-to-face help to students, (b) a more supportive, interactive, collaborative learning environment for distant students that may compensate for their physical isolation. In mixed-mode or complete online courses, the learning materials should, for instance, comprise detailed study guides that serve as references to the course resources and activities (e.g., Fisher, 1999; Bolek et al., 2003). Especially, when social constructivist perspectives are embraced, appropriate scenarios for structured use of the CMS collaborative facilities should be included (e.g., Hanafiah, 2002: Kurubacak, 2003). In addition, in such courses, ensuring distant students' retention is a particular challenge, and, thus, it is crucial that faculty adopt student-centered approaches, foster a sense of community through the CMS, provide human touch, define expectations for participation, and maximize students' online interactions, through appropriate support, feedback and facilitation, in order to prevent feelings of isolation that may cause students' attrition (e.g., Musgrove et al., 2001; Topper, 2003).

In fact, successful mixed-mode or complete online courses have emphasized students' activity and collaborative interactions through the CMS, fostering learning communities that enabled sharing of knowledge and mutual help, and kept students highly involved (e.g., Fisher, 1999; Koosha, Anderson. Madadnia & Akpolat, 2002). It has also been found that distant students particularly need, appreciate and become motivated by fast and constructive feedback from faculty (e.g., Heaton et al., 2002; Zaharias, 2002), and that the more faculty get actively involved in the online interactions, the greater the sense of classroom community created in such courses (Lucking et al., 2001). These requirements result in increased workload for faculty. Finally, as already stressed, CMS still lack sufficient tools to enable faculty to effectively monitor and scaffold students. Compared to oncampus courses, this problem is aggravated in mixed-mode or complete online courses, where faculty also lack the face-to-face contact that would permit them to estimate students' feelings and difficulties (Korpi, 2000).

The examination of the differences among the various modes of CMS-based online learning, which has been presented in this section, indicates that CMS have been used for the development of learning communities within the framework of oncampus as well as mixed-mode and complete online courses, yielding positive student attitudes towards these modes of online learning. However, as physical contact diminishes, the need for activating student-centered collaborative learning strategies and providing human touch among course participants through the CMS increases, and hence, effective collaborative facilities are essentially needed. Faculty workload also increases, which renders the need for the design of appropriate CMS facilities for structuring, monitoring and managing students' interactions more acute.

CONCLUSIONS

This article referred to an investigation, which was aimed at presenting a synthesis of recent research on the applications of CMS in HE institutions, and at evaluating the research directions and findings from a social constructivist perspective. Specifically, the investigation explored the current uses of CMS in HE institutions, the impact of CMS on teaching/learning processes as well as on faculty members and students, and the differences among various modes of CMS-based online learning, focusing on the extent to which CMS currently support the creation of social constructivist online learning environments, with a view to define relevant implications for the design of CMS.

The main conclusions are as follows:

* CMS are currently being used in a great variety of academic disciplines and subjects for online courses. The investigation showed that CMS are being used in oncampus courses addressed to onsite students as well as in mixed-mode and complete online courses addressed to nontraditional students. CMS are, thus, becoming a generalized medium for the provision of online education at the university and college level.

* Students show positive attitudes towards CMS-based learning, appreciating the opportunities for online interaction with faculty and peers, and for anyplace/anytime learning. These positive attitudes are very helpful for faculty who consider creating online courses and putting into practice participatory, social constructivist approaches to learning by means of CMS.

* Faculty use CMS in diverse ways, according to the needs of their courses. A recurring issue in the overviewed studies is the increased workload with which faculty are charged, especially when adopting social constructivist approaches to learning that rely heavily on CMC interactions among course participants. To better support such approaches, it is therefore crucial that appropriate facilities be designed and incorporated into CMS so that faculty can easily and effectively provide structure to online interactions, and also, monitor, scaffold, and assess students on the basis of the quantity and quality of their online interactions. This need becomes more acute in mixed-mode and all the more so in complete online courses, where faculty members lack face-to-face contact with students, and where adoption of student-centered approaches becomes critical and student motivation is particularly dependent on faculty-student feedback.

* CMS currently cannot entirely support alternative forms of assessment compatible with social constructivist approaches to learning. The design of the assessment facilities of CMS should, thus, be improved and extended encompassing sophisticated facilities so that faculty can easily and effectively apply various alternative assessment techniques through these systems.

* Although CMS are currently being used in academic subjects that lend themselves to collaborative discourse-oriented learning approaches, they still fail to entirely support subjects that involve hands-on tasks and intensive, structured instructor-student and peer interactions on the basis of these tasks. Appropriate CMS facilities should be designed and incorporated into CMS so that laboratory-like instructional designs and interactions are accommodated.

* Despite the afore-mentioned weaknesses, CMS enable faculty--even with a significant workload--to apply various instructional strategies, including strategies emanating from social constructivist approaches to learning. The investigation showed that even commercial CMS are moving from the teacher-centered model that they initially supported towards student-centered, collaborative learning modes. The design of CMS is, thus, being influenced by a general tendency for a move of online courses from mere information distribution towards collaborative learning and interaction (Harasim, 2000). In an older study of commercial CMS (Firdyiwek, 1999), it was found that the content presentation and testing facilities, which support the traditional behaviorist instructional model, were fairly well developed in these systems, whereas the authoring, interactivity, and communication facilities, which can support innovative pedagogy, were less developed. It appears that the latter facilities are now beginning to receive more attention.

* Nevertheless, the investigation showed that the collaborative facilities of current CMS are still lacking as regards easy and straightforward support of collaboration and interaction among course participants. The design of these facilities, thus, needs to be improved and extended, beyond the commonly offered discussion boards, comprising sophisticated facilities such as flexible personal and shared workspaces, interactive collaborative authoring tools, functional resource sharing tools, and scaffolds for framing online interactions, to effectively support online learning communities. This will render collaborative learning activities (e.g., inquiry, problem solving, knowledge building, creation, sharing and peer-review of artefacts) easier for faculty to organize and more profitable for students. The need for such facilities is more acute in mixed-mode and complete online courses, where effective online collaboration is essential, given the scarcity or absence of opportunities for physical collaboration.

Finally, it seems that CMS specifically designed on the basis of social constructivist and participatory design philosophy principles, provide interesting alternatives to commercial CMS, and should be considered by HE institutions, as they are more likely to provide innovative functionalities compliant with the contemporary demands for constructivist, collaborative learning environments.

References

Angelo, T. (2000). Transforming departments into productive learning communities. In A. Lucas (Ed.), Leading academic change: Essential roles for department chairs (pp. 74-89). San Francisco: Jossey-Bass.

Bennett, J., & Bennett, L. (2003). A review of factors that influence the diffusion of innovation when structuring a faculty training program. The Internet and Higher Education, 6(1), 53-63.

Bostock, S. (1998). Constructivism in mass higher education: A case study. British Journal of Educational Technology, 29(3), 225-240.

Chiang, G. (2002). What kind of support do they need? An instructional designer's experience in faculty and student support for online courses. Proceedings of the 30th Annual ACM SIGUCCS Conference on User Services (pp, 17-21). New York: ACM Press.

Dede, C. (1996). Distance learning to distributed learning: Making the transition. Learning & Leading with Technology, 23(7), 25-30.

Dougiamas, M., & Taylor, P. (2003). Moodle: Using learning communities to create an open source Course Management System. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003,1, 171-178.

Duderstadt, J. (1998). Transforming the university to serve the digital age. CAUSE/EFFECT, 20(4), 21-32.

Edutools (2003). Providing decision-making tools for the EDU community. Retrieved May 7, 2006, from http://www.edutools.info/

Fetherston, T. (2001). Pedagogical challenges for the World Wide Web. Educational Technology Review, 9(1), 25-32.

Firdyiwek, Y. (1999). Web-based courseware tools: Where is the pedagogy? Educational Technology, 39(1), 29-34.

Gros, B. (2002). Knowledge construction and technology. Journal of Educational Multimedia and Hypermedia, 11(4), 323-343.

Harasim, L. (1999). A framework for online learning: The Virtual-U. IEEE Computer, 32(9), 44-49.

Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. Internet and Higher Education, 3, 41-61.

Harasim, L, Hiltz, S., Teles, L., & Turoff, M. (1997). Learning networks: A field guide to teaching and learning online. Cambridge, MA: MIT Press.

Hayes, R. (2000). Exploring discount usability methods to assess the suitability of online course delivery products. The Internet and Higher Education, 2(2/3), 119-134.

Hazari, S. (1998). Evaluation and selection of Web course management tools. Retrieved May 7, 2006, from http://www.sunilhazari.com/education/webct

Heeler, P., & Hardy, C. (2002). Implementing a web-based master's degree in instructional technology: A progress report. Journal of Computing Sciences in Colleges, 17(4), 72-76.

Heinecke, W., Dawson, K., & Willis, J. (2001). Paradigms and frames for R & D in distance education: Toward collaborative electronic learning. International Journal of Educational Telecommunications, 7(3), 293-322.

Kandzia, P. (2003). Online courses step by step. In Klein, R., Six, H-W., Wenger, L. (Eds.), Computer science in perspective (pp. 206-215). New York: Springer-Verlag.

Kurtz, B., Parks, D., & Nicholson, E. (2002). Effective Internet education: A progress report. Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education (pp. 312-316). New York: ACM Press.

Linge, R. (2003). The good, the bad, and the ridiculous: Putting a course online with Blackboard. Journal of Computing Sciences in Colleges, 18(4), 355-364.

McCormack, C., & Jones, D. (1998). Building a web-based education system. New York: John Wiley & Sons.

Morss, D. (1999). A study of student perspectives on web-based learning: WebCT in the classroom. Electronic Networking, 9(5), 393-408.

Oliver, K. (2000). Methods for developing constructivist learning on the web. Educational Technology, 40(6), 5-18.

Oliver, R., & McLoughlin, C. (2000). Web-based learning and generic skills development. UniServe Science News, 15. Retrieved May 7, 2006, from http://science.uniserve.edu.au/newslettervol15/oliver.html 

Palloff, R., & Pratt, K. (1999). Building learning communities in cyberspace. San Francisco: Jossey-Bass.

Reeves, T., Baxter, P., & Jordan, C. (2002). Teaching computing courses--computer literacy, business microcomputer applications, and introduction to programming online utilizing WebCT. Journal of Computing Sciences in Colleges, 28(1), 290-300.

Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3, 265-283.

Scardamalia, M., & Bereiter, C. (1996). Student communities for the advancement of knowledge. Communications of the ACM, 39(1), 36-37.

Schuler, D., & Namioka, A. (Eds.) (1993). Participatory design: Principles and practices. Hillsdale, NJ: Lawrence Erlbaum.

Wolz, U., Walker, H., Palme, J., Anderson, P., Chen, Z., Dunne, J., et al, (1997). Computer-mediated communication in collaborative educational settings: Report of the ITiCSE '97 working group on CMC in collaborative educational settings. ACM SIGCUE Outlook, 25(4), 51-68.

The references from the AACE DL that were retrieved through the keyword searches are reported in the Appendix

APPENDIX:

List of the articles retrieved from the AACE DL through the keyword searches

a. Adoption of CMS by faculty members

AI-Khatab, A. (2002). Saved by Blackboard: Web-enhanced teaching. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2288-2288.

Arneson, W., & Arneson, D. (2002). Delivery of web-based and web-suppported distance education courses using a freeware version of Blackboard. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 124-126.

Bai, H., Chuvessiriporn, S., & Lehman, J. (2002). Faculty use of WebCT: Impact of P3T3 faculty development initiatives. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1494-1495.

Bermant, B., & Knight, J. (2002). Profession or vocation? The changing roles of college faculty. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 606-607.

Boer, W., & Fisser, P. (2002). Best practices experiences: Successful use of electronic learning environments. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 384-389.

Caris, M. (2003). The deans go back to school: Why don't faculty adopt e-learning? Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 1662-1665.

Chatel, R. (2002). Empowering higher education faculty to use technology to enhance teaching and learning: A peer mentoring program. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 627-629.

Cummings, C., Mehlig, L., & Kalkman, D. (2002). Faculty development and the process of change. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1564-1565.

Freeman, W., Brimley, W., & Rosen, R. (1999). Early experiences in broadening the use of web-based learning to mainstream faculty. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1364-1365.

Gonzales, C., Bussmann, S., & Bovard, B. (2003). Web-based professional development for teachers: A process approach. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 353-356.

Hines, S. (2002). Hand-me-downs: Reusing online courses and building courses for reuse. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 958-958.

Huang, P. (2002). Barriers encountered using WBI to deliver courses. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 823-824.

Mann, B. (2000). Phase theory: A teleological taxonomy of web course management for distance education. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 1448-1450.

McBride, R., Gillan, R., Fuller, F., Tweed, E., & McFerrin, K. (2002). An online solution to educational technology leadership certification: A case study. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 247-254.

Mills, S. (2001). The WebCT certified trainer program: Modeling online teaching and learning with industry driven standards. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 3066-3071.

Moeller, B., Haavind, S., Anderson, L., & Carrigg, P. (2002). Faculty development through online courses: Results from an evaluation of the PT3 netseminars. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2388-2389.

Mortera-Gutierrez, F. (2003). Instructional interactions at a distance versus institutional indicators: Migrating from LearningSpace-Lotus notes to new e-learning platforms, a higher education experience at ITESM-CCM in Mexico City, Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 900-901.

Nealy, S., Williams, D., & Autrey, K. (2002). Faculty institute: A model for online success. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1981-1983.

Payeras, J. (2003). Distance instructor training and support with WebCT. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 2009-2012.

Philhower, S., Allen, J., Simonson, S., Greer, R., Witten, B., & Montalvo, G. (2002). Cyber-cerebration: Edge happenings at Western Illinois University. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2328-2328.

Roberts, C. (2002). Creating an effective training model: One size does not fit all. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed 2002(1), 2719-2720.

Schmidt, K. (1999), Are we creating new teaching environments with web courseware tools? Proceedings of the World Conference on the WWW and Internet 1999(1), 1417-1417.

Serrano, Y., Santiago, J., & Medina, R. (2000). Online interactive learning: An innovative project of the inter American University, Bayamon Campus, developed with the integration of students, faculty and administrative staff. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 162-164.

Solberg, B. (2001). Organizational learning: the venue for institutional change with online technologies. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 746-751.

Vidoni, K., Kongrith, K., & Lamitina, D. (2003). Faculty development in higher education: Graduate assistants helping education faculty with technology integration. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2248-2249.

Xu, C., Sloan, J., & Novikova, A. (2002). Faculty use of Blackboard for course instruction at two Mid-Western universities: A multiple case study. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 330-334.

Zhang, C. (2002). The interactivity issues in using Blackboard. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2452-2453.

b. Evaluation of the use of CMS in the teaching and learning processes independently of discipline

Alexander, S. (1999). Selection, dissemination, and evaluation of the TopClass WWW-based course support tool. International Journal of Educational Telecommunications 5(4), 283-292.

Belvin, L. (2003). A tale of two semesters: It was the worst of times; it was the best of times. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 1646-1649.

Bohley, K. (2002). The student voice: Results of an attitudinal survey. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1973-1977.

Dean, D. (2003). The first three years of Blackboard at Eastern Washington University: Valuable lessons from a highly successful project. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 895-901.

Lazenby, K. (1999). Using WebCT at the University of Pretoria, South Africa. International Journal of Educational Telecommunications 5(4), 293-307.

Magoun, D., Owens, C., Gillan, R., & Stout, L. (2003). A first year evaluation for a distance education initiative (DEI) grant for three Louisiana universities. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2074-2081.

Marghitu, D., Sankar, C., & Raju, P. (2003). Use of web tools to enhance student-instructor interactions: Experiences with WebCT and FrontPage. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 1017-1020.

McClelland, R. (2002). Developing an organisational e-learning strategy: A UK university case study. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 655-660.

McCourt, C., & Widmayer, S. (2000). Using instructional technology: What do the students think? Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 656-661.

McKenzie, B., Bennett, E., Mims, N., & Davidson, T. (2001). Students perceptions of the value of on-line instruction. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2992-2997.

Newland, B. (2003). Evaluating the impact of a VLE on learning and teaching. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 601-603.

Widmayer, S., & McCourt, C. (2000). Faculty reaction to WebCT: One university's experience. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 1173-1176.

c. Evaluation of the use of CMS in the teaching and learning processes in specific disciplines: Teacher education

Akanbi, L. (2001). Improving the teaching of reading, language arts and literacy through WebCT: A work in progress. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2179-2180.

Arquette, C. (2003). Online and face-to-face: Using Blackboard to augment a traditional-style teacher education course. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 1620-1623.

Aufenanger, S. (2002). Evaluation of web based communication and cooperation in a course in teacher education at a German university. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1140-1141.

Bao, J. (2000). Using TopClass to facilitate regular educational research course. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 1645-1648.

Beckett, W., Wetzel, K., Chisholm, I., Zambo, R., Padgett, H., & Odom, M. (2002). Preservice and inservice teachers collaborating with technology in K-8 multicultural classrooms: Year 2. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1501-1507.

Bohlin, C., & Amarasinghe, R. (2003). A comprehensive electronic resource center for mathematics educators in California. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 1896-1899.

Bolek, C., Forsythe, R., Pinhey, K., & Murray, G. (2003). Strategies for enhancing the use of information technology in the P-12 classroom. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 3384-3387.

Boulet, M. (2002a). Designing and redesigning the WebCT part of the undergraduate course "Teaching and learning with music technology". Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2553-2556.

Boulet, M. (2002b). The use of WebCT as a course component of the teaching and music technology undergraduate course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 188-189.

Bruning, M., Larsen, V., & Barry, K. (2003). Exploring multiple web-based technologies to support teacher education preparation. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2359-2361.

Bull, P. (2002). Pedagogical ethnotechnography: A study of the impact of information technology as a pedagogical tool on the attitudes of preservice social studies middle school teachers. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2172-2173.

Cheney, C. (2003). "To make a virtue of necessity" creating electronic portfolios in preservice teacher education using WebCT. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 35-38.

Dyches, T., Smith, B., & Syal, S. (2003). Infusing technology in an introduction to special education course. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 3232-3235.

Fisher, M. (1999). Lotus LearningSpace: A WWW strategy for growth. International Journal of Educational Telecommunications 5(4), 309-329.

Graham, R., & Daniels, T. (2002). Developing an online staff development program using instructional technology for supervisors and administrators. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 369-374.

Heaton, L., Washington, L., & Pauley, R. (2002). The professional development series: Web-based staff development for K-12 teachers. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 182-183.

Jackson, C. (2002). Expanding the boundaries of the music education for the elementary teacher classroom with information technology. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 758-761.

Kasik, M. (2002). Technological applications in a graduate special education program. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 1822-1822.

Moore, W., Moore, W., Henderson, A., & Diack, M. (2002). Increasing teacher certification through a multicampus(Blackboard) praxis network. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1944-1947.

Newcombe, E., Kinslow, J., & Goss, M. (2001). Forming a cadre of learners: Effective educational technology integration in a teacher preparation program. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2039-2041.

Parker, J. (2002). Technology as a practical tool for real world teaching. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1410-1411.

Ravitz, J. (2002). A distance scholarship model for teaching and learning about technology supported assessments. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2393-2394.

Sosin, A. (2003). Infusing technology in literacy teacher education. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 1102-1105.

Stapleton, J., & Newman, J. (2002). Linking history, technology and teacher education: The Allen Parker Slave narrative. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2212-2216.

Strickland, J., & Strickland, A. (2001). Creating a web-based graduate core course in technology for international English speaking students. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 1815-1816.

Tucker, G., Armfield, S., & Blocher, M. (2003). Developing a hybrid site-based distance delivered introductory hands-on technology course. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 541-542.

Weston, H. (2000). Using WebCT to deliver a finite mathematics course to preservice teachers. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 1055-1059.

Wickham, L. (2001). "Electronic/distance preservice teacher education--an example". Pro ceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 3018-3023.

Wolfe, C., Ashley, J., Nancy, E., Taylor, E., & Wolff, J. (2001). Collaborative technology exploration: Bridges between university and K-12 education. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2824-2824.

Other disciplines

Akinyemi, A., Al-Kindi, M., Al-Rawahi, Z., & Al-Rawas, 0. (2002). Information technology and medical students at Sultan Qaboos University. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2535-2536.

Bohlin, R. (2003). The design of virtual reality museums on the Web: A constructivist approach. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 1394-1397.

Burrell, A. (1999). Languages and statistics: Solutions for the support of online learning. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1743-1743.

Combs, L. (1999). Multi-dimensional web-based learning in freshman chemistry. Proceedings of the World Conference on the WWW and Internet 1999(1), 1680-1680.

Croop, F. (1999). A first experience at teaching undergraduate object oriented programming via the World Wide Web using WebCT. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1526-1527.

Davis, N., Li, Q., & Nilakanta, R, (2001). Dilemma analysis of constructive case-based approach to distance learning. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2847-2849.

Denham, S., Henderson, D., Parker, V., Bi, X., & Mehta, S. (2000). How to teach differently: Using technologies for web-based courses. Proceedings of the World Conference on the WWW and Internet 2000(1), 630-633.

Diers, M., & Heywood, B. (2002). Medical education: A multimedia enhanced BlackBoard course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 408-409.

Farley, R., & Tiffany, P. (2001). Using the course management system Blackboard 5 with the computer algebra system maple in the mathematics classroom. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 1937-1939.

Fletcher, J., Branen, L., & Anderson, E. (2001). A North American nationally distributed multimedia course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 504-505.

Jacobsen, M., & Kremer, R. (2000). Online testing and grading: Using WebCT in computer science. Proceedings of the World Conference on the WWW and Internet 2000(1), 263-268.

Keppell, M., Kennedy, G., & Harris, P. (2000). Transforming traditional curricula: Enhancing medical education through multimedia and web-based resources. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 1404-1405.

Koosha, H., Anderson, T., Madadnia, J., & Akpolat, H. (2002). Educators experiences and lessons in e-learning. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2627-2628.

Koosha, H., Madadnia, J., & Anderson, T. (2002). Developing a virtual master of engineering at UTS: A case study. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2623-2624.

Kunkel, R. (2002), Using technology and constructivism in business law: First year lessons. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 1048-1049.

Kurubacak, G. (2003). Designing and implementing WebCT-based courses online: Distance english language teacher training program (DELTT) model. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 390-393.

Legare, G. (2002). Time and resource issues pertaining to the design and the management of course materials for a WebCT-based project with engineering students. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1780-1787.

Lepkowski, F. (2002). Implementing library instruction for freshman composition on WebCT: The Oakland University experience. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1796-1798.

Lu, A., Zhu, J., & Stokes, M. (2000). The use and effects of web-based instruction: Evidence from a single-source study. Journal of Interactive Learning Research 11(2), 197-218.

Masuyama, K., & Shea, A. (2003). Successful integration of technology? A case study of a first year Japanese language course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 1336-1339.

Payeras, J., & Diabate, E. (2002), E-learning in Spanish: A language and culture course and a placement test. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2505-2507.

Porter, A., & Corderoy, R. (1999). Evolving from face-to-face to online teaching for large student groups: Phase 3, restructuring in a computer based student/subject management system. Proceedings of the World Conference on the WWW and Internet 1999(1), 1397-1398.

Richardson, B., & Felkel, B. (2001). Integrating videos into a business calculus class. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2779-2784.

Semrau, P., & Boyer, B. (2002). Collaborative university teamwork in designing a distance education course for the government. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 306-306.

Singh, M. (2002). Elearning translates to experiential learning in a communication theory class--higher education. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1). 864-869.

Thompson-Sellers, I. (2001). From "inches" to "miles": Web-enhanced instruction using WebCT (Version 3.1). Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 294-297.

Watson, R., & Boggs, C. (2003). Warming up the chill: Integrating technology into a general chemistry course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 1881-1882.

Wijngaards, N., Jacobsen, D., Kremer, R., & Shaw, M. (1999). The learning web: A technical evaluation of WebCT in concurrent classroom and distance education sections of a software engineering graduate course. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1340-1341.

Yang, J., & Hsiao, C. (2002). Seeing is believing! Can students learn culture using distance education? Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2778-2779.

d. Evaluation of the use of CMS for collaborative learning activities

Aschermann, J., Buckley, C., & Harrison, D. (2003). Breaking down the barriers: E-learning as a vehicle for facilitating personal and professional development among pre-service teachers in Wales and Missouri. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2986-2987.

Bryan, V., & Musgrove, A. (2003). Creating virtual learning teams with Blackboard. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2362-2365.

Carey, S. (2000). Within student comparison of ESL acquisition--through content between virtual and F/F seminar for ESL and native speakers' negotiated meaning. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 282-284.

Chen, I., Hood, B., & Bhattacharjee, M. (2003). Mentoring technology-based learning community for student interns. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2369-2370.

Freed, S., & Eggers, M. (1999). Interactions examined from multiple perspectives. Proceedings of the World Conference on the WWW and Internet 1999(1), 1695-1695.

Garvin, T. (2003). Using threaded discussion to develop reflective practice in preservice teachers. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 1802-1805.

Giza, B. (2003). Analyzing online professional development: Modeling interaction mathematically to identify proximal social spaces. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2394-2395.

Haack, C., Alagic, M., Gibson, K., Watters, J., & Rogers, G. (2003). Tailoring a website to meet the needs of global learners, Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2396-2399.

Johansson, T., & Ruokamo, H. (2002). The realization on interaction in cognitive science network studies. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 863-867.

Joyce, D., Nodder, C., & Young, A. (2001). Using an electronic discussion board to supplement classroom sessions with postgraduate teacher education students. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2864-2867.

Korhonen, V. (2001). Situated and socially shared cognition in practice: Designing a collaborative network learning experience for adult learners. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 1002-1007.

Leh, A., & Winograd, D. (2002). How telecommunication technologies and Moderating strategies in online instruction may benefit teacher education? Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 2367-2371.

Loving, G. (2000). Role modeling critical thinking in an online course for nurse educators. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 120-123.

Lucking, B., Rovai, F., & Ireland, M. (2001). Classroom community in post-secondary classes: An examination of traditional and distance learning environments. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 2381-2382.

Maeers, M. (2000). Using WebCT to extend learning in graduate educational technology courses. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 686-691.

Malone, S. (2002). The tech mentor project: Technology and the student teaching experience. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 1685-1686.

McShay, J., & Colon, I. (2002). Embracing critical perspectives on multicultural pedagogy in teacher education: An online intercultural exchange. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 395-395.

Sudweeks, F. (2000). To mediate or not to mediate--the role of the mediator in the online classroom. Proceedings of the World Conference on the WWW and Internet 2000(1), 607-607.

Topper, A. (2003). Assessing student learning in a web-based learning environment (WBLE). Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 2526-2531.

Vellom, P., & Mascazine, J. (2002). Utilizing WebCT to monitor pre-service teachers' professional development. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2341-2344.

Verkler, K. (2001), It's a small university after all--reducing distances between colleges via Web CT. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 1788-1789.

Wetig, S. (2002). Utilizing 'Blackboard' to engage teacher candidates in higher-order thinking. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 99-100.

Worrall, P. (2000). Using WebCT in a communications learning community. Proceedings of the World Conference on the WWW and Internet 2000(1), 844-845.

Xu, Y., & Iran-Nejad, A. (2002). Comparing themes of critical reflection from face-to-face and on-line discussion in a course for teacher education students. Proceedings of the Society for Information Technology and Teacher Education International Conference 2002(1), 105-108.

Young, K. (2003). Computer-mediated communication: The importance of alternative learning media in health education and promotion. Proceedings of the Society for Information Technology and Teacher Education International Conference 2003(1), 3931-3932.

Young, S. (2001). Innovative use of bulletin boards in undergraduate and masters level online courses. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 2113-2114.

e. Examination of CMS as to their potential to support innovative instructional approaches

Bailin, A. (2002). Blackboard and beyond. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1159-1162.

Brown, E., & Byrne, R. (1999). Lightweight tools for on-line course development. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1513-1514.

Bullen, M. (1999). Technology meets pedagogy in online distance education. Proceedings of the World Conference on the WWW and Internet 1999(1), 1207-1208.

Carmichael, D. (2001). An educational evaluation of WebCT; a case study using the conversational framework. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 233-238.

Cronje, J. (2002). Putting the action into interaction: What instructors and students do online. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 363-368.

Currie, S., & Campos, M. (1999). An activity theory framework for anticipating the needs of learning communities. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1605-1606.

Danchak, M., & Pedersen, J. (2002). QuikQuiz: Complementing WebCT tools for more learner interaction. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1374-1377.

Hanafiah, M. (2002). Sustaining instructional design strategies to enhance e-learning among learners. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1911-1914.

Ingraham, B., Watson, B., McDowell, L., Brockett, A., & Fitzpatrick, S. (2002). Evaluating and implementing learning environments: A United Kingdom experience. Educational Technology Review 10(2), 28-51.

Korpi, M. (2000). Background for designing net-based learning situations. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 542-545.

Laaksonen, I., & Front, T. (2002). The elearning concept at Helsinki University of Technology, Finland: Working with the 7-step approach on the elearning platform. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 1069-1070.

Laferriere, T., Breuleux, A., & Bracewell, R. (2000). Collaborative inquiries into the networked classroom. Proceedings of the Society for Information Technology and Teacher Education International Conference 2000(1), 852-857.

Mackie, B., & Ziemer, N. (2003). Producing community in an online or web-enhanced environment. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 394-395.

McCartney, B. (1999). Web based teaching and its effect on different learning styles. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 1999(1), 1509-1510.

McMahon, M., & Luca, J. (2000). Courseware management tools and customised web pages: Rationale, comparisons and evaluation. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2000(1), 687-692.

Musgrove, A., Knee, R., Rodney, D., & Musgrove, G. (2001). Using WebCT 3 to create web-based learning for multiple learning styles. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001(1), 1455-1459.

Nelson, W., Palumbo, D., Wellings, P., & Smith, R. (2003). Supporting learning with content, tools and community: Thinking outside of the learning management system box. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2003(1), 405-408.

Oliver, R., Harper, B., Reeves, T., Strijker, A., & Westhuizen, D. (2002). Learning management systems: One size fits all? Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2002(1), 1498-1499.

Pearson, E., & Koppi, T. (2001). Developing more effective access to higher education for people with disabilities: A case study in the design of accessible online courseware. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2001(1), 1470-1472.

Pountney, R., & Aspden, L. (2002). Community and constructivism: Implicit pedagogical models in virtual learning. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 2043-2046.

Tselios, N., Avouris, N., Dimitracopoulou, A., & Daskalaki, S. (2001). Evaluation of distance-learning environments: Impact of usability on student performance. International Journal of Educational Telecommunications 7(4), 355-378.

Zaharias, P. (2002). On-line learning courses: A review and usability attributes. Proceedings of the World Conference on E-Learning in Corp., Govt., Health., & Higher Ed. 2002(1), 1056-1062.

MARINA PAPASTERGIOU

University of Thessaly, Trikala, Greece

[e-mail]