Meeting the assessment demands of networked courses.

The use of the information and communication technologies (ICTs) and, in particular, the penetration of the Internet has given rise to an increase in networked courses for distance education. Such courses are characterised by two-way communication media that allow for interaction between the course writer and the student, between tutor and student, and between students.

Characteristics of such courses include:

1) Increased interactivity: interaction with other students, tutors or course writers offers a much richer learning environment for distance students.

2) Interactive materials: use of interactive assessment tools, simulations and animations are often embedded within the material. This increases the variation in material and the level of activity for the student in the learning process.

3) Flexibility in study patterns: students can study asynchronously and still partake in collaborative work and group discussions and receive timely feedback from their tutor.

4) Access to wider sources of material: use of CD-ROMs and the availability of web-based material means that students of such courses can access a far greater range of material than that traditionally supplied in distance education courses.

These networked courses differ from traditional, print-based ODL courses, not just in their use of ICTs, but also in the manner in which the technologies have influenced the underlying pedagogy, course design, content and even cultural values embodied in the material.

It is widely accepted that assessment must be designed to reflect course pedagogy, aims and objectives. Networked courses require the course designers to rethink the assessment strategy if it is to reflect the aims of the course and appropriately assess the skills developed during the course. At the same time, the use of ICTs offers new possibilities for assessment previously not feasible in a traditional print-based context.

Technology has made a significant impact on assessment in a variety of areas. In some universities, it is in use to automate assignment submission and grade retrieval (van Gorp & Boysen, 1997). Submission of assignments in electronic form is increasingly in use for new courses in many universities and opens up new potential for the use of a variety of formats, for instance, HTML.

Also, technology is widely in use to deliver computer-based assessment, often as an adjunct of computer-based learning packages (Chaloupka, Koppi, & Clark, 1998; Brown, Race, & Bull, 1999) and have a place in progress and achievement testing, and as a diagnostic tool.

Arguably, the most exciting area for innovative assessment in a distance context lies with the increased interactivity afforded by ICTs. This means that online collaborative assessment is now feasible. Interactivity in assessment has been implemented on a wide variety of projects (Collis, 1998; Fiedeldy, 1999; McConnell, 2000).

Networking opens up the possibilities for enhancing formative feedback to students through peer review, when scripts are posted electronically for comment and review (Bos, Kikstra, & Morgan, 1996; Davies & Berrow, 1998). Networks may also be used for delivering model answers to help students to see alternative approaches to written work (Barrett & Paradis, 1988; Mason, 1995). There is the potential for iterative assignment development, where students submit drafts of assignments for comment -- either from the tutor or from peers -- before submitting the final product (McConnell, 1999). Finally, there may be scope for criteria negotiation (Kwok and Ma, 1999), where students engage in discussion of assessment criteria as they work on their assessment. Most of these innovations have been described for small population courses, and the extent to which this experience is transferable to mass distance education is yet to be demonstrated.

At the United Kingdom Open University (UKOU), a wide diversity of courses now have an online component and can be categorised as networked courses. This article explores the manner in which four such courses have developed innovative assessment strategies. These strategies have been devised to reflect particular aspects of course pedagogy and to support students in new ways of studying, whilst exploiting the opportunities offered by networking.

METHODS

The present study is a comparison of assessment strategies used on four networked courses. Two of the course are undergraduate level -- T171: "You, your computer and the Net;" and THD204: "IT and Society" The other two courses are graduate level -- H802: "Applications of IT in Open and Distance Education;" and B823: "Managing knowledge."

The data is drawn from a desk study of course materials and assessment practices for the most recent presentations of all four courses. This includes analysis of the T171 web-based examinable component for 1999, together with evaluation work from the first presentation year described in Mason and Weller (2000), and a study of course materials and assessment on B823.

This was supplemented with empirical data from a doctoral study of assessment on THD204 and H802. A case study of perspectives on the assessment of THD204 was undertaken over a three-year period, from 1996-1998, with three cohorts of students and their tutors, and the findings were compared with a short study of student perspectives on H802. Data were gathered from 21 in-depth interviews and observations, followed by a series of computer conferences to which 800 students and 50 tutors were joined over two years, supplemented by 100 telephone interviews (Macdonald, 1999).

DESCRIPTION OF THE COURSES

The courses included in this study represent a range of levels and are delivered to students at varying stages of their academic career. They differ in the degree of flexibility and openness afforded to students, course aims and pedagogy and the type of resource offered.

T171: You, Your Computer and the Net

This undergraduate foundation course gives an introduction to computers and the World Wide Web. It was presented for the first time in 1999. The next year, a record 12,000 students registered on the course, in two presentations (Weller & Mason, 2000). The course is designed to be delivered to a wide range of students, some of whom may be completely new to computers or to online work, and others who may wish to hone existing skills. Indeed, many of the students in the 1999 presentation intended to take the course as a "stand-alone" and had no intention of further OU study. The course is delivered entirely online, via the Web, and tutorial support is also exclusively online, through First Class.

Course resources include a web-based study guide, with links to further reading, a CD-ROM containing course software, and two set books with introductory booklets designed to help students get started. Whilst students are required to study from custom-written online and print-based course texts, the degree of freedom afforded them increases as the course progresses. So, in the first module, for example, students are simply required to refer to the course web page for the information they need. In the second module, they are expected to make use of some external resources, and there are links to computing magazines. By the third module, they are introduced to a wider variety of resources and are expected to practise skills in evaluation.

The course aims to enculturate students into ICT (Weller 2000) and to provide them with the skills to use the new technologies as effective tools.

THD204: IT and Society

This second-level undergraduate course explores social and technological issues associated with IT developments. It was presented for the first time in 1995 and was the first accredited Open University course to use CD-ROMs as a major course component. In 1998, it was delivered to 1,400 undergraduate students, approximately 70% whom were from technology, math or computing backgrounds.

The course provides students with an innovative environment for resource-based learning by giving them access to an indexed personal library of 300 articles, as well as 350 MB of video clips, CBT and animations on CD-ROM (Jones, Kear & Reilly, 1998), in addition to printed course material. Students are also introduced to sources on the Internet -- in particular, a home page for the course -- which provides links to additional reading and updating material. Networking, using First Class, links students with staff and fellow students for tutorial support, debate on course issues and some online collaborative work.

H802: Applications of IT in Open and Distance Education

This online course forms part of the Open University's Master of Arts degree in Open and Distance Education. The course offers practical experience in online activities and interaction, collaborative work and Internet searching. It was presented for the first time in 1998 to 60 postgraduate students worldwide. It attracts students who are either planning, or already concerned with, the design and delivery of online teaching materials.

The course has very little purpose-written course material. Indeed, three out of four blocks have no printed course material except for a study and assignment guide and set books. It uses a web-based Virtual Campus, which has three categories of resource within an integrated environment: a BBS (Bulletin Board System), an online study guide and links to further information resources. The approach is largely constructivist and collaborative -- the emphasis being on online activities and peer learning, rather than on individual study of course materials -- and students are expected to make use of a range of web resources.

B823: Managing Knowledge

This postgraduate course forms part of the MBA program and was presented for the first time in 2000. The course explores a variety of aspects of managing knowledge, using technologies that support knowledge processing. The course is designed for practising managers who are already involved with knowledge management.

The course is delivered through printed course units, four CD-ROMs, two readings and three audiocassettes. Students attend a residential school, and tutorial support is provided through an initial day school, followed by face-to-face or computer mediated tutorials using First Class, in addition to a real time conferencing package, Lyceum.

INNOVATIVE ASSESSMENT PRACTICES

In line with current practice at the OU, all four courses have a continuous assessment component, together with an end-of-course assessment or exam. All assignments have a summative role, but also play an important formative role in supporting student learning throughout the course. Three out of four courses T171, H802, B823) use the university's electronic assignment submission system.

The account that follows describes the use of assignments to reflect course pedagogy, and highlights ways in which the technology has been exploited to support new forms of assessment.

It is widely accepted that assessment must reflect course aims, objectives and underlying philosophy, and the four courses described here do this in a variety of ways. For example, T171 aims to introduce the Internet to students and makes use of web-based assessment in support of these aims. In THD204, students are introduced to a resource-based approach, and the assessment is designed to help them practise the skills they need to learn in this way. B823 aims to give students first-hand experience of managing knowledge, and the assessment provides an opportunity for them to reflect on their experiences. Finally, H802 relies on experiential and collaborative approaches, and these are integrated with the assessment.

In addition to these course-specific requirements, there are wider considerations that are applicable to other networked courses. The interactivity afforded by networking gives students access to a greater variety of resources, including those of fellow students, than they would have had with conventional, print-based courses. Lauzon (1999) suggests that this can lead to multiple constructions of reality, which makes conventional ideas on the assessment of content increasingly difficult to implement. Indeed, Gipps (1994) suggests that the assessment of any course that adopts a constructivist model of learning probably needs to be flexible in its approach to content:

"Should we still be so concerned with standardisation, with replicability and with generalisation?... the constructivist paradigm does not accept that reality is fixed and independent of the observer, rather reality is constructed by the observer and thus there are multiple constructions of reality. In this paradigm there is no such thing as a 'true score'."

(Gipps, 1994, p. 288)

The courses described here vary in their approaches to choice and flexibility in content and some also have a more flexible approach to content in particular parts of the course. They have adopted assessment strategies to accommodate these varying requirements. For example, in T171, where students are given an increasing degree of freedom as the course progresses, the assignments give them a choice of two topics: one of which is more open ended than the other, thus allowing scope for a range of personal interests (Weller, 2000).

This flexibility allows scope for a variety of approaches to assessment tasks, and room for individual development and initiative. The use of the Web as a resource for assignments gives another dimension to the concept of choice and flexibility, and the more proficient students often take the opportunity to research their topic in greater depth. It appears that many students on T171 appreciated the choice, since it catered for a wide variety of backgrounds and levels of experience (Mason & Weller, 2000).

The adoption of these flexible approaches to the assessment of content brings implications for students because they need the critical skills to manage the information resource, whether this is derived from discussions with peers or from use of the Web. The extent to which the course teaches these skills, or assumes they are prerequisite in students will depend on the level of the course, its aims and target audience. Furthermore, the degree of flexibility and openness in content offered by the course will impact on the level of critical analysis required in students in order to be effective. The extent to which assessment should provide opportunities for the practice of particular skills for information management must depend on an estimate of the skills base already possessed by students taking the course, and the degree to which the acquisition of these skills forms a part of course objectives.

This is exemplified by early evaluation work on THD204, which showed that, in spite of the introductory computer based tutorials, many of these undergraduate students lacked information-handling skills and experienced serious problems with information overload (Macdonald & Mason, 1998). The skills they needed included practical IT skills to negotiate the environment, together with cognitive skills to formulate queries effectively and to assess the relevance of the material they retrieved.

The same study showed that postgraduate students in H802 were in a different position, and a certain level of competency could be assumed in formulating searches and assessing the material they retrieved, although they appreciated practice in new IT skills.

Assignments can be used to help students practise these skills. For example, the following assignment from THD204 requires students to explore the various search methods available on the course CD-ROM and to give an assessment of their use in future course work. They then use the results of their search as source material for an essay on a course theme.

* Compare the use of three different CDROM search methods, describing outcomes, and offer an assessment of the various methods.

* Use the results of the searches to write an essay on a relevant course issue.

(THD204)

This assignment involves the practising of a variety of information handling skills, including operating electronic search tools, navigating through electronic resources, reading appropriate sources selectively, getting the gist of important points, and finally integrating this into written work. The evaluation of this assignment showed that students had benefited from the experience and were more competent in their use of the tools, although many problems still remained with the more complex cognitive skills of selective reading, reflection and integration (Macdonald, Mason & Heap, 1999).

A similar exercise appears in T171 where students are required to review a number of web sites for a home page. In their review, they are asked to apply "clear thinking" principles, including the perspective of the sources, the context in which ideas apply and in separating observations from theories.

The central importance of skills for the management of information is underlined in the course B823, which describes the concept of sensemaking, defined as information gathering, analysis, synthesis, sharing and re-use -- it is closely related to the information handling skills described for THD204. Whilst it is recognised that the students who take this course will already have developed critical thinking skills in previous MBA studies, sensemaking receives overt attention in the assessment, where students are required to reflect on the process as a result of their first-hand experience in using a variety of Internet tools to search for and assess information.

ASSESSMENT WHICH EXPLOITS ONLINE INTERACTIVITY

Networked courses offer the potential for greater and more sustained interactivity in distance courses than has previously been possible. They also offer scope for innovative assessment practices that make use of this interactivity. The following account illustrates ways in which assessment has been used not only to reflect the use of interactive teaching and learning in distance courses, but also to help students develop the skills they need to optimise their use of the medium.

Building online collaborative' skills

To participate effectively in online collaborative activities, there are a variety of skills that students need to acquire. Assessment can be used as a way of encouraging them to participate and to reflect on their experiences. The four courses described here have all used activity-based assessment to a greater or lesser degree. It means that students learn by undertaking a series of tasks and are then required to reflect on their experiences in the related assignment. In this way, they learn by their mistakes and gain a first hand understanding of the issues. Evaluation work from THD204 and H802 showed that students appreciated this integration of activities with assessment, because it guaranteed the involvement of all students in online discussions.

With respect to the skills of online conferencing, at the most basic level, students may be encouraged to participate, simply by awarding marks for a message input into the system, in order to ensure that all students get online at an early stage in the course.

Of course, this is no guarantee that the message submitted will contain information of substantive academic content, or contribute in any way to student learning, and other assessment devices may encourage more meaningful participation. Indeed, three of the courses described here have an assignment for which students are required to discuss a course topic in a small online group. For example, the first assignment in T171 requires students to summarise an article on online collaboration, and then share and discuss their summary online. As evidence of participation, they have to include two of the messages they contributed during the online debate, in their assignment.

Summarise an article on online collaboration.

Post summary to online conference.

Debate issues online.

Marks given for:

* Key points on group work from reading and personal experience; and

* 2 messages submitted to conference.

(T171)

The evaluation of this assignment and experience on other courses has shown that it had a positive effect on the quality of contributions and acted as a focus for tutorial support, as well as being crucial in developing online discussion skills.

This relationship between online participation and assessment has been observed elsewhere (Warren & Rada 1998; Wilson & Whitelock, 1998). It appears that students who are new to computer conferencing may take time before they have developed the various skills needed to feel comfortable interacting in this medium. In this context, assignments may be useful as a way of building skills for online collaboration in an incremental way and in raising awareness of potential pitfalls. This can be accomplished by setting an activity linked assignment and encouraging students to reflect on their experiences, with a view to putting the lessons learnt into practice in a subsequent assignment. These examples demonstrate the significance of encouraging reflection during the assessment as an integral part of experiential learning through 'activity linked' assessment.

For example, evaluation of an online collaborative project assignment on THD204 revealed that whilst many students valued the collaboration, their online collaborative skills were poorly developed, and this hampered the process of collaboration. The course team, therefore, developed an earlier conferencing assignment to encourage reflection on group management by requiring students to produce a strategy for online collaboration (Macdonald, Mason, & Heap, 1999). This had a positive impact on the outcome of collaboration in the later online collaborative project assignment.

Discuss a course topic in a small online group. Marks awarded for:

* five messages contributed to group debate, each supported by another message to illustrate their ability to interact and build on other contributions;

* summary of whole debate; and

* statement of group's proposed strategy for online collaboration, with a personal assessment of the strategy.

(THD204)

A similar opportunity is offered in T171, where students are also introduced to online collaboration in a two-step process. A reflective exercise in the first assignment requires them to draw up a short list of key points on group work from their reading and from personal experience. This is then used as preparation for the next collaborative assignment. However, in this course, the exercise was intended simply to give students a taste of online collaborative work, so even if the group did not function well, they were not penalised.

Summarise an article on online collaboration.

Post summary to online conference.

Debate issues online.

Marks given for:

* Key points on group work from reading and personal experience; and

* Two messages submitted to conference.

(T171)

One of the aspects of collaborative work that caused problems with students on THD204 was in the management of online collaboration, particularly in decision- making tasks such as time management and task allocation. Similar observations are made in some feedback from students in T171, who also commented on the difficulty of collaborating when relying on asynchronous communication (Mason & Weller, 2000). Many groups in THD204 overcame this with face-to-face meetings or audio conferencing, and this observation has been noted elsewhere (Collis, Andernach & van Diepen, 1997). This principle of using synchronous -- working in conjunction with asynchronous -- collaboration is integrated into B823. In this class, students are offered the opportunity to meet face to face at a residential school before their online collaborative work, in addition to the use of real time conferencing using Lyceum groupware. Lyceum provides students with audio communication over the Internet, as well as the use of shared workspace.

Producing a collaborative product

Another form of assessment with networked courses is where students are asked to work collaboratively online to produce a joint product, which then forms part of an assignment. There are two assessable elements to this: the process of collaboration and the collaborative product itself. Inevitably, one of the difficulties in implementing this fairly is in establishing the contribution of the individual.

One strategy is to design an assignment that contains an individual component and to weight marks towards that component. For example, in B823, students are required to work collaboratively to produce a report on knowledge management technologies. As part of the collaborative approach, students have to plan a strategy and divide the information resources between group members. They then produce summaries of the various readings and share these with the rest of the team in synchronous or asynchronous exchanges. In this case, 20% of the marks are awarded for a collaboratively produced summary. The remaining 80% of the marks are awarded for individual contributions to the group report, which cover the costs and benefits of sensemaking and, secondly, a critical reflection of the practice of online collaboration.

Sensemaking in an Internet team. Collaborative work to produce a report that assesses the role of various knowledge management technologies, and to demonstrate an understanding of the process of sensemaking (information gathering; analysis; synthesis; sharing; re-use). Group has to divide up the information resources and individuals summarise their content for colleagues.

Individual marks awarded for:

* Description of Internet tools used, with costs and benefits;

* Reflection on the benefits and costs of working in a 'virtual team.'

Group marks for:

* Summary report on the roles of various Internet tools.

(B823)

This strategy may be extended by awarding marks for the individual contribution to the collaborative process. For example, in THD204, students are required to work on a report that provides a critique of a fictional newspaper article. Students write the core of the report individually and produce a collaborative summary and conclusion. Collaborative marks are awarded for the summary and conclusion, and for the overall quality of the discussion on the online conference. Individual marks are awarded for the individual's contribution to the group effort, as recorded in the conference transcript, as well as to the body of the report. In this way, the process of collaboration and the contribution of the individual are rewarded, as well as the final collaborative product.

Collaborative group project. As a group, provide a critique of a fictional article, based on course themes.

Individual marks for:

* Body of report;

* Contribution to online discussion; and

* Structure, coherence, and use of evidence.

Collaborative marks for:

* Summary and discussion; and

* Discussion of themes in group conference.

(THD204)

Another approach is to avoid giving marks for the collaborative product altogether, but instead to reward the process of achieving and reflecting on collaboration. In T171, students are required to produce a web channel as a collaborative exercise. There are, again, individual and group elements to the final product. Each student is responsible for producing a web page on a particular subject. They also produce an analysis of the effectiveness of group work, thus illustrating the principle of reflection as an important part of skills development. As a group, they need to decide who will do what, and what style of web page to adopt, and they produce a home page, with links to the individual contributions. However, marks are awarded for the individual web pages and the reflective commentary -- but not the home page itself -- so students are not penalised if it does not work successfully.

Group collaboration to create a web channel. The group produce a home page providing links to individually produced web pages. They must decide who will cover which subjects for individual web pages, and agree on a consistent style for all web pages.

Individual marks for:

* Individual web page in HTML with links to a minimum of five relevant web sites, a review of each site and table of contents; and

* Their reflection on the effectiveness of group work and technical/design features of the home page.

(T171)

ASSESSMENT WHICH EXPLOITS ELECTRONIC SUBMISSION

The use of electronic media for submission offers the potential for the inclusion of multimedia in assignments, because they no longer need to be printed out. In practice, however, most students have bandwidth restrictions if they work from home. In the courses described here, students submit assignments in HTML, thus allowing for links to further reading, or to images, and reflecting an emphasis on a more open and flexible approach to information use from a wide variety of diverse sources.

In H802, one assignment requires students to produce a set of hyperlinked pages to the most significant parts of the set book. Students have to select the most relevant sources, summarise their content with a personal commentary on their relevance and link them in a way that reflects the relationships between them.

To produce a hypertext guide to work from the set book on web based instruction, with commentaries on the various sources.

Individual marks allocated for:

* Explicitness and coherence of rationale for selection of sources;

* Design of guide; and

* Quality of comments.

(H802)

A similar philosophy is adopted on the course T171, where the use of HTML is encouraged throughout the course as a means of thinking and working with web material (Weller, 2000). Three of the assignments are concerned with the production of web pages, with two assignments being specified as "web reports," in which students write a report in HTML, with links to other resources.

To produce a Web report, made up of linked HTML documents, which must include essay plan, and further sources such as conference discussion, personal experience, or external documents. Students are given a choice of two titles for the report.

Marks given for:

* Essay plan

* Presentation, including web structure and design

* Content

(T171)

All three examples reflect the use of new kinds of resources: extracts from conference messages, reflections from a personal study log or external web resources. The assignments are assessed not only on content, but also on presentation and functionality in terms of ways in which the links are used, the use of graphics and clear layout and the use of the Internet as a resource.

END-OF-COURSE ASSESSMENT

ICTs offer new options for the end-of-course assessment, which are feasible for networked courses even when students are globally distributed. This is part of a growing trend in the use of alternatives to the traditional examination.

Balanced against this are institutional considerations of plagiarism, particularly for courses that are networked. The ease with which material from electronic sources can be cut and pasted into assignments, coupled with the possibilities for collaboration through networking means that students need new guidelines so that they are able to learn what constitutes justifiable use of resources, including each other's, and where plagiarism begins and ends. Serial presentations of a course within one year may add to the complexities, when the same assignment is used each year.

There are also new challenges in establishing identity when the students are never face to face with tutors or administrators. And traditionally, the exam is the time for distance courses when students would identify themselves with their work.

Two of the courses described here (THD204, B823) have a traditional end-of-course exam, whilst two (T171, H802) have alternative forms of end-of-course assessment, on the assumption that these alternatives to the traditional exam will be more appropriate to course aims, and more rewarding for students.

However, findings from the evaluation work on THD2O4 show that the majority of undergraduate students interviewed placed great value on the preparation for their exam as the opportunity to pull the whole course together. The revision period was the time when they revisited earlier work, and began to acquire an overall understanding of course issues. The impending exam served to heighten this experience and to lend motivation to their efforts. Similar observations have been made in campus-based universities (Entwistle & Marton, 1994).

It does appear that achieving a synopsis is not equally important to all students. From the same study, it was found that the H802 students interviewed were less reliant on the synoptic effect from studying for a final examination. This may be reflective of the type of course and the degree of independence of the students. These students were capable of making their own synopses in response to those elements of the course that were most pertinent to their work. The examinable component for this course requires them to reflect on their readings of web resources, experiences and conference interactions during the whole course to provide an assessment of the role of technology in mediating learning.

An extended essay which assesses the role of technology in mediating learning, to draw on material from all parts of the course.

Marks awarded for:

* Demonstrating an understanding and critical use of course materials;

* Argumentation of the question;

* Analysis of online interactions, personal experience; and

* Style and written presentation.

(H802)

The students were enthusiastic about their final examinable component and did not feel that the course was invalidated by the absence of the exam. They appreciated the increased flexibility it allowed and the time for reflection and integration of ideas.

Whether synopsis and motivation are indeed of significance to students, or to particular students, the form of final assessment must reflect the philosophy of the course. The end-of-course assessment in T171 gives students the opportunity to reflect on their work throughout the course and to demonstrate their expertise in relevant skills. Students are required to produce a web page, with links to two parts: a portfolio of evidence of tasks completed - together with their reflection on these tasks - and a summary and comparison of some of the key issues in two modules of the course. For part one, their evidence must include examples of participation in course conferences: an example of creditable note taking for one week, reflection on feedback from an assignment, and the impact it had on learning. Marks are awarded for presentation, good use of the Web as a resource and a good grasp of the benefits of writing for the Web, as well as for the content of the submission.

Construction of a web site, in two parts: 1) with links to evidence of work completed; 2) a review of content from two units, and an index page.

Marks awarded for:

Part I

* Presentation and functionality of web site;

* Examples of conferencing, note taking, solutions from previous exercises, with commentary on the learning experience; and

* Reflection on one tutor marked assignment feedback.

Part II

Content; and

Imaginative use of web links, use of images, use of the web as a resource.

(T171)

Seven out of the 24 markers gave feedback on this assignment in 1999 and were impressed by the high quality of the submissions. Tutors are required to certify that the portfolio is the students work. And although concerns remained as to the authorship of the work, others felt that identity was relatively easy to establish because of the relationship they had established with the students through online collaborative work. Some tutors also felt that students might have produced reflective notes specifically for the portfolio, rather than as an ongoing process through the course. Whilst students are encouraged to develop a hypertext study journal through the course, they are not assessed on it as they go along, and so there is scope for inventiveness. We do not know whether this form of end-of-course assessment provides the synopsis and motivation which was valued by the undergraduate students in THD2O4, although clearly it presents the opportunity for students to research areas of interest, to encourage reflec tion and to demonstrate their expertise in web page design and development.

CONCLUSIONS

The interactivity offered by online conferencing and the submission of assignments in electronic form offer new and exciting potential for the assessment of networked courses. At the same time, it is important that the assessment should reflect course aims and objectives, and that it should provide a corresponding level of affordance and flexibility in content to that provided in the course itself. In addition, the assessment must reflect the values the course is trying to teach. For example, H802 is initiating students into a constructivist environment for learning and the assessment methods are in tune with this underlying philosophy. T171 enculturates students into the world of ICT, and the web assignments are one of the main means of accomplishing this.

The assessment described here illustrates the feasibility and scalability of a variety of strategies that have previously been used in other universities with small course presentations, and clearly there is potential for a wider implementation within the UKOU. Examples described from the four courses in this article are: collaborative assignments, web-based assignments, assignments to encourage independent learning skills and enhancements to assignment development and feedback.

The assessment strategies on these four courses raise the following six issues, which have a broader applicability:

1. The use of networking on a course allows scope for greater collaboration and interactivity than would have been possible in a print-based course delivery. It also introduces access to wider resources, including fellow students, and this challenges conventional ideas on the assessment of course content. At the same time, course teams need to design assessment that reflects the particular aims and objectives of their networked course.

2. Students in these courses need to develop particular skills, which can be reinforced and valued through the assessment. The use of an experiential approach, by integrating practical activities with assignments, is a useful model in this context. A series of linked assignments is another way to support the incremental development of skills.

3. Networking presents new challenges for designers of assessment in establishing student identity and in ensuring that students are aware of guidelines on plagiarism. It also opens up new avenues for improving the fairness and consistency of marking by offering the potential for tutors to establish a dialogue with course teams and with each other on marking moderation.

4. A variety of assessment strategies have been described here, which utilise the potential offered by online interactivity and electronic submission.

* Assessment can be used to support online collaborative learning in a variety of ways, from encouraging participation in online conferencing to the production of a collaborative product. By combining activities with a reflective component they can be encouraged to learn and build on their experiences.

* The electronic submission and marking of scripts lends itself to the use of a variety of formats in assignments, notably HTML. This opens up new opportunities for innovative assignments that encourage research beyond conventional course materials and cater for a variety of interests and levels of experience.

5. ICT provides opportunities for alternatives to the proctored final exam that are more logistically feasible for online and global courses. They may also be more challenging, rewarding and enjoyable for students. Whatever the choice might be, the importance to students of synopsis and motivation in end of course assessment should receive serious consideration.

6. Where the use of ICT is integral to a course, it should be integral to at least some aspect of the assessment in order to reflect the skills students are using to study the course.

Table 1

Networked courses included in this study

Course                        Level

T171: You, Your computer and  Undergraduate entry level
the Net (30 points)           course, first presented in 1999,
                              delivered to 12,000 students
                              in 2000.

THD2O4: IT and Society        Undergraduate, post-foundation,
(60 points)                   first presented in 1995.
                              1,600 students in 2000.



H802: Applications of IT in   Postgraduate course, first
Open and Distance Education   presented in 1998.
(60 points)                   47 students in 2000.


B823: Managing Knowledge      Postgraduate course, first
(30 points)                   presented to approx. 700
                              students in 2000.

Course                        Resources

T171: You, Your computer and  Online study guide, links to
the Net (30 points)           further resources, set books,
                              CD-ROM of course software,
                              First Class conferencing.

THD2O4: IT and Society        Printed course books, set books,
(60 points)                   CD-ROM personal library;
                              Web page with links to further
                              resources, First Class
                              conferencing.

H802: Applications of IT in   Virtual Campus, including
Open and Distance Education   Bulletin Board, online Study
(60 points)                   Guide and links to further
                              resources.

B823: Managing Knowledge      Printed course books,
(30 points)                   CD-ROMs,
                              First Class conferencing.

Course                        Assessment strategy

T171: You, Your computer and  4 assignments + end
the Net (30 points)           of course assessment



THD2O4: IT and Society        6 assignments (including 1
(60 points)                   double weighted project)
                              + exam



H802: Applications of IT in   3 double weighted
Open and Distance Education   assignments + final
(60 points)                   examinable component


B823: Managing Knowledge      3 assignments + exam
(30 points)

REFERENCES

Barrett, E., & Paradis, J. (1988). Teaching writing in an on-line classroom. Harvard Educational Review, 58(2), 154-171.

Bos, E.S., Kikstra, A., & Morgan, C.M. (1996). Multiple levels of use of the Web as a learning tool. In Proceedings of ED-TELECOM '96 (pp. 31-36), Boston, MA. Charlottesville, VA: Association for the Advancement of Computing in Education.

Brown, S., Race, P., & Bull, J. (1999). Computer assisted assessment in Higher Education. London: Kogan Page.

Collis, B. (1998). New didactics for university instruction: Why and how? Computers and Education, 31(4), 373-393.

Collis, B., Andemach, T., & Diepen, N. van (1997). Web environments for group based project work in higher education. International Journal of Educational Telecommunications, 3(2/3), 109-130.

Davis, R., & Berrow, T. (1998). An evaluation of the use of computer supported peer review for developing higher-level skills. Computers and Education, 30(1/2), 111-115.

Entwistle, N., & Marton, F. (1994). Knowledge objects: Understandings constituted through intensive academic study. British Journal of Educational Psychology 64, 161-178.

Fiedeldy, A. (1999). Online collaboration. Case study from environmental psychology (postgraduate). In C. Morgan & M. O'Reilly (Eds.), Assessing open and distance learners (pp. 214-216). London: Kogan Page.

Gipps, C. (1994). Developments in educational assessment: What makes a good test? Assessment in Education, 1(3), 283-291.

Gorp, M. van, & Boysen, P. (1997). ClassNet: Managing the virtual classroom. International Journal of Educational Telecommunications, 3(2/3), 279-291.

Jones, M., Kear, K., & Rielly, A. (1998). The design, development and use of a CD-ROM resource library for an Open University course. British Journal of Educational Technology 29(3), 241-254.

Kwok, R.C., & Ma, J. (1999). Use of a group support system for collaborative assessment. Computers and Education, 32(2), 109-125.

Lauzon, A. (1999). Situating cognition and crossing borders: resisting the hegemony of mediated education. British Journal of Educational Technology 30(3), 261-276.

McConnell, D. (1999). Examining a collaborative assessment process in networked lifelong learning. Journal of Computer Assisted Learning, 15(3), 232-243.

McConnell, D. (2000). Implementing computer supported cooperative learning (2nd ed.). London: Kogan Page,

Macdonald, J. (1999). Appropriate assessment for resource based learning in networked environments. Unpublished doctoral thesis, Open University, Milton Keynes, UK.

Macdonald, J., & Mason, R. (1998). Information handling skills for resource based learning. Open Learning, 13(1), 38-42.

Macdonald, J., Mason, R., & Heap, N. (1999) Refining assessment for resource-based learning. Assessment and Evaluation in Higher Education, 24(3), 345-354.

Mason, R. (1995). Using electronic networking for assessment. In F. Lockwood (Ed.), Open and Distance Learning Today (pp. 208-217). London, Routledge.

Mason, R., & Weller, M. (2000). Factors affecting students' satisfaction on a web course. Australian Journal of Educational Technology 16(2), 173-200.

Warren, K.J., & Rada, R. (1998). Sustaining computer mediated communication in university courses. Journal of Computer Assisted Learning, 14, 71-80.

Weller, M., & Mason, R. (2000). Evaluating an Open University web course: Issues and innovations. Paper presented at Networked Learning 2000. Lancaster, UK.

Weller, M. (2000). Creating a large scale, third generation, distance education course. Open Learning 15(3), 243-252.

Weller, M. (2000). The use of narrative to provide a cohesive structure to a web-based course. Journal of Interactive Media in Education, July 2000 [Online]. Available: http://www-jime.open.ac.uk

Wilson, T., & Whitelock, D. (1998). Monitoring the on-line behaviour of distance learning students. Journal of Computer Assisted Learning, 14, 91-99.

JANET MACDONALD, ASSOCIATE LECTURER, THE UK OPEN UNIVERSITY IN SCOTLAND

E-MAIL: [e-mail]

MARTIN WELLER, FACULTY OF TECHNOLOGY, THE UK OPEN UNIVERSITY, MILTON KEYNES

E-MAIL: [e-mail]

ROBIN MASON, INSTITUTE OF EDUCATIONAL TECHNOLOGY, THE UK OPEN UNIVERSITY, MILTON KEYNES

E-MAIL: [e-mail]