Markers Assistant--a software solution for the management of the assessment process.

Technology-mediated instruction is changing the educational environment by adding another dimension to teaching and learning. Despite the advances in technology and the way we use technology, little attention has been given to providing technology solutions that help create, conduct, and manage the central task of assessment. Software to manage and conduct assessment presents many challenges. This article examines a stand-alone, flexible software environment designed to provide all the tools required to view, mark, store, and deliver results efficiently while improving the feedback to the students. A description of the software and its functions, the techniques and methods developed to overcome the limitations that software presents when used to conduct assessment is examined. Case studies are presented that describe: (a) the types of assessment the software has been used to conduct and manage; (b) the types of problems that were encountered; and (c) the solutions and techniques that were discovered and used to overcome the problems.

**********

Currently online learning environments are providing new and diverse methods of publishing course material that can be accessed by all students regardless of their geographical location. Students are using discussion groups to learn and share information. Assignments are being completed in electronic form and can be submitted electronically by way of custom submission systems or e-mail. A greater number of students from diverse areas can now participate and learn in ways never experienced before. Most of the effort in developing the tools and applications to facilitate these environments is focused on developing tools to present, deliver, and communicate educational content to the students. Assessment on the other hand is not keeping pace and is not considered a major priority (Anderson, 2001).

There are tools to provide for formative and diagnostic assessment, but very little work has been done to provide the tools to allow the management of summative assessment. The net result is an increase in the marking burden associated with this type of assessment (Martin, 2001) as much of the management is still a manual process (Bull & Stephens, 1999).

ASSESSMENT

Assessment of students is a major task undertaken by all teachers, lecturers, and trainers at all education levels. It can be defined as the process of obtaining information that is used to make educational decisions about students, to give feedback to the student about his or her progress, strengths, and weaknesses, to judge instructional effectiveness and curricular adequacy; and to inform policy (Dietel, Herman, & Knuth, 1991). The assessment process generally consists of a number of tasks. These include:

* creating assessment material;

* distributing the assessment task;

* conducting the assessment;

* collection and storage of the submissions;

* grading and evaluation of the submissions;

* moderation of the assessment;

* recording marks and grades; and

* delivery of feedback and results to the students.

Assessment is more than just a process of evaluation. It is also a management issue where the assessor develops their own customised methods to complete the assessment process and record, manage, and deliver the results (Stephens, Sargent, & Brew, 1999). In an environment of ever increasing class sizes (Thomson, 2001), there is pressure on the assessors to ensure that the marking and feedback are always consistent and completed within the required timeframes. Individual lecturers are trying to deal with ever increasing amounts of assessment within a time bottleneck, which is a consequence of this individual approach to marking (Stephens et al.).

MARKERS ASSISTANT

To meet the demands of assessors, an application titled, Markers Assistant has been developed. Markers Assistant is a Windows-based application that provides a single flexible environment where a variety of assessment tasks can be created and managed while providing all the advantages that exist in having the assessment data in an electronic form. In designing the application it was considered important to develop a tool that was flexible enough to suit a variety of assessment tasks. The application should be able to being used to assess tasks such as programming assignments, essays, short answer and report type formats, databases, spreadsheets, graphical information, and so on. Assessment can vary depending on the type of information to be assessed, the method being used, the media in which the submissions are created, the quantity and complexity required to meet the criteria of the assessment task, and the policies and techniques that exist within the organisation performing the assessment.

Marking Guides

Central to the application is the marking guide. According to the Australian Universities Teaching Committee (AUTC) the process of assessment should provide greater attention to the communication of clear assessment criteria to students through the development and use of marking guides (James, McInnis, Devlin, & AUTC, 2002). They go on to state:

* the need for high quality and individual feedback;

* an ability to assess a diverse range of students; and

* an ability to manage a high volume of assessment and coordinate an assessment team.

Markers Assistant provides the facilities to create a structured marking guide to suit an assessment task (Figure 1). An item in the marking guide may be a heading item that has no mark associated with it but provides information to explain or group items within the marking guide or it may define assessment criteria and have an associated mark or weighting that represents the maximum marks that can be applied for that marking criteria item.

The marking guide criteria is more efficient and informative if organised in hierarchical groups where the high level criteria must be met before other items within that criteria are considered. Grouping allows marking guides to be set up that orders information into related topics, making it easier to interpret, speeding up the marking process.

Once a marking guide is created, Markers Assistant displays the marking guide in the form of checkboxes organised in its hierarchical configuration (Figure 2). Markers identify the higher-level criteria and work down through the criteria as required. If the high level criteria are not met then the remaining items in the group are disabled and are not considered and marks deducted accordingly. Each item in the marking guide can be switched off by selecting its check box or an adjustment made to the allocated mark by selecting the item with the right mouse button.

An item dialog is displayed where the marker can adjust the allocated mark and add a comment clarifies the expected outcomes for that particular item (Figure 3). Once the marking guide is complete and loaded into the application, a list of students can be loaded or individual students added by way of an add student dialog. Individual students are identified by information supplied through a student list consisting of a unique identifier, such as student number or ID and an associated e-mail address if results are to be delivered by way of e-mail.

[FIGURE 1 OMITTED]

Each student loaded into the application can be found either by identifying them in a drop down menu or by searching all or part of their ID or e-mail address through a search dialog. The student identifier is used through out the process to identify a particular student and their associated view of the marking, their allocated marks and comments.

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

Automation

A major strength of the Markers Assistant is the ability to take advantage of the inherent nature of electronic submissions. Files submitted electronically can be organised in structured ways enabling an application to automatically find, retrieve, and display these files using software designated by the assessor. Many of the submission tools available and used in online learning environments allow for submissions to be organised into folders identified by a unique identifier.

Markers Assistant provides the facilities to setup the environment where the student submissions can be retrieved by matching the student unique identifier to the corresponding folder and one or more files accessed and displayed through an application that the assessor determines (Figure 4).

Marking, Comments and Feedback

Marking consists of checking the marking guide items where the student has met the criteria, adjusting the allocated mark to one that represents the assessor's judgment of the criteria and adding any comments and adjustments to the submission. Comments can be added against an individual item in the marking guide or as a general comment that relates to the submission of a particular student. General comments may also have an adjustment value associated with them that will either increase or decrease the overall mark for a student. All general comments entered are stored in a general comments repository (All Comments) and can be applied to a different student if required (Figure 5).

[FIGURE 4 OMITTED]

From the very beginning of the development of the application, it was considered that the software must maintain a clearly defined focus to improve the management of the assessment process and also improve the feedback that is delivered to students. The AUTC states that "Timely, individual feedback is central to guiding learning. But to provide such feedback to hundreds of students simultaneously within a timeframe that ensures such feedback can be incorporated into student learning is a daunting prospect." (James et al., 2002, p. 32).

[FIGURE 5 OMITTED]

To illustrate the use of the application the following describes the assessment process where students have submitted a written report on a given subject. The assessor prepares a marking guide and assigns marks for individual items within the marking guide (see Figure 2). The assessor then loads the application with the students that are required to complete the assessment task. This results in a single project file consisting of the marking guide and a list of all students.

If for example, the marking is to be distributed to three markers, the assessor can then use the application to divide the marking project file into three project files that are then saved and given to the markers. The application allows the distribution of marking based on two options. Students can be allocated equally among the markers or through the markers allocation list comprised of a student's ID and associated marker ID. Each marking project created is tagged with the markers ID to enable future moderation of the marking when complete (Figure 6). For moderation purposes a selected number of students can be included in all project files created.

[FIGURE 6 OMITTED]

The Marking Process

Assessors then receive their marking project files containing a copy of the marking guide and their designated students. They load the marking project file into the Markers Assistant application and setup the environment to suit their desired configuration using the command line dialog (see Figure 4) and setting the preferences as required (Figure 7).

If the environment is set correctly, the first student submission will be found and displayed. The application checks the file creation date and compares it with the assignment submission date and alerts the marker of a late submission. In this example we will assume the students have submitted report formatted in Microsoft Word. Therefore the markers would use Microsoft Word (MS) to display the submitted MS Word documents as specified in the assignment specifications. It should be noted that the application does not restrict the marker to use just one application to view submissions. Facilities are provided to deal with the variations in submission file types that do occur. Where submissions are paper based, this facility can be disabled.

A marker then begins to examine the submission against the criteria set within the marking guide. By default the marking guide assumes the student has full marks and the marker will take marks from the student rather than starting from a zero mark. This is based on the assumption that most students will have completed more of the assessment task than not. It was found to be a more efficient approach to the marking and reduced the overall amount of time taken to fill in the marking guide.

[FIGURE 7 OMITTED]

If a particular criteria is not met, the marker can select the items check box, removing the tick and consequently deducting the associated mark from the total for the assessment. Where the marker feels the student has partially met the criteria the marker selects the item with the right mouse button and adjust the marks accordingly by way of the Item Adjustments dialog shown in Figure 3.

Feedback

At this point, the marker can also make a comment that relates to the particular item. This comment does not relate to the particular student being marked but focuses on informing what was expected or possible correct answers that could have been supplied for the particular item. Case studies, described later, found that markers tended to provide critical feedback to individual students rather than solutions. This tended to slow the marking process down and did not provide the basis for quality feedback.

Markers are asked to make comments about the issues the students were misunderstanding, misinterpreting, or provide examples of good work they considered useful and relevant. This information would later be merged with the other markers providing a collection of observations and examples that the assessor or moderator can view, add, and refine prior to the results being released to the students.

Comments and Adjustments

In addition to the item comments, the markers can also provide additional feedback through specific comments to specific students. These comments are only seen by that student. This facility enables information regarding elements within their submission to be highlighted and if required an adjustment be made to the marks allocated. The adjustment marks can either add or remove marks from the student or simply be a comment with no adjustment applied. The comments allow for areas within the marking guide that may be lacking to be addressed where required. For example, a comment and associated adjustment can be applied where the submission was late, or perhaps marks are awarded to reward work that was supplied but not considered in the marking guide. Each comment added is stored against the particular student and is also placed in a general repository where it can be reused if it is relevant to another student during the marking process. A list of all comments made for all students is available for analysis.

Merging Projects

When the marking is complete the assessor saves the results and returns the completed project file to the principle assessor for moderation. The marking project files are then merged into one marking project file ready for moderation, analysis, refinement and distribution to the students (Figure 8).

[FIGURE 8 OMITTED]

Moderation

At this stage the moderation process can begin. Moderation is the process of evaluating the marking from all markers for accuracy and consistency. Markers Assistant provides a number of tools to enable various views of the marking to be created. These include statistics relating to all or groups of results, results summaries, and the sorting and viewing of groups of selected students (Figure 9).

For instance a view of the overall marking statistics can provide an insight into the performance of the students as a group and an analysis of the individual items in the marking guide can reveal either areas where the students need more clarification or where the assessment criteria was vague and potentially misinterpreted by the students (Figure 10). This type of analysis is rarely available using traditional manual assessment processes.

A typical moderation will consist of viewing each markers completed marking statistics and comparing them with the other markers to ensure there is consistency.

Delivery of Results

Once the moderation is complete, the assessor examines each items comments entered by the markers. The assessor then refines these comments to provide adequate information that addresses any potential queries that may arise where students have lost marks for an item. It should be noted that the item comments will only appear within the student's results where they have lost marks for that item. Therefore, a student with low marks will receive more feedback than a student that did well. This is considered targeted feedback. This type of approach is used based on research that suggests that students do not read their feedback and are more concerned with the final grade where their result is comparable to their perceived level of effort. In these cases the student will accept the result and not investigate why (Wotjas, 1998). It is only those students that feel the grade does not reflect their perceived effort that will demand feedback. Targeted feedback therefore provides this type of information by providing an explanation of the desired or expected result only where the student requires an explanation. The feedback is not a criticism but a collection of solutions that the students use to discover the areas that need refinement, thereby encouraging them to think of the solution rather than the errors they made. (Rossling & Freisleben, 2000; Coats 2002). This is a subtle approach that seems to benefit both the student and the assessor. Once the item comments are complete, the assessor can then send the results to the students through their e-mail address (Figure 11). Results can also be saved as text or HTML files that can be printed and sent to students.

[FIGURE 9 OMITTED]

[FIGURE 10 OMITTED]

Where the student lost marks for a particular criteria, a "<" is used to indicate they lost marks and a number representing the associated item comment is displayed. The student can then map that number to information within the Special Comments to determine in more detail the reasons why they lost marks.

CASE STUDIES

Markers Assistant has been developed over a four year period and has been used within a variety of subjects within the School of Information Technology at Deakin University. Based on previous studies (Wells, 2003), the following case studies outline two types of assessment tasks managed using the application. They highlight the various issues encountered and the solutions that were developed to meet the needs of the assessment tasks.

Software Development

This unit was based around the assessment of students' ability to apply programming concepts using the Visual Basic programming language. As part of the course the students were required to submit eight programming assignments in a 13-week period. Around 430 students were enrolled in the course over four campuses. Processing and marking the 430 * 8 assignments required a solution that automated as many components of the marking process as possible, while maintaining and improving the assessment feedback to the students.

Each students' submission consisted of a Visual Basic (.vbp) project file and one or more source files depending on the requirements of the assignment. Assignments were submitted electronically through a custom submission system developed at Deakin University. Assignments were grouped in directories identified by: Assignment Number-> Student Number-> [assignment files].

The software is designed to accept a series of parameters that identified the location of the files, an application that would be run to view the files (in this case VB.exe) and any command line arguments the application may require. Student assignments were automatically retrieved and presented based on the current student being marked. In the Visual Basic example, the student's project file was used to compile, run, and present their assignment to the assessor. Facilities were provided to view the contents of the each file in the student's submission directory in text form. This enabled the assessor to inspect the code in all files quickly and simply, further speeding up the marking process (Figure 12).

Marking was distributed to the 10 markers in the form of a project file created by the software. Equal numbers of students were allocated and stored in project files and distributed to the markers. Each marker completed their assessment then saved and returned the project file to the principle assessor for compilation and analysis. The ability to create, distribute then compile project files returned from markers enabled the primary assessor to examine and adjust the assessments as a group providing a greater level of consistency and accountability.

[FIGURE 12 OMITTED]

A potentially time consuming component of the assessment process is the management of student queries regarding the results received. Students often either do not agree with the marking or require clarification regarding elements of the assessment. Regardless, the assessor must retrieve the student's results, check the assessment and either make adjustments or provide additional feedback to the student to address their concerns. The Markers Assistant software was extremely valuable when managing these queries within this unit. The assessor simply searches for the student, is presented with the completed marking guide and associated comments and adjustments, and is automatically presented with the student's compiled submission. Adjustments can be made and results returned to the students as required and the updated data saved and stored in an efficient manner.

Computers and Society and Professional Ethics 

This unit was conducted in 2002, 2003, and 2004 and consisted of between 370 to 480 students based over four campuses. Students were required to submit three written assignments in essay and short answer formats, submitted through electronic means as Microsoft Word formatted documents.

The assessment of essays and short answer questions requires the assessor to make judgments regarding the student's submission against criteria that are not easily definable. Assessment of essays using detailed marking guides has the potential to focus the assessor on structure and specific content and where the assessment is distributed to more than one marker, a greater variation in the results may occur (Kuisma, 1999). Assessment criteria should therefore be less specific and associated criteria marks reflect a range that allows the assessor to make summative judgements for each criteria. Formative assessment should then be added to each criteria item through the use of comments, examples, and instructions to coach and guide the student (Jackson, 1995). To facilitate the use of both summative and formative assessment, adjustments were made to the software to allow the associated mark for each item in the marking criteria to be adjusted and comments entered. The item dialog enabled the marker to adjust the mark associated with the item to between 0 and the maximum value assigned in the marking criteria. In addition facilities were added to enter a comment associated with the item (see Figure 3).

This enabled the assessor to build a collection of information or formative assessment that explained and quantified the information that would have been considered desirable to answer the particular item. This tends to educate the student, providing them with the information that relates to how they are doing and how specifically they might be able to improve upon what they are doing (Stefani, 1998; Coats, 2002), rather than the traditional method, which merely explains where and why they lost marks (Bruner, 1974). This also compensated for the inability of the software to annotate and make comments within the submission text.

Surveys at the end of the subject were conducted in 2002, 2003, and 2004 to gauge the students' beliefs regarding the assessment and the techniques used to define the assessment and provide feedback (Figure 13).

In the 2003 year, a lack of resources meant the assessment workload resulted in poor turn around times and limited feedback provided with each assessment returned to the students. This may account for the differences in the results compared to 2002 and 2004.

In the 2004 year, students were also assessed on their participation within the unit through six assessable tasks. These were not managed in the same manner as the three assignments, which were managed using the Markers Assistant. This resulted in a lack of feedback provided with the participation assessment. Responses regarding the lack of feedback may therefore have focused on participation feedback as all assignments that were assessed using the Markers Assistant contained detailed feedback and all students would have received the feedback within their results. Where students did receive feedback, most felt the feedback was above average.

Students were also invited to make comments regarding their assessment and the feedback they received.

  I particularly appreciated example answers for the questions that I
  got wrong. (2004)

  I was satisfied with the feedback I got as the mark was broken down
  and I could see where I went wrong and could also rate my performance
  to the average class performance. (2004)

  Assignment marking feedback was clear but I would like to have some
  kind of feedback on the online participation marks. (2004)

  Too generic. Felt like I was talking and receiving information from a
  machine. Seemed very computer generated. (2004)

  When we did receive the feedback it was excellent, it was in depth,
  pointed out the weaknesses and suggested ways to improve it. The
  unacceptable factor was the time frame in returning these assignments.
  At the time of completing this survey, I have completed all three
  assignments yet received feedback for only the first one. (2003)

  Overly wordy, possibly longer than the assignments I submitted! (2003)

  Some was too in depth to understand were you went wrong. (2003)

  The marking should cover feedback on both the areas that students have
  done well & also on areas that the students are slack. (2003)

  Perhaps a personalised input from the marker will be good. (2002)

  The assignment feed back was excellent. From the feedback, I was able
  to see where I went wrong and how I could have fixed the problem.
  Excellent part of the unit. (2002)

  This unit had more that I have seen in the last five. (2002)

Providing quality, consistent, prompt assessment in an environment where class sizes are large and resources limited presents many challenges. The development and use of the Markers Assistant in this unit has resulted in a more efficient assessment process, where the focus has shifted from issues associated with the management of the assessment process to providing more meaningful, useful, consistent, and prompt results and feedback. This has enabled the exploration and use of alternative, potentially more valuable types of assessment that previously could not have been considered due to resource constraints.

Conclusion

According to an AUTC report, a survey conducted by the University of Queensland in 2001 identified various assessment issues associated with large classes (AUTC, 2001). These included:

* excessive marking loads;

* the circumscription of valid assessment by necessity of management;

* giving sufficient prompt feedback;

* monitoring cheating and plagiarism;

* maintaining quality and consistency of marking; and

* lack of higher order thinking examined by certain assessment tools.

Addressing these issues using traditional methods of assessment management requires more resources. Using the Markers Assistant to manage the process and redirecting the resources to providing quality, consistency, and prompt assessment has resulted in an improvement in all cases where the software was used.

The use of this type of application has an enormous capacity to speed up and improve the management and assessment of student submissions. If you consider the assessment process is not just the marking of submission but a management issue with many associated tasks, the software proved very valuable in all components of the assessment process, especially when the class size is large.

In all cases where the software was used, it was believed that the turn around time from submission to delivery of the results was shorter than using traditional means. Assessment review requests were dealt with more efficiently and overall the management of the results was not a task in addition to the marking process but a consequence of using the software.

Further research is being conducted to guide the development of the software and to investigate the potential of automating more of the assessment processes and tasks.

A version of the software may be downloaded at http://www.surreal.com.au

References

Anderson, J. (2001). Final report: Flexible learning leaders. Research investigation on online assessment as an integral part of flexible online delivery. Retrieved January 24, 2006, from http://flexiblelearning.net.au/leaders/fl_leaders/fll01/finalreport/janice.doc

AUTC (2001). Teaching and assessment in large classes. Retrieved January 24, 2006, from http://www.tedi.uq.edu.au/largeclasses/pdfs/LitReview_3_Teach&Assess.pdf

Bruner, J. (1974). Beyond the information given. London: George Allen & Unwin Ltd.

Bull, J., & Stephens, D. (1999). The use of question mark software for formative and summative assessment in two universities. Innovations in Education and Training International, 36(2) 128-136.

Coats, M. (2002, July). Enhancing student learning through the assessment of outcomes: Developing and demonstrating essay writing skills. Paper presented at the Association for the Study of Evaluation in Education in South Africa (ASEESA) Conference, Johannesburg, South Africa.

Dietel, R., Herman, J., & Knuth, R. (1991). What does research say about assessment. Retrieved January 24, 2006, from http://www.ncrel.org/sdrs/areas/stw_esys/4assess.htm

Kuisma R (1999) Criteria referenced marking of written assignments. Assessment and Evaluation in Higher Education, 24(1), 27-40.

Jackson, M. (1995, July). Making the grade: The formative evaluation of essays. The experience of quality in higher education. Papers from the Experience of Quality in Higher Education Symposium, Griffith University, Brisbane, Australia.

James, R., McInnis, C., Devlin, M., & AU TC (2002). Assessing learning in Australian universities. Canberra: Australian Universities Teaching Committee. Retrieved January 24, 2006, from http://www.cshe.unimelb.edu.au/assessinglearning/docs/AssessingLearning.pdf

Martin, N. (2001). Web-based assessment. Testing Times, 1(4), 1-7

Rossling, G., & Freisleben, B. (2000, September). TOPKAPI: A tool for performing knowledge tests over the WWW. Paper presented at the Interdisciplinary Conference on Electronics and Computer Engineering in the Third Millennium, Davos, Switzerland.

Stephens, D., Sargent, G., & Brew, I. (1999). Comparison of assessed work marking software: Implications for the ideal integrated marking tool (IMT). Paper presented at the 5th International Computer Assisted Assessment Conference (CAA 01).

Stefani, L. (1998). Assessment in partnership with learner. Assessment and Evaluation in Higher Education, 23, 339-350.

Thomson, A. (2001). Growth is blamed for fall in standards. Times Higher Educational Supplement No. 1486 http://www.thes.co.uk/search/story.aspx?story_id=78988

Wells, J. (2003, August). Assessment Management using Software. Lecture Notes in Computer Science, 2783, 411-422

Wotjas, O. (1998) Feedback? No, just give us the answers. Times Higher Education Supplement, Sep 25 1998. Retrieved January 24, 2006 from http://www.thes.co.uk/search/story.aspx?story_id=53518

JASON WELLS 

Deakin University, Victoria Australia

[e-mail]

Deakin University, School of Information Technology
SCC883 Information Technology in Society
Assignment 1--Research Paper

Marks for John_Smith

Title page                                            2 / 2
Contents page                                         0 / 2 <
Abstract--Executive Summary                           3 / 5 <
Introduction                                          2 / 3 <
Body--Contents
  Clear--Concise--Easy to read                        3 / 3
  Structure--Layout--Presentation                     4 / 4
  Use of point form, graphics and tables              3 / 5 <
  Appropriate use of titles and section headings      2 / 3 <
  Writing style, creative language                    3 / 5 <
  Audience orientation                                4 / 5 <
  Use of scientific and technical terms and jargon    3 / 3
Conclusion                                            3 / 3
Harvard Style References
  Correct--Relevant                                   0 / 3 <
  Reflect a wide variety of sources                   0 / 4 <
Evaluation of the Research                           35 / 50 <:17
Sub Total                                            67

General Comments and Adjustments:
1. Your presentation was poor. Presentation is important and where
   possible use formatting, tables and graphics to layout and
   communicate your research.
2. Your assignment was late. You where told that you would lose 10 marks
   for each day the assignment was late: -10

Adjustments Sub Total                               -10
Total Mark                                           57 (Out of 100)

Legend: < - indicates marks lost
        :<number> - Comments have been made, refer to SPECIAL COMMENTS

SPECIAL COMMENTS
Refer to the following for explanations on why you may have lost marks
for individual items within the marking guide.

17. Evaluation of the Research
To research is to make knowledge. As a researcher, you will:
1. gather knowledge produced by others in the past,
2. produce new knowledge for yourself, and
3. present your new knowledge to your readers.
Often novice researchers leave out the second step, which I consider the
most important. You as the researcher should be contributing something
new to the conversation on your topic: an insight, a connection, a
question, a complexity that has not been offered before. Although most
of you did investigate in some depth the major areas many did not
present any indication that they really understood the main issues. It
is not good enough to simply state the facts and not drawn any
conclusions.
What did you learn from the facts ??.......

Figure 11. Student results

Which of the following do you agree with regarding the Assignment
Marking?

                                        2002  2003  2004

The marking was fair                    48%   28%   38%
The marking was consistent              28%   10%   16%
Feedback was provided                   58%   36%   34%
I did not understand were I went wrong  18%   31%   34%

How do you rate the assignment marking feedback you received?

           2002  2003  2004  Average

very poor   6%   16%    9%   10%
poor        7%   14%   12%   11%
average    35%   46%   26%   36%
good       38%   21%   39%   33%
very good  15%    3%   14%   11%

Figure 13. Students' feedback