Printer Friendly

Now that your students have created web-based digital portfolios, how do you evaluate them?

With the recent influx of new teaching and learning technologies, schools are implementing digital portfolios. The program at lona College developed a four-point rubric to evaluate web-based digital portfolios. A web-based portfolio, as used in this article, is a digital portfolio that incorporates web-based materials into teaching and learning. The three main elements evaluated were form (design and aesthetics), function and usability (ease of use), and components (presence and communication of the required samples). This rubric has allowed an objective, systematic, and reliable evaluation of the portfolios by the college supervisors.

**********

Student created portfolios are commonly used in teacher 'preparation programs to demonstrate teaching skills and expertise. As test scores alone lack the comprehensive scope needed for effective assessment and evaluation, portfolios can be implemented to interpret/make decisions regarding learning of teaching competencies (Cole & Ryan, 1998). With the recent influx of new teaching and learning technologies, schools are implementing digital portfolios. A web-based portfolio, as used in this article, is a digital portfolio that incorporates web-based materials into teaching and learning. This purposeful collection of work, captured by electronic means, is an exhibit of individual efforts, progress, and achievements in one or more areas (Wiedmer, 1998) that combines aspects of traditional and electronic or digital portfolios.

The program at College has begun to implement web-based digital portfolios with student teachers. Since clarity about the end product is the starting point for any excellent teacher preparation program (Campbell, Melenyzer, Nettles, & Wyman, 2000), it was necessary to develop a systematic plan for evaluating these portfolios. For this purpose, we created rubrics, which are scoring guides with sets of criteria that describe levels of performance or understanding. They are formulated to assist in clarifying, communicating, and assessing teacher expectations (Custer, 1995; Burch, 1997), and can be powerful tools for teaching and assessment (Goodrich, 1997).

The matter of evaluating portfolios is a critical one. Several issues were considered before the process of designing the rubrics began (adapted from Barrett, 1995):

* purpose and the focus of the portfolio ("why" and for "whom"--who is the audience and for what purpose is it being created?);

* nature of outcomes expected ("what"--the institution's expectations of what will the student be able to do?);

* nature of evidence ("how" will the expected outcomes be shown-- which sample documents and products are to be included?); and

* time span ("when"--expected over what period?).

One needs to specify the components of the portfolio that will be evaluated as well as decide what constitutes an acceptable level of proficiency. The incorporation of web materials into the portfolio adds another dimension to be evaluated.

USING RUBRICS FOR EVALUATION

The creation and use of rubrics can ease the burden of portfolio evaluation. In any given rubric, key elements, traits, or dimensions to be evaluated are listed. Criteria discriminate among different levels related to the understanding of content, proficiency of a skill or process, and/or quality of a product or performance (Marzanno, Pickering, & McTighe, 1993). Rubrics place a value on the information that is gathered using assessment tools such as projects, products, and performances. The challenge is in designing a rubric that specifies the criteria needed but is flexible enough to allow differences in styles without stifling creativity (Burch, 1997).

In the rubrics, a set of expectations are provided for students, describing what will be assessed and the standards used in the assessment, enabling students to become more thoughtful judges of their own work (Goodrich, 1997). The authors have found these rubrics to increase consistency in the rating of performances, products, and achievements by enabling teachers to focus on what elements of a performance or product are most important and not be distracted by subjective concerns. Thus, the thorny scoring issues for the portfolio were weighed and settled before the evaluation process began.

Rubrics must be valid, reliable, systematic, and practical if they are to be used. Some suggestions for developing an appropriate rubric are (Marzanno, Pickering, & McTighe, 1993):

* decide on the number of levels of performance needed;

* describe the standards of performance to use as benchmarks;

* use demonstrative verbs that describe observable behaviors; and

* reflect and revise.

The rubric should use language that is clear and positive with articulate gradations of quality (Goodrich, 1997). Words that are vague should be avoided and precise descriptive language for each level of performance should be used (Rolheiser, Bower, & Stevahan, 2000). A four-point rubric is frequently used in evaluating student work. A typical format that can be adapted is found in Table 1:
Table 1

Four Point Rubric

Score Level of Performance

4 Demonstrates exceptional performance
 or understanding; exceeds requirements
3 Demonstrates acceptable performance or
 understanding; is the standard
2 Performance or understanding is emerging or
 developing but does not meet required standards
1 Clearly does not meet the standards; makes
 serious errors or omissions


A Sample Rubric

A rubric, based on the four-point model, has been developed to evaluate the required components of the web-based digital portfolio. (Figure 1) The three main elements being evaluated are:

* form--design and aesthetics;

* function and usability--ease of use; and

* components--presence and communication of the required artifacts.

Please note that the weights given to the three main elements can be adjusted to reflect the importance of each element as deemed appropriate for your program.

The required components for our students were: (a) philosophy of education; (b) unit of lessons following the required lesson plan format; (c) unit overview; and (d) instructional strategies used and an explanation of how these were used.

The four levels of performance proficiencies have been defined using clear descriptive terms. The inclusion of web-based materials has necessitated the addition of this aspect to the evaluation process.

Examples

The following three examples of student portfolios illustrate the unit overview page, which is the home page of this portfolio (Figures 2, 4, and 6) and sample student-created web-based lessons (Figures 3, 5, and 7). These are sample pages from the digital portfolios created by three student teachers.

The first student portfolio sample pages (Figures 2 and 3), received a score of 3 in form, a score of in function and usability, and a score of 2 in components. A score of 3 in form was given because, even though the pages were attractive, there was nothing unique (the BD factor--Figure 1) in the presentation that would make the website stand out. A score of 4 in function and usability resulted from the convenient navigation, ease of use and effective links between pages. The student made sure that all pages were appropriately linked, information was presented in a way that one could find it easily and the overall layout of the page facilitated ease of use. The student received a score of 2 for the components because even though the ideas were well communicated, some required components were missing.

On the second portfolio (Figures 4 and 5), the student received a score of 4 in form, a score of 4 in function and usability, and a score of three in components. A score of four in form was given because the pages were not only attractive and easy to read, but also the student designed the page in a manner that would highlight the unit theme. This was a unique feature seen only in this particular student's portfolio. A score of four in function and usability resulted from effective and easy navigation between pages. All pages were well connected through links that were easy to find and there were no broken links throughout the portfolio. A score of three for components was because all required artifacts were present and well communicated. However, there were a few grammatical errors and typos, therefore, precluding a score of four.

The third portfolio sample pages (Figures 6 and 7), received a score of two in form, a score of three in function and usability, and a score of three in components. The portfolio web pages were overall not well-formatted displaying poor color combination between background and text making the pages difficult to read, and irrelevant and un-proportionate use of images on the pages, thus the score of two on form. The portfolio web pages were somewhat easy to navigate; however, there were two broken links and therefore, received a score of three in function and usability. Finally, a score of three on components resulted because even though almost all necessary components for the portfolio were presented, some details were missing therefore precluded a score of four.

SUMMARY

This rubric is an example of a scoring guide to evaluate web-based digital portfolios. The performance proficiencies can be altered to meet the needs of an individual program. The rubric should be considered a draft since assessment is an ongoing process-change it as needed. The key idea is in the definition-a rubric is a guide. As students begin to design web-based digital portfolios, they must be assessed on all aspects of the portfolio. This rubric has enabled the supervisors and faculty to evaluate the portfolios systematically and reliably.
Figure 1

Rubric for portfolio web page(s)

Scale Elements

 Form -- Design and Aesthetics
 Visually appealing, organization,
 intuitive and clear layout, proper
 format, clear text, neat presentation.

Score

 4 Very Appealing and Imaginative:
 Exceeds Main points clearly presented; creative,
Requirements attractive, appealing graphic elements
 OR included appropriately; good color
Exceptional combination; well organized page layout;
 well divided sections; neat; has the
 "biblio-didly (BD)" factor. *

 3 Appealing:
 Meets Followed guidelines -- contains all the
Requirements required elements of a good design but
OR Standard does not have the "BD" factor.




 2 Somewhat Unappealing:
 Close to Some errors in formatting; dull color
 meeting combination and layout; plain, somewhat
 standards messy presentation; random use of
 graphics; some omissions from
 guidelines.

 1 Very Unappealing:
Clearly does Significant errors in formatting;
 not meet inappropriate color combination and
 standards messy presentation; inappropriate
 use of graphics; significant omissions
 from guidelines.

Scale Elements

 Function and Usability
 Convenient navigation, suitable
 download time, effective links
 without errors, overall ease of use.

Score

 4 Very Easy to Navigate and Use:
 Exceeds Links make excellent use of the web's
Requirements timeliness, colorfulness and
 OR layout is easy to navigate;
Exceptional suitable download time; no errors.



 3 Somewhat Easy to Navigate and
 Meets Use:
Requirements Links are somewhat effective;
OR Standard presence of no outdated links;
 somewhat longer but acceptable
 download time; convenient
 navigation.

 2 Somewhat Difficult to Navigate
 Close to and Use:
 meeting Few links and/or errors in links; some
 standards navigation difficulties and
 awkwardness; long download time.


 1 Extremely Difficult to Navigate
Clearly does and Use:
 not meet Critical errors and omissions in links;
 standards definite navigation difficulties and
 awkwardness; extremely slow in
 downloading.

Scale Elements

 Presence and Communication
 of the "Components"
 Presence and clear
 communication of components.

Score

 4 Complete and Extremely
 Exceeds Well Communicated:
Requirements All relevant components
 OR presented in an extremely
Exceptional coherent, organized, and well-
 scripted fashion.


 3 Complete and
 Meets Communicated Adequately:
Requirements All relevant components
OR Standard presented in an adequately
 coherent and organized fashion.



 2 Incomplete and Inadequate
 Close to Communication:
 meeting Some components missing; not
 standards well written; presented in an
 unorganized and unpolished
 fashion.

 1 Significantly Incomplete and
Clearly does Awkward Communication:
 not meet Significant omissions of
 standards components; presented in an
 awkward and poorly written
 fashion.

* Biblio-diddly is our special word for that special essence that sets
the product above the rest.

Copyright 2001 Dianne Goldsby and Minaz Fazal


References

Barrett, H. (1995). Technology support for alternative assessment [Online]. Available: http://transition.alaska.edu/www/portfolios/TechPortl.html [1999, June 1].

Burch, C. Beth. (1997). Creating a two-tiered portfolio rubric. English Journal, 86(1), 55-58.

Campbell, D.M., Melenyzer, B.J., Nettles, D.H., & Wyman, R.M. (2000). Portfolio and performance assessment in teacher education. Boston: Allyn and Bacon.

Cole, D.J., and Ryan, C.W. (1998, February). Documentation of teacher education field experiences of professional year interns via electronic portfolios. Paper presented at the 78th annual meeting of Association of Teacher Education. Dallas, TX.

Custer, R.L. (1995). Rubrics: An authentic assessment tool for technology education. Technology Teacher, 55(4), 22-37.

Goodrich, H. (1997). Understanding rubrics. Educational leadership, 54(4), 14-17.

Marzanno, R., Pickering, D., & McTighe, J. (1993). Assessing student outcomes. Performance assessment using the dimensions of learning model. New York: Elsevier Science.

Rolheiser, C., Bower, B., & Stevahan, L. (2000). The portfolio organizer. Succeeding with portfolios in your classroom. Alexandria, VA: Association for Supervision and Curriculum Development.

Wiedmer, T.L. (1998). Digital portfolios: Capturing and demonstrating skills and levels of performance. Phi Delta Kappan, 79(8), 586-589.
COPYRIGHT 2001 Association for the Advancement of Computing in Education (AACE)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2001, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

Article Details
Printer friendly Cite/link Email Feedback
Author:Fazal, Minaz
Publication:Journal of Technology and Teacher Education
Date:Dec 22, 2001
Words:2080
Previous Article:Future online teachers' scaffolding: What kind of advanced technological innovations would teachers like to see in future distance training projects?
Next Article:Living in the present tense: Student teaching telecommunications connect theory and practice.


Related Articles
Expanding into the digital universe.
Implementing Web-Based Digital Portfolios in Three Teacher Preparation Programs.
Teaching Portfolio 101: Implementing the Teaching Portfolio in Introductory Courses.
How useful are portfolios to assess online learning.
Challenges of electronic portfolios: student perceptions and experiences.
Integrating information literacy into the Virtual University: a course model.
NetLert releases N-Contact. (New Products).
ACGI.
Action research on student portfolios.
Web-based digital teaching portfolios: fostering reflection and technology competence in preservice teacher education students.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters