Now that your students have created web-based digital portfolios, how do you evaluate them?
Student created portfolios are commonly used in teacher 'preparation programs to demonstrate teaching skills and expertise. As test scores alone lack the comprehensive scope needed for effective assessment and evaluation, portfolios can be implemented to interpret/make decisions regarding learning of teaching competencies (Cole & Ryan, 1998). With the recent influx of new teaching and learning technologies, schools are implementing digital portfolios. A web-based portfolio, as used in this article, is a digital portfolio that incorporates web-based materials into teaching and learning. This purposeful collection of work, captured by electronic means, is an exhibit of individual efforts, progress, and achievements in one or more areas (Wiedmer, 1998) that combines aspects of traditional and electronic or digital portfolios.
The program at College has begun to implement web-based digital portfolios with student teachers. Since clarity about the end product is the starting point for any excellent teacher preparation program (Campbell, Melenyzer, Nettles, & Wyman, 2000), it was necessary to develop a systematic plan for evaluating these portfolios. For this purpose, we created rubrics, which are scoring guides with sets of criteria that describe levels of performance or understanding. They are formulated to assist in clarifying, communicating, and assessing teacher expectations (Custer, 1995; Burch, 1997), and can be powerful tools for teaching and assessment (Goodrich, 1997).
The matter of evaluating portfolios is a critical one. Several issues were considered before the process of designing the rubrics began (adapted from Barrett, 1995):
* purpose and the focus of the portfolio ("why" and for "whom"--who is the audience and for what purpose is it being created?);
* nature of outcomes expected ("what"--the institution's expectations of what will the student be able to do?);
* nature of evidence ("how" will the expected outcomes be shown-- which sample documents and products are to be included?); and
* time span ("when"--expected over what period?).
One needs to specify the components of the portfolio that will be evaluated as well as decide what constitutes an acceptable level of proficiency. The incorporation of web materials into the portfolio adds another dimension to be evaluated.
USING RUBRICS FOR EVALUATION
The creation and use of rubrics can ease the burden of portfolio evaluation. In any given rubric, key elements, traits, or dimensions to be evaluated are listed. Criteria discriminate among different levels related to the understanding of content, proficiency of a skill or process, and/or quality of a product or performance (Marzanno, Pickering, & McTighe, 1993). Rubrics place a value on the information that is gathered using assessment tools such as projects, products, and performances. The challenge is in designing a rubric that specifies the criteria needed but is flexible enough to allow differences in styles without stifling creativity (Burch, 1997).
In the rubrics, a set of expectations are provided for students, describing what will be assessed and the standards used in the assessment, enabling students to become more thoughtful judges of their own work (Goodrich, 1997). The authors have found these rubrics to increase consistency in the rating of performances, products, and achievements by enabling teachers to focus on what elements of a performance or product are most important and not be distracted by subjective concerns. Thus, the thorny scoring issues for the portfolio were weighed and settled before the evaluation process began.
Rubrics must be valid, reliable, systematic, and practical if they are to be used. Some suggestions for developing an appropriate rubric are (Marzanno, Pickering, & McTighe, 1993):
* decide on the number of levels of performance needed;
* describe the standards of performance to use as benchmarks;
* use demonstrative verbs that describe observable behaviors; and
* reflect and revise.
The rubric should use language that is clear and positive with articulate gradations of quality (Goodrich, 1997). Words that are vague should be avoided and precise descriptive language for each level of performance should be used (Rolheiser, Bower, & Stevahan, 2000). A four-point rubric is frequently used in evaluating student work. A typical format that can be adapted is found in Table 1:
Table 1 Four Point Rubric Score Level of Performance 4 Demonstrates exceptional performance or understanding; exceeds requirements 3 Demonstrates acceptable performance or understanding; is the standard 2 Performance or understanding is emerging or developing but does not meet required standards 1 Clearly does not meet the standards; makes serious errors or omissions
A Sample Rubric
A rubric, based on the four-point model, has been developed to evaluate the required components of the web-based digital portfolio. (Figure 1) The three main elements being evaluated are:
* form--design and aesthetics;
* function and usability--ease of use; and
* components--presence and communication of the required artifacts.
Please note that the weights given to the three main elements can be adjusted to reflect the importance of each element as deemed appropriate for your program.
The required components for our students were: (a) philosophy of education; (b) unit of lessons following the required lesson plan format; (c) unit overview; and (d) instructional strategies used and an explanation of how these were used.
The four levels of performance proficiencies have been defined using clear descriptive terms. The inclusion of web-based materials has necessitated the addition of this aspect to the evaluation process.
The following three examples of student portfolios illustrate the unit overview page, which is the home page of this portfolio (Figures 2, 4, and 6) and sample student-created web-based lessons (Figures 3, 5, and 7). These are sample pages from the digital portfolios created by three student teachers.
The first student portfolio sample pages (Figures 2 and 3), received a score of 3 in form, a score of in function and usability, and a score of 2 in components. A score of 3 in form was given because, even though the pages were attractive, there was nothing unique (the BD factor--Figure 1) in the presentation that would make the website stand out. A score of 4 in function and usability resulted from the convenient navigation, ease of use and effective links between pages. The student made sure that all pages were appropriately linked, information was presented in a way that one could find it easily and the overall layout of the page facilitated ease of use. The student received a score of 2 for the components because even though the ideas were well communicated, some required components were missing.
On the second portfolio (Figures 4 and 5), the student received a score of 4 in form, a score of 4 in function and usability, and a score of three in components. A score of four in form was given because the pages were not only attractive and easy to read, but also the student designed the page in a manner that would highlight the unit theme. This was a unique feature seen only in this particular student's portfolio. A score of four in function and usability resulted from effective and easy navigation between pages. All pages were well connected through links that were easy to find and there were no broken links throughout the portfolio. A score of three for components was because all required artifacts were present and well communicated. However, there were a few grammatical errors and typos, therefore, precluding a score of four.
The third portfolio sample pages (Figures 6 and 7), received a score of two in form, a score of three in function and usability, and a score of three in components. The portfolio web pages were overall not well-formatted displaying poor color combination between background and text making the pages difficult to read, and irrelevant and un-proportionate use of images on the pages, thus the score of two on form. The portfolio web pages were somewhat easy to navigate; however, there were two broken links and therefore, received a score of three in function and usability. Finally, a score of three on components resulted because even though almost all necessary components for the portfolio were presented, some details were missing therefore precluded a score of four.
This rubric is an example of a scoring guide to evaluate web-based digital portfolios. The performance proficiencies can be altered to meet the needs of an individual program. The rubric should be considered a draft since assessment is an ongoing process-change it as needed. The key idea is in the definition-a rubric is a guide. As students begin to design web-based digital portfolios, they must be assessed on all aspects of the portfolio. This rubric has enabled the supervisors and faculty to evaluate the portfolios systematically and reliably.
Figure 1 Rubric for portfolio web page(s) Scale Elements Form -- Design and Aesthetics Visually appealing, organization, intuitive and clear layout, proper format, clear text, neat presentation. Score 4 Very Appealing and Imaginative: Exceeds Main points clearly presented; creative, Requirements attractive, appealing graphic elements OR included appropriately; good color Exceptional combination; well organized page layout; well divided sections; neat; has the "biblio-didly (BD)" factor. * 3 Appealing: Meets Followed guidelines -- contains all the Requirements required elements of a good design but OR Standard does not have the "BD" factor. 2 Somewhat Unappealing: Close to Some errors in formatting; dull color meeting combination and layout; plain, somewhat standards messy presentation; random use of graphics; some omissions from guidelines. 1 Very Unappealing: Clearly does Significant errors in formatting; not meet inappropriate color combination and standards messy presentation; inappropriate use of graphics; significant omissions from guidelines. Scale Elements Function and Usability Convenient navigation, suitable download time, effective links without errors, overall ease of use. Score 4 Very Easy to Navigate and Use: Exceeds Links make excellent use of the web's Requirements timeliness, colorfulness and OR layout is easy to navigate; Exceptional suitable download time; no errors. 3 Somewhat Easy to Navigate and Meets Use: Requirements Links are somewhat effective; OR Standard presence of no outdated links; somewhat longer but acceptable download time; convenient navigation. 2 Somewhat Difficult to Navigate Close to and Use: meeting Few links and/or errors in links; some standards navigation difficulties and awkwardness; long download time. 1 Extremely Difficult to Navigate Clearly does and Use: not meet Critical errors and omissions in links; standards definite navigation difficulties and awkwardness; extremely slow in downloading. Scale Elements Presence and Communication of the "Components" Presence and clear communication of components. Score 4 Complete and Extremely Exceeds Well Communicated: Requirements All relevant components OR presented in an extremely Exceptional coherent, organized, and well- scripted fashion. 3 Complete and Meets Communicated Adequately: Requirements All relevant components OR Standard presented in an adequately coherent and organized fashion. 2 Incomplete and Inadequate Close to Communication: meeting Some components missing; not standards well written; presented in an unorganized and unpolished fashion. 1 Significantly Incomplete and Clearly does Awkward Communication: not meet Significant omissions of standards components; presented in an awkward and poorly written fashion. * Biblio-diddly is our special word for that special essence that sets the product above the rest. Copyright 2001 Dianne Goldsby and Minaz Fazal
Barrett, H. (1995). Technology support for alternative assessment [Online]. Available: http://transition.alaska.edu/www/portfolios/TechPortl.html [1999, June 1].
Burch, C. Beth. (1997). Creating a two-tiered portfolio rubric. English Journal, 86(1), 55-58.
Campbell, D.M., Melenyzer, B.J., Nettles, D.H., & Wyman, R.M. (2000). Portfolio and performance assessment in teacher education. Boston: Allyn and Bacon.
Cole, D.J., and Ryan, C.W. (1998, February). Documentation of teacher education field experiences of professional year interns via electronic portfolios. Paper presented at the 78th annual meeting of Association of Teacher Education. Dallas, TX.
Custer, R.L. (1995). Rubrics: An authentic assessment tool for technology education. Technology Teacher, 55(4), 22-37.
Goodrich, H. (1997). Understanding rubrics. Educational leadership, 54(4), 14-17.
Marzanno, R., Pickering, D., & McTighe, J. (1993). Assessing student outcomes. Performance assessment using the dimensions of learning model. New York: Elsevier Science.
Rolheiser, C., Bower, B., & Stevahan, L. (2000). The portfolio organizer. Succeeding with portfolios in your classroom. Alexandria, VA: Association for Supervision and Curriculum Development.
Wiedmer, T.L. (1998). Digital portfolios: Capturing and demonstrating skills and levels of performance. Phi Delta Kappan, 79(8), 586-589.
|Printer friendly Cite/link Email Feedback|
|Publication:||Journal of Technology and Teacher Education|
|Date:||Dec 22, 2001|
|Previous Article:||Future online teachers' scaffolding: What kind of advanced technological innovations would teachers like to see in future distance training projects?|
|Next Article:||Living in the present tense: Student teaching telecommunications connect theory and practice.|