Challenges of electronic portfolios: student perceptions and experiences.

With the implementation of national standards addressing technology, teacher preparation programs are faced with the issues of preparing teachers to effectively use and to seamlessly integrate technology across content areas. A team teaching approach at one major southeastern university required its methods students to produce electronic portfolios. The teaching team consisted of secondary education language arts and social studies faculty, inservice teachers, instructional technology faculty, and graduate students from both disciplines. This effort of modeling technological best practices resulted from numerous team meetings, intensive planning, and consistent project evaluation. The preservice teachers were required to attend technology seminars as part of regular classroom and methods work. Students were evaluated on their electronic portfolios that consisted of web-sites, digitally edited teaching episodes, databases, concept maps, and more. Through pretest and posttest surveys, the students were assessed on their perceptions of an electronic portfolio's value and their ideas of how technology can enhance teaching and learning in future classrooms. This article presents a discussion of the results from these assessments, procedural details, and the challenges and successes experienced by the teaching team and the students.

**********

With the development of National Educational Technology Standards for Students and Teachers (NETS) through the cooperation of the U.S. Department of Education, International Society of Technology in Education (ISTE), and other organizations, states are beginning to integrate technology standards for teachers and students within their courses of study. Institutions of higher learning are cooperating through consortia efforts with other colleges and K-12 faculty to increase the preservice technological experiences for students. Fulton (1998) noted that teachers must be technologically savvy if students are expected to graduate with adequate workplace skills. According to McKenzie (1999), only 20% of teachers report feeling very well prepared to integrate educational technology into classroom instruction, which seems to indicate that preservice education and extended staff training are needed. The development and use of electronic portfolios by preservice teachers may be one method to introduce the students to s kills for teaching and learning in future classrooms.

LITERATURE ON ELECTRONIC PORTFOLIOS

Lankes (1998) defined electronic portfolios as a "purposeful collection of student work that exhibits the student's efforts, progress and achievements" (p. 18). Electronic portfolios differ from traditional portfolios in that information is collected, saved, and stored in an electronic format (Barrett, 1998). Electronic portfolios allow students to demonstrate problem-solving and critical thinking skills using authentic and performance based assessment (Campbell, Cignetti, Melenyzer, Nettles, & Wyman, 1997; Meyer, 1992). While more research is required to assess the effectiveness of electronic portfolios with preservice teachers, existing research suggests that there are benefits to be gained from use with preservice teachers (Herman & Morrell, 1999; Polonoli, 2000). Numerous advantages related to the use of electronic portfolios are suggested by the literature. Bull, Montgomery, and Kimball (2000) wrote that electronic portfolios promote learner self-evaluation even as they maximize the use of diverse learni ng strategies. Barrett (1997) agreed with this assertion adding that electronic portfolios allow students to demonstrate problem-solving skills even as they are compelled to take responsibility for their learning. Campbell et al. (1997), argued that out of this learner responsibility comes a degree of control over the learning process and over the process of becoming a professional educator.

Students must make decisions and be prepared to analyze information and to demonstrate a body of knowledge while developing their electronic portfolios (Herman & Morrell, 1999). Throughout the process students are actively involved in their own assessment (Cole, Ryan, Kick, & Mathies, 2000).

The effective use of the medium requires ongoing evaluation (Barrett, 1998; Cole et al., 2000), both on the part of the instructor and of the student. Cole et al. (2000) concluded that the central advantage of all portfolios, not just electronic, is that the instructor is able to assess the student's process of learning. Evaluation, requiring teamwork, creative thinking, and reflection (Bull et al., 2000), goes beyond the limits of the traditional classroom. Herman and Morrell (1999) argued that electronic portfolios shift the balance from teacher-centered learning to student-centered learning. Multiple sources of evaluation combined with self-evaluation encourage preservice teachers to recognize and address individual strengths and weakness (Barrett, 2000; Corbett-Perez & Dorman, 1999; Herman & Morrell, 1999). Herman and Morrell (1999) noted that electronic portfolios encourage students to review their individual teaching values.

Barrett (1998) and Lankes (1998) maintained that electronic portfolios are an attractive method of alternative student assessment. However, issues relating to this electronic format must be addressed before requiring students to engage in this type of assessment (Corbett-Perez & Dorman, 1999). Research suggests that the implementation of an electronic portfolio project requires considerable investment of time and effort on the part of the instructor and the student (Cole et al., 2000; Linn & Baker, 1992).

PURPOSE AND METHODS

The southeastern university in this study schedules methods blocks for its preservice teachers each semester. During the fall semester, a team of teacher education professors, inservice teachers, instructional technology professors, and graduate students from both disciplines began meeting to prepare for technology infusion in the following spring language arts and social studies methods blocks. The team met extensively and formalized plans for the students' use of electronic portfolios. The goal was to integrate various available technologies that were traditional, such as database and word processing, and to incorporate additional contemporary uses, such as digital video editing. The electronic portfolios for both classes would introduce students to technologies they could use in their future classrooms, but also technologies that could enhance their career potential. Common applications for both methods blocks included database, word processing, presentation software, digital camera use, website developmen t, and digital editing. The students voluntarily participated in pretest and posttest surveys that assessed current level of technology knowledge and comfort level on technologies to be used during the semester. The students were also asked if the additional technology elements associated with the electronic portfolios were worth doing. Student comments were noted throughout the semester both informally by the teaching team members and formally through semester end surveys. Workshops provided early in the semester focused on the specific applications and skills students would need to produce their electronic portfolios. Additionally, online tutorials were available throughout the semester and the technology team members visited the classes frequently and offered support through open labs and scheduled workshops. An additional lab assistant was hired to assist with the digital editing process and to accommodate longer lab hours toward semester's end.

Students developed their portfolios over the course of the semester. The first step in developing the portfolio required the students to identify a metaphor for becoming a teacher. Once the students had developed this metaphor, they created an electronic presentation illustrating the metaphor. These presentations ranged from three to six slides. Video, audio, and graphics were used in the presentation to illustrate their metaphors.

Another assignment required that students create a 30-second video statement. The statement was a reflection of their philosophies of social studies or language arts education. Students first had to formulate a philosophy before using a digital camera to record their statements. As the students identified research problems related to the process of becoming a teacher, they were required to look for additional elements to include in their electronic portfolios. Instructors emphasized the dynamic nature of the electronic portfolios by encouraging students to revise documents and the portfolios throughout the semester. Students were also encouraged to continue collecting data/artifacts throughout the semester. Some examples of additional artifacts included were: observation evaluations, journal entries, reflective poems or illustrative artwork, copies or photographs of student work, lesson plans, tests prepared and/or student test data, videotapes, unit plans, other research projects, notes to or from students/p arents/teachers, and photo galleries illustrating teaching progress.

Throughout the process, students were encouraged to support one another. Support groups allowed students to share ideas and work together to master the technologies. An electronic discussion forum provided students with another way of discussing successes and problems as well as sharing project ideas.

RESULTS

Twenty-five students participated in the pretest survey. One social studies student dropped during the semester and another had a baby prior to the last few weeks of class, therefore, 23 students were surveyed during the posttest period.

Students rated their current level of technology knowledge on a scale of 1 to 3, with 1 = limited, 2 = fairly knowledgeable, and 3 = very knowledgeable. Pretest and posttest surveys indicated 16% rated themselves as a I and 16% as a 3. Sixty-four percent indicated fairly knowledgeable in the pretest and 60% as fairly knowledgeable at the posttest.

When asked to rate their comfort level on the technology to be used/or used during the methods block, 64% indicated fairly comfortable at the pretest assessment, with 20% indicating very comfortable. At posttest, 56% chose fairly comfortable, with 32% very comfortable.

The students were also asked if they believed the additional technology elements integrated into the methods block are/were worth doing. Four percent indicated no at both pretest and posttest assessments. Ninety-two percent indicated yes at pretest with 88% noting yes at posttest.

The students were asked to provide written feedback to two queries on both the pre and the posttest. One query asked the students to briefly state their philosophy regarding technology use in a future classroom. Another asked the students to complete the following sentence: Compared to traditional assessment methods (i.e., examinations, writing a paper, etc.) electronic portfolio assessment _____. The students' responses to these questions were analyzed for emerging patterns and trends using constant comparative analysis (Miles & Huberman, 1984). Several sub-themes emerged during data analysis. The next section, organized around these two broad questions, illuminates the emergent themes: technology as a tool in teaching and learning, motivation, future technology plans in teaching, and technology's role as assessment. Student names have been changed to aliases.

Technology as a Tool in Teaching and Learning

The overwhelming majority of the students made positive comments on both the pre and the posttest. Many of these students did, however, indicate varying degrees of trepidation on the pretest. Even though many were initially frightened about the prospect of using technologies with which they were unfamiliar, they were committed to learning how to use the technologies. To illustrate the contrast, comments from the pretest are presented and then posttest comments follow for each student.

On the pretest, comments like the following were common:

Tim: "I have so much to learn... but I know that the use of technology is absolutely vital..."

Mary: "Technology is a great idea; it is important to use technology in order to gain the students['] interest."

Leigh: "Technology is potentially a great experience for us as teachers."

Margaret: "Technology should be a valuable learning tool that I will use in the future."

Dana: "I believe this will be more concise and much more useful."

The students who made these comments summed up their experiences on the posttest with the following comments:

Tim: "Technology should be integrated as much as possible in order to have students comfortable with and knowledgeable about this vital component of their present and future."

Mary: "Because it is very important to use technology in instruction, I want to use it as much as possible."

Leigh: "I feel that technology in my future classroom is necessary to increase students' interest. I will use technology as much as possible in my future classroom."

Margaret: "Technology in the classroom will allow my future students to broaden their ideas and thoughts....I think that technology will be a big part of my classroom. I hope to be introduced to new software in the future."

Dana: "I enjoyed this more because it is more beneficial. I hope to incorporate technology daily."

One student, David, who was initially pessimistic, changed his opinions over the course of the semester. David initially said that the assignments using technology seemed "...new and confusing to me." However, on the posttest, David stated, "technology is the way of the future in education...I intend to use as much technology in my classroom as possible. I believe it makes teaching more interesting."

Karen remained optimistic throughout, but she was also cautious about the time commitment the integration of technology takes. On the pretest, she stated, "integrating technology is very time consuming with all of the other examinations, papers, etc..."

On the posttest, she commented, "I enjoyed using the technology of today, but I find it too time consuming and more difficult to use if the person I am working with does not understand the technology."

Finally, one student, Leslie, began and ended with a pessimistic attitude about her abilities to use technology. She stated on the pretest that technology would, "challenge me to increase my computer knowledge"; she summed up the semester's work with the comment, "the work was an incredible stress considering my lack of computer knowledge."

Motivation

The students indicated that technology is a useful way to reach their students. Several commented that technology is one way to not only motivate students to learn, but also to address diverse learning styles. Illustrations include the following statements.

Dianne: "I believe that computers and audio-visual technology not only appeal to students, but encourages them to learn."

Johnny: "Technology lets you incorporate more learning styles, allowing students to display their particular skills....This is a technological world, and these skills are important for my students to learn."

Eve: "...bringing technology into the classroom is extremely important because it will motivate students. Not only will the students find the lessons more interesting, but they will also benefit from the technology use by becoming familiar with everything."

Gina: "Technology facilitates whole language and student-centered classrooms and helps assess different learning styles."

Future Technology Plans

The majority of the students articulated their commitment to using technology in their teaching. One recurring theme in the students' discussions about their future plans to use technology was its usefulness for preparing their future students for life beyond high school. A case in point is Patty's comment that she will use technology because "...it is essential for students to have a working knowledge of technology...it is a vital aspect of our society."

Similarly, Meredith stated, "In order to prepare my future students for entry into the workforce, they must get accustomed to technological instruments while they are still in school. I want to provide practical applications in my classroom."

Further, Melissa articulated her plans to continue using technology because "...it will be helpful to my students." Melissa further stated, "It is my goal to prepare my students for the world which means incorporating technology."

Bethany's comment is one final illustration of this commitment to helping their future students use technology: "I will do my best to incorporate much of the material I learned in methods in my classroom. Technology will be a huge part of my curriculum. Students today need to have an understanding of the new technology that is available."

Even those students who were initially skeptical about the feasibility and usefulness of technology discussed their desire to continue learning how to use technology in their internships and beyond. For example, Leslie stated, "I will probably take summer workshops to apply technology in the curriculum... Lord help them! I will do as much as possible to increase my technology knowledge, but I am positive my students will be more knowledgeable."

Finally, Eve simply stated, "I will try to incorporate technology as much as possible as equipment permits."

Some students talked more concretely about their plans to use technology. They pointed to specific projects and activities they could assign to their future students. As a case in point, Margaret stated, "...my students will use word processing and the Internet for projects and research."

Further, Jennifer stated, "...technology in English language arts can help provide both teachers and students with more expanded opportunities to explore literature. I will use various software and the Internet for research in my classroom."

Greta added that she would "...continuously look for ways to incorporate technology" and that she would "teach students to edit videotapes of play performances in my language arts classroom." Cassidy discussed how relevant technology could be used in her future history class "...to look up historical information and find original documents."

Other students were less able to articulate specific plans to integrate technology. One illustration was David. Although he believes "...it makes teaching more interesting," he was not sure what kind of resources he would use in his classroom. He commented, "I am not sure what I would use in my classroom at this date."

Technology's Role in Assessment

Students consistently commented that the electronic portfolio is a better method to assess what they know. For example, Tim stated that the portfolio was a "living example of our work." Jennifer initially stated, "The electronic portfolio is wonderful. It will allow me to learn more about technology as well as to create a neat and informational way to present myself." Jennifer concluded the semester with the simple sentence, "The portfolio is a wonderful alternative."

Bethany commented that the prospect of developing an electronic portfolio was "a scary idea at first." In the end, she noted, "...I am very glad we used this method of assessment." Melissa believed that the portfolio allowed her to "...use technology and keep a better collection of my qualifications of being a potential teacher." She further believed that the portfolio "...gave me an opportunity to express why I should be a teacher." Melissa also stated, "I am not a great 'test-taker,' and this provided me an excellent opportunity to demonstrate my ideas in a variety of ways." Mary noted that the portfolio" ...was easier to do and showed my knowledge better than a traditional method." Julie commented, "The portfolio is a very interesting method that can be used to compile assignments throughout the semester." Although Anne initially believed that technology "added a new and difficult element to the process," she later stated that developing the portfolio was "...more enjoyable and a more exciting approach to the evaluation of knowledge." Finally, Meredith commented on the pretest that the electronic portfolio would "provide a broader range of evaluation based on my abilities and effort." At the end of the semester, Meredith still believed that the portfolio provided a "broader base for evaluation" and "was easier to transport."

Two students, Sam and Dianne, offered a more complete critique of the process. Sam was enthusiastic about the portfolio as a method of assessment on both the pre and posttest. He noted on the pretest, "...the electronic portfolio should be a new and exciting method of assessment...."

On the posttest, he said that the portfolio "was very beneficial to me." But, he added his concern that "...it could have been explained better and made to be better understood."

Dianne stated, "I plan to incorporate technology, but not run a computer lab." Dianne was also concemed about student access to technology. She stated,

"...this use of technology is a new way of teaching future teachers skills they will use in the schools. Unfortunately, the technology needed to accomplish this project was not readily available in the open computer labs nor could anything be done at home and converted to the lab computers."

Dianne's critique was similar to some students' concerns voiced during our classes that their future students would be unable to use new technologies because of a lack of technological accessibility at some schools. However, no other students stated these concerns on the pre or posttests.

DISCUSSION

The students' reflections upon this project have contributed to a stronger teacher preparation program and have assisted the team in planning and adapting for future technological integration. While both the language arts and social studies sections implemented the same technology elements in the electronic portfolio (i.e., database, presentation, digital editing, etc.), students in one section were given specific instructions on what to include in their electronic portfolios; the other section was allowed to choose independently. One teaching team member, a graduate assistant, was also reassigned during the semester, which resulted in some confusion for one content group. While both situations could have contributed to the students, as a whole, indicating lower percentages at posttest in knowledge of technology, overall, the teaching team believes that the preservice teachers simply had preconceived notions on what the methods block normally entailed versus what this particular semester required with the int roduction of the electronic portfolios. The group did, however, indicate they were more comfortable with the technologies at semester's end than they were at the beginning.

As past research has indicated, electronic portfolios require an investment of time and effort from both instructor and student (Cole et al., 2000; Linn & Baker, 1992). The teaching team that facilitated this first semester effort began planning early and evaluated the process consistently throughout the semester. Some problems with application compatibility had to be addressed, additional lab hours were needed, and additional workshops were added. Some students had higher learning curves in adapting to the technologies than others and needed additional time in the lab. However, it is clear that the preservice teachers were encouraged and motivated by electronic portfolio use and have overwhelmingly indicated their support for learning these technologies in a preservice educational setting. Several students commented on the usefulness of the experience with technology relative to their marketability as future teachers. For instance, Patty commented that the knowledge she gained will be, "helpful when I begin looking for a job."

Similarly, Claire stated, "...technology is a good way to improve your job chances and improve the classroom." Sam added, "...the semester's experiences will be very useful when I start interviewing for a job. "Gina commented that she would be able to show, "my technology skills to an employer." Finally, Greta believed that she could, "...use the knowledge to impress administrators."

While many of the preservice teachers were positive about the value of electronic portfolios as a form of assessment, a small number indicated concerns that the electronic portfolio assignments were not presented in a clear manner and that the technologies were not always readily available. While every effort was made by the teaching team to ensure clarity and access, departure of one team member did create confusion. Team members learned from this experience and have taken steps to clarify common assignments and applications for future classes. Additionally, the team has planned for software compatibility and for additional equipment access for the preservice teachers.

It was refreshing for the team to note one theme that appeared consistently throughout the semester. The preservice teachers seemed to enjoy the experience of integrating technology within their teaching and learning activities. Representative comments included:

Johnny: "Rocks! This is fun because you can do so much with it--creatively!"

Eve: "The technology is more useful and fun to do."

Dana: "I did enjoy this more because it is more beneficial."

Tim: "The use of technology was more creative and fun and [it] challenged me to 'think outside the lines'."

CONCLUSIONS

Implementing electronic portfolios required time, commitment, extensive planning, and ongoing evaluation. These challenges were expected and consistent with research (Cole et al. 2000; Linn & Baker, 1992). The process was successful because the students internalized another alternative for collecting and exhibiting their achievements (Lankes, 1998). The students articulated positive opinions about the utility of technology in teaching and learning beyond superficial applications. As Barrett (1997) suggested, the electronic portfolios prompted the students to think carefully, work consistently, and choose wisely. These are desirable characteristics to facilitate among all preservice teachers; therefore, electronic portfolios will be continued in the methods courses, the outcomes will be studied, and slowly expanded into other facets of the teacher education program. Specifically, students in the first teacher education course will begin developing pieces for the electronic portfolio, continue adding pieces in methods, and complete the portfolio during their internship. This extended experience will hopefully further stimulate the students' development as professional educators (Campbell et al. 1998).

FUTURE DIRECTIONS

The team teaching approach assisted in the overall success of the electronic portfolio integration project and has continued to be a model at the institution of this study. The evaluation process was important and has helped to enhance the accessibility of and support in using the technologies for teaching and learning. Efforts have continued and since technology is ever changing, the evaluation process of how to effectively use technology as tools for teaching and learning must continue. To that end, the teaching team remains committed to reflective planning for, using, and evaluating technology in instruction. As efforts grow to include more faculty, inservice teachers, and preservice teachers in other content areas, the authors are excited about the possibilities and understand the challenges. The focus remains good pedagogy; therefore, technology efforts will continue to be research-based, reflective, and appropriate across the various content areas.

References

Barrett, H. (1997). Collaborative planning for electronic portfolios: Asking strategic questions. [Online]. Retrieved February 19, 2002 from the World Wide Web: http://transition.alaska.edu/www/portfolios/planning.htm

Barrett, H. (1998). Strategic questions: What to consider when planning for electronic portfolios. Learning and Leading with Technology, 26, 6-13.

Barrett, H. (2000). Electronic teaching portfolios: Multimedia skills + portfolio development = Powerful professional development. Proceedings of the Society for Information Technology and Teacher Education, (pp. 1111-1115). Charlottesville, VA: Association for the Advancement of Computing in Education.

Bull, K.S., Montgomery, D., Overton, R., & Kimball, S. (2000). Developing teaching portfolios. Quality University Instruction Online: A teaching Effectiveness Training Program. Retrieved from the Worle Wide Web February 19,2002, http://home.okstate.edu/homepsages.nsaf/toc/

Campbell, D.M., Cignetti, P.B., Melenyzer, B.J., Nettles, D.H., & Wyman, R.M. (1997). How to develop a professional portfolio: A manual for teachers. Boston: Allyn & Bacon.

Cole, D.J., Ryan, C.W., Kick, F., & Mathies, B.K. (2000). Portfolios across the curriculum and beyond. Thousand Oaks, CA: Corwin.

Corbett-Perez, S., & Dorman, S.M. (1999). Technology briefs. Journal of School Health, 6(69), 247.

Fulton, K. (1998). Learning in a digital age: The skills students need for technological fluency. T.H.E. Journal Online. Retrieved October 12, 1999 from the World Wide Web: http://www.thejournal.comlmagazinel vau1t/A2016.cfm

Herman, L.P., & Morrell, M. (1999). Educational progressions: Electronic portfolios in a virtual classroom. T.H.E. Journal, 26(11), 86.

Lankes, A.M.D. (1998, April). Portfolios: A new wave in assessment. T.H.E. Journal, 25, 18.

Linn, R., & Baker E. (1992). Portfolios and accountability. CREST. Retrieved May, 2000: http://www.cse.ucla.edu/CRESSTfNewsletters/CLFALL92.PDF 

McKenzie, J. (1999). Reaching the reluctant teacher. Retrieved February 19, 2002: http://www.fno.org/sum99/re1uctant.html

Meyer, C.A. (1992). What's the difference between "authentic" and "performance" assessment? Educational Technology, 33(3), 34-45.

Miles, M.B., & Huberman, A.M. (1984). Qualitative data analysis: A source book of new methods. Beverly Hills, CA: Sage Publications.

Polonoli, K.E. (2000). Defining the role of the digital portfolio in teacher education. Paper presented at the annual meeting of the West Virginia Network, Morgantown, WV. (ERIC Document No. ED 447 806)