Do technology investments pay off? The evidence is in!

A five-year evaluation of the Multimedia Project shows how technology can improve teamwork, communication and problem-solving skills.

The March/April 1999 issue of this magazine featured an article about the Challenge 2000 Multimedia Project, one of the original 19 Technology Innovation Challenge Grant projects funded by the U.S. Department of Education. That article described the project's design and some of the preliminary findings. This month, we're pleased to share the results of the final evaluation of the project, which provides solid, empirical evidence about the value technology investments can add to education when they include teacher training, technical support staffing and curriculum materials development.

In a five-year evaluation of the $6.6 million project, SRI evaluators showed that technology-using students surpassed their non-technology-using peers in developing some of the most critical skills for the new economy of the 21st century. In addition, these same students equaled their non-technology-using peers in learning the basic skills measured in standardized tests. In short, students in technology-using classes not only "got the basics," they got more.

The extra edge that the Department of Education's Technology Innovation Challenge Grant gave students focused on the skills of communication, teamwork and problem-solving.

These are the very skills identified by the U.S. Department of Labor (1999) as fundamental to the new technology-based workplace of the future, but which are not tested in the familiar standardized tests in our classrooms today.

Results of the Multimedia Project

Growth in communication skills, teamwork and solving complex problems is not measured by the typical multiple-choice achievement test. To measure student learning in these areas, SRI gave Multimedia Project students and a matched comparison group of students a project, and studied how they completed it.

Working in teams, students had to produce a brochure targeted at school officials to inform them about the problems faced by homeless students. Students were given documents about homeless youth, then asked to use this information in their brochure. Judges, who were blind to the students' participation in the Multimedia Project, scored the brochures on three dimensions:

* how well they accurately represented the key content of the documents provided;

* how well they addressed the likely concerns of their audience; and

* how well their design integrated text and graphics into an eye-catching and convincing presentation.

The Multimedia Project students out-performed the comparison students on all three dimensions of good communication and presentation skills: content mastery, sensitivity to their audience, and coherent design integrating multiple graphical and textual elements.

Even people who applaud results such as these nonetheless worry that time spent on such projects will detract from students learning basic skills. On the contrary, SRI found that students in the Multimedia Project made the same progress as their counterparts in comparison classrooms on standardized test scores. Multimedia Project students learned the basics and more: they learned the kinds of skills that are most sought-after by employers and that will contribute to students' development as life-long learners.

What happened in the classroom

In systematic observations, SRI researchers found that the teaching in Multimedia Project classrooms looked very different from teaching in comparison classrooms. In project classrooms, teachers were more likely to take the role of facilitator or coach. They spent less time lecturing. They asked a greater number of open-ended questions and let students discuss ideas. They encouraged students to solve problems independently. They helped students develop new knowledge and integrate it into their multimedia presentations.

For their part, students in Multimedia Project classrooms were more likely to be actively engaged in small-group collaborative activities. They were more likely than students in comparison classrooms to be engaged in long-term assignments that lasted a week or more and therefore gave students an opportunity to explore a topic in depth. They were also more likely to participate in small group discussion led by one of their peers. Finally, they were more likely to be engaged in solving problems that people face when they are designing something that will be reviewed by others.

Students in the Multimedia Project were working on high-level reading and presentation skills. They also thought deeply about the audience for their presentations and discussed issues such as audience interests, knowledge and preferences. In short, the observational study found that students in Multimedia Project classrooms were doing extensive "higher-order" thinking -- that is, planning, problem solving and making decisions -- all focused around their involvement in project-based learning with multimedia.

Investing in teacher professional development

Investing in teachers is at the core of the Multimedia Project's strategy for transforming teaching and learning. The investment takes several forms. Teachers receive professional development in the form of intensive summer institutes and follow-up meetings during the school year. Teachers who develop project proposals are eligible for competitive minigrants for specialized hardware and software needed to do multimedia projects. Teaching colleagues with expertise in project-based learning and multimedia provide technical support and personal coaching.

Rewards and recognition are provided through cash honoraria and an annual multimedia exhibition and festival. Teachers can tap on-line resources and support from the Multimedia Project's award-winning Web site (http://pblmm.k12.ca.us) and take advantage of regional networking opportunities designed to foster the development of a learning community among the teachers and ensure the sharing of best practices across schools and districts.

What should school leaders do?

In the current frenzy to raise test scores and reach school API goals, principals and other administrators need to be careful not to throw the baby out with the bath water. It's easy to understand how we might discourage project work in favor of direct instruction, or limit the use of technology to drill and tutorial types of programs. But if we do so, we will shortchange our students.

Teachers need to use a balanced blend of instructional approaches. Administrators should encourage the use of technology in rigorous, well-designed project-based learning. Implemented properly, technology can be a powerful partner in preparing students for work and for being effective, active citizens in the digital age.

Bill Penuel and Louise Yarnall are researchers at SRI International in Menlo Park. Michael Simkins is director of the Challenge 2000 Multimedia Project and chief operating officer of Joint Venture: Silicon Valley Network in San Jose.