New report: handheld computers play a productive role in K-12 teaching and learning: teachers are "overwhelmingly positive" about using handheld computers in their classroom, according to an SRI International report.

With so many new programs being created in K-12 education these days, the hardest part may be trying to evaluate what is working and what isn't.

In early 2001, Palm initiated its Palm Education Pioneers grant program, the first large-scale, objective and systematic evaluation of handheld computers in K-12 education. Handheld computers had already proven to be effective tools for business users. Could they provide similar benefits for K-12 teachers and students? And if so, how and why? Palm and its research partner, the nonprofit SRI International, set out to discover some answers.

More than 100 educators (grades 2-12) participated in the classroom teacher awards of PEP. Each received a classroom set of Palm handhelds to use in projects wholly determined by the teachers themselves. While they varied widely in subjects taught, years of teaching experience, pedagogical style and more, most had no prior experience in using handheld computers for teaching.

With grants awarded in February and June 2001, these teachers have now used the Palm handhelds for at least a full school year. SRI's researchers, who conducted surveys and site visits at the end of each semester, issued their final evaluation report this fall. Their conclusion: "Handheld computers have a productive role across a wide spectrum of instructional topics and purposes, grades and student populations.

"There is something new and unique about handheld computers in education," states the report. "For the first time, students can have a truly portable and personal low-cost, anytime/anyplace general learning device that can be used in any number of private or collaborative learning activities."

The full report--available at www.palmgrants.sri.com--offers many details on the impact of handheld computers on K-12 teaching and learning. Some findings might be somewhat expected. For instance, nearly 92 percent of PEP teachers state that Palm's computers "contribute positively" to their students' learning.

Other outcomes, however, may surprise. For example, elementary school teachers proved to be the most enthusiastic about using handheld computers for instruction, followed by middle school and then high school teachers.

SRI's 100-plus page evaluation report analyses use by grade level, academic subject, teaching style and more. It details the impact on students' motivation and attitude about learning. The handhelds' effect on teaching practices, quality of learning activities, and even on the use of desktop computers are all described.

Key Findings

PEP teachers are "overwhelmingly positive" about the effectiveness of handheld computers as tools for teaching and learning, concludes SRI's researchers. Teachers report that handhelds are easier to integrate into the "flow of classroom activity" than desktop computers.

Fully 91 percent of teachers who had their students use the Palm handhelds for writing assignments feel it can improve the quality of the activity. Being able to supply every student in class with the tool, with its IR beaming and word-processing software, clearly makes a difference. Teachers also rate add-on keyboards as being the most valuable peripheral for handheld computers and DataViz' Documents To Go software as the second-most valuable software package.

Yet having the handhelds did not significantly alter how much time students spent using desktop or laptop computers. Just 15 percent of teachers say it changed "a lot" while 44 percent report no change at all.

Conversely, teachers responding that their experiences with handheld computers did (or will) change the kinds of instructional activities they plan for students "a lot" is 45 percent, with another 33 percent choosing" a little." As one teacher commented to researchers: "Palm [handhelds] have not replaced the time kids spend in the computer lab--they have added to their technology experiences."

Relating to Curricula and Constructivism

Along with writing tasks, science investigations especially benefit by the availability of handheld computers, says the report. Across the sciences, an average of 96 percent of PEP teachers agree the classroom set of handhelds improves the quality of learning activities. Various probes and sensors easily attach to Palm handhelds, plus there is plenty of software available for data analysis. ImagiWorks' ImagiProbe, for instance, tops teachers' rankings of most-valuable software packages.

From 70 percent to 75 percent of teachers involved in reading and social studies projects agree that handhelds can improve the quality of instructional activities. Though still compelling, this is at the low end of the scale and may reflect a lack of materials that specifically address these curricular subjects, notes the report.

Interestingly, 80 percent of educators doing projects with English language learners report handhelds can enhance the quality of activities, as do 85 percent of teachers conducting projects for English. SRI's researchers explain that these learning activities rely mainly on eBooks, dictionaries, thesauri and writing tools. Thus, they fall under the umbrella of benefits that a classroom set of handheld computers brings to writing assignments.

SRI researchers also found that high-constructivism teachers do tend to evaluate handheld computers more positively than their medium- and low- constructivism counterparts. However, the trend is not statistically significant. While handhelds prove "particularly effective" for inquiry-based science and open-ended writing tasks, considered constructivist activities, PEP teachers also report success in using handhelds for flash cards and multiple-choice quizzes.

Measuring Subtle Effects

The advantages handheld computers offer to K-12 students encompasses both tangible and intangible effects. For example, teachers report that the handhelds enable students to use technology more often and more intensively than was previously possible, resulting in greater technology proficiency.

"This allows students to be familiar with other forms of technology beyond their X-Box and desktop computer," commented one teacher, adding that it also "shows [students] how technology can be used in the `real world' through data collection and analysis applications."

Access to handhelds certainly motivates students, but researchers were surprised that the effects of motivation appear to increase over time. They had expected that as the novelty wore off, such effects would decline. Instead, motivation remained "robust" throughout the second semester of projects. Students are first motivated by the "coolness" factor, explains one teacher, but after that wears off, motivation comes from having no papers to lose or get messed up.

Students are also likely motivated by how well handheld computers perform the role of personal learning tool. The device's portability gives them true anytime and anywhere access to wide range of computing capabilities. This, in turn, seems to empower students to take more responsibility for their own learning.

Finally, teachers report that Palm handhelds' IR beaming function boosts both collaboration and communication by students. Specifically, it supports formal collaboration in small-group work as well as spontaneous sharing and collaboration. As a result, many PEP teachers indicate they will integrate collaborative projects more frequently into class work.

In addition to confirming an array of benefits that handheld computers bring to K-12 classrooms, SRI's evaluation also dispels some common concerns. Theft and breakage, though they occurred, were not serious problems, even at the earliest grade levels. And inappropriate usage of the handhelds could usually be managed by teachers.

In fact, a number of teachers report having a classroom set of handhelds seemed to enhance the "learning partnership" between themselves and their students.

SRI researchers say the PEP teachers' high levels of enthusiasm "stems from the fact that handheld computers finally allow unobtrusive, affordable 1-to-1 computing in the classroom." Instead of supplanting desktop computers, the researchers believe handhelds may actually increase the use of existing computers on campus. As technology is integrated throughout the curriculum, teachers and students can choose the most appropriate tool for a given activity, the researchers say.

TEACHERS EVALUATE HANDHELDS IN THE CLASSROOM

                                     Disagree/Strongly    Neither agree
                                         disagree          or disagree

 Palm computers are an effective           2.4                 8.5
 instructional tool for teachers

  I plan to continue to use Palm           4.9                 4.9
      computers in instructional
     activities with my students

       Having a classroom set of           4.9                11.1
 handheld computers will improve
     the quality of the learning
 activities I can implement with
                     my students

     Handheld computers are more           9.8                18.3
     easily used in the "flow of
classroom activity" than desktop
                       computers

        Use of Palm computers in           2.4                 4.9
     learning activities has the
    potential to have a positive
    impact on students' learning

       Having a classroom set of           4.9                 7.4
  handheld computers will have a
  positive effect on my teaching
                        practice

                                      Agree/Strongly
                                           agree

 Palm computers are an effective           89.1
 instructional tool for teachers

  I plan to continue to use Palm           90.2
      computers in instructional
     activities with my students

       Having a classroom set of           84
 handheld computers will improve
     the quality of the learning
 activities I can implement with
                     my students

     Handheld computers are more           71.9
     easily used in the "flow of
classroom activity" than desktop
                       computers

        Use of Palm computers in           92.7
     learning activities has the
    potential to have a positive
    impact on students' learning

       Having a classroom set of           87.7
  handheld computers will have a
  positive effect on my teaching
                        practice

TEACHERS RATE HANDHELDS SPECIFIC BENEFITS

                                          Not applicable/    Not at all
                                            Don't know

Increased proficiency with technology                           1.2

     Increased motivation or attitude           1.2             1.2
                        toward school

                Increased cooperation           2.4             3.7
                    and collaboration

                Increased self-esteem           7.3             3.7

               Ready-at-hand computer           3.7             7.3

            Increased self-directness           9.8             6.1

      Student initiative in using the          14.6             8.5
       handheld computer for learning

    Increased opportunity to learn at          12.2            11
                       their own pace

   Increased organization, in general          17.3            12.3

                                          Somewhat    Quite a bit/
                                                       Extremely

Increased proficiency with technology      28            70.7

     Increased motivation or attitude      28            69.5
                        toward school

                Increased cooperation      31.7          62.2
                    and collaboration

                Increased self-esteem      36.6          52.4

               Ready-at-hand computer      41.5          47.5

            Increased self-directness      40.2          43.9

      Student initiative in using the      36.6          40.3
       handheld computer for learning

     Increased opprtunity to learn at      39            37.8
                       their own pace

   Increased organization, in general      38.3         32.1

Terian Tyre, [e-mail], is special projects editor.