Preface: positioning evaluation and research within PT3 projects.
This special issue of JTATE is focused on evaluation and research studies conducted under the PT3 program of the U.S. Department of Education. PT3, Preparing Tomorrow's Teachers to Use Technology, is a program designed to ensure that new teachers are prepared to use computers and other technology when they reach the classroom and, in the effort to prepare teachers, that education faculty will be induced to change the content and process of preservice teacher programs. Begun in 1999, the program is a response to the need to improve teacher preparation, recognizing the changing technology environment in K-12 schools. It is, at its core, an effort to develop human resources in education and introduce new technologies to the preparation of new teachers and to the classroom practice of these new teachers in schools across the country.
Evaluation: Problem or Problem Solving
As a component of each PT3 grant, recipient schools were required to have an evaluation conducted on their efforts and the impact of the program. They were also strongly encouraged to work with an external evaluator. Evaluation (other than student evaluation or evaluation for program accreditation) is not a common element in university programs, especially when conducted by someone who is not part of the department or college. It can be threatening for those project directors who do not feel confident that their efforts are paying off. It is a foreign body in their lives, unpredictable in its form and function. Even those project directors who had conducted evaluations themselves were reluctant to fully engage with outside evaluators or were resistant to their participation in project activities.
For some grant recipients, evaluation was a necessary evil, to be tolerated and ignored. The program demands it, we'll let it happen; if we don't encourage them, perhaps it won't be a disaster. For other faculty, evaluation was seen as a set of judgments that people make about a program or activity. It implied accreditation and happened after the fact, was focused on the outcomes, but not on the process of the activities. We'll be finished with our work when they give us the report. Some projects saw evaluation as benign and somewhat irrelevant. We're going to do what we said we would do, so why bother. If there are some good findings coming out of the evaluation, we'll take credit for them and, if not, we can ignore them.
There were other PT3 projects that understood from the start how evaluation could contribute to their efforts. By design, the evaluation became a means of capturing information from participating faculty and students that could be used to modify the project, that verified strategies, that tested alternatives, and that built knowledge about the preservice preparation process or about the efforts required to gain the participation of faculty in the change process. These PT3 sites and staff were most willing to explore the questions that were posed by their evaluator (or that they offered to the evaluation team), and they were also more willing to explore the larger questions and hypotheses about technology, preservice education, and faculty change. Often these projects established a strong collaborative relationship between the project staff and the external evaluator, leading to highly productive efforts.
Many PT3 projects and project directors were pleasantly surprised by the value added by evaluation. It provided independent feedback, new ideas, and opportunity to fine tune the preservice program around technology. For these PT3 participants, the benefits of their evaluation findings helped improve the project and changed their view of evaluation.
Replacing Evaluation with Research
The PT3 program began prior to the time the Bush administration took office and before the passage of No Child Left Behind (NCLB). Both events have had major impacts on K-12 schools and higher education; theoretically what is important may have stayed the same, how it is to be handled has changed.
The past few years have produced a sea-change in how educators pose questions, plan studies, and collect, analyze and report data. The demands for studies in education to approximate those in medical research--regardless of appropriateness--is causing consternation among those interested in new products, variations in instruction and changes in pedagogy. Scientifically-based research is the non-negotiable issue for accepting conclusions about the success or impact of any program, product or project. Random assignment to various treatments is touted as the gold standard of research.
However, at the time that the PT3 program started, the role of evaluation as a service to projects meant that evaluators would assist projects in meeting their goals by providing useful feedback and information, as well as assess the outcomes and impacts of such endeavors to help the U.S. Department of Education provide findings to Congress about what had been learned about preservice teacher education and technology. The strategy, for the most enlightened PT3 projects, was a combination of formative evaluation in the early stages of implementation, moving to a more summative approach for the final year of the project.
Most recently, throughout the education enterprise, the practice of evaluation has moved closer to that of research, and the pressure for conducting scientifically-based research has superseded the effort to inform and improve each individual project. Not only is formative evaluation regarded as less important, summative evaluation has more stringent rules than it has had in the past. While NCLB is focused on K-12 schools, it has a direct bearing on postsecondary education as well. Since PT3 projects focus on preparing teachers for the K-12 classroom, NCLB has clear relevance for participants. New teachers emerging from a college of education need to know more about standards, assessments, and demonstrably-proven strategies and materials for instruction than they might have in the recent past. Without commenting on the appropriateness or value of this requirement, we can say that the context for both the efforts of the Preparing Tomorrow's Teachers to Use Technology program, and the evaluation that has been a part of it, has undergone dramatic change.
Nevertheless, the projects reporting in this issue began their work prior to the NCLB, and thus developed their studies from the interests of the project staff and the external evaluators without the pressures of the Institute for Education Sciences and the call for more scientifically-based research. The projects can be excused for not providing examples of highly-rigorous, scientifically-based research. It was not demanded, nor would it have been highly sought after by project staff. Imposing the political and administrative demands of rigorous research methods would have stalled most initiatives.
While the questions that need to be answered by scientifically-based research strategies will be well worth asking, this grant initiative was neither the place for it nor was the necessary funding and support available. PT3 grantees took on reasonable challenges and tried to handle them well. PT3 is practical, not theoretical; it is about getting better teachers into K-12 schools, year after year.
What We've Started to Learn
Among the more important questions explored by the articles in this issue are those that focus on influencing faculty to change their classroom processes to incorporate technology--both to build a more effective course and to model the application of technology in instruction. The evidence from these research efforts not only demonstrates that school of education faculty can do more than just talk about change, but that they can modify their use of resources, their instructional practice, and even the content of their course to account for technology as a teaching and learning tool, and as a component of the content they are teaching to preservice teachers. Faculty were able to re-conceptualize their course designs to acknowledge that computers and other technologies had a role to play in what their education students learned and how they learned it. Furthermore, education faculty also developed new perspectives on what technology knowledge and skills education students needed, both for using it to meet the requirements of a course, but also to become successful teachers out in the K-12 classrooms of their communities.
In addition, these studies point to the contributions that technology-savvy, practicing teachers make by working with preservice teacher placements and by working side-by-side with and advising education school faculty. They bring to the discussion not only practical experience with integrating technology into diverse classroom settings, but also the set of technology skills that are needed to do successful work in today's schools. By informing faculty about the reality of the K-12 classroom and the contributions of (and required efforts for developing) technology for instruction, preservice course syllabi have changed to reflect a new sense of practice and a new set of expectations for student-teacher performance.
Because the evaluation and research studies included in this journal have explored a complex of contextual and program issues, they have succeeded in identifying many of the elements in a program that contribute to the achievement of project goals. The studies have no single silver bullet to offer those interested in replicating successful programs. They acknowledge the difficulties of working with stakeholders having diverse backgrounds, needs, and interests. They recognize the limitations of organizational constraints and finite resources. And they bring to bear the learning of earlier efforts on the problems of institutional change. They attempt to substantiate the collection of procedures and efforts that led to modification of their institution's instructional program and to improved knowledge and skills for both faculty and preservice teachers.
The multiple methods used by these studies in productive and thoughtful ways illustrate that there is no single strategy for designing a useful study to assess the impact of a complex change program. Taking a rigorous, scientifically-based approach to the PT3 initiatives may have provided a narrow answer to one of the important questions, but a single right answer to some of the important questions would not easily inform the variety of programs to prepare teachers. These studies may not represent the best of research, of teacher preparation, or of faculty change. But they do offer well-supported and valuabe insights into how the best of all possible worlds might be approached.
The articles here are asking some of the more interesting and important questions. The methodologies used range from the qualitative to the descriptive to the quasi-experimental, providing the range of insights that come from approaching complex questions with multiple research methods. The goal of these studies was primarily to inform the projects of the progress they were making in developing a faculty who used technology for teaching and who prepared preservice teachers to infuse technology in the K-12 classrooms in which they were placed. A secondary goal, far behind the primary one, was to build the body of knowledge about technology and its value in the teacher induction process. Therefore, the research and evaluation focus was more on process and assessing the initial outcomes in practice than it was on the deep and difficult policy issues of changing how schools of education prepare teachers or the best way to prepare novice teachers to integrate technology into their early practice. The researchers are to be commended for building or evolving thoughtful studies into what is basically an applied project.
Opportunities for Additional Research
Like all federal programs, there are implicit goals and assumptions about this initiative and the targets of its activities. The need for the PT3 program emerged during a period of enormous growth in the availability of computers and other technologies in schools and the demand to use this expensive resource in effective ways. Most practicing teachers were unprepared to engage these new technologies with any confidence or sophistication. They had not been prepared in schools of education to incorporate technology in instruction, nor had they been taught to use technology while in college. In-service programs took on the task of training the practicing teachers, while the colleges and universities tried to catch up. The PT3 program stimulated this effort with three-year grants to bring faculty and curriculum up to speed.
A common strategy undertaken by PT3 grantees was to change the way education faculty teach by helping them learn how to use technology for themselves and how to apply it in their instructional practice. At the same time, education schools helped preservice teachers develop a core of technology skills while learning about the application of technology for instruction. The program encouraged the twin goals of changing how faculty taught and what preservice teachers learned. That was the focus of the past efforts. Now we can see additional opportunities to move from evaluation to research that are more closely aligned with the expressed demands of the Department of Education and the Institute for Education Sciences. PT3 projects started down a path that can be converted from evaluation to research. Let me offer a few ideas.
Some implicit beliefs about teacher education were instantiated within many of the PT3 projects. These included:
* the value and power of constructivist teaching and learning for both K-12 and higher education,
* the importance for teachers at any educational level to master a broad set of technology skills for personal and professional use,
* the value of applying technology for teaching at the college and K-12 levels, and
* the positive impact of creating instructional lessons and units on the beliefs and future practice of preservice teachers.
Some of the other, perhaps naive, assumptions adopted within many PT3 grants were:
* that college faculty would embrace change,
* that participation in a federally-funded change program would be desired by all faculty members, regardless of incentives,
* that teachers who mentor student teachers already use and will be delighted to see their students use computers in their classrooms, and
* that students preparing to become teachers will be eager to apply technology in their student teaching program.
Incidental goals and realized outcomes of the PT3 program were also based on hypotheses about how technology could be used and could influence faculty and teacher practice. These included:
* provuding good K-12 technology-using teachers as models from whom higher education faculty can learn,
* taking advantage of the motivational power and productivity of linking technology to university classroom instruction,
* shifting some of the leadership for changing instructional practice from senior faculty to junior faculty,
* strengthening the link between methods and curriculum content through technology,
* seeking collaboration within divisions of universities and across universities, and
* bringing evaluation (if not research) into academic departments not used to external scrutiny of internal activities.
These are all testable hypotheses, having a bearing on both higher education policy and practice, yet few PT3 projects had the opportunity to explore them within the confines of the grant. Many of these issues are central to the evolution of schools of education, linking what preservice teachers learn with the ubiquitous presence of various technologies in their lives (and the lives of the students they will meet), changing the preservice programs in ways that affect post-secondary pedagogy and course content. These are powerful issues, worthy of good research.
The relatively well-funded PT3 program was accessible to a wide range of IHEs (Institutions of Higher Education) with teacher education programs. They were able to accomplish some of their hopes, move ahead with some fraction of their goals. Given what they have done, how could we take advantage of the opportunity that this grant program has provided to do further research? What can we learn that can change our efforts to prepare students to become teachers? To become effective teachers integrating technology into their pedagogy? What could we learn about our ability to change schools of education in meaningful ways? To bring the applications of contemporary technology into the curriculum and university instructional process? And how can the important questions be examined in ways that instill confidence in our conclusions? How do we design scientifically-based research into our preservice preparation programs? And how do we design our preservice preparation programs in ways that are amenable to scientifically-based research?
Exploring some of the hypotheses noted and answering these questions should become the next line of research.
SAUL ROCKMAN, GUEST EDITOR
Rockman Et Al, Inc
|Printer friendly Cite/link Email Feedback|
|Publication:||Journal of Technology and Teacher Education|
|Date:||Jun 22, 2004|
|Previous Article:||Connecting the university with a professional development school: pre-service teachers' attitudes toward the use of compressed video.|
|Next Article:||Learning through CyberApprenticeship (LTCA): robust evaluation predicated on stakeholder engagement.|