Two new evaluation instruments for collaboration.

Abstract

Elementary teachers, a principal, and professors co-designed a curriculum alignment project to integrate state social studies and reading standards. Data were collected using a philosophy of education scale (Pryor, C. R., 2003) and an attitude and belief questionnaire (Pryor, B. W. 2003). Data were analyzed using descriptive and analytic statistics. Results indicate that if project objectives change during collaborative consulting, use of quantitative instruments can guide participant philosophies and beliefs about changing roles.

This project was funded by collaborative and fellowship research grants from the Texas A&M University System's Regents Initiative for Excellence in Education, Institute for School-University Partnerships.

Introduction

Benefits of university-school collaboration have long been a part of the education literature. Among the most outstanding of these benefits are: (a) an expanded base of expertise for the project (McCall & Restow, 2001), (b) a sense of professional renewal for participants (Lewison & Holliday, 1999), and (c) multiple perspectives in problem solving (Sirotnik & Goodlad, 1998). Once the collaborative team is identified, however, several roadblocks to these benefits often appear--some of which are less noticeable by a collaborative team. Three often reported roadblocks include a lack of clear project objectives, insufficient time or energy for the project, and disappearing administrative support (Johnson, Johnson, & Holubec, 1992; Lieberman 1992). Different beliefs held by collaborative participants can be roadblocks that are often not obvious until a project is completed (e.g., McCall & Restow, 2001). These beliefs can affect participants' perceptions of: (a) their role in carrying out the project, (b) the design of the project deliverable, and (c) their ratings of the usefulness of the collaborative project (Markowitz & Crane, 1993).

Two New Measures of Participant Perspective

Typically, qualitative methods such as journals, meeting notes, or interviews are used at the end of a project to report participants' perspectives of project worth (Vare, 1997). In reviewing reports of collaborative curriculum projects, McCall and Restow (2001) found the different perspectives held by participants (e.g., philosophical goals) can affect the development of curriculum, such as selection of teaching methods or resources to be used in the project (McCall & Restow, 2001). The differences in philosophical approach should not be surprising. Teachers and professors work in different cultures, with different job expectations and status (Brookhart & Loadman, 1990). The expectations of the roles of those who collaborate, therefore, vary according to the philosophical beliefs, expectations and attitudes of the project participants. Without some measure of this difference, project goals might not be clear, leaving otherwise well planned projects bereft of the real benefits of collaboration expanded expertise and professional renewal.

The purpose of this article is to describe two instruments that can be used in university-school collaborative projects. This article also describes lessons learned in curriculum collaboration and suggests constructs for use of questionnaires.

Method

Project Goals-Integrating Discourse Skills in Social Studies/Reading

Early in the project, teachers and researchers decided to focus the curriculum project on children's skills in discourse to increase meta-cognition skills--talking about learning. Discourse has been defined as the way in which ideas are exchanged--particularly of what ideas entail (Schram & Rosean, 1996). The benefits of teachers learning to integrate strategies of discourse (e.g., hypothesize, explain, expand, question, probe) into a social studies and reading curriculum are: (a) classroom environment of reasoned inquiry (Schram & Rosean, 1996), (b) increased conceptual development (Cobb, Boufi, McClain, & Whitenack, 2000), (c) increased knowledge of a domain area (Latham, 1997), and (d) improvement of task related knowledge, such as memory improvement (Memke & Presley, 1994). Several studies have identified how integration of these domain areas can enhance reading skills. These benefits are attractive to teachers because they (a) increase knowledge in the domain area, (b) more highly engage students (Klingner, Vaughn, & Schumm, 1998), and (c) foster "deeper-richer" classroom "talk" (Roser & Keehn, 2002). These benefits were important to the collaborative team.

Phases of the Collaborative Process

Phase one: University-school needs. The project was an outgrowth of a district mandate to align social studies and reading standards for the fourth grade. This mandate changed to mathematics alignment after the grant funded. However, because alignment of social studies and reading domain areas was a goal of the grant and of interest to the collaborative team, the project proceeded as a curriculum-planning grant. The team identified three school needs. First, increase the teaching of state history. Second, increase student recta-cognition skills using discourse (i.e., self-talk) in history lessons. Third, seek resources to increase teacher, researcher, and student knowledge of state history. To enhance historical knowledge, a history professor developed a bibliography of readings for the project team.

(see website http://tlac.coe.tamu.adu/faculty/facpages/pryor.html)

Phase two: Planning for standards alignment. To learn more about students' meta-cognitive abilities, teachers decided to use a test generated outside the district. A reading professor provided samples of analytical inventory and a professor-developed test of meta-cognition. Teachers identified sections to administer for formative evaluation.

Phase three: Changing focus. Teachers-researchers collaborated on lessons that could be developed after administration of the two tests of meta-cognition. The grant provided The Texas Weekly newspaper for all fourth grade students. Teachers used this newspaper for discussion, critique of current events, and written narrative responses. Globes, maps, and a limited number of library and trade social studies books were available at the school. Teachers reported that the school library and classrooms were limited in historical trade books and biographies about development of Texas during 20th century.

Phase four: Struggling efforts and conflicting perceptions. Because the project was designed to align standards, and did not include a student-testing component, teachers began to regard administration of tests as "out of pocket" or extra work. Given the amount of time needed for mandated district benchmarking tests in preparation for state mandated achievement tests, the implementation of two additional tests, even sections of two tests--became understandably unacceptable to the teachers. It became evident that the teachers' perceptions of the utility of additional formative testing had changed, and the researcher role concomitantly shifted from the provider of information about formative testing and curriculum alignment back to providers of teaching resources for standards alignment.

Phase five: Co-developers' curriculum decisions. Teachers selected children's books that could be provided to the school by the grant. To determine various approaches to teaching using these books, teachers were urged to review their Philosophy of Education Scale (POES, Pryor C. R., 2003). Humanist, (student-centered) teachers might decide to use books differently than executive (behaviorist) teachers.

Participants

The study was conducted by university faculty members in collaboration with fourth grade teachers at a K-4 elementary school. The teacher alignment team was composed of four teachers and the school principal. District shift in principal school assignments occurred the week the project funded.

Roles and Responsibilities

The principal investigator was responsible for organizing collaborative team planning and integrating aspects of the alignment project into university methods course syllabi. University faculty members provided expertise in reading, state history and developing/administering two evaluative questionnaires. The researchers' primary role was that of domain area experts in identifying books, statewide resources and testing materials. The teachers' primary role was to implement and evaluate resources. The principal coordinated details of school-university organization.

Instruments

Philosophy of Education Scale (POES)

(see author website http:://tlac.coe.tamu.edu/faculty/facpages/pryor.html)

POES is composed of 84 items, each evaluated on a 5-point scale that measures 1 of 7 variables: role of the teacher, role of knowledge, lesson planning, student evaluation (grading), and three measures of classroom organization--environment, management, and activities. The scale items are categorized by use of a test key that corresponds to four philosophical approaches to teaching: (a) executive (manager), (b) humanist (facilitator), (c) classicist (subject expert), or (d) informationist (interpreter/technologist). Two sets of curriculum theory graduate students (n=45) conducted content validity of the POES.

Attitude and Belief Questionnaire (ABQ) (Pryor, B.W., 1990; 2003)

The theory of reasoned action (e.g., Fishbein & Ajzen, 1975) guided the development of the ABQ. This theory holds that the performance of any given behavior is determined by an intention to perform the behavior, and behavioral intention is determined by an attitude toward performing the behavior, and a subjective perception of normative (i.e., what other people think) influence concerning the behavior. The ABQ is a seven-point bipolar evaluative and probability scale with four sections; each section measures one of the four dimensions of the collaborative curriculum alignment project: importance of resources provided by the grant, utility of the project, belief in the importance of the project and belief that important others think the project is important.

The POES was administered to the fourth grade teachers at the beginning of the project so that they could begin to identify their philosophical approach to curriculum. The POES was again administered to these teachers, along with the three researchers at the end of the project. The ABQ was administered to both groups at the end of the project. The principal and one professor did not complete either scale. The three professors who did complete the ABQ, did not complete the "beliefs about utility of curriculum integration" section since they do not teach fourth grade.

Results and Discussion

Comparison of Teacher-Researcher Differences on the POES

Although the researchers held similar philosophies, classicist (content-focused), and humanist (person-centered), the teachers were more diverse in their overall philosophical approach. Teachers' ratings of executive (behaviorist) and informationist (technological) were higher and more spread out than those of the researchers. This comprehensive philosophical approach among the teachers was reflected in their development of classroom activities (e.g., use of student-centered projects). Teachers' philosophical approach (e.g., use of activities and materials) to curriculum design was important to the university preservice teacher methods faculty. The researcher team wanted to translate the activities used by the teachers (books and newspaper) into activities that could be integrated into university syllabi. The fourth grade teachers and researchers agreed that the alignment of university curriculum (and professor philosophical approach) with that of the field-based schools in which preservice teachers are placed during the semester of their methods courses can help preservice teachers develop skills in integrating curriculum and learn the culture of school curriculum decision-making (Metcalf & Kalich, 1996). Table 1, shows the comparison of teacher and researcher philosophical approach to teaching. See issue's website <http://rapidintellect.com/AEQweb/fal2003.htm>

Comparison of Teachers-Researcher Differences on the ABQ

Utility of Project Resources Teachers and researchers differed on their ratings of three items: Use of Texas Weekly, description of the Institute of Texan Cultures, and use of the POES. These items were rated highly by the researchers, as were all resources; other resources (e.g., bibliography of Texas historical readings) were highly rated by teachers. Teachers explained that the newspaper arrived mid-term so they did not have full use of this resource; and the other two items were perceived as too abstract and lacking information on how to use them. To enhance personal knowledge, teachers ordered a book on Texas History and highly rated the item "bibliographic resources." The teachers also highly rated books they had ordered for future use with the fourth grade classes.

Utility of the Project Overall The teachers rated the utility of the project as "low" on all items. Researchers, however, rated two of the items highly: "project helped fourth grade students" and "project helped to integrate into other subjects". Given the shift of project focus, from alignment to formative evaluation, and back again to alignment, it is not surprising that perceptions of utility differed. However, belief in the co-development of this type of project holds potential for continued work, as seen in the teacher-researcher belief ratings.

Belief in the Alignment of Social Studies and Reading Standards Since they were not teaching fourth grade, researchers did not complete the "belief about alignment" section of the scale. As indicated in Table 2, teachers highly rated additional staff development for future alignment projects. They also highly rated collaborative planning. Without information gained from belief items such as those on the AEQ, projects can easily overlook an important aspect of collaboration: participant need for added knowledge. Table 2, portrays teachers-researcher differences in perceptions of the project. See issue's website < http://rapidintellect.com/AEQweb/fal2003.htm>

Conclusion: Lessons Learned

As teachers and researchers co-develop curriculum alignment plans as a means for using state mandated standards in teaching, the following findings appear important. Quantitative data can be used to evaluate goals and philosophical approaches that are usually not collected before and after collaboration projects. By using instruments such as the POES and ABQ, we were able to identify the strengths of the project as well as the gaps between teachers' expectations and those of the researchers. We suggest that researchers employ these instruments to increase understanding of goals and perspectives among participants in collaborative projects.

References

Brookhart, S. M., & Loadman, W.E. (1990). School-university collaboration: Different workplace cultures. Contemporary Education, 61 (3), 125-128.

Cobb, P., Boufi, A., McClain, K., & Whitenack., J. (1997). Reflective discourse and collective reflection. Journal for Research in Mathematics Education, 28, 258-277.

Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

Johnson, D., Johnson, R., & Holubec, E. (1992). Circles of learning. Alexandria, VA: Association of Supervision and Curriculum Development.

Klingner, J. K., Vaughn, S., & Schumm, J. S. (1998). Collaborative strategic reading during social studies in heterogeneous fourth-grade classrooms. The Elementary School Journal, 99 (1), 3-22.

Latham, A. (1997). Asking students the right questions. Educational Leadership, 54, 84-85.

Lewison, M., & Holliday, S. (1999). Investigating a school-university partnership through the lenses of relationship, self-determination, reciprocal influence and expanding power. In D. M. Byrd & D. J. McIntyre (Eds.), Research on professional development schools (pp. 79-96). Thousand Oaks, CA: Corwin.

Lieberman, A. (1992). School/university collaboration: A view from the inside. Phi Delta Kappan, 74, 147-154.

Markowitz, N., & Crane, B. (1993, April). A case study in collaboration for curriculum reform. Paper presented at the meeting of American Educational Research Association, Atlanta.

McCall, A., & Restow, T., (2001). Connecting families and state history: A teacher educator and classroom teacher collaborate. In M. Christensen, M. Johnson, & J. Norris (Eds.), Teaching together: School/university collaborative to improve social studies, Bulletin 98 (pp. 47-63). Washington DC.: National Council of the Social Studies.

Memke, D. J., & Pressley, M. (1994). Elaborative interrogation: Using "why" questions to enhance the learning from text. Journal of Reading, 37, 642-645.

Metcalf, K., & Kalich, P. (1996). Laboratory experiences as transition from campus to field. In D. J. McIntyre & D. M. Byrd (Eds.), Preparing tomorrow's teachers: The field experience (pp. 79-93). Thousand Oaks, CA: Corwin.

Pryor, B. W. (1990). Predicting and explaining intentions to participate in continuing education: An application of the theory of reasoned action. Adult Education Quarterly, 40, 146-157.

Pryor, B. W. (2003). Attitudes and beliefs towards integration of social studies and language arts questionnaire. Project development for the Texas A&M University System Board of Regents, Academy Partnership for Excellence in Education.

Pryor, C. R. (2003, February). Helping preservice teachers develop a philosophy of education statement. Paper presented at the meeting of the Association of Teacher Educators, Orlando, FL.

Roser, N. L., & Keehn, S. (2002). Fostering thought, talk, and inquiry: Linking literature and social studies. The Reading Teacher, 55 (5), 416-426.

Schram, P., & Rosaen, C. (1996). Integrating the language arts and mathematics in teacher education. Action in Teacher Education, 18, 23-38.

Sirotnik, K. A., & Goodlad, J. L. (1988). School-university partnerships in action: Concepts, cases, and concerns. New York: Teachers College.

Vare, J. (1997, January). Research as relationship: Ethics and complications in school-university relationships. Paper presented at the annual International Conference on Qualitative Research in Education, Atlanta.

Caroline Pryor, Texas A & M University at College Station

Rui Kang, Texas A & M University at College Station

Dr. Pryor is an Assistant Professor and Regents Fellow. She coordinates the elementary social studies program and is a 2003 Wye Fellow of the Aspen Institute. Kang is a graduate student specializing in social studies and economics education. She is a PhD student with two master's degrees, in curriculum and finance.