Multiple paths to critical reflection: a flexible model of teacher learning and its impact on student achievement.
This research evaluated impact of the Omaha Public Schools' Urban Systemic Program professional development model on mathematics and science teacher change and student achievement. The model offered various participation pathways, focused teachers' learning in three areas (beliefs, content, and pedagogy) and required teacher reflection during classroom strategy implementation. To determine teacher change, observations, interviews, action research, pre-post perception profiles, retrospective pre-post surveys (beliefs and understandings), and exit surveys were completed. Participants' action research determined impact on students' understandings. Criterion Referenced Tests, as well as leadership pre- and post- surveys, action research and interviews determined school change. To evaluate program impact, participant and non-participant AYP (Annual Yearly Progress) data were compared. Pathway comparisons used mean AYP Science Scores and Average Standards Mastered. Data indicate that changing beliefs and critical reflection were essential to change. Participants showed mean increases in scores, though none were significantly larger than non-participants and impact varied by path. However, with the commitment of leadership and 70% of teachers, schools significantly impacted achievement. Research implications include 1) the importance of the school as the unit of change to impact achievement and 2) the necessity of reflection and work-embedded professional development to impact teacher change and student achievement.
The most important factor determining student achievement is teacher quality (Darling-Hammond 1999, Darling-Hammond 2000, National Commission on Mathematics and Science Teaching for the 21st Century 2000). It is essential to provide quality teachers in the classroom, with teacher quality based on research that outlines what is meant by effective teaching.
In addition, during the past two decades research has advanced understanding of how people learn. The National Research Council (1999) defined important next steps in the research agenda. Research indicated that student learning is promoted when there is focus on learning for understanding, building on preexisting knowledge and facilitating active learning. Later NRC work outlined what the culture of learning might look like in mathematics and science classrooms (National Research Council 2005) and stressed the importance of providing learner-centered, knowledge-centered, assessment-centered and community-centered environments.
Understanding of quality professional development as it impacts teacher change has advanced. Yet enduring challenges of professional development remain, including: 1) raising the performance of all students in mathematics and science while reducing achievement gaps, 2) enhancing the goals of student learning from formulaic to promoting understanding, 3) promoting better teaching, and 4) developing new organizations that are flexible, organized for improvement and focused on student achievement results (Loucks-Horsely et al 2003).
This research addresses these challenges and particularly focuses on the implementation of a professional development model and its effectiveness at teacher and school change as well as its impact on student learning.
Prior Work: The Foundation for Change
The state of mathematics and science teaching and learning in the Omaha Public Schools (OPS) has changed dramatically over the past ten years of National Science Foundation support. When the Urban Systemic Program (Banneker 2000: CEMS) began in Omaha, the district had successfully completed a five-year, National Science Foundation-funded Comprehensive Partnership for Math and Science Achievement (CPMSA) award. Key achievements during the CPMSA built a foundation for work during the USP; foundational work included the following:
1. Prior to the CPMSA, attention was paid to mathematics at the elementary level, but science instruction was often lacking. By the end of the CPMSA, elementary teachers were paying more attention to science instruction but still lacked the confidence and conceptual knowledge that would help them teach science more effectively.
2. District graduation requirements included only two years of mathematics and two years of science and accepted many non-core courses. The CPMSA was successful in eliminating courses that did not position students to take advanced courses in high school and be competitive at the postsecondary level; it also laid the foundation for a policy change to establish three-year graduation requirements in mathematics and science that expected core-course completion by all students in the district.
3. Enrollment and achievement of underrepresented students in mathematics and science was positively impacted, but achievement remained an issue.
4. Effective means to increase student enrollment and success in these courses were established:
* Working with parents to better understand importance
* Providing meaningful support to students--tutoring and enrichment.
5. CPMSA-funded, action research informed aspects of portfolio expectations and goals central to USP professional development.
By the transition year between the CPMSA and the USP, significant progress had been made on policies, convergence of resources, and standards. At the end of the CPMSA it was evident that teachers needed support to work with students for whom they had never been responsible; as a result, USP professional development targeted teacher belief systems and expectations, content and instructional pedagogy. The five years of USP work focused on continued change in policy, convergence of resources, standards-based curriculum and instruction, partnerships, and teacher professional learning to support educators more effectively as they worked to increase student achievement.
Theoretical Frameworks: The Basis for Current Work
To meet the expectations of policy changes and new standards successfully, a professional learning environment was required that would not only enhance teacher learning but also impact school change. Implementing the standards successfully required teacher and school change, a complex process since it strikes at teachers' beliefs and philosophies. The theoretical framework used to design this unique, flexible professional development model built on adult learning theory (Knowles et al. 1998), change processes (Fullan and Stiegelbauer 1991, Hargreaves and Fullan 1998, Lortie 1975), student-centered instruction (McCombs and Whisler 1997, National Research Council, 1999), and previous research on action research during the CPMSA (Koba et al. 2000, Koba and Clarke 2002).
The Professional Development Model
To translate this research into practice, the OPS specifically designed and implemented a professional development model during the USP, Banneker 2000: Community of Excellence in Mathematics and Science (CEMS). Rather than establish a traditional trainer-of-trainers model where only alpha teachers further their learning, the USP focused on the school as a unit of change to establish professional learning communities that engaged a critical mass of teachers focused on student learning (see Figure 1). As shown in the figure, all schools in the district were required to establish professional development plans for mathematics and science and were considered Planning Schools in the CEMS model. For a school to receive financial and intellectual support from CEMS, the principal was required to involve, within a three-year period, 70% of their 4th-9th grade teachers in the USP-designed, intensive professional development. The school was identified then as a Developing School, and teachers and leadership chose the professional development options best suited to them. Once the school reached its 70% teacher participation goal and impacted student achievement consistently, the school was named an Exemplary School and now serves as a model site in the OPS.
CEMS professional development was designed around the National Science Education Standards (National Research Council 1996) and included an emphasis on standards-based practices, with specific focus on inquiry and the nature of science, coupled with content learning. Participants committed to work-embedded learning and demonstrated their learning through a portfolio with four sections: 1) beliefs and philosophy, 2) content, 3) curriculum and instruction, and 4) action research. Understanding the complexity of change, we defined the parts of the portfolio with examination of beliefs and philosophy at the core. Beliefs were the focus of Part I but were also explicit in the teachers' reflective action research (Part IV), requiring teachers to be intellectual, reflective practitioners and to inquire into teaching and learning methods. Portfolio completion also required demonstration of content learning (Part II) and enhanced pedagogy (Part III: Curriculum and Instruction), expecting teachers to integrate theory and practice in the school setting. They demonstrated their learning in units of study they developed, implemented, videotaped and reflected upon. Data were collected between rounds of implementation, analyzed, and used to facilitate changed instruction, integrating theory and practice in the school setting. Finally, their research results were included in a database of resources available to teachers, honoring participants as producers of knowledge about teaching.
To assure flexibility in the program concurrently, a variety of approaches to the experience were available (see Figure 2). Teachers following the Individual Path chose to complete their learning through 18 hours of graduate work at the University of Nebraska Omaha or opted to work with the support of a CEMS Professional Development Specialist (PDS). Following the Team Path, teams of two to four members collaborated to learn and compile a portfolio; their learning plan was based on composite team needs. Finally, the School Wide Path resulted in school portfolios based on school needs, but not at the expense of teacher needs. In all cases, participation was voluntary.
Each approach had the participant develop and implement a learning plan, resulting in a variety of professional development activities. These plans were based on personal learning needs, school improvement goals and student achievement needs. To determine the participants' learning needs, we worked with McREL (Mid-continent Research for Education and Learning) to develop an online Profiler, a set of Likert scale statements to which participants responded. This online instrument clustered teachers' responses around categories (i.e., inquiry, equity, motivation, etc.) and compared results to exemplars. Discrepancies were identified and used by participants to develop learning goals, and databases of linked standards and research-based practices helped participants find learning resources. They completed an online plan to which their PDS responded. As these long-term plans were implemented, participants completed their online portfolio, supported electronically by the PDS in an interactive manner.
In summary, salient features central to the CEMS professional development model included:
1) Involvement of a "critical mass" (70%) of teachers at each school;
2) The active involvement of the leadership at each school;
3) Intensive study, based on teacher and student needs and focused on beliefs and philosophy, content, and instructional pedagogy;
4) Ongoing (12-18 months), consistent, and naturally embedded work with each teacher; and
5) A process for teachers' critical reflection regarding their beliefs and practices (action research).
[FIGURE 1 OMITTED]
This study was established both to evaluate Banneker 2000: CEMS and to add to the research base. The focus of the study was to determine the impact of the CEMS model (standards-based professional development with flexible options for teacher engagement and focused on the school as the unit of change) on teacher and school change and on student achievement To further define the research, the following questions were identified.
1. How does a flexible but intensive and ongoing professional development program promote teacher change?
2. How does teacher learning during this program impact student achievement and understanding in that teacher's classroom?
3. How do commitments of and participation by school leadership and a critical mass of teachers in the school impact school change and school-wide student achievement?
4. Which professional development approach/pathway in the CEMS model was most effective for teacher change and student achievement?
[FIGURE 2 OMITTED]
To respond to the first question on the impact of the model on teacher change, data were gathered through the use of pre- and post-profiles of teacher perceptions, and pre- and post-retrospective surveys of teachers' beliefs and understandings, observations, and interviews.
Pre- and post-profiler: The Profiler, previously described, was administered to all CEMS participants. Data were collected online at the beginning of the program and as participants completed work (pre- and post-profiler). These data were used by teachers to reflect on their growth and by the USP to determine change in teachers' perceptions in three categories: beliefs, content and pedagogy. Though data were collected on those three categories, the data reported here are the beliefs data, since teacher beliefs were a core area of focus during the USP and essential to the change process. A series of questions was asked to which teachers responded, scores for questions related to various beliefs categories were clustered, and composite scores were reported. The clusters include general pedagogical approaches (constructivism), expectations for students (expectations/equity), and pedagogical content knowledge (inquiry/problem solving). Change in the composite scores between the pre- and post-Profiler for each of the three categories was determined for a sample of teachers (n=25). This sample included participants that varied across grade level, professional development path, cohort, and discipline (mathematics and science).
Retrospective pre- and post-surveys: Teachers' portfolios required reflection on changes in Profiler scores at the end of their learning experience. Participants sometimes noticed declines in scores and attributed these declines to increased understanding in the area. Many participants felt that if they had known more about the topic when they completed their initial Profile, their scores would have been lower. Based on these responses from early cohorts, it was decided that retrospective pre- and post-surveys on both beliefs and understandings (see Figure 3 for a sample) would be administered to confirm accuracy of Profiler data and to inform the research.
The survey's Beliefs scale contained 11 items and the Understandings scale contained 12 items. Respondents were asked to think about the issues addressed in each question and to what degree it represented their understanding before and after participation in Banneker. Response categories included the following: 1) None, 2) Very Low, 3) Moderate, 4) High, and 5) Very High. The scale's reliability was calculated using Cronbach's Alpha. For this scale, a = .86, sufficiently high to indicate that the scale measures a cohesive concept. The scale reliability with items deleted indicated that deleting one or more of the items did not raise the scale reliability. Reliability analysis was calculated using post-Banneker impressions.
Program Completion Surveys: At the close of the initiative, surveys were mailed to all portfolio completers (n=454). A 10% return on surveys (n=45) provided data used in this analysis. Paired sample t-tests were used to identify significant differences between scores measuring teachers' impressions of their understanding and beliefs before and after Banneker.
Observations and Interviews: Observations and interviews were conducted to determine fidelity of learning as it translated into practice and to identify the "determiners" of exemplary teachers. Among respondents to the retrospective pre- and post-survey, participants were identified that represented various cohorts, learning paths, grade levels and disciplines. Twelve teachers were identified to interview and observe; full data for nine teachers were collected and included in the analysis. These teachers represented various cohorts, both mathematics and science, and each grade band (primary, intermediate, middle school and high school).
Two observations of each participant were conducted between January and May of 2006. Observation methods included: 1) global scans to establish general classroom atmosphere and 2) pre-selected rubrics (Llewellyn 2001) to discern implementation fidelity of CEMS-taught inquiry based practices. Specific rubrics included lesson presentation, communication, student engagement, classroom organization, and questioning skills. Various indicators in each area scored a teacher as using teaching approaches that ranged from traditional to practicing inquiry. Rubrics were scored to deliver a composite score in each category to determine where in this range teachers were positioned (traditional, exploring inquiry, transitioning to inquiry or practicing inquiry). The boundary between "traditional approach" and "exploring inquiry" was rated a "1." The boundary between "exploring inquiry" and "transitioning to inquiry" scored a "2," while the boundary between "transitioning" to "practicing inquiry" ranked a "3." A perfect "practicing inquiry" score was a "4." In addition, scores were examined to determine in which areas teachers had most fully implemented inquiry.
Interviews were completed with each participant after all observations of that teacher were completed. The first interview question (How do you now see yourself as a teacher in the classroom?) related to teacher change. Interviews were coded for common themes and used to determine impact of learning on teacher change.
Student Achievement and Understanding
To respond to the second research question, "How does teacher learning during this program impact student achievement and understanding in that teacher's classroom?" data were gathered from three primary sources: 1) participant vs. non-participant classroom CRT (Criterion Reference Tests) results, 2) teacher action research during strategy implementation to determine impact on student achievement and 3) teacher interviews to explore teachers' perceptions and knowledge of both student understanding and the impact of teacher learning on student understanding.
Participant vs. non-participant assessment results: A cohort of Banneker teachers who completed a portfolio was compared to a group of non-participant teachers with similar school demographics. Participants included 68 Pre-Kindergarten through 7th grade Math and Science teachers, 34 of whom completed a portfolio in 2003-2004 and 34 of which did not participate in the program. All participants in the CEMS sample were pursuing one of the individual learning paths (CEMS, independent, team or university); no teams or school-wide participants were part of the sample. Inclusion in the treatment and sample was limited to those teachers that had student CRT scores in 2002-2003, the baseline year, and who had taught in the same school, at the same grade level and in the same grade/discipline for the three consecutive years for which data were collected. These three years included the year prior to participation, the year during participation, and the year after participation. Control (non-Banneker) participants were matched to treatment (Banneker) participants according to school demographics, specifically the percent of minority students present in a school and the percent of free/reduced lunch students in a school.
Student CRT scores in Math and Science were used to tabulate the Average AYP (Adequate Yearly Progress) Score, Average Number of Standards Mastered, and Success Rate for each teacher. These same scores were tabulated in Math and Science for the years of 2002-2003, 2003-2004, and 2004-2005. Success rate was calculated by dividing the sum of the AYP scores by the sum of the AYP test parts.
Action Research: Teachers' reflective action research required that they implement strategies/approaches in the classroom that they learned during their personal program and reflect on student impact during implementation. Teachers gathered achievement data on both district criterion referenced measures and classroom measures of understanding. Summaries of sample action research results for each cohort were compiled to demonstrate impact.
Interviews: The second interview question (If someone were to look at your classroom now what would they observe?) was designed to elicit teacher feedback on both classroom atmosphere and student learning. Teacher interview responses were coded for common themes.
Leadership and Critical Mass
Data for the final research question were drawn from leadership action research, Exemplary School CRTs, principals' retrospective pre- and post-surveys, and teacher interviews.
Leadership action research: Principals and other instructional leaders in each developing school were given the opportunity to complete the portfolio process themselves, individually or as a team. These action research summaries served as qualitative and quantitative data for school level impact on teacher and student learning.
Exemplary School CRT data: As a school fulfilled its commitment to include 70% of the teachers in CEMS professional development, the school's CRT results were analyzed, comparing that year's results with the previous year. If consistent increases in CRT results across grade levels were demonstrated, the school was named Exemplary. These data served as the primary measure of impact of teacher learning on student achievement at the school level.
The first Exemplary School was named during the first year of CRT implementation so their designation was based on California Achievement Test (CAT) results and on differences between school and district CRT results. Three years of CRT data are available for the next nine schools gaining exemplary status, and two years of data are available for the remaining schools, named during the last year of CEMS. Samples of these data are printed below.
Leadership retrospective pre- and post-surveys: A ten-item, retrospective survey of leadership understandings was administered. Respondents were asked to think about the issues addressed in each question and to what degree it represented their understanding before and after participation in Banneker. Response categories included the following: 1) None, 2) Very Low, 3) Moderate, 4) High, and 5) Very High. At the close of the initiative, surveys were mailed to principals in all Developing and Exemplary Schools (n=58). A 28% return on surveys (n=16) provided data used in this analysis. Respondents included principals from eight Developing Schools and eight Exemplary Schools. Data were compiled for all respondents and analyzed for differences in response between principals in Developing and Exemplary Schools.
Teacher interviews: The final interview question (What are your professional relationships with teachers both in your school and outside the school and district?) provided qualitative data to inform the impact of leadership and critical mass on school change and student achievement. Teacher interview responses were coded for common themes.
Multiple data sources were used to determine which professional development approach in the CEMS model was most effective for teacher change and student achievement. These sources included: 1) retrospective pre- and post survey results by path, 2) CRT results by path, 3) program completion rates by path, 4) teacher interviews, and 5) exit questionnaires.
Retrospective pre- and post survey results by path: The retrospective pre- and post-survey was previously described. A One-Way ANOVA was used to identify differences in reported understandings and beliefs according to teachers' Banneker pathway.
CRT results by path: These data were gathered as described in the "Student Achievement" section. In the sample used for this evaluation only CEMS (N=8) and Team (N=14) pathways had a large enough sample to allow a comparison, due to the rigorous requirements for inclusion in the sample. Paired sample t-tests were used to assess whether either group exhibited a significant mean increase from baseline to one year after program instruction.
Program completion rate by path: A program "completer" was defined as an individual who submitted a portfolio at the end of their program. The definition of "participant" was an individual who committed to participation, completed the initial profiler and began program work. It excluded individuals that committed to involvement, only to drop out prior to initiation of work. The total "completer" number by path was determined, numbers were compared to the total number of original participants by path, and percent completion was determined.
Teacher interviews and exit questionnaires: As described previously, teacher interviews were conducted with a sample group. Since references to pathway experiences emerged during the interviews, transcripts were included as qualitative data. Exit questionnaires were administered to all Banneker participants, both completers and drops. Responses to these open-ended questions were coded, along with teacher interview results, for themes. These themes served as qualitative data to inform our understandings of path effectiveness.
RESULTS AND DISCUSSION
Pre- and post-profiler: The beliefs component of the Profiler included question clusters that focused on general pedagogical approaches (constructivism), expectations for students (expectations/equity), and pedagogical content knowledge (inquiry/problem solving). Positive impacts on beliefs were apparent in all three categories (Figure 4). It is interesting to note that participants were expected to focus on only one of these areas during their program but often showed growth in all three areas. However, there were participants who measured declines in scores; many felt these declines were due to learning more about what they previously did not know and felt that their original responses were inflated. As stated earlier, the retrospective pre- and post-survey was included as a data source as a result of these responses.
Retrospective pre- and post-surveys: Figures 5 and 6 share these data and the questions on which these data were based. The paired sample t-tests indicated a significant change in beliefs for each individual question and for composite before and after scores (t(45)= 11.12, p< .05). Consistent positive impact was demonstrated on teachers' beliefs (Figure 5), in this case about their teaching. The highlighted questions showed the greatest change and indicate the importance of collaborative, teacher reflection in the context of their work (curriculum, strategies and materials). Figure 6 shares similar data for teachers' understandings. Paired sample t-tests indicated a significant gain in understanding for each individual question and for composite before and after scores (t(45)= 18.91, p<.05). While consistent and positive changes are demonstrated, the greatest increase was on questions related to teachers' own reflective practice, specifically the ability to implement research-based strategies, reflect during implementation and help students take more control of their inquiry-based learning.
Interviews: Interview data (see Table 1) support results of pre- and post-profile data and retrospective pre-post survey data, and provide teachers' rationales for changes in understandings and beliefs. Themes evident in the interviews include the importance of required reflection and use of research to their professional growth, enabling them to "discover" their own learning and translate that into classroom practice. A strong message of increased teacher efficacy and willingness to try new things emerged, and participants often related this to their changed beliefs--most strongly those beliefs about the role of inquiry in the classroom and the release of teacher control to allow student decisions and inquiry in the classroom.
Observations: Observation data showed learning environments in which students were engaged in lessons and teachers facilitated the learning. All participants attempted to allow student ownership of learning by releasing some control and providing inquiry opportunities. Degree of inquiry implementation varied, but all made steps toward an inquiry-based classroom.
Rubric scores confirmed these patterns (see Table 2). With the exception of one teacher in one category, all were at least transitioning to inquiry, as evidenced by scores of two or greater in all categories. Lesson presentation, communication and engagement of students scored the highest, each with composite scores between 2.8 and 3.0 (strongly within the transitioning to inquiry category and close to practicing inquiry). The lesson presentation rubric evaluated teachers' practices that reflect the role of teacher as facilitator and implementation of inquiry in the classroom, as well as use of whole-group, small-group and individual instruction, flexing their approach when unexpected results occur. The communication rubric defines the teacher as "practicing inquiry" when she clearly defines expectations, expects student-to-student as well as student-to-teacher dialogue, and facilitates that communication by movement through the room, monitoring discussions and making eye contact. The student engagement rubric defines practicing inquiry as classrooms where the teacher engages students in discussion, investigation and reflection, there is frequent self-engagement by students, students are consistently active in hands-on and minds-on activities, and the teacher frequently and effectively solicits information from students. Scores in each of these categories provide evidence that the observed teachers are strongly transitioning to inquiry (and in half the cases practicing inquiry). Slightly lower composite scores occurred for classroom organization and questioning skills, but all but one teacher scored at least as transitioning to inquiry. In some cases, classroom organization was fixed with no flexibility on the teachers' part, often resulting in a lower score on that rubric. Overall, the observation data confirm that teachers involved in this research were clearly transitioning to inquiry and, in three cases, consistently practicing inquiry in the classroom.
Math T-test Analyses: T-tests to compare the Average AYP intercepts for Math at baseline were found to be non-significant. The control and treatment groups did not have significantly different starting points in terms of their students AYP scores.
A Paired sample t-test indicated no significant difference from Average Math AYP Scores at baseline to Average Math AYP Scores at the third measurement for either participants or nonparticipants. Neither group showed evidence of increased success as measured by Average AYP math scores.
A Paired sample t-test indicated a significant difference from Average Math Standards Mastered at baseline to Average Math Standards Mastered at the third measurement for participants of Banneker (t(25)= -2.55, p<.05). No significant increase for Nonparticipants was found when examining Average Standards Mastered at time 1 and Average Standards Mastered at time 3. The participant group showed evidence of increased success as measured by Average Standards mastered in Math.
Science T-test Analyses: A Paired sample t-test indicated a significant increase for participants (t(25)= -2.11, p<.05) and non-participants (t(24)= -2.25, p<.05) when examining the difference from baseline to third measurement of Average AYP Scores in science. Non-participants showed a slightly larger mean increase in student's average AYP score than participants in the Banneker program.
Paired sample t-tests indicated a significant difference from Average Science Standards Mastered at baseline to Average Science Standards Mastered at the third measurement for both participants (t(25)= -4.866, p<.05) and non-participants (t(25)= -3.14, p<.05) (See Figure 7). Participating teachers of Banneker exhibited a slightly larger mean increase in the average number of standards mastered by their students than teachers that did not participate in the program.
[FIGURE 7 OMITTED]
Discussion of T-test Results: Banneker trained teachers showed significant mean increases in Math Standards Mastered, Math Success Rate, Average AYP Science Scores, Science Standards Mastered, and Science Success Rates from the years 2002-2003 to 2004-2005. None of these increases were significantly larger than the increases demonstrated by Non-Banneker trained teachers. The absence of a significant difference between these two groups suggests the possibility of variables other than Banneker participation accounting for student achievement increases in the three years examined for this study. While classroom and school factors, such as percent of minority and free/reduced lunch students, have been controlled for through control/treatment matching, factors such as teacher motivation, expertise, and experience have not been measured or controlled. Also not accounted for are professional development hours taken by Non-Banneker teachers that may have affected classroom behavior or effectiveness. Finally, the range restriction of the five-point CRT scale makes this metric difficult to use when demonstrating change or differences due to variables. Fewer individuals score at the extremes and group means tend toward the center of the range, making it difficult to discern significant differences and/or changes in scores. As a result, these data must be interpreted in the context of additional information.
Action Research: Over 85% of teachers' action research projects detailed enhanced achievement and understanding of their students, as measured by Criterion Referenced Tests and/or teacher assessments. Table 3 shows results for a sample drawn from the same cohort year previously reported in the science and math t-test results. Common teacher research results across action research projects included 1) consistent and positive impact on California Achievement Test (CAT) scores, Criterion Referenced Tests (CRT's) and/or classroom assessments, 2) increased demonstration of student understanding in journals, student work and dialogue, 3) enhanced problem-solving and questioning, and 4) other positive impact on attitudes and behaviors that increased student opportunity to learn. As teachers completed their required teacher reflection, gathering data and focusing on what students did and said, they reported looking more closely at individual students and what each student understood, rather than looking at class averages. This usually led to modifications of instruction for the entire class and/or individual students, further enhancing understanding. Inconsistencies in CRT reporting between action research and t-test results may be due to individual teachers sampled, as well as the action research focusing on one year as compared to three years' scores from three cohorts measured in the t-tests.
Interviews: Themes that emerged from interview coding indicated that student understanding was deeper and more conceptual and that their abilities to engage in this type of thinking were enhanced (see Table 4). Teachers felt their students understood more and that, while CRT scores improved (in these particular classrooms), the more important results were student ownership of learning, collaborative efforts, and more questioning and thinking on the students' parts.
Leadership and Critical Mass
Leadership Action Research: One example of a site with both leadership commitment and a critical mass of teachers (all teachers) was Pinewood Elementary. Pinewood's effectiveness can be attributed to these two important aspects since the principal and instructional facilitator completed a leadership portfolio. During their work, they learned more about inquiry and facilitated professional development for their entire staff and also supported the purchase and use of classroom materials. Table 5 shows results drawn from their leadership action research. These results from a survey of all Pinewood teacher participants indicate that teachers' level of comfort with inquiry-based science increased significantly. Not only did the comfort level increase but also the teachers' love of teaching inquiry based science increased.
In addition, all students of participating teachers were surveyed. At the end of the professional development period, a majority of the students liked science. It was also clear from the student data that they perceived that their teachers liked science teaching. More than seventy-five percent of the students felt that they would pass their CRT.
Every school in which a principal or other instructional leader completed a leadership portfolio achieved Exemplary status. Recall that this required 70% teacher participation and positive impact on CRTs. Not all schools that met participation requirements were named Exemplary, but all schools where leaders were involved gained this status.
Exemplary School CRT Data: During the life of the USP, 17 Exemplary Schools were named. Data from 10 Exemplary Schools reported here. With the exception of the single high school named exemplary, these schools serve populations with high percentages of lower socioeconomic status students, English as a second language learners, and students of color. Six of these schools were named Exemplary in the summer of 2004, based on consistently improved CRT results between 2003 and 2004. Data from 2005 are included for these schools; while some scores continued to increase, some declined. Results from four schools named Exemplary in 2005 includes only two years of data, 2004 and 2005.
Central Park Elementary (Figures 8 and 9) shows an increase each year in grades one, two, four and six. Grades three and five showed an increase in science CRT results between years one and two and a decline during year three. Central Park showed increased mathematics success on the CRT during years one through three for grades one, two, four, five and six. There was a decrease in the mathematics CRT for grade four. The principal was supportive from the beginning of the initiative both in terms of personal participation in leadership meetings and in supporting teachers in their efforts.
Liberty Elementary (Figure 10) was involved in a school-wide professional development. The principal was instrumental in leading teachers to this point. She bought into the idea of school-wide professional development and helped, in conjunction with the instructional facilitator, to orchestrate implementation of the work. As a result, Liberty was successful in science CRT's in grades one, two, and five across all three years, though decreases in CRT success occurred during the third year at the third, fifth and sixth grade levels.
Lothrop Spanish, Science, and Technology Magnet School had increased CRT success over the three years in grades one and four (Figure 11). There were increases in grades two and three the first year but a slight dip between years two and three. The school leadership at Lothrop was very supportive and involved with the partnership from the beginning of the grant.
Minne Lusa Elementary (Figure 12), like Liberty, joined the partnership as a school-wide project. The principal engaged grade level teams to complete the professional development. Time during the school day was allocated for teams to work together, and the resulting CRT success rate was very positive. In general, each year all grade levels except grades one and four increased the CRT scores. However, there was just a slight decrease in the Science CRT scores in grade one from year two to three. Grade four showed the largest dip in CRT scores during years two and three.
Another Exemplary School, Saratoga Elementary, showed significant increases in CRT success each of the three years in grades two, four and six (Figure 13). All grade levels increased in CRT success from year one to two, but grades one, three and five showed a decrease in success between years two and three.
Burke High School (Figure 14) was the lone Exemplary high school in this study. Students at Burke increased CRT success each year in biology and chemistry. Physics success increased between years one and two but slipped slightly between the second and third years.
Franklin Elementary (Figure 15) was named Exemplary and distinguished itself in both mathematics and science. There was an increase in mathematics CRT success each year. The science CRT success also increased across all grade levels except for a slight decrease for 5th grade.
King Science Center (Figure 16), a middle school magnet center, showed progress in mathematics and science CRT's each year in mathematics (algebra, geometry, pre-algebra and math 5-7). The CRT science success rate increased each year in 7th and 8th grade and Biology but showed a slight decrease from year one to two in grades five and six.
Bryan Middle School (Figure 17) was successful each year in mathematics (Math 7, Algebra, Geometry and Pre-Algebra). Another Exemplary middle school, Hale (Figure 18) showed success each year in math 7, geometry, and pre-algebra. There was a decrease in CRT success in algebra.
Leadership retrospective pre-post surveys: Figure 19 shows the results from the USP principals' retrospective pre- and post-survey of understandings. The highlighted questions are those that showed the greatest change for "all" respondents though there were positive changes in all cases. It is interesting to note the questions on which Exemplary School principals showed greatest growth (questions 1, 2 8, 9 and 10). These questions relate to understanding science curriculum and instruction at the classroom level and the role of leadership in effecting change. Most of the remaining questions address areas of understanding common to other disciplines required of all leaders in the OPS, especially monitoring data. One might infer that the more intimate understanding of science standards, inquiry and materials, as well as leadership capacity to affect change in math and science, were essential to the schools' Exemplary status. Further research in this area is warranted.
Teacher interviews: Data emerged during teacher interviews indicating the importance of expanded networks and how the networks enhanced participants' leadership capacity (Table 6). Though teacher leadership development was not the focus of this work (i.e., not a trainer-of-trainers model or leadership development effort), teacher leaders emerged. Principals often partnered with them and recognized their growth. Many became formal or informal leaders in their school, enhancing the work of the collaborative community, and several took on leadership roles at the district level.
Retrospective pre- and post survey results: The One-Way ANOVA analysis showed that on only one question did teachers identified by path show a significant difference in their level of understanding after participation in Banneker. On question seven of the survey, instruction to develop students' conceptual understanding, as supported by research, University path teachers reported a significantly higher level of understanding after Banneker participation than Team path teachers. Composite scores on the beliefs instrument after participation showed no significant differences when comparing path.
CRT results by path: Pathway comparisons appeared to show that CEMS-supported teachers outperformed Team instructed teachers on all measures of science achievement. While these differences were not significant, CEMS-supported teachers exhibited increases in student achievement more often than Team instructed teachers. Pathway comparisons for other paths are not possible due to the rigorous criteria for inclusion in the study sample.
Program completion rate by path: Completion rates varied by path. Rates by path were: CEMS-supported (50%), University (52%), Independent (59%), Team (63%), School-wide (93%) and Workshop participants (96%). Completion was high for the school-wide path since the principal committed support systems to assure this. Reasons for ceasing participation in other paths was often reported in exit questionnaires as due to constraints in participants' personal lives rather than from the experience itself.
Teacher interviews and exit questionnaires: While completion rate was low for CEMS-supported and University paths, teacher feedback from interviews and exit questionnaires indicated a deeper level of change and commitment on the part of these teachers. Specific examples include extensive, positive feedback on experiences of the university cohorts, including confidence in content understanding, models for classroom application and the strong network of colleagues established during their 18 months of study together. They also spoke highly of university professors and the attention provided to them by faculty. This same type of personal experience was cited as a positive aspect of the CEMS path, especially when working with certain professional development specialists. The opportunity to work with their PDS both inside and outside the classroom made application of theory to practice much easier for this group. Coupled with significantly greater CRT success as compared with Team pathway participants, this indicates that personal interactions with the PDS/mentor allowed greater translation of participant learning into effective classroom practice.
While some respondents spoke positively of team and school-wide experiences, it was not uncommon to hear that some team members carried more of the burden of work and, as a result, often learned more in the process. While every school that chose the schoolwide option reached exemplary status, there was more resistance from some teachers when the principal required participation. Exemplary Schools that had the least resistance were those where the principals encouraged participation, allowed various paths to completion of participation, and provided support for teachers work (pay and/or release time).
The strengths of the individual pathways (university and CEMS-supported) and the team pathways (small teams and school-wide teams) vary. Individual paths had lower retention rates but deeper learning while team paths allowed greater participation and retention, but increased resistance and, in some cases, produced only cursory learning. Exit questionnaires consistently included statements that stressed the importance of pathway choice for completion of work and successful teacher learning.
CONCLUSIONS AND RECOMMENDATIONS
This research evaluated the impact of the Banneker 2000: CEMS professional development model on teacher and school change and determined the degree of impact on student achievement. Extensive qualitative data and initial quantitative data were used to evaluate and summarize the initiative's impact. The results were based heavily on the qualitative data and further organization and analysis of our quantitative data is warranted. However, the intent of this work is to summarize general impacts using the data already available. One area of concern is the low return rate of on-program completion surveys. This rate is likely due to teacher receipt of surveys in the summer, a time at which responsiveness might be limited. In the future, return rate might be enhanced by administration before the school year ends.
Research results on teacher change include evidence that teachers were able to change their previous belief systems about how students learn, who can learn and how to support student learning through inquiry-based approaches. All data sources (pre- and post-profile results, retrospective pre-post surveys and teacher interviews) provide evidence that the program served as an effective change intervention. Teachers' action research into inquiry-based (standards) instruction over an extended time proved central to this experience in which teachers acted as researchers and reflective practitioners. Their action research experiences required that they look at individual students' understandings and disaggregate data in their classroom rather than focus on mean results. Their required, reflective research forced teachers to listen to students, and listening opened doors to changed practice.
Letting students take center stage during inquiry experiences meant challenging teachers' previous beliefs and practices. Data consistently confirm that relaxing rigid control in the classroom allowed students to take center stage and teachers to understand students' understandings and abilities more fully--understandings that often exceeded the teachers' original expectations. This increased both student and teacher efficacy since student engagement and understanding improved using this approach.
Another essential factor that promoted teacher change, as reported by teachers during interviews and confirmed by the retrospective pre-post survey, was exposure to and training in the use of research, both their own action research and our research. Teachers learned to select research-based strategies for classroom use and to modify instruction based on their ongoing action research.
Finally, enhanced content and pedagogical content (especially inquiry) understandings made teachers more confident as they implemented new strategies. Elementary teachers often avoided science instruction because they lacked science content background, but this changed as they learned more content and strategies to teach that content effectively. Secondary teacher participants most often credited enhanced efficacy to improved pedagogical content knowledge.
Independent work by the program external evaluator supported our findings. Her case study results found the initiative was successful at promoting teacher change due to four factors.
1. It taught inquiry through inquiry (action research).
2. It examined teacher beliefs about teaching and learning.
3. It helped teachers establish professional development goals that were attainable.
4. It provided teachers with a positive self-efficacy (showed teachers they can make a difference in student learning.
This research also demonstrated that teacher change translates in some degree to student achievement. Full implementation of new policies and practices takes many years, and even when support systems are in place teacher implementation takes several years (Bybee 1997). Though participating teachers outperformed non-participating teachers on some district CRT measures, none of these differences were significant. However, the researchers felt that the five point CRT scale and inability to control all variables influenced these results. In addition, CRT results through the years were for different student cohorts. This, coupled with the length of time it takes to fully implement new learning, would encourage further research in the classrooms of these teachers in the future.
The impact on student understanding was more evident in teacher action research and researcher observations. Action research results consistently showed deeper conceptual understanding by students as evidenced in formative and summative classroom assessments not always assessed in the CRTs. Classroom observations during this research found classrooms with students at the center engaged in questioning, explanation and dialogue, grappling with ideas and searching for understanding.
Program impact was most evident in schools where a high percentage of teachers completed CEMS studies. Almost all schools that met participation goals also impacted student achievement at the school level. The work in both the CPMSA and the USP has proven that school leadership and a critical mass of teacher participants are important.
The Exemplary Schools that experienced both teacher commitment and student achievement were the sites where school leadership was significantly involved. Examples are Pinewood, Liberty, Lothrop, Minnie Lusa, and Central Park. In each of these schools the principals were engaged in and communicated with the teachers about the professional development work. As a result, the principals understood the work that the teachers were doing.
Though not as intricately involved in the professional development experience, principals at King Science Center and Nathan Hale collaborated to determine the staff needs and met with them to establish the positive resolve required to take action. This type of leadership is significant because it provides support for teachers that include resources and time. Principals took the time to think about ways of assisting the teachers in their work.
However, leadership can impact school change positively or negatively; positive support and participation by school leadership are essential for school change that maintains involvement of all teachers and increases achievement. However, participation demands by leadership without support systems lead only to resistance. Several Developing Schools began with teacher commitment but lost teachers over time, and thus never met participation requirements, as a result of this type of leadership.
Research related to pathway options suggests that no one pathway represented the ideal. Rather, different pathways worked best for different individual teachers. While there were some teachers who decided to leave the program, their reasons were rarely associated with the pathway they chose. Pathway options took into consideration the teacher's personal and professional circumstances and learning goals, thus provided effective professional development experiences for participants. Prior experiences did not allow teachers to make such choices. Teachers were told what they needed to do and they complied. Learning path flexibility was an effective way to implement the current teaching/learning model.
Professional development effectiveness alone will change only the teacher. While individual pathway choices met needs of individual teachers, pathways demonstrated varying retention rates. Efficiency requires engaging enough teachers in a school to make an impact. Support from the principal and collaboration among teaching peers are critical for teachers who want to implement best practices successfully. This study supports the idea that individual teachers participating in professional development and returning to their schools and expecting to produce an impact does not occur to the same degree as when critical masses of teachers are involved.
This research demonstrates that both professional development program effectiveness and efficiency can be addressed simultaneously. Path choice allows effective teacher learning, while involvement and support by leadership allows participation by many teachers, regardless of pathway. To balance effectiveness and efficiency requires options for professional learning from which teachers can choose, as well as school-wide support and focus over a long term on the impact of teacher learning on student understanding.
Based on these findings, recommendations for school systems considering use of an effective professional development model include:
1. Teacher beliefs must be examined in order to impact student learning.
2. Critical reflection on the impact of teaching practice on individual students is central to changing teacher beliefs.
3. Content and pedagogical content knowledge are essential to improved practice but must be addressed in concert with reflective practice in order to impact teacher and school change.
4. Choice-is necessary for effectiveness of and significant participation in an intensive and ongoing professional development program.
5. A critical mass of teachers must be involved in order to improve achievement consistently at the school level.
6. Positive support and participation by school leadership are essential for school change that impacts achievement.
Bybee, R. 1997. Achieving Science Literacy: From Purposes to Practices. Portsmouth, Heinemann: 232 pages.
Darling-Hammond, L. 1999. Teacher Quality and Student Achievement. Washington, Center for the Study of Teaching and Policy, University of Washington: 46 pages.
Darling-Hammond, L. 2000. Teacher quality and student achievement: a review of state policy evidence. (Website for Education Policy Analysis Archives). Retrieved from the internet 30 May 2006: http://epaa.asu.edu/epaa/v8n1/
Fullan, M. and S. Stiegelbauer. 1991. The New Meaning of Educational Change (2nd ed.). New York, Teachers College Press: 354 pages.
Hargreaves, A. and M. Fullan. 1998. What's Worth Fighting for Out There? New York: Teachers College Press: 127 pages.
Knowles, M.S., E. R. Holton, and R. A. Swanson. 1998. The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development (6th ed.). Woburn, Butterworth-Heinemann: 263 pages.
Koba, S., W. Clarke, and C. T. Mitchell. 2000. Action research: collaborative efforts in teacher change. In D. L. Jordan, M. A. Henry, and J. T. Sutton (eds), Changing Omaha Classrooms: Collaborative Action Research Efforts. Aurora, McREL: 90-102.
Koba, S., and W. Clarke. 2002. Action research: a tool to promote inquiry-based teaching and learning. In: D. L. Jordan and J. T. Sutton (eds.), Action Research in Science Education. Aurora, McREL: 1-40.
Llewellyn, D. 2001. Inquire Within: Implementing Inquiry-Based Science Standards. Thousand Oaks, Corwin Press: 146 pages.
Lortie, D. 1975. Schoolteacher: A Sociological Study. Chicago, University of Chicago Press: 244 pages.
Loucks-Horsely, S., N. Love, K. E. Stiles, S. Mundry, and P. W. Hewson. 2003. Designing Professional Development for Teachers of Science and Mathematics. Thousand Oaks, Corwin Press: 336 pages.
McCombs, B. and J. Whisler. 1997. The Learner-Centered Classroom and School: Strategies for Enhancing Student Motivation and Achievement. New York, Jossey-Bass: 228 pages.
National Commission on Mathematics and Science Teaching for the 20th Century. 2000. Before It's Too Late: A Report to the Nation from the National Commission on Mathematics and Science Teaching for the 21st Century. Washington, DC, U.S. Department of Education: 47 pages.
National Research Council. 1996. National Science Education Standards. Washington, DC, National Academy Press: 245 pages.
National Research Council. 1999. How People Learn: Brain, Mind, Experience, and School. Washington, D.C., National Academy Press: 240 pages.
National Research Council. 2005. How Students Learn History, Mathematics, and Science in the Classroom. Washington D.C., National Academies Press: 590 pages.
Susan B. Koba
Omaha Public Schools
1219 North 54 Street
Omaha, Nebraska 68132
Carol T. Mitchell
Department of Teacher Education
University of Nebraska Omaha
Omaha, Nebraska 68182
Table 1. Teacher change themes and supporting participant quotes drawn from teacher interview data. Theme Teacher Comments Required * Nowhere do you have to look so reflection closely. Wouldn't have done it without Banneker. * I've always reflected, but I didn't really know what to reflect on or about. I'm more targeted now--styles of teaching, who I'm teaching. * Ability to reflect and modify instruction Teacher * Confidence to try things efficacy * I have confidence to do my job. * More willing to try non-traditional lessons. I have control and confidence in decisions and judgments about it (cooperative learning). * I am a much better teacher today because of Banneker. Before Banneker I just did not teach science. I was afraid to teach it because I felt that I didn't understand it. I avoided scheduling science or placed it at the end of the day. Many times we just didn't get to it. * I am more confident in my science teaching and more willing to take risks when planning investigations. I welcome the many questions that my students ask. * I feel more comfortable doing science with my students now. * I have always loved science. I just needed more work on understanding science concepts and delivery of science...I am more turned on to science now than ever before because of Banneker and the NASA work. Using * Learned to determine the value of research something before trying it and during implementation--from action research--not just because it was a trend. * Reading the research pushed me to do more group work * My teaching wouldn't be where it is. I wouldn't be reading research. I was in survival mode. Now I can sort research and determine what's valid. Inquiry: * Science literacy and inquiry--Doing Think, science, not just learning about it question * Banneker definitely caused me to focus and on thinking--inquiry--not just doing but explain thinking--"What "are you thinking?" "Explain yourself." Tell me more." * Teaching for understanding--not for the test. * My classroom has changed over the years. Today I am including more hands-on science than I did when I first started at Skinner. I have moved from just reading science to science investigations. Teacher * More inquiry, freedom, choices--less control rote * I was a lot more rigid before. Thought there was a correct way and a wrong way--with little in between. Now no one way--with every group of kids I must change because they're different and respond differently. * I was too quick to help students ... Before, I felt confident in content and felt I had materials to use in the classroom ... but letting go of control--that's all Banneker. * Strengthened my ability to engage students in science; Less teacher-directed * Students as active participants instead of just bystanders--Know what they're doing and why they're doing it--more on task--fewer discipline problems My * Discovering my own learning again learning * Know more about how kids learn and how to use that to build lessons * Increased content & pedagogical content knowledge, especially nature of science Table 2. Observation data derived using scoring rubrics to determine degree of inquiry-based instruction in CEMS participants' classrooms (n=9). Rubric Category Lesson Communication Engagement Teacher Presentation of Students A 2.5 2.5 2.5 B 2.5 3.0 3.0 C 2.5 2.5 3.0 D 3.0 3.5 3.0 E 3.5 3.0 2.5 F 4.0 3.5 3.5 G 3.0 3.0 2.5 H 3.5 3.5 3.5 I 2.5 2.0 2.5 Composite Results 3.0 2.9 2.9 Rubric Category Classroom Composite Teacher Organization Questioning Score A 2.5 2.0 2.4 B 2.0 2.5 2.6 C 2.0 2.5 2.5 D 3.0 3.5 3.2 E 2.5 2.5 2.8 F 3.0 2.5 3.3 G 3.0 3.0 2.9 H 3.5 3.0 3.4 I 2.0 1.5 2.1 Composite Results 2.6 2.6 2.8 Table 3. Sample action research results drawn from 2003-2004 cohort. Grade and Discipline Research Focus Achievement Impact 1st, Math Problem-based Using PBL--95% Advanced on Learning CRT vs. no PBL-50% Advanced, 48% Proficient and 2% Progressing. 1st, Science Graphic Increased performance on organizers CRT's 1st, Math Cooperative Higher success rate on Learning CRT's than previous year 1st, Math Manipulatives Increased performance on to scaffold CRT's over previous years learning 2nd, Science Journals to 100% Advanced on all CRT's process hands- on science 3rd, Science Literacy Not reported centers-- science vocabulary and concept growth 3rd, Science Cooperative Increased performance on learning CRT's 4th, Science Learning cycle CRT Round 1 (4-04)--64% Proficient or Advanced; CRT Round 2 (4-03)--90% Proficient or Advanced; Round 3 (4-03 Form B)--95% Proficient or Advanced. 5th, Science Performance- % of students Proficient or based Advanced increased from 77% to assessment 95% 5th/6th, Math Problem-solving CRT success (Proficient or in Science Advanced) 15t CRT--42%; 2nd CRT --59%; 3rd CRT--71% 6th, Science Science Not reported journals to enhance conceptual understanding 7th, Science Inquiry and Increased performance on self-direction CRT's Grade and Discipline Other Impacts 1st, Math Increased self-direction in problem- solving; improved perceptions of students and parents; increased confidence in problem-solving 1st, Science Enhanced abilities to communicate understandings. 1st, Math Increased student discussion of work; positive perceptions. 1st, Math Better understanding of mathematical concepts; moved away from dependence on manipulatives 2nd, Science Improved participation in science; student--reported increase in concept understanding 3rd, Science Increased median scores on classroom assessments; improved understanding of science concepts; increase in self- directed learners 3rd, Science Improved cooperation 4th, Science Increased student confidence and risk- taking; more detailed and mature journal answers; more engaged students with more positive behaviors 5th, Science Improved work habits and engagement; increased confidence to answer questions, discuss and interact. 5th/6th, Math Increased student engagement 6th, Science Better science writers; journals indicated increased understanding; increased engagement and enthusiasm 7th, Science Increased engagement and time on task; improved self-direction. Table 4. Student understanding themes and supporting participant quotes drawn from teacher interview data. Themes Teacher Comments Working * Students doing more of the questioning and together to thinking question and * Questioning. Kids are using white boards, doing explain research, working together to answer their own questions, explaining to each other. * Kids are more likely to and free to ask questions. They respect other people's thoughts even if wrong, and let them explain. Responsibility * Their understandings were deeper because they for thinking had to struggle through the thinking aspects (i.e., through inquiry). * Not all regurgitation. Great to know facts, but it's more important to explain. * Kids are the ones thinking rather than me pouring things in their heads. * My students are independent learners and work effectively individually and in groups. * My students understand their roles. Changing * Through CEMS I learned tools to aide the teacher environment in facilitating that type (i.e., inquiry) of environment for students, leading to deeper understanding * Considering changes last year (departmentalization), our 4th graders are now 5th graders-I believe their CRT's are 100% for every student. Such a difference in knowledge the bring to the curriculum compared to past 5th graders. Table 5. Pre-and Post-Survey results from a leadership team's action research at the first Exemplary School, Pinewood Elementary. Initial Final Survey % Survey Focus of Survey Question Response % Response Teacher level of comfort with 15% 76% inquiry-based science Teacher love of teaching 5% 72% inquiry-based science Student's level of "liking" 55% 90% science Student response--whether their 40% 89% teacher liked teaching science Students thought they would pass 30% 80% their CRT Table 6. Leadership development themes and supporting participant quotes drawn from teacher interview data. Theme Teacher Comments The * Greatly opened a network I didn't know importance of was there before--of teachers so networking passionate and excited about science; The community challenged a lot of us beyond our comfort zone--almost like a sorority--dedicated to teaching. * Network of people to communicate with; common base knowledge to communicate with. * My professional relationships with teachers in my building are minimal relative to science. Many of them are not teaching science. The other Banneker teacher and I do talk about what is going on during science lessons. I have more communication with teachers in the district who have had Banneker classes with me and those who have been in Banneker workshops. We share information and ideas frequently. * My relationship (professional) is with another teacher in my building who was also a part of the Banneker program. Other colleagues that I communicate are those who were in the Banneker classes. * I sometimes discuss what I am doing with other Banneker teachers in my building. Took on a * Gave me more confidence in what I was leadership doing. I took on more leadership... role wanted to share and take the next step * Want to share; feel a responsibility to do so. Did more * I've done so many things I never than I expected to do...anything is possible expected of * I didn't know what I was getting into. myself It totally exceeded my expectations. The Banneker process took me way further than I ever dreamed of. I had career goals, but never thought they'd be happening this early. * I don't know if Teresa and I would have pushed departmentalization without CEMS. Figure 3. Sample retrospective pre-and post-survey instrument to evaluate participants' understandings and abilities. Directions: Read each of the statements in the first column and rank yourself in the second column (shaded gray) by thinking back to your understanding BEFORE your participation in Banneker 2000: CEMS. Next, think about your level of understanding about each statement AFTER your participation in Banneker and indicate that understanding (i.e., NOW) in the third column. Circle the appropriate numbers using the following key: 1=no understanding 2-very low level 3-moderate level 4-high level 5=very high level My Level of Understanding or Ability How would you describe BEFORE BANNEKER your level of understanding of or Very Very ability in the following: None low Moderate High High 1. Teacher research to 1 2 3 4 5 promote change requiring personal growth in attitudes and skills. 2. Teacher research to 1 2 3 4 5 successfully implement learned teaching strategies. 3. Critical reflection on 1 2 3 4 5 instruction to encourage student success in math or science. 4. Implementation of 1 2 3 4 5 specific teaching strategies to support student inquiry. 5. Teaching big ideas and 1 2 3 4 5 concepts as well as facts. 6. Instruction to elicit 1 2 3 4 5 and address students' prior knowledge, as supported by research. 7. Instruction to develop 1 2 3 4 5 students' conceptual understanding, as supported by research. 8. Instruction to help 1 2 3 4 5 students think about and take control of their own learning, as supported by research. 9. Formative assessment 1 2 3 4 5 strategies to promote inquiry and student understanding. 10. Ability to map units 1 2 3 4 5 of study by unpacking standards to improve instruction. 11. Ability to analyze 1 2 3 4 5 and enhance lessons to promote inquiry and student understanding 12. Ability to engage all 1 2 3 4 5 students in learning math and/or science. My Level of Understanding or Ability How would you describe AFTER BANNEKER your level of understanding of or Very Very ability in the following: None Low Moderate High High 1. Teacher research to 1 2 3 4 5 promote change requiring personal growth in attitudes and skills. 2. Teacher research to 1 2 3 4 5 successfully implement learned teaching strategies. 3. Critical reflection on 1 2 3 4 5 instruction to encourage student success in math or science. 4. Implementation of 1 2 3 4 5 specific teaching strategies to support student inquiry. 5. Teaching big ideas and 1 2 3 4 5 concepts as well as facts. 6. Instruction to elicit 1 2 3 4 5 and address students' prior knowledge, as supported by research. 7. Instruction to develop 1 2 3 4 5 students' conceptual understanding, as supported by research. 8. Instruction to help 1 2 3 4 5 students think about and take control of their own learning, as supported by research. 9. Formative assessment 1 2 3 4 5 strategies to promote inquiry and student understanding. 10. Ability to map units 1 2 3 4 5 of study by unpacking standards to improve instruction. 11. Ability to analyze 1 2 3 4 5 and enhance lessons to promote inquiry and student understanding 12. Ability to engage all 1 2 3 4 5 students in learning math and/or science. Figure 4. Pre- and Post-Profiler results for teacher beliefs in three categories (n=25). Number above bar = mean responses from sample. Pre Post Constructivism 2.9 3.3 Expectations/Equity 3 3.3 Inquiry/Problem Solving 2.9 3.2 Note: Table made from bar graph. Figure 5. Results from the teacher participants' "beliefs" retrospective pre- and post-survey and associated questions (n=45). Highlighted questions showed greatest change. Mean response Question # 1 2 3 4 5 6 Before 3.3 3.3 3.4 4 3.6 2.9 After 4.4 4.6 4.4 4.8 4.5 4.3 Mean response Question # 7 8 9 10 11 Before 2.8 3 3.4 3.8 3.9 After 4.3 4.4 4.6 4.7 4.7 Questions--Rank your level of agreement with each statement: (Note: This was on a scale of 1-5, with 5 being the most positive. The number in parentheses after the question is the change in mean response for the question) 1. All students can learn science and mathematics subject matter through inquiry and/or problem solving. (+0.9) 2. Personal and critical reflection on my own instruction is essential to improved student understanding and achievement. (+1.3) 3. Assessing each student's understanding of concepts is essential if all students are to master mathematics and science. (+1.0) 4. The way in which I teach impacts the success of all my students. (+0.8) 5. The role of high quality instructional materials is to support both what is taught and how it is taught.. (+0.9) 6. Mapping instructional units helps me develop an understanding of how concepts are developed. (+1.4) 7. Mapping units of study by unpacking standards clarifies and prioritizes content and develops lessons that promote student understanding. (+1.5) 8. Modifying instructional materials should involve reflecting on research about how people learn (my own research and that of others). (+1.4) 9. Developing instructional materials in collaboration with others enhances unit and lesson quality and promotes teacher learning. (+1.2) 10. Conducive learning environments are created when positive students attitudes about learning science and mathematics are developed. (+0.9) Creating a classroom environment conductive for learning includes recognizing a student's progress and effort. (+0.8) Figure 6. Results from the teacher participants' "understandings and abilities" retrospective pre- and post-survey and associated Questions (n=45). Highlighted questions showed greatest change. Mean Response Question # 1 2 3 4 5 6 Before 2.7 2.76 3.02 2.87 13.07 2.85 After 4.11 4.2 4.35 4.24 4.20 4.2 Mean Response Question # 7 8 9 10 11 12 Before 2.67 2.7 2.67 2.78 2.78 3.28 After 3.98 4.25 4 4.15 4.28 4.46 Questions--Rank your level of understanding of: (Note: This was on a scale of 1-5, with 5 being the most positive. The number in parentheses after the question is the change in mean response for the question) 1. Teacher research to promote change requiring personal growth in attitudes and skills. +1.41) 2. Teacher research to successfully implement learned teaching strategies. (+1.44) 3. Critical reflection on instruction to encourage student success in math or science. (+1.33) 4. Implementation of Specific teaching Strategies to support student inquiry. (+1.47) 5. Teaching big ideas and concepts as well as facts. (+1.13) 6. Instruction to elicit and address students' prior knowledge, as supported by research. (+1.35) 7. Instruction to develop students' conceptual understanding, as supported by research. (+1.31) 8. Instruction to help students think about and take control of their own learning, as supported by research (+1.55) 9. Formative assessment strategies to promote inquiry and student understanding. (+1.33) 10. Ability to map units of study by unpacking standards to improve instruction. (+1.37) 11. Ability analyze and enhance lessons to promote inquiry and student understanding (+1.5) 12. Ability to engage all students in learning math and/or science. (+1.18) Figure 8. Science Criterion Referenced Test results for Central Park Elementary, named Exemplary based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 1 70 75 84 2 81 88 94 3 61 73 69 4 67 76 79 5 32 71 63 6 58 70 81 Note: Table made from bar graph. Figure 9. Mathematics Criterion Referenced Test results for Central Park Elementary, named Exemplary based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Math 02-03 Math 03-04 Math 04-05 1 93 92 95 2 86 87 95 3 78 83 80 4 72 80 83 5 65 74 82 6 71 76 82 Note: Table made from bar graph. Figure 10. Science Criterion Referenced Test results for Liberty Elementary, named Exemplary in science based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 1 70 83 94 2 77 89 97 3 78 78 67 4 90 91 72 5 28 61 63 6 49 71 58 Note: Table made from bar graph. Figure 11. Science Criterion Referenced Test results for Lothrop Magnet School, named Exemplary in science based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 1 74 89 89 2 64 88 82 3 39 76 70 4 57 81 87 Note: Table made from bar graph. Figure 12. Science Criterion Referenced Test results for Minne Lusa Elementary, named Exemplary in science based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 1 88 97 95 2 57 91 95 3 63 84 93 4 95 98 84 5 69 85 91 6 47 72 80 Note: Table made from bar graph. Figure 13. Science Criterion Referenced Test results for Saratoga Elementary, named Exemplary in science based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 1 84 97 90 2 64 91 94 3 50 73 70 4 65 69 83 5 16 69 61 6 71 70 79 Note: Table made from bar graph. Figure 14. Science Criterion Referenced Test results for Burke High School, named Exemplary in science based on consistent increases in CRT results between 2002-2003 and 2003-2004. Grade Level Science 02-03 Science 03-04 Science 04-05 Biology 59 67 75 Chemistry Course 54 69 73 Physics 76 89 86 Note: Table made from bar graph. Figure 15. Mathematics and Science Criterion Referenced Test results for Franklin Elementary, named Exemplary based on consistent increases in CRT results between 2003-2004 and 2004-2005 Grade level Math 03-04 Math 04-05 Science 03-04 Science 04-05 1 91 97 96 99 2 96 96 83 93 3 92 99 85 89 4 70 92 57 75 5 78 82 66 63 6 49 77 47 75 Note: Table made from bar graph. Figure 16. Mathematics and Science Criterion Referenced Test results for King Science and Technology Magnet School, named Exemplary based on consistent increases in CRT results between 2003-2004 and 2004-2005. Course 2003-04 2004-05 Math 5 76 92 Math 6 64 75 Math 7 26 59 Algebra 52 93 Geometry 54 100 Pre-Algebra 50 80 Science 5 57 52 Science 6 54 53 Science 7 62 80 Science 8 52 61 Biology 19 67 Note: Table made from bar graph. Figure 17. Mathematics Criterion Referenced Test results for Bryan Middle School, named Exemplary in mathematics based on consistent increases in CRT results between 2003-2004 and 2004-2005. Course Math 03-04 Math 04-05 Math 7 72 81 Algebra 81 97 Geometry 100 Pre-Algebra 73 87 Note: Table made from bar graph. Figure 18. Mathematics Criterion Referenced Test results for Hale Middle School, named Exemplary in mathematics based on consistent increases in CRT results between 2003-2004 and 2004-2005. Course Math 03-04 Math 04-05 Math 7 44 51 Algebra 77 73 Geometry 72 80 Pre-Algebra 54 62 Note: Table made from bar graph. Figure 19. Results from the principals/ "understandings and abilities" retrospective pre- and post-survey and associated questions (n=16). Highlighted questions showed greatest change. Change in Mean Response Question # 1 2 3 4 5 All Schools 0.63 0.82 0.5 0.69 0.87 Exemplary Schools 0.86 1.14 0.43 0.68 0.86 Developing Schools 0.37 0.50 0.5 0.63 0.87 Change in Mean Response Question # 6 7 8 9 10 All Schools 0.69 0.81 0.69 1.25 0.75 Exemplary Schools 0.71 0.86 0.86 1.43 1 Developing Schools 0.62 0.75 0.5 1.12 0.5 Questions--Rank your level of understanding of: 1. Science and math standards 2. Science inquiry as it is conducted in the classroom 3. Problem-solving strategies 4. Best practices in science and math 5. Effective science and math leadership 6. Monitoring science and math achievement 7. Analysis and use of science and math achievement data 8. Identifying effective science and math materials 9. Developing teacher leaders in math and science 10. Leadership's role in effecting change in math and science
|Printer friendly Cite/link Email Feedback|
|Author:||Koba, Susan B.; Mitchell, Carol T.|
|Publication:||Transactions of the Nebraska Academy of Sciences|
|Date:||Aug 1, 2008|
|Previous Article:||Intelligent modular tool for emergency surgical treatment during payload-critical missions.|
|Next Article:||Predicting student performance in a statistics course using the Mathematics and Statistics Perception Scale (MPSP).|