Making Informed Choices: A Model for Comprehensive Classroom Assessment.

Abstract

An interdisciplinary team of sixteen faculty designed a research-based, comprehensive classroom assessment model which provides professors with a framework for making informed choices about assessing student learning. The three-year inquiry resulted in the development of a model with a five-step process, supportive planning grids, and resources to facilitate implementation of comprehensive assessment in the college classroom. Diagnostic, formative, and summative assessment methods and scoring tools are linked to learning outcomes, levels of thinking, and student characteristics. This article describes the process involved in the design and implementation of a campus-wide assessment model and its impact on the teaching/learning culture.

Introduction

Historically, assessment at the college/university level has been used to judge the quality of student work and assign grades (Guskey, 1988). While this is a necessary function in most institutions of higher learning, it should not be the only one. Assessment in the college classroom must have the fundamental purpose of improving student learning.

Student learning encompasses the ability to understand and apply the knowledge and skills learned to new contexts. If students truly understand, they should be able to demonstrate, explain, and interpret information in a variety of ways. Understanding requires students to view new content through the filter of prior knowledge and personal values, integrating it using a consciously critical perspective (Wiggins & McTighe, 1998). This type of student understanding goes beyond being able to reproduce facts and summarize content; thereby, expanding the role of assessment further than merely the determination of a grade. When assessment is linked to understanding, it becomes a complex, multi-faceted process, one in which both the student and the teacher are actively involved (Angelo & Cross, 1993). Change in how assessment is viewed is closely linked to increased understanding about how students learn. The current thinking is supported by research on learning styles (e.g., Dunn, 1990; Silver, Strong, & Perini, 1997; Smeaton, P., Mueller, S. & Waters, F., 2000), multiple intelligences (Gardner, 1993; Goleman, 1995), and brain-based learning (Jensen, 1998; Sylwester, 1995). When the academic content is embedded in an authentic context in a variety of ways, students remember it longer and have a richer understanding, allowing them to connect discrete pieces of the information to the larger ideas (Caine & Caine, 1991). The research and theories also indicate that instructors need to employ a variety of teaching strategies to meet the diverse learning needs of students. As professors utilize a variety of instructional methods, they begin to see the need for comprehensive classroom assessment and, in fact, realize that assessment and instruction are interwoven.

Research on teaching and learning strongly supports the movement toward comprehensive and non-traditional assessment (Angelo & Cross, 1993; Costa & Kallick, 1995; Kane, Khattri, Reeve, & Adamson, 1997). There is overwhelming evidence suggesting a need to shift from relying solely on traditional summative assessments, such as written tests and term papers to ongoing use of alternative assessments, such as simulations, portfolios, case studies, performances, and projects (Johnson, 1996; Wiggins & McTighe, 1998). Alternative assessments such as these mirror the more student-centered instructional strategies that have been shown to increase student achievement (Fogarty 1997). The utilization of these alternative methods gives a clearer picture of what students actually understand rather than what they have just memorized. Providing a variety of assessment methods and student choice allows students to demonstrate their knowledge using vehicles that are responsive to their learning needs.

Individual efforts in implementing alternative and comprehensive assessment techniques have been documented by college and university teachers throughout the country (Malinowski, 1994; Monson, 1997; Moscovi & Gilmer, 1996; Reynolds, 1998; Sparapani 1997; and Wolcott, 1999). These include diagnostic, formative and summative assessments as well as a variety of scoring tools. Angelo and Cross (1993), concerned about improving college teaching, compiled over fifty ways to assess learning in the classroom. These methods actively involve students in the assessment process and are linked to learning outcomes and the levels of thinking used in instruction. Clearly, there is movement on campuses to experiment with different types of assessments to meet the diverse learning needs of college students and to improve teaching and learning.

The evidence is clear, but how do college campuses put a more comprehensive view of assessment into practice? The purpose of this article is to share the East Stroudsburg University Comprehensive Classroom Assessment Model (ESU Comprehensive Task Force, 1999) and the inquiry process used to construct it.

The Process

In the fall of I997, the ESU Center for Teaching and Learning secured a Strategic Planning Grant that brought together sixteen faculty from ten different departments to begin a three year inquiry process of designing a university classroom assessment model. The inquiry process had four phases:

* Research: review of current research and literature regarding university classroom assessment and the characteristics of university learners.

* Design: synthesis of research findings and design of a framework for selecting assessments to match course outcomes and learner characteristics.

* Review: internal review of the model by faculty across campus and external review by an assessment authority to finalize and publish the model.

* Dissemination: presentations of the model to multiple audiences (university, school, department) and inclusion of the model in the Conceptual Framework of the teacher education unit.

The Model

Information gleaned from the research phase yielded a common language and led to the following definition: Comprehensive assessment is the gathering and synthesizing of information about student learning over an extended period of time and using a variety of data gathering methods and tools (ESU Comprehensive Task Force, 1999). The synthesized research also led to the development of a list of characteristics of comprehensive classroom assessment (see Figure 1). A comprehensive view of assessment incorporating diagnostic, formative, and summative methods of evaluation ensures meaningful student learning.

Figure One: Characteristics of Effective Comprehensive Assessment

* Is directly linked with course and daily outcomes and standards

* Includes opportunities for reproduction of factual knowledge and the application and/or creative production of skills, knowledge, or concepts

* Includes opportunities for self-assessment, goal setting, and personal/professional development through reflection

* Includes a variety of tools that take into consideration the characteristics of the learners and the structure of the content

* Includes performance-based tasks or projects that have a real-life context

* Provides vehicles for specific feedback to the student about strengths and areas for improvement

* Is congruent with methods of instruction

* Occurs frequently to allow correction and redirection

Based on these characteristics, a model that outlined a five-step implementation process was developed in the design phase (see Figure 2).

Figure Two: Comprehensive Classroom Assessment Model

Step One: Review student learning outcomes and level of thinking desired. What should students know and be able to do? At what level of thinking is understanding to occur? Analysis? Synthesis? Rote memory?

Step Two: Determine when to integrate diagnostic, formative, and summative assessments in the instructional process.

Step Three: Determine the class demographics. Review the class profile of learner characteristics: Are there mid-career students? Special needs students? What are the hemisphericity and perceptual values?

Step Four: Choose the assessment method. Examination of the levels of thinking and learner characteristics will enable you to choose among the assessment methods or determine how to provide options within or among assessment methods to match the outcomes and characteristics of learners.

Step Five: Determine the scoring tool. After having chosen the assessment method, choose a way to identify and communicate the critical attributes for mastery.

This model can be used by faculty to make informed choices about assessing student learning as they move away from the more traditional, almost exclusive use of summative assessment. The scope and complexity of comprehensive assessment made it necessary to create a model that was systematic, informative, and relatively easy to use. From experience, most faculty engaging in comprehensive assessment seem to go through the five steps sequentially. However, faculty are encouraged to adapt the model to meet the needs of the program, students, or instruction.

Step One: Review Outcomes and Level of Thinking

Many colleges and universities, including ESU, are now using outcomes to help define learning. If instruction is to be goal-directed, it is imperative to link assessment to outcomes. Attention also needs to be paid to appropriate levels of thinking (i.e., recall and comprehension, application, problem solving, synthesis, and reflection) in order to ensure that the assessment method matches the level of thinking practiced during instruction and required by the outcome.

Step Two: Determine When to Integrate Diagnostic, Formative, and Summative Assessments

Diagnostic assessments, completed prior to instruction, are important for determining what students already know so that instructional decisions about content and time frame can be made. Formative assessments are important throughout instruction to determine if students are learning what is being taught so that, if necessary, instructional adjustments can be made. Summative assessments, occurring at the end of instruction to determine mastery of outcomes, include a range of traditional and performance-based options.

Step Three: Determine the Class Demographics

After identification of the demographics, the impact on the demonstration of student learning should be ascertained. Demographics include a variety of elements, including the age and experience level of the student, documented physical and learning needs of students, diversity characteristics, and individual needs based on learning styles. Fortunately ESU was able to build upon a prior initiative related to learning styles. First year students at ESU have routinely completed the Dunn and Dunn Learning Style Preference Survey (Dunn, Dunn & Price, 1991) which provided information for students and professors about individual and class learning style needs. This step in the model may need to come first if a significant gap occurs between student potential and actual student performance.

Step Four: Choose the Assessment Method

Methods are the vehicles that allow the professor to gather data about student learning at various points in the teaching/learning process. Assessments methods are chosen based on course outcomes, thinking levels, and class demographics. A planning grid for each type of assessment was designed to help the professors make these decisions (see Figure 3). While the list is not exhaustive, it does offer choices in all three categories of assessment: diagnostic, formative, and summative. This step in the model might be undertaken first if there are departmental requirements for a particular method of assessment, such as a common final exam.

[Figure 3 ILLUSTRATION OMITTED]

Step Five: Determining the Scoring Tool

Not only must assessment be viewed differently, but so must scoring. A simple grade at the top of a paper or test is insufficient. A tool must be chosen to fully communicate with the student the critical attributes for mastery. Included in the model disseminated to ESU faculty were samples of scoring tools already in use by their colleagues, such as rubrics, rating scales, and checklists (see Figure 3).

The Model into Practice

During the review phase the model went through substantive changes as task force members met with representatives from their departments to obtain feedback on content and usability. In addition, suggestions were given for supporting resources that would aid their implementation of the model. An external review was then conducted by Dr. Jay McTighe, director of the Maryland Assessment Consortium and co-author of Understanding by Design (Wiggins & McTighe, 1998).

The task force decided on a "grass-roots" approach for the dissemination phase, conducting small group sessions about the model as a whole and on the specific steps, such as developing scoring tools. Task force members also presented the model at university, state, and national forums. Assessment information, including executive summaries of research and sample methods and tools, was made available through the Center for Teaching and Learning website and professional library. The purpose of this widespread dissemination effort was to engage faculty in critical dialogue about assessment as well as to foster ownership of the model university-wide.

One area where the model was integrated into practice was by faculty in the teacher education unit. The model was seen as an essential piece of the teaching and learning process; therefore, it became an integral part of the Conceptual Framework for all teacher education programs. The faculty infused the language of assessment into daily practice; set professional development goals of creating new diagnostic, formative, and summative assessment methods and scoring tools; and shared these with their colleagues. Teacher education students are being required to use this model as they plan instruction in clinical settings.

A second area, the Nursing Program, implemented the model by developing two scoring tools to measure specific communication skills that nurses use with patients. In the past the department had taught the skills in the first year experience and assessed them traditionally. They discovered that the traditional assessment did not provide enough information about which skills each student had and had not mastered. They now formatively assess the skills using role-play methodology and a checklist. The faculty works with individual students to provide feedback on specific skills. Later in the program the students are summatively assessed using a rubric that clearly delineates levels of performance.

Progress of implementation of the model has been slow but steady. Some faculty have been more receptive to changing assessment practice, finding both the model and resources extremely helpful. As others see the benefits of assessing comprehensively and begin to feel the pressure from students to follow the tenets of comprehensive assessment, the task force expects that reluctant faculty will begin to embrace the model of comprehensive assessment.

Lessons Learned

Upon reflection and evaluation of the model-building process, the task force ascertained three important elements that should be considered whenever the university undertakes a campus-wide innovation.

* Be open to the benefit of the knowledge available in areas beyond the university. The assessment team expanded the literature search to include basic education and adult learning in venues such as health care to gain useful data and practical ideas.

* Be open about the amount of effort the change will consume. Comprehensive classroom assessment is time-consuming and challenging. Professors are discovering that there are many practical issues that surface when they move toward comprehensive assessment, not the least being finding the storage space for all the projects that can be submitted in a single day. Although not all hurdles can ever be predicted, if the task force had anticipated some of those specific obstacles, a section of the handbook could have been developed to provide suggestions regarding logistical issues. Currently members of the task force attempt to problem solve by providing suggestions during dissemination and support sessions. Although the growth process remains a struggle and the use of comprehensive assessment takes more time, professors are beginning to reap the tangible reward of increased student achievement.

* Do not avoid the "sticky" issues or at least have a plan to address them at a later date. The external reviewer on the project, Dr. Jay McTighe, challenged the task force about the absence of information on two issues, grading and review practices. Neither are popular topics on a university campus, and the task force was concerned about adding more controversy to the document. However, the task force did develop a plan to address the issues, including follow-up seminars and a campus-wide retreat. These two topics continue to generate lively debate and are not yet ready to be included as an addendum to the assessment model. However, campus members are uncovering their beliefs about these topics, a first clarifying step toward inclusion in the model.

New Directions

Faculty members at East Stroudsburg University are at various levels of implementation of the Comprehensive Classroom Assessment Model. As more faculty embrace and experiment with the model, a fifth phase of inquiry focused on validation has begun to emerge as the conversation about assessment filters into daily practice. The validation phase affords the potential for various inquiry projects and action research in the college classroom. Research paths include surveys of current practices, policies, and structures on campus; case studies; and comparative studies to determine the impact of assessment on student achievement. Responsive action in designing and/or modifying assessment methods and scoring tools can then be taken. The Comprehensive Classroom Assessment Model is providing a catalyst for scholarly activity and continuous improvement in teaching and learning, thus contributing to the university mission of academic excellence.

References

Angelo, T. & Cross, K. (1993). Classroom assessment techniques (,?nd ed.). San Francisco: Jossey-Bass Publishers.

Caine, R., & Caine, G. (1991). Making connections: Teaching and the human brain. Alexandria, VA: Association for Supervision and Curriculum Development.

Costa, A. & Kallick, B. (1995). Assessment in the learning organization.' Shifting the paradigm. Alexandria, VA: Association for Supervision and Curriculum Development.

Kane, M., Khattri, N., Reeve, A., & Adamson, R. (1997). Assessment of student performance: Studies of educational reform. OERI, United States Department of Education

Dunn, R. (1990). Rita Dunn Answers Questions on Learning Styles. Educational Leadership 48 (2), 15-19.

Dunn, R., Dunn, K., & Price, G. (1991). Productivity Environmental Preference Survey. Lawrence, KS: Price Systems Inc.

ESU Comprehensive Task Force (1999). Achieving academic excellence.' The ESU model for comprehensive classroom assessment. Unpublished manuscript, East Stroudsburg University of Pennsylvania.

Fogarty, R. (1997). Brain-compatible classrooms. Arlington Heights, IL: Skylight Training and Publishing, Inc.

Gardner, H. (1993). Multiple intelligences.' The theory in practice. NY: Basic Books.

Goleman, D. (1995). Emotional Intelligence. NY: Banton.

Guskey, T. (1988). Improving student learning in college classrooms. Springfield, IL: Charles C. Thomas Publisher.

Jensen, E. (1988). Teaching with the brain in mind. Alexandria, VA: ASCD.

Malinowski, P. A. (1995). Issues in assessment: Implementation. Finger Lakes Community College, NY: ERIC Document 393874.

Monson, T. D. (1997). Two-try testing. The Teaching Professor, 11, 10.

Moscovi, H. & Gilmer, P. (1996). Testing alternative assessment strategies. Journal of Science College Teaching, 15 (5), 319-323.

Reynolds, B. E. (1999). Designing assessment activities to encourage productive collaboration, cooperative learning and college teaching. New Forums.com. CA.

Sparapani, E. F. (1996). Portfolio assessment.' A way to authentically monitor progress and evaluate teacher preparation. Paper presented at the annual meeting of Association of Teacher Educators, St. Louis, February, 1996.

Silver, H., Strong, R. & Perini, M. (1997). Integrating learning style and multiple intelligences. Educational Leadership 55 (1), 22-27.

Smeaton, P., Mueller, S., Waters, F. (2000). The forgotten many: How media educators can address learning styles in the college classroom. Feedback (41) 3.

Sylwester, R. (1995). A celebration of neurons: An educator's guide to the human brain. Alexandria, VA: ASCD.

Wiggins, G. & McTighe, J. (1998). Understanding by design. Alexandria, VA: ASCD. Wolcott, S. K. (1999). Developing & assessing critical thinking and lifelong learning skills through student self-evaluation. Assessment Update 11 (4), 4-5. San Francisco:Jossey-Bass.

Suzanne S. Mueller, East Stroudsburg University
Faith H. Waters, East Stroudsburg University
Patricia S. Smeaton, East Stroudsburg University
Patricia Pinciotti, East Stroudsburg University

Mueller, Ed.D., is a Professor of Movement Studies and Exercise Science, and co-director of the Center for Teaching and Learning <[e-mail]>. Waters, Ed.D., is a Professor and department chair of Professional and Secondary Education, and co-director of the Center for Teaching and Learning <[e-mail]>. Smeaton, Ed.D., is an Assoc. Professor of Secondary Education, and co-director of Professional Development Schools-Secondary <[e-mail]>. Pinciotti, Ed.D., is a Professor, and chair of the department of Early Childhood and Elementary Education, and author of Art as a Way of Learning: Explorations for Teachers. E-mail: <[e-mail]>.