Performance assessment in CTE: focusing on the cognitive, psychomotor ... and affective domains.
Background Terminology and Context
Throughout this article we will be referring to curriculum alignment, direct/indirect/process/product assessments (Erickson & Wending, 1988), authentic assessments (Custer, Schell, McAlister, Scott, & Hoepfl, 2000), Assessment FOR Learning (Stiggins, 2004), and student performance in the three learning domains as identified by Bloom, Engel-hart, Furst, Hill, and Krathwohl, (1956). Rather than formally defining these terms, we'll present examples in a brief vignette from a course many of us took in high school: driver's education.
Driver's Education with Mr. Brady
Let's call our driver's ed (DE) instructor Mr. Brady. One of the activities in DE was to change an automobile tire according to the manufacturer's recommendation (learner objective). Mr. Brady communicated the objective to the class, used the appropriate instructional resources (the DE car, tools, parking lot), taught us to change a tire (instructional strategy), and then planned on assessing us using the manufacturer's instructions for changing the tire. Mr. Brady planned to assess and document our processes (procedural steps) along the way rather than only assessing the final work product. Up to this point, this activity had curriculum alignment. The objectives, resources, instructional strategies and planned assessments were connected.
But then Mr. Brady realized the dynamic of our class during this activity and how much instructional time we could waste if we really applied ourselves. Short on time, he was forced to modify his assessment. Rather than each of us actually changing a tire (psychomotor assessment) for our summative exam (which would have been a direct assessment of the objective), Mr. Brady revised the assessment where we explained how to change a tire on a written test (cognitive assessment of the psychomotor objective). So while Mr. Brady was not directly assessing the psychomotor objective, he was indirectly assessing our knowledge of the psychomotor process.
While indirect assessment may jeopardize curriculum alignment, Mr. Brady realized that indirectly assessing the psychomotor objective trumped no assessment. Furthermore, he required us to perform on the cognitive assessment by synthesizing our knowledge and practicing correct grammar (using short answer and essay questions) rather than using lower-level matching, true/false (T/F), and multiple choice (M/C) items. Finally, by observing us throughout the instructional process using Assessment FOR Learning-type practices (Stiggins, 2004), he was providing formative feedback throughout instruction while documenting our performance. Even if we didn't have our psychomotor test on actually changing the tire, he had documentation (formative assessment data) on what every student knew, could do and valued.
Investigating the Three Domains of Learning
Using the above tire-changing vignette, it is proposed that we (Mr. Brady's students) were performing in all three learning domains: cognitive (knowledge), psychomotor (fine/gross motor skills), and affective (appreciations, values) (Bloom et al, 1956; Krathwohl, Bloom & Masia, 1964). We also propose performance is not tied only to the psychomotor domain. Upon reflection, don't our students perform in all domains almost daily? Perhaps they do not perform in the psychomotor domain every day, but we believe students perform cognitive and affective tasks daily. We must seize the opportunity in documenting student performance in all three learning domains as evidence of their learning.
For example in the cognitive domain, students perform daily by using correct English and grammar when they create memos, resumes, technical documents, charts and solutions to problems, etc. In the psychomotor domain, they perform fine and gross motor skills by completing physical tasks like welding, constructing a structure, and manipulating a mouse/pointing device. In the affective domain, they perform employability skills daily as evidenced by being on time, getting along with others, and spending time on task. But back to Mr. Brady's tire-changing activity: While we performed in all three domains, were Mr. Brady's assessments authentic (Custer et al., 2000)? Did his assessment mirror the tasks we would be performing in the real world when we would graduate?
Authentic Student Performance in All Three Learning Domains
When determining the authenticity of our assessments, Custer, et al (2000) recommend this litmus test: Will our students be evaluated this way in industry? We further suggest investigating Custer's recommendation at the domain level. Cognitively, will students take recognition-response-based T/F, matching and M/C tests? Perhaps M/C items are relevant as many certification exams use scenario-based M/C items. But we believe that if a student can construct an answer (constructed-response items), they can recognize an answer (recognition-response items). Psychomotorly, will students be expected to perform the tasks of their craft with fine motor skills or gross motor skills? From an affective/employability perspective, are we preparing model employees? We may answer "yes" to these questions, but what do authentic assessments and the resulting documentation actually look like?
Regarding documentation, when authentically assessing student performance, we recommend two main types: student-completed and rater-completed (where the instructor, peer or guest rater observes and documents student performance). While not exclusive, the cognitive domain may generally include student-completed items while the psychomotor/affective domains generally include rater-completed items.
The Cognitive Domain
Many inservice and preservice teachers indicate using recognition-response items that are easy to grade (T/F, M/C, matching). While we must value teacher time (haven't we all created assessments with ease-of-grading in mind?), we also must ensure students are authentically performing in the cognitive domain that mirrors how they will perform in industry. When they gain employment, will they be describing and explaining procedures to a patient's family members? Will they be charting or journaling occurrences of a daily shift or customer repair order? Or will they be taking T/F, M/C, and matching tests? While recognition-response items may have their place for informal daily checks of understanding and reading quizzes, we propose student cognitive performance in higher-level assessments (chapter exams and above) should mirror how students will perform in their careers, using as many writing and public speaking activities as possible.
The Psychomotor Domain
Documenting student performance in the psychomotor domain (direct assessment) may shift from student-completed to rater-completed items. Whether the assessment is an actual project being constructed, a fictional job sheet, or a collaborative capstone project, Erickson and Wending (1988) recommend using Likert-type rating scales (1-5); graphic rating scales (pictures or text that describe each Likert rating digit); a binary checklist (either yes or no); or organized, well-written notes. Rubrics (grading grids) with multiple criteria may also be used. Regardless of which tools we use, all items' criteria should be objective for rater consistency.
The Affective Domain
We encourage using daily checklists to document student performance and behaviors in the affective domain, which also include employability skills. Using checklists, students earn points (we don't encourage giving or taking away student points) in areas such as arriving ready to learn, getting along well with others, following all safety procedures, staying on task, and not leaving early. In an over-simplified example, if students earn one point daily for each employability criteria listed above (five points total per day), over a typical 175-day school year, students can earn up to 875 points. If a student is late every day (unexcused tardies) but earns all other daily points, they would earn 700 daily points (80% average). This documentation of employability skills could justify a student's grade during parent-teacher conferences, but it must be communicated in the course syllabus. Yes, this requires dedicated documentation, but do we and our advisory committees believe employability skills are important? Absolutely. Moreover, as Custer and Claiborne (1992, 1995) discovered, employers value students' employability skills even more than we as educators think they do. So isn't it equally important for us to assess and document these required affective and employability skills?
However, the affective domain can be difficult to measure because we cannot see inside our students. Sometimes we only see what the students want us to see, which brings us to Eddie Haskell and The Eddie Haskell Syndrome. Eddie was the tormentor of Theodore "Beaver" Cleaver and a friend of Wally (Beaver's older brother) in the television show "Leave It to Beaver." Eddie always seemed to harass Beaver and was considered by many to be the "dark angel" of Wally's conscience. However, when Ward or June Cleaver (Beaver's and Wally's parents) were around, Eddie was a model citizen. Comments like, "My, what a lovely dress, Mrs. Cleaver!" would transition to "Hey Wally, where'd your old lady find that ugly dress?" when Mrs. Cleaver would leave the room. So cognitively, Eddie would tell adults what he thought they wanted to hear. But affectively, his true beliefs and values came out when he thought the adults weren't present; hence the term The Eddie Haskell Syndrome (EHS).
While it's difficult to accurately cite the origin of EHS, we've used it with our Missouri preservice and inservice teachers since 1995. Students like Eddie may cognitively perform by writing essays on topics such as "Why safety and personal protective equipment (PPE) are important." Using only the cognitive domain, we may measure and evaluate Eddie's knowledge and determine he values safety because he wrote it correctly in an essay. However, if we could observe unobtrusively, we may see Eddie chiding other students for wearing their PPE while not wearing his own. So what would that tell us about his true appreciations fir safety and PPE? Sure, Eddie may have scored well on the cognitive assessment he knows the importance), but does he truly appreciate the importance, based on his actions when he thinks we're not observing him?
Our challenge is this: If we ask our students affeetive questions on a cognitive (written) assessment (thus indirectly assessing the affective domain), they may tell us what they think we want to hear or properly recite our expectations. But how do they act when they think we're not observing them?
Student performance assessment can occur within the entire instructional process and should not be reserved for only the cognitive domain or summative assessments, nor occur only on test days. So to conclude, here are questions and challenges that hopefully summarize our main points.
1. How can we create (or revise as needed) our learner objectives to include all three learning domains? When hired, our students may be evaluated daily in all three domains, so shouldn't we provide the most thorough preparation possible?
2. How do we ensure curriculum alignment between our objectives, instructional strategies, resources and assessments, as all elements equally contribute to student learning?
3. Flow can we adapt (or increase.) Assessment for Learning strategies that document student performance in all three domains, not waiting until test day or only using assessment of student learning?
4. How can we ensure our students will authentically perform in the cognitive domain as they will in their careers? When hired, will they be recognizing answers from a list of potential answers, or will they be constructing responses via written and spoken words?
5. In the affective and psychomotor domains, how can we improve or develop new tools (binary checklists, rating scales, graphic rating scales, rubrics; that document student performance throughout the instructional day? Furthermore, how can we incorporate process assessments where formative feedback is needed, and product assessment where there could be multiple safe and equally-efficient correct solutions?
Finally, we hope you were asking yourself' while you were reading this if all three domains actually work together when assessing C'I'E students. We believe they do. For example, when a student is performing in the psychomotor domain, we believe the student is also performing in the cognitive domain (sequencing steps, evaluating the situation) and in the affective domain (appreciating a job well done, quality control, safety).
As Dabney Doty, former instructor at the University of Central Missouri accurately stated, "There is more headwork than handwork in shopwork." So we believe the key for CTE instructors is authentically assessing and documenting students' performances in all three learning domains and providing appropriate feedback to increase their learning.
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H. & Krathwohl, D. R. (1956). Taxonomy of Educational Objectives; The Classification of Educational Goals. Handbook I: Cognitive Domain. New York, NY: David McKay Company, Inc.
Custer, R.L (Ed), Schell, J.W., McAlister, B., Scott, J. & Hoepfl, M. (2000). "Using Authentic Assessment in Vocational Education." information Series No. 381. ERIC Clearing House on Adult, Career, and Vocational Education. Columbus, OH: The Ohio State University. Obtained from www.eric.ed.gov
Custer, R.L. & Claiborne, D.M. (1992). Critical skiffs cluster for vocational education. Journal of Vocational Education Research, 17(4), 15-40.
Custer, R.L., & Claiborne, D.M. (1995). "Critical Skills Cluster for Vocational Education: The Employers' Perspective: A Replication Study. Journal of Vocational Education Research, 20(1), 7-33.
Erickson, R.C., and Wentling, T.L. (1988). Measuring Sudent Growth: Techniques and Procedures for Occupational Education (Revised edition). Urbana, IL; Griffon Press.
Krathwohl, D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of Educational Objectives; The Classification of Educational Goals. Handbook 11: The Affective Domain. New York, NY: David McKay Company, Inc.
Stiggins, R. (2004). "New Assessment Beliefs for a New School Mission." Phi Delta Kappan, 86(1), 22-27.
Dr. Bart Washer will be presenting this 1-hour Idea Lab, "Performance Assessment in CTE: Focusing on the Cognitive, Psychomotor and Affective Domains," at CareerTech Vision 2012. For more programming information, visit www.careertechvision.com.
Bart Washer, Ph.D., is an associate professor and CTE graduate program coordinator in the Dept. of Career and Technology Education at the University of Central Missouri. He may be reached at firstname.lastname@example.org.
Lori Cochran is an assistant professor and coordinator of the Bachelor of Science degree in occupational education in the Dept. of Career and Technology Education at the University of Central Missouri. She may be reached at email@example.com.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Assessment; career and technical education|
|Author:||Washer, Bart; Cochran, Lori|
|Date:||Sep 1, 2012|
|Previous Article:||Advocacy strategies.|
|Next Article:||The economic vitality formula of success.|