Approaching assessment through Critical-Theory-Oriented Reflection Diaries (SRDs). (The scholarship of teaching and learning).

Abstract

This essay is based on a PowerPoint presentation made for the AAHE Assessment Conference 2000 in Charlotte, NC on June 17, 2000 that was scheduled as a Concurrent Interactive Session. It discusses the "Student Reflection Diary" (SRD), an assessment approach developed by the authors in the past five years. Before proceeding to read the entire essay, readers are advised to acquaint themselves with appendices A & B.

**********

In order to ground the discussion on assessment in scholarship, we review relevant literature (Shermis, 1992; Paul, 1993; wa Mwachofi et al 1995; Angelo, 1999; Mentowski & Associates, 2000). In his concluding remarks, Shermis (1992) observes: "The only hope for substantial curricular reform and for adoption of reflective inquiry in American schools is for teachers to redefine [italics added] the basic pedagogical concepts that they have inherited" (p. 73). And that teachers "must substitute for self-defeating, authoritarian, and unreflective assumptions and practices an harmonious theory of critical thought that will do what John Dewey declared in 1910 to be the most crucial need in our society: that students be taught to reflect upon what they learn in school" [italics added] (p. 73).

Whether or not Dewey's call is heeded depends on what metaphor defines the education philosophy. In their essay entitled "Reflections on the `Student-As-Customer' Metaphor," wa Mwachofi et al., (1995) observe that, "The idea that university students are `customers' who pay for their education and are therefore entitled to be satisfied with the `services' they receive has become widely accepted in American society" (p. 1). Questioning such acceptance, the authors argue that, "Learning is not shopping, and" that "the employment of such model obscures the pedagogical process" (p. 1). Paul (1993) points out that knowledge is not a "thing" that a teacher can pass on to a student. Rather, knowledge is that which a thinking mind creates as a result of engaging critical thinking. In our view, teaching and learning should be about the critique of life; students and teachers reflect on the choices humans make in everyday living, wa Mwachofi et al., (1995) observe that, "When we tell students that knowledge is a commodity, we tell them that learning requires no effort beyond paying for it" (p. 2). They argue that, "In representing students as customers, we teach them an ethic in which they are beholden only to themselves, not the larger community of which they are already members" (p. 2).

The Critical Theory Behind Student Reflection Diaries (SRDs)

Offering his assessment guidelines, Angelo (1999) observes that, "assessment practices should help students develop the skills, dispositions, and knowledge." He suggests that students should engage their academic work "actively -- intellectually and emotionally, ... set and maintain realistically high, personally meaningful expectations and goals" (italics added), and "[b]ecome explicitly aware of their values, beliefs, preconceptions, and prior learning, and be willing to unlearn when necessary" (http://www.aahe.org/Bulletin/angelomay99.htm).) Likewise, Mentkowski and Associates (2000), in their discussion about "transforming learning," point out that, "When students integrate performance and self-reflection they get a sense of `what I can do across settings, and how I can improve'" (p. 196). And that that as students "develop their self assessment capacities, they take charge of their own learning" (p. 196). In similar manner, our SRD approach to assessment encourages students to go beyond casual observations and seek explanations and reasons why phenomena are the way they are (see Figure 1 below). The graphic illustrates levels of thinking, "surface-structure" and "deep-structure" for "shallow" and "deep" thinking respectively. See issue's website <http://rapidintellect.com/AEQweb/spri02.htm>

In his sixth guideline, Angelo suggests that students should "Seek and find connections to real-world applications of what they're learning." As illustrated above, we seek explanations: we ask "WHY" questions about experiences we generally take for granted in life. For instance, regarding a "price drop" in an economics class, if students ask the WHY question, they can understand the "ripple effects" of a price drop in relations to various human choices. The SRD approach helps clarify the point that a "price drop" is a symptom whose causes (relevant connections) need to be articulated broadly and systematically. It also prompts students to seek interconnections like the implications to the supermarket prices, the incentive or disincentive to current and would be farmers, and the choices farmers will make given their shrinking incomes. If students merely memorize, for a test, the principle that prices of agricultural products drop when farmers experience a boom harvest, that is shallow (surface-structure) approach. Angelo's fifth guideline suggests that our assessment practices should help students to "work in ways that recognize (and stretch) their present learning styles or preferences and levels of development." If teachers can help students learn to frame the "why" question, students will learn to reckon with the interconnections and consequences of the phenomenon in question. The long-term goal is for student to internalize the habit of seeking deeper explanations to phenomena. For a communication or gender studies class, a teacher could ask: "Why do women aspire to have the `perfect' body?" "What constitutes `the perfect' body and who determines the criteria for perfection?" And for a math class, the question could be: "why is it that for any double-digit number made of numbers 1 to 9, if the two are subtracted from each other, we get an answer that is divisible by 9? For example, if 72, is transposed to 27, (72-27=45 and 45/9=5). Critical thinking encourages understanding not memorizing.

To assess is to estimate or interpret students' levels of reflection (process), not their recollection (content only). If the "student-as-customer" metaphor prevails, students treat content learning as an end in itself rather than the means to an end; they ask questions like, "Excuse me professor! Is this going to be in the test?" Content is often a quantifiable list. Our SRD approach treats process and content as two sides of the same coin because students engage in thinking (a process) about a fact (content). Facts (content" are the point on which reflection (process) begins. But as scholars speak about "content" and "process," language has created a false "dichotomy" that implies actual separation of the two. We clarify assessment in the next section, once students have written their Reflection Diary Entries (RDEs).

The SRD And Assessment

In their discussion of "how a person grows in relation to educational processes," Mentkowski and Associates (2000) observe that, "Critical thinking involves distinctive higher-order thinking skills, closely resembling the reasoning abilities traditionally identified as central to higher education" (italics in the original) (p. 180). The authors identify the "four long-standing aims of education" as follows: "(1) developing thinking skills in relation to the declarative knowledge structure of the disciplines, (2) developing the whole person's capacity to make meaning, (3) developing self-reflection and cultural identity, and (4) developing capacity to perform in work, family, and civic settings" (p. 180). The SRDs assessment approach helps students document their self-recognition episodes. Students are encouraged to base their reflections on learning encounters (lectures, group discussions, readings, and other relevant experiences.). The teachers' role is to latch on to those episodes and provide students with a systematic method for self-recognition.

By design, the Reflection Diary Guide (RD guide)provides a structure, a formal and systematic guideline for writing Reflection Diary Entries or (RDE). The objective is to get students to internalize the reflection process from the beginning of the semester. Soon, the eight elements postulated in the RD guide become second nature. This process is ongoing: further reflection results in additional self-recognition. As students get better at reflecting, they can tease out issues from the peers' RDEs for class discussions. Typically, most students master the process by the third week in the semester. Students share their RDEs with peers and give each other feedback. (To encourage clarity, the professor may select an example of a good RDE, make copies for the entire class, and discuss it step by step.). Below is a sample student RDE from "Issues In Media, Literature & Art," a class taught by the first author. The course is required for all students in the college of Arts and Sciences at Florida Gulf Coast University during the Fall 1997 to Spring 2001. It examines the role and impact of media, literature and art in society. The RDE is about a class session in which the critical meaning of "culture" was introduced through the concept of "culture as expectation." Reflecting on the session, the student wrote:

   When we talked about `culture as expectation,' I came to the realization
   that although I had thought, all along, that I new what culture is, I had
   really never examined the concept deeply. I am reminded of when I was six
   years old, my grandmother asked me to dress up so she can take me to a
   play. When I questioned her why it was necessary to dress up, she said that
   she wanted her senior friends to have a good impression of the family. Now,
   I see that she was following the "expectations of culture" that we
   discussed in class today.

The "Trigger Effect," (see Appendix A, Figure 1) helps to explain how students encounter learning. In the above entry, the student engages critical thinking: She identifies "culture as expectation" as the concept that clarified to her that although people often think they are acting independently, in actuality, they are following the "expectations" of culture. The four-step approach for assessing students RDEs helps: We ask: "Did the student 1) identify a concept, 2) pick a real life example, 3) interpret (by superimposing 1 and 2), and 4) draw an explicit conclusion about specific understanding accomplished?" In our assessment of the above RDE, indeed, the student: 1) identified the concept (culture as expectation), 2) offered a practical example from her own personal life (the story of the grandmother asking her to dress up), 3) interpreted the link between first and second steps by offering an explanation of the meaning of "culture as expectation." The explanation did not regurgitate what was explained in class; in her own terms, she explained her experiences, and 4) concluded that she now understood how her grandmother followed the expectations. This student points to what happened, and, more important, she explains why it happened the way it did. Paul (1993) suggests that any assessment should entail confirming that students elevate their thinking from previous levels is illustrated here. The RDE confirms improvement of the student's critical skills. Her argument of what is meant by "culture as expectation" proves that she understood logic. Her assessment of the role of media, literature and art in society proves that she understood the logic of the discipline. Her insight about her grandmother offers her class, an opportunity to discuss additional examples of "agents" of culture that create and maintain the "dress codes" (e.g., the fashion industry and television.).

Concluding Remarks

The SRD approach transforms students into partners in learning and assessment. When students' personal experiences are accepted as legitimate references for explaining phenomena, the intellectual enterprise is demystified and students are empowered. Students reconceptualize the role of education and the role of the teacher: They see learning as the process of creating new insights, as a result of relating concepts and ideas to their everyday living experiences. They see teaching as the noble profession; teachers are the `good guys' (partners) that help students sharpen their intellectual skills, not the `bad guys' that could jeopardize their GPA. They also appreciate the narrowness of placing the value of education on simply acquiring a job. Students learn to see connections among seemingly unrelated phenomena, because they understand complexity and logic; they realize that problems have multiple explanations and, consequently, require multiple solutions. The reflection process teaches students as much about themselves as the phenomena they reflect upon. It also prepares students to become intellectually active citizens, in accord with Dewey's proposition that (cited in Shermis, 1992), "students be taught to reflect upon what they learn in school." Open-minded teacher learn from students' perspectives. Finally, it is our hope that we have introduced more questions than answers to the discussion on assessment. We hope the conversations continues.

Appendix A

This is an excerpt of the handout entitled "Approaching Assessment Through Critical-Theory-Oriented Reflection Diaries (SRDs)."

What are Reflection Diaries?

SRDs are records of students' experiences in their various episodes of critical thinking. Thus, a SRD entry documents thoughts that provoke students' minds, as a result of encountering specific experiences in the course (e.g., lectures, readings, class discussions, and other relevant observations students experience both in and out of the classroom). See issue's website <http://rapidintellect.com/AEQweb/spri02.htm> Thus, SRDs represent thoughts that a student considers critical. These are thoughts that go beyond one's casual thinking, thoughts that one has not considered before, or thoughts that one has considered before, but without venturing deep enough.

Question: So, How Do Reflection Diaries Work?

Answer: They provide the student the opportunity to participate in the assessment of their thinking progress, as they track their knowledge-acquisition path. Thus, good entries:

1. provide evidence, rather than claims of where the student is, regarding their critical thinking experiences.

2. engage course subject matter deeply by identifying, for analysis, important issues that pertain to real life situations and at the same time link theories, concepts and models introduced in the class. Students are encouraged to recognize the legitimacy and importance of their personal experiences in their learning endeavors. In this sense, students refer to Self and see knowledge as "in here" rather than "out there."

3. place the student at the center of discussion. Students writes clearly and directly about their practical experiences in critical thinking terms as they enter the discussion and engage the issues as insiders and in their own terms.

4. link the issues explicitly (not implicitly) to real life situations.

5. attempt to reflect on the experience of learning rather than merely recall what was "learned". Please do not simply describe what you did in class. That amounts to giving me your class notes. Instead, write about the thoughts you are experiencing, as the result of what you learned. (See the Trigger Effect model).

6. attempt to, if possible, extend, re-conceptualize, question, affirm or negate the various theoretical positions proposed in class. Students' unique practical experiences place them in unique perspectives to do so.

7. avoid arguments for their own sake (arguments that do not connect to specific subject matters). Remember that your reflection is not about debate where you agree or disagree; it is about what and how you understand the nature and consequences of the specific social dynamics. Therefore, avoid moralizing: For example, instead of stating, "I can't believe how much time people spend on the Internet," you are more effective if you evaluate the specific causes or effects of spending time on the Internet. You could, for example, link your explanation to other aspects of society such as exposure to ads that urge people to consume mindlessly.

8. provide the teacher the arena for evaluating students' levels of learning (sophistication).

Appendix B

This documented is handed and discussed on the first day of class.

Our Teaching Philosophy

1. The university is a place where ideas, including our own (i.e. students' and professors'), are challenged.

2. Cooperative Learning: Both professors and students learn form one another. Through dialogues, they expand their horizons and gain valuable critical insights.

3. Theoretical knowledge is not useful on its own and should not be the end in itself. Rather, it should be connected to our everyday practical lives and challenges such that, theory informs and helps refine practice, and vice versa.

4. "Unexamined life is not worth living." In subjecting issues into rigorous scrutiny, students participate in the assessment and recording of their own intellectual developments. They compare their everyday living with the theories and concepts they encounter.

5. Our primary concern with learning is not about who is right or wrong, or finding the right or the wrong answer. Rather, it is about how we reach such conclusions ("Rightness" and "Wrongness"). It is about the "process," the philosophy and the logic behind conclusions. We seek to understand the fundamental origins of conclusions, beliefs, and culture, a function of beliefs, conclusions, or expectations.

6. Students' unique perspective will broaden our overall understanding of the relentlessly complicated world web. Therefore, you are invited and encouraged to join in as full and legitimate members of class.

References

Angelo, A. T. (May, 1999). Doing assessment as if learning matters most AAHE Bulletin (Available on the web at http://www.aahe.org/Bulletin/angelomay99.htm).

Mentkowski, M., and Associates. (2000). Learning that lasts: integrating learning, development, and performance in college and beyond. Jossey-Bass, San Francisco.

Paul, R. (1993). [video tape] How to teach students to assess their own work. The Tactics. Foundation for critical. Thinking.

Shermis, S. S. (1992). Critical thinking: helping students learn reflectively. Eric Clearinghouse on Reading and Communication Skills, Edinfo Press.

wa Mwachofi, N; Gilbert, P.; Strom, M.; Cohen, H. (May, 1995). Reflections on the "Student-as-customer" metaphor. Teaching Forum: The undergraduate Teaching Improvement Council. UW System. 16(1), 1-3.

Ngure wa Mwachofi, Ideas Unlimited International, FL Ari Katini Mwachofi, University of Arkansas

Ngure wa Mwachofi, a former professor, is President/Founder of Ideas Unlimited International (IUI) a consulting company. Ari K. Mwachofi is Assistant Professor of Agricultural Economics in the Department of Agricultural Economics. Her research interest revolves around agricultural economics policies and politics. She is also a poet, novelist and playwright. The authors owe Dr. Bill Cerbin of the University of Wisconsin System gratitude for introducing the first author to the use of student portfolios in Summer 1994 at the University of Wisconsin Teaching Fellows Workshop.