Assessing writing in a physical therapy unit.

Abstract

A case study detailing one student's involvement with supervisors, coworkers, and patients in a hospital physical therapy unit yields significant information about factors involved in the creation of documents to solve a personnel problem. Student documents reveal the student's efforts and encumbrances as he creates viable options to address the problem as an insider. Analysis of data is useful in assessing students' experiences and promoting teacher accountability.

Introduction

As a new faculty member incorporating service learning in all my courses and facing third year review, I am especially concerned with the experiences of my students. The kinds of comments that students record in course evaluations indicate that changes occurred for them. One student, who worked with an investigator in the identification unit of the local police department reports, "I saw the impact of their reports and began looking more carefully at my own writing." Such assessment is insufficient. Jo Allen cautions that a single measure "tends to compromise the strength and veracity of the measure" (96). She recommends a triangulation of methods (96). The nature of the class assignments pointed to the case study as a suitable method to use. Many students in my technical writing courses have jobs, and they undertake projects at their workplaces. They are immersed in the organizational culture as insiders. I chose to study their experiences and their learning outcomes situated within four key questions:

1. Do student activities reflect theoretical/philosophical goals that frame assignments?

2.What do students learn about writing in a professional setting and the influences on that writing?

3. How are their writing abilities strengthened?

4. How are students affected as they address a need or problem that has ramifications at many levels and that is intertwined with status, reputation, workload, evaluation, and compensation?

Reviewing the Philosophical Framework

Laura Julier notes that conversations regarding service learning reflect a wide range of theoretical assumptions and philosophical foundations (133). In shaping and evaluating ssignments I was influenced by theorists espousing democratic citizenship and moral and civic literacy. For Bruce Herzberg determining whether students become involved in understanding what needs to be done to effect social change is an important part of assessment (58-59). Herzberg refers to the goals of Campus Compact, whose concern is that service learning programs be designed to promote critical analysis of social problems (58). According to Herzberg, having students merely write personal responses to community service experiences "is not sufficient to raise critical or cultural consciousness" (59). Paul Heilker advocates a form of service learning in composition courses that moves away from the curriculum to help students "understand writing as social action" (74) Students actually construct writing products that help support an organization's philosophy and achieve its mission. Talking about this gained literacy, Peck, Flower, and Higgins assert, "Community literacy grounds social action in a culture of learning and problem solving where strategies of collaboration, planning, argument, and reflection are explicitly negotiated" (213).

Identifying the Learning Behaviors for Evaluation

Allen identifies specific categories and exemplary student behaviors (103-106). I have rearranged these and added my own criteria to assess student work within organizations.

* Analysis of the topic/issues. Clearly identifies the challenging issues/need to be solved within an organization

* Mission of organization. Demonstrates a clear understanding of the philosophy or mission that directs the activities of an organization/agency

* Interacting with organization representatives. Establishes a working relationship with employees and decision makers

* Audience analysis. Demonstrates appropriate consideration and accommodation for audience's background, needs, and stance on the topic

* Information gathering. Analyzes research, statistics, and findings related to the topic/issue

* Knowledge of techniques or strategies of persuasion. Demonstrates knowledge of numerous techniques and options for creating persuasive texts such as proposals, reports, memos

* Application of techniques. Demonstrates application of techniques to promote the persuasive nature of the text. Articulates welt-reasoned and well-crafted objections to contrary opinions of evidence

* Synthesis of points. Exhibits the ability to integrate the findings related to the issue from multiple sources

* Organization. Clearly demonstrates an understanding of the best organizational approach for presenting the persuasive argument for the audience and topic

* Conclusion. Moves the audience to the reasoned conclusion stated in the document.

The Case Study as an Effective Assessment Tool

I studied the work of Larry, a 21 year-old-senior who works forty to fifty hours a week as a technician in a physical therapy unit of a local Catholic hospital, a not-for-profit organization that serves a large Hispanic population. His five-week project, "Rehab Technician Utilization," addresses a chronic problem that threatens the viability of the department: patients are not receiving the therapy they need because technicians are not being used effectively. The problem causes increased tensions between the physicians and the unit directors, mistrust between technicians and physical therapists, the loss of jobs, and inattention to patients who require treatment to regain their health. Furthermore, not addressing the problem means possible financial instability for the unit and the hospital.

Collecting the Data

I collected Larry's preliminary proposal for the project, summaries of meetings with hospital personnel, a log of activities, progress reports, the documents created for the unit, the final report to the decision makers, and reflection memorandum in which he assessed his work within his physical therapy unit. I studied my notes regarding Larry's oral presentation of his project at the end of the semester and my interview with him at the physical therapy unit at the hospital after the semester ended.

Analyzing the Data: Student Behaviors

Identifying a community need/gathering information

In a report to the unit director, Larry calls attention to the problem and to the layers of effects resulting from that problem. Larry is familiar with the unit's documents, and he is cognizant of the attention to details that characterize the expectations of his audience, the director of the physical therapy unit. He describes the situation in regards to the time measurement tool, the paid hour stat:

   According to the Paid Hour stat statistical evaluation for the
   fiscal year 2002-03 and the ongoing patient missed issue, a rehab
   utilization problem exists. The beginning of the fiscal year was
   yielding anywhere from .90-1.04 paid hours per stat. This was too
   high. The minimal rate for a productive day should be at around
   .86. On paper, it was appearing as if there were just too many
   employees, technicians in particular, which was putting pressure
   on the [unit] as well as the hospital financially. On top of that,
   patients were not being seen. This, to the majority of the
   physicians, is quite troubling, as you well know.

Because the technicians, who assist the physical therapists, were not being used effectively, some patients missed their treatment. Also, the data indicated that too many technicians were employed. In his report to the unit director, Larry notes that news of the data (relayed above) upset the technicians:

   Attitudes and emotions began to flare up and soon, the number of
   technicians began to decline. A couple transferred out to other
   departments, some opted to take earlier than expected retirement,
   and a few took it to the extreme of being terminated due to lack of
   cooperation.

With fewer technicians to assist them, the physical therapists were forced to increase their already heavy patient workload, and again the situation posed the risk of patients missing their needed treatments. Having expressed to the unit supervisor his interest in conducting a study of the rehab technician utilization problem, Larry was invited to meet with the director and chief executive officer and physical therapists. According to Larry, the problem had been broached before in the Process Improvement (PI) committee, traditionally made up "only of physical therapists." In this meeting Larry experienced first hand the process professionals use to address a problem. In his reflective memo, Larry writes, "I was allowed to sit in on an administrative meeting with the CEO of Physical Therapy and Rehabilitation to get some insight on her views of the issue." During the meeting Larry was given the task of gathering information and developing a plan to address the problem. His insider status opened research opportunities for him. He was allowed access to records that technicians usually do not see. He reviewed therapists' daily planners, daily staffing sheets, employee work records, and departmental reports. Larry comments on the cooperation he encountered:

   The department strongly makes use of this program [Excel] to store
   80% of their records. One of the department's secretaries noted
   that Excel is the best program to use when saving valuable figures.
   Also she was quite an asset to me in providing me with paid hours
   per stat and patient missed files, plus showing me how to use the
   program.

In addition, Larry was able to gather information during interviews with physical therapists and rehab technicians.

Creating a proposal/presenting viable options

Because of the focus on the problem, the unit expanded the PI team to include supervisors, therapists, and technicians, a move that Larry reports, "gives everyone a chance to express their ideas on the issues as a few technicians feel overpowered and helpless." In his report to the unit's director, Larry advises, "The team will need to design a plan, do the improvement, check the results, and act to hold the gain and continue to improve the process." In the detailed recommendation Larry describes one of the options, the use of the technician daily log sheet for the PI team's plan. He notes,

   Due to its structure and design, it would be the most detailed
   document available for analysis and data collection purposes. The
   need poses the question, How do we impact change? How can the PI
   team make an impact on technician support in different areas? The
   facts show that it must first be determined what the technicians
   are doing on a daily basis. The log sheet illustrates where the
   technicians are at, who they are working with, what they are doing,
   and the time it takes them to complete their tasks. All these
   questions have to be answered to determine where the need lies.

Having thought out the challenges to the new procedure to be implemented, Larry outlines the path to help the procedure succeed, and he notes the broad effects of his documents. His recommendation to the readers points out the sensitivity of the political climate that constitutes part of the problem; he considers the emotional responses of the technicians and the therapists. In accommodating the needs of the readers, he becomes, as Melody Bowdon and J. Blake Scott term, "the audience's advocate" (29). His recommendation reveals a critical analysis of the complex situation that is necessary to effect change:

   With all this information the PI team will be able to determine
   which area is in need of technician assistance, which therapists are
   utilizing technicians properly, and which technicians are being
   adequately productive in assisting the therapists. Granted, all the
   technicians may need to be advised on how to fill out the document,
   which can be easily done by providing an in-service. And finally,
   when technicians are assured that this project is to benefit them as
   opposed to suppress them, they will be more cooperative in providing
   assistance. If they feel overworked, this project may be a stepping
   stone to providing a door for new job openings for rehab
   technicians.

Experiencing the demands of professional writing

Larry experienced first hand how professional writing is created. He engaged in the creation of literacy in this unit, and he experienced the factors that shape that literacy. Larry explains:

   Among other challenges was taking on the task of producing two
   documents, the Therapist Log Sheet and the Technician Log Sheet, for
   the PI team to use for its study. The challenge behind the task was
   that I was given a day and a half to design and present the document
   to the unit's director. After a couple of revised drafts, my
   documents were ready to be distributed to certain associates for the
   study.

Also, Larry became aware of the discursive practices used to solve problems:

   Regarding the options and recommendation for the resolution to this
   need, the director was well pleased with my research and
   presentation of options. Actually, during the PI meeting, I was able
   to voice an option, which came to be the Tech Log Sheet. The
   director read my report and agreed with my recommendation and is
   strongly leaning on the Tech Log Sheet to find a resolution to this
   need.

With his improved effectiveness in writing came empowerment and authority for Larry. His active role in helping the unit reach its objectives granted him visibility and enhanced his status. His descriptions of the economic and emotional impact of the problem on the technicians reveal an increased sense of professional involvement and personal responsibility and effort which constitute social action. This goal is evident in his concern for the patient who is caught in this personnel utilization problem:

   Without therapy, a patient's battle with disease and sickness will
   only be prolonged. A prolonged hospital stay is not what physicians,
   patients, or families members enjoy. The documents to improve
   efficacy among employees will improve the attention given to
   patients.

Coping with affective elements in problem solving

Along with the positive comes the negative. For Larry the tensions among the technicians--his peers--became more real for him, particularly in his interviews with them. He learned well how they felt--their fears, their suspicions. To be able to develop a solution, he analyzed and evaluated the cold facts recorded in the unit's data banks and looked out from the perspective of a problem solver. Larry knows that the document he created has already caused additional anxiety for some technicians who believe the detailed activity record makes them a target, monitors their behaviors throughout the work day, and makes them responsible for the quality of treatment given to the patients. However, he realizes that the activity record is necessary to secure concrete information to be used to meet the needs of patients. Larry learned in a personal way what Peck, Flower, and Higgins highlight: "The literacy of social and cultural critique openly addresses issues of power, defining social relationships in terms of economic and ideological struggle" (204)

Using the Information from the Assessment

I have been sending my students into the community armed with a focus on writing and researching abilities. The study of Larry's project made me aware of the enormity of the responsibility we assign to students in service learning with its emphasis on social action. The responsibility involves complex interactions, writing and speaking engagements, and emotional encumbrances that may affect the personal life of a student and his/her position. I felt a tension between my desire to help my students engage in professional writing that promotes changes in the community and my concern for their emotional and economic welfare. What is my responsibility in preparing my students to undertake these projects? What are the ethical implications of not addressing the challenges they face?

As a response I will invite students from previous classes to talk about the factors that shaped the documents they created and the emotional impact of their undertaking. Second, I will encourage students to make me aware, in their progress reports, of any emotional stresses. Finally, I will provide opportunities for students to discuss with the class any emotional and attitudinal tensions within their efforts. Other sources are useful in gathering information to assess the students' experiences. Zlotkowski points to student journals, focus groups, and presentations. Gelmon, Holland, Driscoll, Spring, and Kerrigan recommend using surveys, interviews, and analyses of syllabi. Bowdon and Scott provide ample guidelines for securing feedback from community supervisors and clients.

Conclusion

The future of the service learning projects in my courses depends on evaluations of my teaching by my director and faculty review groups. The qualitative data that answers the four key assessment questions can be included in yearly self-evaluations and portfolios to help them understand service learning and its impact on student learning. In short, Larry, the student, learned the mechanism of a professional discourse as the vehicle to influence the lives of his fellow employees and the patients served by the physical therapy unit. Being actively involved in assessment of student outcomes and in accountability will ensure the continuation of service learning endeavors to meet students' needs as future professionals and engaged citizens.

References

Allen, Jo. "The Impact of Student Learning Outcomes Assessment on Technical and Professional Communication Programs." Technical Communication Quarterly, 13. 1 (2004): 93-108.

Bowdon, Melody, and J. Blake Scott. Service-Learning in Technical and Professional Communication. New York: Longman, 2003.

Gelmon, Sherril B., B. Holland, A. Driscoll, A Spring, and S. Kerrigan. Assessing Service-Learning and Civic Engagement: Principles and Techniques. Providence, RI: Campus Compact, 2001.

Heilker, Paul. "Rhetoric Made Real: Civic Discourse and Writing beyond the Curriculum." In Writing the Community:Concepts and Models for Service-Learning. Eds. Linda Adler-Kassner, Robert Crooks, and Ann Watters. Washington, D.C.: American Association for Higher Education, 1997.

Herzberg, Bruce. "Community Service and Critical Teaching." In Writing the Community: Concepts and Models for Service-Learning. Eds. Linda Adler-Kassner, Robert Crooks, and Ann Watters. Washington, D.C.: American Association for Higher Education, 1997.

Julier, Laura. "Community Service Pedagogy." In A Guide to Composition Pedagogies. Eds. Gary Tate, Amy Rupiper, and Kurt Schick. New York: Oxford UP, 2001.

Peck, Wayne Campbell, Linda Flower, and Lorraine Higgins. "Community Literacy." College Composition and Communication, 46.2 (1995): 199-222.

Zlotkowski, Ed. Faculty Service Learning Workshop. Texas Campus Compact. Del Mar College. Corpus Christi, Texas. 2003.

Diana Cardenas, Texas A&M University-Corpus Christi

Diana Cardenas, Ph.D., is director of the technical and professional writing program.