Is service learning making the grade?

Abstract

Studies that have examined the effect of service learning on academic learning have produced equivocal findings, feeding the reluctance of some teachers to change their customary instructional methods. This article presents the methods and findings of a classroom study in motor development that tested the robustness of service learning. Specifically, did the use of service learning or traditional "hands-on" activities as the instructional method in a concurrent laboratory result in better grades for students in lecture?

**********

Instructional Methods Influence Student Learning

The purpose of learning is to use what is learned (Dewey, 1938). Traditional lecture-laboratory course design is built on this premise. For example, in the laboratory component students typically follow precise instructions to perform "hands-on" activities that reinforce content being concurrently taught in lecture. Student lab write-ups serve as the evaluation piece wherein students are expected to demonstrate their understanding of lecture content by linking it to practice. The degree of understanding, however, is not solely a function of the students. Student learning of course content is affected significantly by the instructional method that faculty choose to use with those students, in this case "hands-on" activities. In essence, student performance (e.g. letter grades, exam percentages, etc.) is not simply student performance. The end product is the result of students experiencing the course content within the instructional framework designated by the instructor, which begs the question--To what degree is the instructional framework chosen by the teacher influencing students' grades?

Teachers choose certain instructional frameworks (e.g. problem-based learning, "hands-on" lab activities, service learning) to deliberately target higher levels of learning, such as application, integration, or synthesis. Service learning, for example, is an instructional method that engages students in community-based experiences and then through structured reflection students relate their community service experiences to the academic content of a course. In numerous service-learning research studies (Balazadeh, 1996; Cohen & Kinsey, 1994, Eyler & Giles, 1999; Fenzel & Leafy, 1997; Foreman, 1996; Kendrick, 1996; Markus, Howard, & King, 1993; Miller, 1994; Oliver, 1997), students and faculty report (via attitudinal surveys or questionnaires) that service learning positively develops academic learning, such as ability of students to apply course content to new situations. Documentation of these attitudinal changes, however, offer little with regard to the extent that service learning improves the academic performance of students (Eyler, 2000), or the process by which cognitive change occurs.

Studies that have examined the effect of service learning on academic learning have produced equivocal findings. For example, some studies have shown that service learning affects measures of academic performance (i.e. students' grades) in a course (Balazadah, 1996; Berson & Younkin, 1998; Markus et al., 1993) whereas other studies also using course grades as measures have shown no such effects (Kendrick, 1996; Miller, 1994). Broad generalizations about the impact that service learning has on students' academic performance, therefore, have been difficult. Consequently, many teachers are reluctant to attempt a community-based approach to teaching and learning and continue with their customary instructional methods (e.g. "hands-on" activities in a lab section), possibly at the expense of students' academic performance. Furthermore, their reluctance may be exacerbated by potential deterrents identified by teachers who use service learning, such as time, funding, and logistics (Abes, Jackson, & Jones, 2002). For teachers to be more confident in choosing to use service learning as an instructional methodology, service learning research must take a much more systematic approach. For example, the systematic repetition of similar studies (i.e. replication) is essential to furthering the direction of this line of inquiry. The independent "replication of a significant difference in several experiments is more convincing than a significant difference in a single experiment" (Keppel, 1982, pg. 74). Ambiguous findings of previous service learning studies (e.g. see Balazadah, 1996; Berson & Younkin, 1998; Kendrick, 1996; Markus et al., 1993; Miller, 1994) are not surprising. These studies were far from similar. Replication identifies the strength of a phenomenon under similar conditions.

With hesitancy, I approached service learning in higher education by beginning to test its robustness or the strength of its repeatability. I replicated Markus et al.'s (1993) methodology as closely as possible but in a different discipline. My willingness to "buy-in" to a service learning framework was dependent on whether it would have a positive impact of student learning, and if that impact would be significantly more than the "hands-on" activities currently used in the lab component of a lecture-laboratory designed course. What follows is what I did and what I found.

Methodology Used in Two Similar Classroom Studies

At California State University, Northridge, motor development is an undergraduate Kinesiology course designed in a lecture/laboratory format. Students meet for lecture twice a week and for a required lab class once a week. The study by Markus et al. (1993) also had a similar course design in political science; however, the concurrent laboratory class was described as a discussion section.

In both studies, students were randomly assigned to either the experimental (service learning) or control (traditional) labs/discussions sections taught by graduate students. The political science discussion section was described as "section meetings devoted largely to discussions of the readings and lectures" (Markus et al., 1998, pg. 61). For the motor development labs, a graduate student implemented a set curriculum intended to target course objectives common to both the service-learning and traditional lab experiences. Students were expected to use information from the readings and lectures to complete assignments in small groups. Students in the service learning lab/discussion class provided 15-20 hours of service to various community groups with structured reflection activities. The other traditional laboratories/discussion sections used methodology customary for these course designs. For example, in the motor development class the students in the traditional lab class performed small group active learning activities that reinforced concurrent lecture content. Conversely, in small groups the motor development students were assigned to community sites that provided services to different age groups (preschoolers, first graders, pregnant teens, and older adults). These service-learning students designed and implemented developmentally appropriate movement activities and were required to complete weekly journal reflections in which they were asked to (a) objectively describe what happened at community site; (b) describe what they felt about the encounter; and (c) connect what they were learning concurrently in motor development lecture with their experience at the community site. In lecture, for instance, students may be learning about changes in the skeletal system across the lifespan. In the traditional lab, small groups of students would use pre-existing data to plot height and weight percentiles of individuals representing different points of the life span and then answer questions pertaining to these plots. Service-learning students would actually measure, record, and plot height and weight percentiles of people at their community site and then, as a structured reflection, answer the same assignment questions used in the traditional lab. The concurrent lecture component was independent of students' laboratory or discussion experience. All students received identical lectures, readings, and exams. Faculty who taught the lecture classes had no knowledge of students' lab/discussion experience. The effect of the two different pedagogies (service learning or traditional) being used in the lab/discussion component on students' academic performance in lecture was examined in both studies. The motor development study looked at students' performance (percentage score) on three lecture exams, while Markus et al. (1993) looked at letter grade performance (grade scale) of the political science students on two lecture exams.

Similar Findings Between the Two Classroom Studies

Students enrolled in a service-learning motor development laboratory scored an average of 4.8 percentage points higher across three lecture exams than students who received traditional in-house laboratory experiences (F (1, 52) = 4.28, p < .05). In essence, the percentage difference between groups could potentially advance a service-learning student approximately one-third of a letter grade past a student who participated in the traditional "hands-on" lab activities. These results corroborated Markus et al.'s (1993) findings, in which the service-learning students outperformed students who experienced the traditional discussion by one-third of a letter grade. Changes in the affective domain of students who served in the community were also measured via self-administered questionnaires (5-point Likert-scale items) before and after either the laboratory or discussion experience in the two respective studies. Although the same questionnaire was not used in both studies, both studies focused on measuring pre- to post-course attitudinal change in beliefs, values, and community awareness. The present study also revealed that students who participated in service learning reported a positive change (p < .05) on items pertaining to (a) understanding of course material; (b) community awareness; (c) self-awareness; and (d) commitment to civic responsibility. Again, these results parallel similar positive effects in the affective items measured in Markus et al.'s (1993) political science study. Political science students who participated in service learning demonstrated positive pre- to post-course attitudinal changes in personal values, orientations toward their community, and beliefs that their performance in the course approached their potential, whereas students in the traditional section showed no significant change.

Where Do We Go From Here

The lack of systematic attempts to detangle the effects of service-learning and subsequent reflection on academic learning outcomes is of concern to service-learning researchers (Eyler, 2000). The present replication study in a motor development class begins to address this concern. Attempts were made to replicate as closely as possible in a motor development class a previous service-learning research study in a political science class (Markus et al., 1993). Both studies found similar results. The combined findings of both studies demonstrate the positive impact service learning has on academic learning and affective development appears to be robust under similar conditions. More to the point is that the use of service learning as the instructional method in a concurrent laboratory/discussion resulted in better grades for students in lecture.

On the other hand, results from the present study also beg the question--do grades and exam percentages tell us anything about how students' learning has changed? Teachers who have experienced the powerful effects that service learning pedagogy has had on the quantity and quality of student engagement in their classroom intuitively expect far grander differences than 4.8 percentage points (Eyler, 2000). A frequent comment from teachers who use service learning is that it substantially improves their students' critical thinking skills. Likewise, service-learning faculty report that their strongest motivation to continue to use service learning are the positive outcomes on student learning (Abes at al., 2002). One would expect, therefore, a more compelling performance by service-learning students on exams. The use of traditional methods (e.g. exams) for assessing student learning, however, may not be measuring what is in fact changing. Eyler stated that "traditional grades measure 'replicative learning' or memory of content, [and] they are weak measures of the outcomes we expect to see enhanced by service-learning" (2000, pg. 13). Research (Ramsden, 1992) has also indicated that students may use different approaches to learning (i.e. surface versus deeper) in different situations (e.g. exam vs. small group discussion). "Even in classes where faculty have designed tasks to engage students more deeply in learning, a number of classroom conditions can discourage students from deeper learning. Testing methods, deadline pressures, and excessive workloads may prevent students from taking time to use deeper approaches to learning" (Cross & Steadman, 1996, pg. 125).

It appears that the pedagogical challenge for service-learning researchers and faculty is the disconnect between traditional measures of assessment (e.g. written exams) and the deeper learning that is taking place in service-learning experiences. My reluctance to use service learning has since vanished and been replaced by creating new ways to assess deeper learning, such as critical thinking, in my students. I have also witnessed the powerful effects that service learning has on the quantity and quality of student engagement in my classroom. Service learning is now the instructional method used in all my motor development laboratory classes.

References

Abes, E. A., Jackson, G., & Jones, S. R. (2002). Factors that motivate end deter faculty use of service-learning. Michigan Journal of Community Service Learning, 9 (1), 5-17.

Balazadeh, N. (1996). Service-learning end the sociological imagination: Approach and assessment. Paper presented at the National Historically Black Colleges and Universities Faculty Development Symposium, Memphis, 'IN.

Berson, L S. & Younkin, W. F. (1998). Doing well by doing good: A study of the effects of a service-learning experience on student success. Paper presented at the American Society of higher Education, Miami, FL.

Cohen, J. & Kiusey, D. F. (1994). Doing good and scholarship: A service-learning study. Journalism Educator, 48(4), 4-14.

Cross, K. P. & Steadman, M. H. (1996). Classroom research: Implementing the scholarship of teaching. San Francisco: Jossey-Bass Inc.

Dewey, J. (1938). Experience and education. West Lafayette, IN: Kappa Delta Pi.

Eyler, J. S. (Fall, 2000). What do we most need to know about impact of service learning on student learning? Michigan Journal of Community Service Learning, Special Issue, 11-17.

Eyler, J. S. & Giles, D. E. (2002). Beyond surveys: Using the problem solving interviews to assess the impact of service-learning on understanding end critical thinking. In A. Furco and S. H. Billig (Eds.) Service-learning: The essence of pedagogy. Greenwich, Connecticut: Information Age Publishing.

Fenzel, L. M. & Leaty, T. P. (1997). Evaluating outcomes of service-learning courses at a parochial college. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL.

Foremen, C. W. (1996). Service-learning in the small group communication class. Paper presented at the Annual Meeting of the Speech Communication Association, San Diego, CA.

Kendrick, J. R. (1996). Outcomes of service-learning in an introduction to sociology course. Michigan Journal of Community Service Learning, 2, 72-81.

Markus, G. B., Howard, J. P. F., & King, D. C. (1993). Integrating community service and classroom instruction enhances learning: Results from an experiment. Educational Evaluation and Policy Analysis, 15(4), 410-419.

Markus, G. B., Howard, J. P. F., & King, D. C. (1998). Integrating community service and classroom instruction enhances learning: Results from an experiment. Advances in Education Research, 3, 59-67.

Miller, J. (1994). Linking traditional and service-learning courses: Outcome evaluation utilizing two pedagogically distinct models. Michigan Journal of Community Service Learning, 1, 29-36.

Oliver, H. T. (1997). Taking action in rural Mississippi: Uniting academic studies and community service through Project D.R.E.A.M.S. Paper presented at the Annual International Conference of the International Partnership for Service-Learoing, Kingston, Jamaica.

Ramsden, P. (1992). Learning to teach in higher education. New York: Routledge.

Jennifer L. Romack, California State University, Northridge

Jennifer Romack, Associate Professor, teaches courses in motor behavior in the Department of Kinesiology and the Faculty Fellow in the Center for Community Service Learning.