The endless quest for scholarly respectability in service-learning research.
Patti H. Clayton, Robert G. Bringle, & Julie A. Hatcher, Editors
Sterling, VA: Stylus, 2013
"Important objectives in the development of the research agenda include:
1. to agree on a working definition of service-learning;
2. to identify critical research questions needing further research;
3. to discuss methodological problems of doing service-learning research;
4. to develop strategies for encouraging and supporting research on critical questions;
5. to identify ways to expand the dissemination of existing and future research;
6. to encourage action research models through the collaboration of researchers with teachers, community representatives, and students."
(Giles, Honnet, & Migliore, 1991)
"Having a common set of questions is a necessary step for furthering research in service learning, but that is not enough. We also need to learn more about theory, design and gathering of data. We need consensus on the domain of service-learning, and precise, measureable constructs. As in much of contemporary social science research there is a paradigm debate in service-learning about ways of knowing. 'Objective' research methods are seen by many in the field as antithetical to the personal and experiential epistemology that service-learning represents, whereas many skeptics see the narrative, qualitative dimensions of service-learning research as anecdotal and unconvincing. We argue for a multi-method approach at this stage of service-learning research. Both quantitative and qualitative methods have much to offer. In what follows we identify the types of studies that might be designed to answer the ten key questions raised in this chapter." (Giles & Eyler, 1998, p. 70)
"It has been our goal that the articles in this special issue of the Michigan Journal of Community Service Learning will stimulate service-learning researchers--both those well-established and those just beginning--to embark upon answering some of the questions posed herein." (Howard, Gelmon, & Giles, 2000, p. 9)
We need to "insist on quality of research design, whether quantitative or qualitative. We need to move away from large-scale surveys with poorly specified measures of service-learning and service-learning outcomes" to "more focused study that gives us deeper understanding to direct our practice...We also need to work with research scholars from related fields and to bring some theoretical rigor to the design of our research programs" (Eyler, 2002, p. 12)
"The best chance for the research to have an impact on the practice of service-learning, on higher education, and on the disciplines and professions is for the research to be based on theory, to test theory, and to develop theory." (Bringle, 2003, p. 18)
During the 35 years or so since service-learning became a recognized field (Sigmon, 1979), there have been continuous efforts to improve the quality of research. The quotes that begin our reflections are drawn from just a few of the published attempts focusing specifically on improving the rigor of research conceptualization and design and on identifying critical questions to guide research. In the 1980s, as the emerging field of Experiential Education, as represented by the National Society for Internships and Experiential Education (NSIEE), matured, it became concerned with a research base for its arena of practice and its research committee began to work on research bibliographies and a research agenda. At the same time, Professor John Duley and his colleagues at Michigan State University were writing about how to assess the learning in service-learning and other forms of experiential learning, forming a foundation for future research agendas (Yelon & Duley, 1978). The work that the NSIEE research committee developed became the basis for the first research agenda and subsequent agendas as service-learning was emerging from a subfield of experiential education to a separate arena of practice. During this decade, the research committee sponsored research sessions at NSIEE conferences. Although these sessions grew in attendance, it became clear that a practitioner conference was not a venue likely to draw serious researchers. Several members of the NSIEE Research Committee then formed and led a Special Interest Group (SIG) in the American Educational Research Association (AERA) and used it as a then rare arena to present research about service-learning as one type of Field Education at the AERA annual meetings. Today this Special Interest Group has evolved into "Service-Learning and Experiential Education." The failure of these efforts to draw substantial interest from more than a handful of researchers was part of the motivation for the efforts to create the research-focused International Association for Research on Service-Learning and Community Engagement (IARSLCE) in 2001. A continuing theme of keynotes and publications from this Association has been the need to improve research quality.
The first service-learning research agenda was the product of a Wingspread Conference sponsored by the Johnson Foundation and organized by NSIEE. Forty eight participants, including researchers, educators, service-learning practitioners, students, representatives from government agencies, foundations, and national associations crafted the agenda to "identify critical research questions that need further research" (Giles, Honnet, & Migliore, 1991). This agenda included detailed questions and areas of research framing the broad thematic questions: (a) What is the effect of service-learning on the intellectual, moral, and citizenship development of participants? (b) What is the effect of service-learning on the advancement of social institutions and democracy? (Giles et al., 1991)
In November 1997, Campus Compact convened a conference comprised mainly of service-learning researchers held at the Education Commission of the States' office in Denver. While the conference themes stated two objectives relating to the pursuit of research, most objectives were much more advocacy-oriented, using verbs and phrases like advance, build, strengthen, further catapult, and inform funders (Howard, Gelmon, & Giles, 2000). One of the outcomes of this Denver meeting was to produce a special issue of the Michigan Journal of Community Service-Learning focusing on what was needed in service-learning research and to be underwritten by Campus Compact. The final product, Strategic Directions for Service-Learning Research (Howard et al., 2000), presented articles on a variety of topics and outcomes related to service-learning, each of which reviewed existing research and research needed. The topics ranged from areas of study such as students, community partnerships, faculty, diversity, disciplines, and institutional impacts as well as perspectives such as research in relation to practice, a university president's view, and methodological debates. What is distinctive about this volume is the interweaving of literature reviews, research questions, and strategic directions from key researchers and practitioners in the field.
A "Top Ten Unanswered Questions Agenda" was in draft form for the November 1997 Denver meeting and was published the next year (Giles & Eyler, 1998). Based on a review of the research to date, this agenda concluded that we knew quite a bit about student learning outcomes and community satisfaction with service-learning though less about the motivation for faculty involvement in service-learning, institutional policies and effects, types of community partnerships and their effects, and the impact on society both in general through the development of social capital as well on the long-term commitment to engaged citizenship among service-learning students. While this agenda was subtitled as a research agenda for the next five years, no subsequent agendas were later developed for the field as whole. In fact, one of us recently hypothesized that that once a field of research gains its own momentum, as is true with service-learning and campus-community engagement, there is much less need for collaboratively-constructed agendas or research-based agendas outside of specific topics that have become differentiated in the field. (Giles, 2010 (1)).
Because we have participated in many of these efforts to improve scholarship in the field as well as in the efforts to create a professional association devoted to service-learning research in the International Association for Research on Service-Learning and Community Engagement, we approached yet another work designed to review the literature and tell service-learning researchers how to proceed with some sense of deja vu. We wondered if the best use of the enormous effort entailed in engaging 30 service-learning scholars in such an enterprise was to continue to talk about doing better research rather than organizing to do better research. We wondered who the audience for another research agenda would be. We wondered if this work would take us someplace new. And we wondered if, after 30 years of exhortations to improve research quality, such modest advances in the research had been made that these arguments needed to be made again.
Volumes 2A and 2B on Research on Service Learning
This two-volume set is the second installment of the IUPUI Series on Service Learning Research. One previous book addressed International Service Learning and future books are planned on Student Civic Outcomes and Service Learning, Diversity and Persistence. We examined both volumes in this set: Volume 2A focuses on Research on Service Learning: Students and Faculty and Volume 2B on Research on Service Learning: Communities, Institutions and Partnerships. The overall goal of this effort is
to enhance the breadth, depth and quality of research conducted in the context of service-learning. Establishing a solid research agenda based on theory and building on prior work in order to improve the quality of subsequent research is the ultimate aim. (p. 4)
This effort began with a symposium of the editors and chapter authors held in Indianapolis in 2009 under the sponsorship of the Community Service Learning Research Collaborative sponsored through the Signature Center Initiative at IUPUI. The group collaborated to create a common frame of reference for their work using a model that views research as sitting at the intersection of practice, theory, design, and measurement. In the book, they distinguish research from evaluation, particularly in its emphasis on theory, but suggest that assessment might perform multiple functions in instruction, program design, and in research. They also created a common template for the development of their chapters to help provide coherence to what is a fragmented research field. Each chapter is organized around a seven-item template: (a) Introduction and scope, (b) Theoretical and conceptual framework, (c) Critical evaluation of past research, (d) Measurement approaches and instruments, (e) Implications for practice, (f) Future research agenda, (g) Summary and conclusions, and (h) Further reading. The four primary emphases of each chapter are (b) through (e) above, and form the core of each chapter's structure.
Defining Service-Learning and High Quality Research
Each of the two books in this set begins by repeating the two introductory chapters so that, although conceived as a single work, each may also stand alone. The first introductory chapter narrows the scope of service-learning to what Rice (Rice, Stacey, & Langer, 1997) aptly referred to as 'academic service-learning' to distinguish it from co-curricular and community-based efforts outside of academic courses. This places some boundaries on the project and leaves other forms of service-learning efforts for later volumes in the series. The authors adopt Bringle and Hatcher's (1996) well-known and well-respected definition of service-learning as
a course or competency-based, credit-bearing educational experience in which students (a) participate in mutually identified service activities that benefit the community, and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of personal values and civic responsibility, (p. 221)
While the initial chapter frames the book in the context of this definition of service-learning, the discussion of the nature of service-learning is somewhat cursory. One of the great weaknesses of the research in this field has been the vague specification of the experiences students actually have in their service-learning classes. It is still common to see articles submitted for review in which service-learning is the independent variable in a study which defines it simply as students participating in some sort of loosely-defined service to distinguish it from the comparison group. It is not surprising that so few of these studies show interesting or strong results (Eyler, Giles, Stenson, & Gray, 2001). This chapter also summarizes a variety of forms of service-learning, and notes the importance of using theory to conceptualize and then explore and document all the elements of the definition when doing research. Future researchers might benefit from a more thorough discussion of service-learning, including suggestions for operationalizing its characteristics for research, before moving on to outcomes and the roles of the various actors in the process. Service-learning is just one example of a pedagogy within higher education; it can be viewed as a form of experiential education (Eyler, 2009) or as a form of problem-based learning (Furco, 1996; Savery, 2006), and needs to be situated within those bodies of research. It cannot be viewed in isolation from the wider higher education experience or in isolation from the broader goals of higher education (Eyler, 2009; Pellegrino & Hilton, 2012). The assessment of the effective implementation of key elements of service-learning are critical for strengthening research in this field.
It would be helpful to explore the practices that embody the definition. How often and how does the service-learning class actually incorporate a process that genuinely creates a project that is mutually designed and beneficial for the students and community members? How often is the student's actual experience in the field monitored and held consistent with the academic purpose for the service activity? What little research in experiential education that has pursued this has not been heartening (Moore, 1981; Parrilla & Hesser, 1998). What faculty desire for their students and their actual service experiences may very well diverge. What does a placement that is mutually beneficial and in which students have valued experiences involving agency and responsibility look like, and most importantly, how can we measure these community experiences when doing research?
The same detailed analysis of options for creating effective reflection is needed. We know that service-learning simply defined has limited impact on students and that the quality of the experience in the field and the quality of faculty-led reflection are what makes a more powerful difference (Eyler & Giles, 1999; Vogelgesang & Astin, 2000). A volume that intends to lay out the description and measurement of key variables in service-learning research needs to begin with the deconstruction of the service-learning experience itself and provide descriptions of some of the varied ways these experiences are developed, students are prepared, the sites are managed and monitored, and the reflective processes are designed.
Because the books are organized around the actors involved in service-learning--students, faculty, communities, partnerships, and institutions, and in the case of students, outcomes of their service-learning--specifying the service-learning experiences being tested is left to the authors of the chapters primarily concerned with outcomes and the topic does not receive its own careful exploration. Since one of the clearest indicators from the literature on service-learning is that quality of the experience matters (Eyler & Giles, 1999; Vogelgesang & Astin, 2000), a thorough examination of the many forms of experience and academic uses of that experience would have strengthened the books.
The second chapter of the book is basically a quick course in research design 101, summarizing the elements of good research design and is notable for its thorough review of qualitative research design. In a field in which qualitative research may often be confused with anecdotal, a researcher tempted in this direction is given some clear pointers about how to proceed. The chapter also exhorts the investigator to ground the work in testing or generating theory (See Figure 1.2.1). This does raise the issue of audience, however. Since most academic scholars have studied and presumably demonstrated some competence in basic research design principles, this kind of information must have primary relevance to practitioners who may be collaborating on research programs or those whose training was not oriented to research. There is probably too much information for trained research scholars and too little for those without this training.
Since one of the primary goals of this book is to bring more theoretical rigor to this enterprise, and since the field is noted for its lack thereof, it would be helpful to spell out more clearly what theoretical rigor looks like. What does theory-testing look like in the context of service-learning research that is oriented to improving practice? A clear example to illustrate the model introduced in Figure 1.2.1 might help clarify to researchers what is meant by using theory in the research process. Show us how attention to theory in design can improve practice. The remainder of this volume also would benefit from more explicit examples of the uses of theory in designing research. Most chapters list a few theoretical perspectives that might be useful in the research, but few go into any detail about how these theories would be useful in creating logical models of relationships among variables, and there is little evidence of how theory could be tested or generated. If this is important to the field, we need clearer examples of what this would look like.
Challenges of Research on Student Outcomes
Volume 2A of this work focuses on students, addressing the traditional areas of cognitive and academic impact, personal and citizenship development, and a welcome less studied category of intercultural competence. Most service-learning research has focused on student outcomes, and in these chapters the authors have documented a steady increase in the quantity and occasionally the quality of this research.
In Chapter 2.1, Peggy Fitch, Pamela Steinke, and Tara Hudson examine the research and theoretical perspectives on student cognitive development outcomes. They assert that paying attention to cognitive student development is necessary to assure students' ability to deal with complex issues in the world after college, and they reference theories of development that have been employed in framing service-learning research such as those of Perry, King & Kitchener, Kuhn, and Belenky. They stress the importance of using theory to connect learning objectives with instructional design and subsequent measures of the impact of the instructional intervention. They also suggest that the scaffolding used in instruction for improved cognition can also be used for rubrics to assess student work and measure student progress, citing both the Problem Solving Analysis Protocol (P-SAP) (Steinke & Fitch, 2007) developed by two of the authors as well as the DEAL model for critical thinking (Ash, Clayton, & Moses, 2009) as exam pies. Their suggestions for future research include using rubrics for problem-solving that can be applied to student work across multiple campuses as well as the use of predictive modeling to gauge the relationships among variables. In addition to the developmental theorists, the authors might have incorporated some of the work related to developing expertise which contrasts with Piagetian developmental models like those cited here (Eyler, Root, & Giles, 1998).
In Chapter 2.2, Jessica Jameson, Patti Clayton, and Sarah Ash address academic learning, which they define broadly to include outcomes associated with learning specific subject matter as well as critical thinking outcomes (that also were addressed in chapter 2.1). They distinguish between basic acquisition of foundational knowledge in a discipline and higher order disciplinary thinking, as well as between the kinds of cognitive skills at the top of the Bloom taxonomy and critical thinking. They stress the importance of linking learning objectives within these various domains to the subject matter of the course, the process of instruction, and then measuring with rubrics applied to student demonstrations of their mastery. An example is given of how Bloom-based rubrics might be applied to a discipline-specific scenario to both guide students through and assess their mastery of higher-level disciplinary skills and understandings. They also discuss ways in which this instructional and assessment process allows exploration of links between learning outcomes such as personal and civic development as well as between teaching and learning.
In Chapter 2.3, Richard Battistoni addresses civic learning outcomes. Much of this chapter is devoted to the genuine confusions about the goals of citizenship education. As is apparent from even casual attention to the political scene in the U.S., what constitutes good citizenship is not at all a settled issue. In chart form, this chapter presents a variety of useful perspectives on civic learning from various social scientists, as well as views of citizenship outcomes from several disciplines. Past research in this area, reviewed in this chapter, has been rather thin, generally relying on student-reports and surveys. In discussing the need for future research, Battistoni describes a model of active citizenship used in Britain as a potential example of linking curriculum, assessment, and research on student outcomes, and mentions the need for longitudinal studies that track not only knowledge and skills that are learned but civic engagement over the course of the life cycle.
In Chapter 2.4, Jay Brandenberger investigates personal development outcomes of service-learning. He notes that there is general consensus that service-learning promotes personal growth, and summarizes the research on the development of agency and identity, perspective transformation and ways of knowing, moral development and spirituality, sociopolitical attitudes, citizenship and leadership, and career development. Very brief attention is given to theories of Dewey, Piaget, Erikson, and others. Unfortunately, most of this work has been superficial, measuring these outcomes with student self-reports on surveys, and the chapter doesn't give much attention to describing more powerful approaches that have been made or how more convincing measures might be developed. Future categories of research are described and the need to embed such research in theory is stated; this would have been a good place to show a clear example or two about how critical areas like identity development or efficacy might be studied, how they might be framed theoretically, and how the links to service-learning might be explored.
In Chapter 2.5, Darla Deardorfif and Kathleen Edwards explore the possibility of using intercultural competence (ICC) as an outcome of service-learning. ICC would seem like a good fit with service-learning given the focus of each on working with diverse people in diverse settings. The authors describe the elements of a consensus model of ICC, including attitudes, knowledge, skills, and interpersonal competencies, and suggest that many of the processes involved in reflective service-learning may facilitate acquiring these capacities. They also briefly describe Allport's contact theory and describe how previous research has attempted to assess these outcomes using this theory. For future research, they stress aligning measurement with the goals of ICC and the specific objectives of the course or experience being examined. Again, it would be helpful to go beyond a list of research questions to more specific examples of theory-based research design and measurement.
While most of service-learning research has focused on student outcomes like these, this research has been weakest in both conception and method, where it mattered the most in defining the service-learning experience, linking it logically to desired outcomes, and measuring those outcomes convincingly (Eyler, 2000; Eyler et al., 2001). These problems still persist in the confused conceptualization of citizenship development and the weak measurement of personal, civic, and intercultural competency described in these chapters. The field has not, for the most part, moved beyond student self-reports, and surveys and research designs have been weak. While these chapters do an admirable job of summarizing efforts to date, and noting questions to pursue in the future, we have a long way to go to design high quality research in these areas.
That said, there are some grounds for optimism, however, in conceptualizing and measuring academic outcomes including higher order objectives like critical thinking and problem-solving. In the chapters on cognitive development and academic learning, the authors document the modest but increased use of measures of what students can do rather than what they say they can do. By using examples from programs that integrate instruction, formative assessment and feedback, and measurement of outcomes, they show the promise of assessment as a tool for both design of effective instruction and authentic measurement of outcomes. While the authors noted that much of this research still uses weak measures, they show us a number of examples of more powerful performance measures. These measures have been used more to assess higher order thinking than disciplinary learning but also have the potential for documenting complex subject matter learning. They also suggest some approaches to future research embedded in the instructional design/assessment process. For example, a "common multi-disciplinary problem scenario might be posed to students, or that students might be asked to identify a relevant problem, but in either case they would then engage with the problem using pre-determined prompts designed to evoke and document the levels of thinking desired" (p. 98). A similar use of the DEAL model associated with rubrics built around a hierarchy of learning goals is offered as another example measuring competence. While the kinds of designs suggested here, that draw on measures using course assignments developed around a hierarchy of increasingly complex learning objectives, offer promise for both cost effective and more powerfully grounded research, this approach does have its limitations.
While it is important to document that students can demonstrate deep understanding of subject matter and higher levels of critical thinking, we are concerned not just whether students can perform, but whether they do perform. Alfred North Whitehead, who gave us the concept of 'inert knowledge' (1929), also observed in a text on mathematics (1911) that "Civilization advances by extending the number of important operations we can perform without thinking about them" (p. 46). This notion is as applicable to personal and civic issues our students will face as they are to mathematical problems. The real test of deeper understanding of subject matter and critical thinking capacity is that students automatically use their knowledge and capacities when confronted with new situations. The ultimate goal of education is to change the way students think and behave once they have left the classroom. Thus, just as assessment of learning must go beyond what students say they have learned to what they demonstrate they have learned, it also must go beyond demonstrations of learning when prompted to demonstrated dispositions to use such learning appropriately in new settings without cues and scaffolds to which they were previously exposed.
The authors of all of the chapters in this section make reference to the importance of longitudinal study, but measurement of higher order thinking and understanding is likely to be unfeasible in such a study. The major longitudinal work in higher education has been the studies by Astin and colleagues of college graduates over time using surveys; even with this approach, the response rate drastically diminished over time. The suggestions in this book for improved integration of design around clear learning objectives, continuous assessment, and use of those assessments to provide rich-textured data for research might have been enhanced by making transfer of learning the organizing principle.
Transfer of learning is discussed several times throughout these chapters but never fully integrated. It would be a good framework for managing authentic assessment across a curriculum. While measuring student performance is a step forward, many of the suggestions involve assessment using the framework used in instruction, e.g., students practice analysis of issues using the DEAL model and then are assessed using a series of prompts from the model. We discovered years ago that our apparently well-designed internship was not achieving the goals we planned for it despite students' ability to apply materials and models learned in each of our classes when asked to do so; when presented with a situation where use of the material was not cued, they fell back on knee jerk responses to problems. Using that information, we redesigned the internship to provide more practice with multiple situations and problems and found that the use of skills and knowledge in an uncued situation increased dramatically (Eyler, 1993).
The suggestions to integrate service-learning experiences across multiple courses and to use rubrics based on P-SAP, DEAL, or others that grow from articulation of a hierarchy of academic and personal objectives to assess multiple student products has tremendous potential to contribute genuinely useful information to the field. But it is also critical to put students in situations where they have the opportunity to use their knowledge and abilities without scaffolding with prompts. There is some tension between uncued assignments and the goals of an integrated curriculum where from beginning to capstone, expectations of performance are made clear to students. Do students use the analytical tools they learned through reflection using models described in these chapters when they have an assignment in another class for which the tools would be relevant? Creating such measures requires collaboration beyond the service-learning course to other coursework in the program, and to the wider student experience, again making it important to see service-learning as one aspect of a student's college education.
Research on Faculty and Service-Learning
We are not likely to see powerful research on student outcomes in service-learning without a better understanding of how to enhance implementation of effective practice by faculty. Although most studies of service-learning do not carefully measure students' actual experiences, the quality of service-learning has consistently been linked with desired outcomes (Eyler et al., 2001). The impact has been slight and the measures of quality weak, but when the goals are higher order thinking, the quality of what goes on in the classroom outweighs the simple addition of service to a class (Eyler & Giles, 1999). Section 3 of these volumes focuses on faculty as a key actor in service-learning. The section addresses research on faculty development, faculty motivation, and the faculty member as a learner.
In Chapter 3.1, Nancy Chism, Megan Palmer, and Mary Price describe several frameworks for viewing faculty development for service-learning, including examining these changes through the lens of individual learning and development theories, theories of organizational development, and theories of teaching and learning. They then synthesize these approaches drawing heavily on the work of Lewin (1947) and Schon (1983) into a five-step experiential learning process that begins with a felt need to change, formulates and experiments with the new practice, and then observes and reflects on the potential of this practice for future teaching. They also describe alternative approaches, including communities of practice and consultative models. They survey the literature and find "scant attention . . . paid to the impact of various approaches to faculty development for service learning, much less to comparative investigations of those approaches" (p. 197). They also found no research framed by theoretical rationales for the interventions. The authors create the rough outline of a logical model linking faculty adoption of service-learning to student outcomes, and suggest future research might strengthen both faculty development and instructional design by linking faculty and student outcomes systematically.
In Chapter 3.2, KerryAnn O'Meara explores the issue of faculty motivation to become interested in and adopt service-learning, and notes that this has been an active of area of study for the past 20 years. She organizes the results of some of this research into a handy chart (Table 3.2.1) that summarizes individual, institutional, and environmental factors influencing faculty motivation and links them to key research. She also creates a systems graphic synthesizing what she has gleaned about inputs, processes, and outputs of faculty motivation to participate in service-learning (Figure 3.2.1). While critiquing much of the past research for an overreliance on faculty self-report surveys, she also describes several qualitative studies that have helped delve below broader categories of gender or status to explore some of the nuances of faculty motivation. Service-learning has often been embraced more fully by marginalized faculty within the academy, e.g., nontenured faculty, faculty of color, and women. O'Meara also discusses the changes in faculty security and status and the fact that these changes pose both opportunities and challenges. There is little theory-driven research in this field and little theoretical direction offered here. Recognizing this, O'Meara suggests that one direction for future research is to better mobilize theory from the social sciences; this would have been a great place to do that. Other directions include examining relationships among inputs, outputs, and processes of faculty motivation; exploring the relationship of faculty structure and supports to service-learning engagement; looking at the impact of the intersection of individual, institutional, and environmental factors such as the accreditation process; and doing more ethnographic work that does not rely on faculty self-reports and memory.
In Chapter 3.3, Patti Clayton, George Hess, Audrey Jaeger, Jessica Jameson, and Lisa McQuire put the emphasis back on the reciprocal relationships in service-learning where all participants both serve and learn from each other. This conception of the service-learning process entails instructors developing an identity as co-learners rather than as knowledge dispensers. The authors discuss the ways in which faculty learning parallels student learning as they increase mastery of subject matter, pedagogy, and intellectual skills, and experience their own personal, social, and cognitive development. They explore the concepts developed by Argyris and Schon (1978) on learning through reflective practice, and describe the levels involved in triple loop learning. They examine studies involving community-engaged scholars' efforts, learning communities, and instructor autobiographies. Before listing possible research questions, they give some examples of how the same rubric-driven measurement suggested for measuring student outcomes can be used to assess and provide research data on faculty development.
These chapters provide a useful review of where we are in faculty development research in service-learning and suggest broad categories for future work. It is surprising that there is not more linkage with the voluminous literature on organizational change, adoption of innovation, the characteristics of early and late adopters, as well as designing learning organizations. Organizational change and adoption of collaborative learning processes are not unique to service-learning. Most of the theories that ground collaborative learning and implementation of organizational change began to be seriously developed and implemented over 60 years ago. Chapter 5.1 of the second volume does focus on organizational change, so perhaps this was simply a division of labor.
There is also an enormous literature on the study of the importance of collaborative work among teachers and on creating learning communities in the public schools that may provide helpful insights. Creating learning communities has been seen as a tool for education reform and teacher development, and it has been one means public schools use to break down the isolation common for teaching professionals by creating formal structures that enable teachers to observe and provide feedback for each other (Bambino, 2002; Brown, 1997; Vescio, Ross, & Adams, 2008). This work provides a solid theory and research base for exploring service-learning which, as the authors in this section note, is by its very nature a good fit for an institution seeking to create a collaborative work environment.
Focus on Community
Volume 2B of this work focuses on service-learning research related to community, institutions, and partnerships. In Chapter 4.1, Roger Reeb and Susan Folger present a Systems Theory approach to studying community outcomes of service-learning. As with most of the literature on service-learning research on community, the authors begin by decrying the lack of research in this area (Cruz & Giles, 2000; Stoecker, Tryon, & Hilgendorf, 2009). From this commonly held observation they move to the presentation of a theoretical framework as the solution to the dearth of community research without analyzing why this deficiency has continued to exist. The PESM theoretical model, published elsewhere by the authors, is a useful model for locating research on specific levels of analysis but it is not a theory of community systems per se. If we were to accept that the major reason why research is scarce in this area is lack of theory (we are unconvinced about that), it seems it would be for lack of theory of community systems and not a research systems theory. The most helpful section of the chapter is table 4.1.1 which presents a detailed analysis of community outcomes of service-learning studies by service site. This table is a must read for anyone contemplating research on community outcomes. It fills a much needed gap in the field of a mega literature review/annotated bibliography because the "At a Glance" review (Eyler et al., 2001) is out of date. The 19 studies are each analyzed for data source, outcome measures, and community outcomes. In the final section of the chapter there is an extended discussion of designs for community studies, mostly emphasizing quantitative approaches for theory testing. There also is discussion of random assignment to treatment, which of course has been an area that has been very rare and very difficult to do in student learning outcomes research. One example that triggered some ethical questions for us involved random assignment of homeless people to treatment and comparison groups. We wondered if many in the service-learning field would question or react negatively to this and ask if the goal was better research or better service and learning. As a result of this striking suggestion we would like to see more discussion of ethics and the protection of human subjects (Wendler, 2012).
Fortunately, Chapter 4.2 by Laura Littlepage and Beth Gazely contains a specific community-based model that is missing in the first chapter of this section. They focus on studying community organization capacity as it relates to both successful student placements as well as community agency. We agree with this emphasis as they argue that the 'burden' of service-learning is on community nonprofit agencies. The model is a balanced one and emphasizes both challenges and benefits. While this approach to community theory may be only one model that will be useful for research, any theory that helps academics understand the dynamics of communities is likely to increase the possibility of community-sensitive and rigorous research. This chapter presents a rich array of possible questions and useful research approaches. We would have appreciated including even more specific examples of this approach.
Taken together these two chapters provide an excellent review of the literature, some possible models and a number of approaches to research design. They also provide some thought-provoking and perhaps contested approaches to addressing the lack of community research in service-learning. In short, they are a valuable addition to the current body of literature in the field. Their limit, however, is that we are left puzzling as to why this lack continues and wishing for some assurance that these discussions will in fact change this longstanding deficit in service-learning research.
The Quest for and Questions of Institutionalization
Three chapters comprise Part 5, "Research on Institutions of Higher Education and Service Learning." The first, by Andrew Furco and Barbara Holland, argues that using theories of organizational change will improve research on the institutionalization of service-learning. Much of the chapter is a very basic and thorough review of the literature on organizational behavior and planned change theories. Indeed, it could be useful to practitioners trying to use the literature to inform institutionalization efforts at their institutions. The latter part of this chapter synthesizes organizational and change theories into a comprehensive framework for assessing institutionalization. In table 5.1.1, extant research is summarized and given a common structure but it is not reviewed in any great depth. As with other chapters in the book, there is the standard call for more theory in service-learning research. The concluding section on a future research agenda provides a very useful table (5.1.2) on the existing rubrics for assessing institutionalization. The call for change theory to be used for institutionalization research is a useful one in the context of the organizational literature reviewed here. One wishes, however, that more studies had been presented in detail and reviewed and critiqued. Perhaps the most lasting and thought-provoking part of this chapter is the observation in the beginning where the authors point out that much of the research on institutionalization has focused solely on the engagement efforts in isolation from their contribution and relationship to other organizational missions. Hence, they recommend that service-learning institutionalization research be more tightly connected to broader research in organizational change in higher education institutions.
Chapter 5.2, "The Engaged Department," by Kevin Kecskes, summarizes previous work on the engaged department, especially that done by Campus Compact. It also give us another case study in the literature on the transformation that has occurred at Portland State University (PSU) by including the very thorough and detailed six-page appendix of the rubric developed at PSU. Kecskes' previously developed cultural theory is linked to the literature on planned change, some of it duplicative of the theories in the previous section. While the research has affirmed the importance of the engaged department, it seems that many new and cross-departmental structures are emerging, so new emphasis on these is warranted in future research agendas focused on the engaged department or campus.
In Chapter 5.3, Lorilee Sandmann and William Plater present and expand their previous work on leadership in the institutionalization of engagement. This is a careful and thorough review of the dimensions of leadership anchored in the literature of higher education with a focus on service-learning. The chapter ends with a very comprehensive seven-page research agenda that covers the who, what how, under what conditions, and content of leadership. Particularly helpful is the call not only for the research but for the research to be able to hold institutional leaders accountable for their accomplishment of engagement goals on a campus.
As we write this in the fall of 2013, we are aware that a number of campuses are working on their classification or reclassification applications for the Carnegie Community-Engaged Institution designation. Arguably, the Carnegie Classification is the single most important force for institutionalization of service-learning and community engagement on campuses since the formation of campus Compact since 1985. What strikes us is the absence of a focus on what should have been another chapter on this important external factor that calls for focused research. While several studies on the Carnegie Classification are referenced in the chapters, and Sandmann and Plater have written elsewhere about this process, it seems that this press for institutionalization by a prestigious outside foundation ought to be the top priory for the research agenda on institutionalization and worthy of its own treatment.
Pursuing Partnerships as the Unit of Analysis
In Part 6, "Research on Partnerships and Service Learning," three integrated treatments of the topic form the call for research in this area. In 6.1, Robert Bringle and Patti Clayton present and analyze conceptual frameworks for partnerships. This critical, thorough, and detailed analysis uses the SOFAR multi-partner model of partnership structures and the TRES instrument for measuring dimensions of relationships in partnerships. The viewpoint and theoretical considerations of this chapter are based in interpersonal relationships and use exchange theory, equity theory, and identity. This lens provides a fairly comprehensive review of the developing body of studies in this area, although some more recent studies do not receive the critique and analysis that earlier ones did. The limits and the strength of this approach and chapter are its focus exclusively on partnerships as interpersonal interactions. The reader is left wondering about the organizational dimension lens and partnerships between organizations, where the individuals might be the agents of the partnership but only one part of a larger set of interactions.
Chapter 6.2, "Organizational Partnerships in Service Learning" by Emily Janke, is the perfect supplement for the lack of any organizational partnership focus in the previous chapter. In fact, the reader would do well to read the two chapters together. Using the field of interorganizational relationships, (IOR), Janke argues that the organizational perspective is needed to go beyond the interpersonal dimension to understand what makes partnerships successful (or not) over time as individual partners and actors change. This chapter provides a thorough and analytical view of literature in IOR and how it can be applied to service-learning organizational partnerships; this is critical since very little literature exists at the organizational level for studying service-learning partnerships, so Janke is pushing the field forward here in an important direction.
The previous two chapters are a paired set of views on community-university partnerships, but as Barbara Jacoby points out in Chapter 6.3, the importance of student partnerships in service-learning has largely been ignored in the research. The arguments here round out this set of three dimensions of partnerships. Jacoby does this by presenting the theories of Dewey and Freire as foundational in understanding partnerships and student learning. Creatively, the Community Campus Partnerships for Health's (CCPH) partnership model and principles are applied to student partnerships. As might be expected, the review of existing literature is very short; as Jacoby points out, only the work of Fisher and Wilson makes the case for research on student partnerships, but no empirical studies have tested their model. The chapter ends with a very thorough and comprehensive future research agenda covering process, roles, effects, program models, and conceptual bases for 'jump-starting' the research in this neglected area.
These three chapters act in concert to cover the levels and dimensions of partnerships in service-learning. All three agree that the recent increase in studies of partnerships in service-learning is attributable to the call to compensate for the lack of and difficulty in community research by using the partnership as the 'unit of analysis' (Cruz & Giles, 2000). These chapters illustrate that this development seems to be fruitful and growing. As noted earlier, the lack of community research in service-learning needs to be addressed. One way to address the gap noted above about the lack of community theory in Chapter 4 might be to begin to develop community theory using a 'field of partnerships' concept as the starting point, such as the community field theory of the 1970's began to do (Wilkinson, 1970).
The Search for a New Generation of Service-Learning Research
If we were teaching a graduate seminar in service-learning, this two-volume set would be our text. The authors have done a masterful job in summarizing the current findings in service-learning research and suggest questions that still need to be pursued. The work is comprehensive and many of the chapters include charts summarizing previous research, tentative models linking variables, or suggested rubrics for measurement. This work will be useful to graduate students casting about for a way to narrow their research focus for their dissertations and points them to useful theoretical perspectives to consider as they develop their proposals. As several authors noted, a powerful way to advance research might be to incorporate scholars from related disciplines; this work will provide a foundation of common knowledge for a team that might want to work across disciplines. While there have been countless research agendas and lists of future research questions published, they are not always easy to locate, particularly for researchers not already immersed in the field. These volumes succeed in pulling these questions together in one easily accessible work.
The summary of elements of research quality is also a useful touchstone for students as they analyze journal articles and potential research projects, and provides a helpful introduction to defining quality in qualitative research. Similarly, discussions of alternative ways to measure results beyond self-report measures may stimulate practitioners to strive for more powerful ways to assess successful instruction or successful implementation of programs with community partners.
This volume repeats the call we have heard for the last 25 or 30 years to anchor research in theory and to contribute to theory development and testing. Most of the chapters tiptoe up to the challenge by referencing potential theories that might inform design, but it would have been helpful to take a theoretical perspective and show us what a strong research design using that theory would look like. A single, well developed study that tests and refines use of theory for each of these topic areas would have been a powerful tool for those designing future research. It is certainly wise to note that theories from the disciplines should infuse service-learning research; it would be more useful to show us in some detail how that might be done. There is always the tension in a work such as this between comprehensively surveying the field and developing recommendations in a more focused way. Given the relative stagnation of the research in this field, choosing depth might have taken us farther toward productive future research.
A new wave of research, with the qualities called for in these volumes that advance theory and our understanding of how to translate theory to practice, is needed. Given the limited resources available to service-learning researchers, large experimental studies or longitudinal studies tracking students to see if the transformative character of service-learning is sustained through behavior of students as citizens, is not likely to be forthcoming. Many of the ideas here, including the suggestions of incorporating research into assessment and teaming with colleagues on other campuses to leverage what might be a small local study into a larger multi-campus effort, have potential for using the integrative potential of service-learning to generate convincing and useful results.
In a recent book review essay in this Journal about disciplining service-learning, Clayton, Edwards, & Brackmann (2013) reflect on the future of the field using the concept of the "next generation engagement" (NGE). While not specially related to research, NGE poses many of the same challenges to service-learning as we see in the discourses on research. We think it is fruitful to puzzle about our questions and concerns using the related concept of Next Generation Research (NGR) because both engagement and research concern the future of service-learning. Parallel to engagement, it can be argued that this book reflects first-generation service-learning research.
The first issue for NGR is the role of theory in research. It is one of the emphases given to all chapter authors in the two-volume set, is almost universally considered absent or poorly done in all areas of service-learning research, and explains why the mantra of the chapters is a plea for more and better theory. Since this is not unique to these volumes but is, as we argue above, a theme or characteristic of the enterprise for the past 30 years, we have to ask, Is it time to rethink the repeated calls for and lamentations about theory in research? No answer is readily apparent, but we think this conversation needs to be shifted. Perhaps this call for testing and building theory is misdirected. Service-learning is a field of practice, and theory is primarily useful in clarifying what we look at, how we measure it, and how we make logical connections. We are not generating theory but using it as a tool to help build more powerful research. Recognizing this, rather than the grand notion that we might generate theory, might we encourage less hand wringing about theory and more clear presentations of the theoretical constructs that underlie the processes we study. If the field of practice is not primarily about building or testing theory, then what does that mean for the accompanying research enterprise?
Another possible direction for this conversation is derived from the observation that each area (students, faculty, community, institutions, and partnerships) seems to use a different theoretical lens, raising the question that even if an area of research used well developed theory, how would these various theories fit together? In partnerships, for example, as we pointed out, there are two separately well-developed theoretical perspectives that are not integrated. Maybe the task is larger than developing theory for research--maybe we need a larger conversation about a full theoretical frame for the whole enterprise of service-learning and community engagement that links all stakeholders, processes, and actors in a full conceptual overview so that research on a specific area can be related to the whole. A daunting task for sure, but so was the first research agenda that only dared to hope to see the amount of research that has been undertaken, the number of practitioner researchers doing research, the establishment of a number of journals dedicated to service-learning research, and an international association focused on this work. Given this, we see that this larger conversation is possible as well as necessary. By taking either of these paths to forging NGR, we may be less likely to wake up on another groundhog day, doing more of the same research, listing more of the same research questions, and lamenting the lack of theoretical rigor in the field.
Argyris, C. S., & Schon, D. (1978). Organizational learning. Reading, MA: Addison-Wesley.
Ash, S. L., Clayton, P. H., & Moses, M. G. (2009). Learning through critical reflection: A tutorial for service-learning students. PHC Ventures.
Bambino, D. (March, 2002). Redesigning professional development: Critical friends. Educational Leadership, 59(6), 25-27.
Bringle, R. G. (2003). Enhancing theory-based research on service-learning. In S. Billig & J. Eyler (Eds.), Deconstructing service-learning: Research exploring context, participation, and Impacts (pp. 3-21). Charlotte, N.C.: Information Age Publishing.
Bringle, R. G., & Thatcher, J. A. (1996). Implementing service learning in higher education. The Journal of Higher Education, 67(2), 221-239.
Brown, A.L. (1997). Transforming schools into communities of thinking and learning about serious matters. American Psychologist, 52(4), 399-413.
Clayton, P. H., Edwards, K. E., & Brackmann, S. M. (2013). Disciplining service-learning and the next generation engaged campus. Michigan Journal of Community Service Learning, 19(2), 80-88.
Cruz, N. I. & D.E. Giles, Jr. (2000). Where's the community in service-learning research? Michigan Journal of Community Service Learning, Special Issue, 28-34.
Eyler, J. (1993). Comparing the impact of alternative internship experiences on student learning. Journal of Cooperative Education, 29(1), 41-53.
Eyler, J. (2000). Studying the impact of service-learning on students. Michigan Journal of Community Service Learning, Special Issue, 11-18.
Eyler, J. (2002). Stretching to meet the challenge: Improving the quality of research to improve the quality of service-learning. In S. Billig & A. Furco (Eds.), Service-learning through an interdisciplinary lens (pp. 3-13). Charlotte, NC: Information Age Publishing.
Eyler, J. (2009). The power of experiential education. Liberal Education, 95(4), 24-31.
Eyler, J. & Giles, D.E. Jr. (1999). Where's the learning in service-learning} San Francisco: Jossey-Bass.
Eyler, J., Giles, D.E. Jr., Stenson, C., & Gray, C. (2001). At a glance: What we know about the effects of servicelearning on college students, faculty, institutions and communities, 1993-2000. (3rd Edition). Retrieved from http://www.serviceleaming.org/filemanager/download/ aag.pdf
Eyler, J., Root, S., & Giles, D.E. Jr. (1998). Service-learning and the development of expert citizens. In R. Bringle & D. Duffey (Eds.), With service in mind: Concepts and models for service-learning in psychology. American Association for Higher Education. Washington, D.C.
Furco, A. (1996). Service-learning: A balanced approach to experiential education. Expanding boundaries: Serving and learning, 1, 1-6.
Giles, D. E. Jr. (2010). Journey to service-learning research: Agendas, accomplishments, & aspirations. In J. Keshen & B. Moely (Eds.), Research for What? (pp. 203-221). ICharlotte, NC: Information Age Press.
Giles, D. E., Jr., & Eyler, J. (1998). A service-learning research agenda for the next five years. In R. Rhoads & J. Howard (Eds.), Academic service-learning: A pedagogy of action and reflection (pp. 6S-12). San Francisco: Jossey-Bass.
Giles, D. E., Jr., Honnet, E.P., & Migliore, S. (Eds.). (1991). Setting the agenda for effective research in combining service and learning in the 1990s. Raleigh, North Carolina: National Society for Internships and Experiential education.
Howard, J., Gelmon, S., & Giles, D.E. Jr. (2000). From yesterday to tomorrow: Strategic directions for service-learning research. Michigan Journal of Community Service Learning (Special Issue), 5-10.
Lewin, K. (1947). Feedback problems of social diagnosis and action. Human Relations, 1(2), 147-153.
Moore, D. T. (1981). Discovering the pedagogy of experience. Harvard Educational Review, 51(2), 286-300.
Parrilla, P. F., & Hesser, G. W. (1998). Internships and the sociological perspective: Applying principles of experiential learning. Teaching Sociology, 26(4), 310-329.
Pellegrino, J. W., & Hilton, M. L. (Eds.). (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Washington, D.C.:National Academies Press.
Rice, D., Stacey, K & Langer, G. (1997). Academic service-learning faculty manual. Office of Academic Service Learning, Eastern Michigan University, Ypsilanti, MI.
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-based Learning, 7(1), 3.
Schon, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic books.
Sigmon, R. (1979). Service-learning: Three principles. Synergist, #(1), 9-11.
Steinke, P., & Fitch, P. (2007). Assessing service-learning. Research & Practice in Assessment, 2, 25-29.
Stoecker, R., Tryon, E. A., & Hilgendorf, A. (Eds.). (2009). The unheard voices: Community organizations and service learning. Philadelphia: Temple University Press.
Vescio, V, Ross, D. & Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student Learning. Teacher and Teaching Evaluation: An International Journal Of Research and Studies, 24(1), 80-91.
Vogelgesang, L. & Astin, A.W. (2000). Comparing the effects of community service and service-learning. Michigan Journal of Community Service Learning, 7, 25-34.
Wendler, R. (2012). Human subjects protection: A source for ethical service-learning practice. Michigan Journal of Community Service Learning, 12(2), 29-39.
Whitehead, A. N. (1911). An introduction to mathematics. New York: Holt & Company.
Whitehead, A. N. (1929). The aims of education and other essays. New York: The Macmillan Company.
Wilkinson, K. P. (1970). The community as a social field. Social Forces, 48(3), 311-322.
Dwight E. Giles, Jr.
University of Massachusetts, Boston
(1) The preceding paragraphs reviewing the calls and actual agendas for service-learning research are more thoroughly reviewed in this article.
Yelon, S. L., & Duley, J. S. (1978)). Efficient evaluation of individual performance in field placement (No. 14, Guides for the Improvement of Instruction in Higher Education). East Lansing, MI: Michigan State University Media Center.
DWIGHT E. GILES, JR. (email@example.com) is professor of Higher Education Administration and senior associate at the New England Resource Center for Higher Education (NERCHE) in the Graduate College of Education, University of Massachusetts, Boston. He has co-authored numerous books and articles on service-learning research and community engagement. His current research focuses on the emergence of service-learning in Hong Kong and the emergence of spirituality as a topic of research in Higher Education.
JANET EYLER (firstname.lastname@example.org) is professor emerita of the Practice of Education at Vanderbilt University. She is the co-author with Giles of Where's the Learning in Service-Learning? and has written many other books and articles on servicelearning and service-learning research. She currently conducts faculty workshops on effective design, evaluation, and use of reflection in service-learning.
|Printer friendly Cite/link Email Feedback|
|Author:||Giles, Dwight E., Jr.; Eyler, Janet|
|Publication:||Michigan Journal of Community Service Learning|
|Article Type:||Book review|
|Date:||Sep 22, 2013|
|Previous Article:||Do university communications about campus-community partnerships reflect core engagement principles?|
|Next Article:||Service-learning: bridging the past and the future for university foreign language programs.|