Professionalizing community-based research.

Abstract

Does community-based research emphasize professional-level data gathering that can positively impact community members? This article explores this question through case studies from environmental education and history, and discovers that a positive answer rests on the skill level of and access to suitable resources. We argue that in order to produce high quality data that is useful to the public faculty must place student skill development at the core of their practice. To do otherwise may unwittingly exacerbate the problematic trend of youth disengagement.

Introduction

As the service-learning movement in the United States modifies its trajectory in favor of civic education and other public outcomes--indeed, on politically re-engaging a generation of disaffected youth--it is no longer sufficient to point solely to increased service-learning activity or other community-based participation as a primary measure of success (Battistoni, 2002). Counting student contact hours in community settings won't suffice. Therefore, many practitioners are now focusing on integrating civic learning outcomes into their service-learning practices in order to better expose students to the mechanisms of local, state, and national policy making (Gottlieb & Robinson, 2002; Howard, 2001; Kahne & Westheimer, 2003). Additionally, in the past decade, many faculty have shifted their curricula to expose students to community-based research (CBR) practices (Polani & Cockburn, 2003). CBR activities can build and sustain community-university partnerships as well as build civic engagement in students. However, CBR experts readily acknowledge that CBR is not appropriate for all research projects, that methodology must be modified in order to account for students' and community members' varied research skills, and that ensuring student preparedness is a primary concern (Strand et al, 2003). Undergraduate students with CBR experience also readily note the importance of appropriate student skill development for project success (Willis et al, 2003).

Today, faculty interested in building civic commitment in students must not only intentionally and regularly discuss curricular connections to public policy, but also in the case of CBR, we must ensure that student findings are seriously considered by community leaders and other public policy makers. But what if student-generated findings aren't considered; or what if they are used and then later discovered to be inaccurate? We suspected that student-generated research data were not being sufficiently utilized to inform public policy. Our response has been to begin an investigation of what was happening, and what could be done in the academy to ensure that students' CBR has sustained public impact.

This paper will briefly discuss the Scott Peterson murder trial, a recent high-profile event in California in which students were utilized as community researchers with unfortunate results, and then explore two case studies from Portland State University (PSU) where faculty members focus specifically on generating high quality, usable data. The first study traces a PSU environmental educator's efforts to ensure high quality, student-generated biomonitoring data, and outlines a methodological strategy intended to increase the accuracy of these data. The second case study outlines curricular innovations that protect and honor a PSU oral historian's deep respect for narrators who are conscientious objectors from the Second World War.

Poor Data and the Scott Peterson Murder Trial

Consider the recent Scott Peterson murder trial. According to the Modesto Bee ("Trial Moved: Survey Flawed?" January 9, 2004), in fall 2003, 65 California State University at Stanislaus students in Professor Steven Schoenthaler's criminology class were given a service-learning survey assignment that, in the words from the course syllabus, would help students "gain hands-on experience working on a science project involving a current homicide case."

At issue was the fairness of the trial that, Scott Peterson would receive if the proceedings took place in a location too close in proximity to the victim's hometown. Schoenthaler's students were asked to randomly poll 1,175 prospective jurors by telephone from several of the largest counties in California in late November and early December, 2003 to test their knowledge and attitudes concerning the case. According to press accounts, over the past 14 years, Schoenthaler has overseen 31 change-of-venue surveys, using student surveyors for three of the highest-profile cases. This time there was a problem: on January 8, 2004 at least six students admitted making up every answer they submitted for the survey that was factored into Judge Al Girolami's decision to move the Peterson capital murder trial out of Modesto, CA ("Students Faked Peterson Trial Survey" Associated Press, January 13, 2004).

This case, among other things, points to the need for instructors to ensure that appropriate data quality control measures are in place for CBR courses. If data appear incomplete--or worse, inaccurate--then community partners and policy makers will appropriately be disinclined to use the results. They, as well as students, could also unfortunately view CBR as irrelevant in terms of its impact on public policy. The following case study outlines an instructor's attempt to address concerns about the accuracy of biomonitoring data.

Case One: Student-generated Biomonitoring Data

This case examines efforts to improve the accuracy of biomonioting data collected by college students. The class, "Water in the Environment" (WIE), is an introductory science course specifically designed for non-science majors. The goal of this course is to introduce these students to regional water quality issues and teach them how to conduct related environmental research. Service-learning partnerships provide a framework to accomplish this goal and can connect students to public forums where water quality is a concern. In WIE, aquatic insect biomonitoring is used as a strategy for engaging students in water quality research and awareness. Aquatic Insect biomonitoring is method for water quality analysis in which stream insects (and other macroinvertebrates) are used as indicators of pollution. Despite several years of student enthusiasm for collecting and studying aquatic insects within a CBR framework, generating accurate student-derived data that is both scientifically and politically useful for a community partner remained a challenge.

Service-Learning and Biomonitoring Data--Life on the Shelf

Aquatic insect biomonitoring is well suited for service-learning activities. Collecting insects is easy and inexpensive, insects are accurate barometers of water quality, and a day in the field collecting insects effectively instills in students a sense of environmental awareness and stewardship. Further, accurate aquatic insect data are theoretically more appropriate for a community partnership than are traditional chemical parameters such as dissolved oxygen, temperature, or pH. In just a couple of hours, using only nets and waders, CBR students can collect enough aquatic insect data to make a relatively accurate claim about water quality conditions. As a result, biomonitoring CBR projects have become widely popular with educators and community agencies that work with universities, schools, and volunteer organizations to assemble high quality aquatic insect data sets.

However, the value of student-collected biomonitoring data is directly tied to identification accuracy at the taxonomic level at which the study is conducted. As the taxonomic focus moves towards species, the accuracy increases, but correct identification becomes more difficult (PLACE FIGURE 1 HERE). High quality community-based biomonitoring requires that the instructor determine the appropriate balance between student ability to accurately identify insects and the desired level of accuracy. Regardless of taxonomic scope, inaccurate identification of insects invalidates the results of a study.

In an ideal service learning research project, community agencies would incorporate student-collected data into the public arena as a way to track stream water quality and support related policy decisions. The troubling reality is that the data rarely, if ever, leave the shelf. An unspoken rule in the professional scientific community is that the data are really collected for the sake of collection, not to inform research and policy. To clarify this issue, we queried professional scientists and discovered a general concern that, at the order level, the data are too broad to be useful for water quality monitoring; furthermore, there is considerable doubt about the accuracy of the students' identifications. If the scientific community does not take student work seriously, how can our students view their own work as relevant? If we are to build reciprocity into our CBR partnerships, we must think explicitly about the quality and ultimate use of our students' research. In the case of aquatic insect biomonitoring, the framework supporting this type of CBR had to be reexamined.

A New Approach to Education-based Biomonitoring

A close look at the partnerships that support education-based biomonitoring reveals two things. First, virtually all of the projects conducted in the name of CBR occur at the order taxonomic level. Second, the only field guides available to educators are focused at the order or genus level (skipping the family level). As we noted earlier, a biomonitoring study becomes more accurate as the difficulty of identification increases. For this reason, order level biomonitoring is most common in the educational setting.

However, our experience with biomonitoring in both K-12 and university level courses demonstrates that students are quite capable of identifying insects to the family level. In fact, if you were to go into the field with a group of students, collect insects, and ask the students to sort them based on how they look, they would naturally sort them to family level. Yet, the taxonomic resources available for educators are not focused at the family level. Conducting biomonitoring projects at the order or family level using genus-level guides does not make sense and represents a disconnect between the identification resources available to educators and the goal of generating high quality, student-derived data. If the scope of student work and related resources were focused on the family level, we could improve the overall accuracy of student identifications. (INSERT FIGURE 2 HERE). If we don't realign the taxonomic scope of community-based biomonitoring resource materials to family level, students will continue to misidentify insects at unacceptable rates or their work will be too taxonomically broad (order level) to be scientifically useful.

Recognizing these substantial problems with community-based biomonitoring activities, PSU supported the creation of aquatic insect identification field guide specifically designed for community-based biomonitoring activities called the Aquatic Macroinvertebrate Field Guide and Reference Manual Educator's Field Guide to Aquatic Insect Biomonitoring. The Guide contains more than 100 pictures of live insects from 20 different families. It is focused almost entirely at the family level and includes information and images that help students accurately identify aquatic insects for community-based biomonitoring activities. A preliminary investigation of two different courses and seventy-five students determined that the Guide significantly increases the accuracy of biomonitoring data. Over time and with the continuing input of community partners and environmental policy makers, this initiative will deepen reciprocity between educational and community entities, add value to student research, and develop a sense of civic efficacy and responsibility among students.

The next case discusses the importance of intentional curricular modifications in order to ensure ethical conduct and consistently high quality data gathering on the part of student oral history practitioners.

Case Two: Oral History as Civic Engagement: Professionalizing Collection

Community-based oral history projects can engage students in historical events and methodology, foster community-school partnerships, and enlarge the historical record. [1] However, if students lack the skills necessary to conduct ethical, technically proficient interviews, community-based projects can reduce civic engagement, undermine community-school partnerships, and only minimally enlarge the historical record with substandard materials. The benefits of student conducted oral history projects are tremendous but so are the potential pitfalls.

To avoid those pitfalls one must consciously build oral history technique into course content. This requires that faculty give at least equal time to oral history skills development as to history content. At Portland State University, most oral history courses utilize service-learning as a teaching pedagogy for this reason. Students conduct interviews to meet several pre-identified needs of the community partner. The courses specifically enhance students' skills for conducting interviews. Unlike with the criminology professor in the Peterson murder trial case who spent "over an hour" training his student surveyors ("Fallout Continues from Survey Scandal in Peterson Case" January 13, 2004), more than half the course time is spent in technical preparation for the interviews.

Interviewing Conscientious Objectors

Incorporating service-learning practices into oral history courses changes the focus of the teaching and learning endeavor. In 2003, fifteen college students in partnership with the U.S. Forest Service collected the histories of World War II conscientious objectors based at Civilian Public Service (CPS) Camp 56 in Waldport, Oregon. The objectives of this annual course included developing a greater appreciation for the history of pacifism and the techniques of oral history. Although historical content was important, its primary purpose was to serve the project; the main goal of the course was to produce proficient, ethical interviews. This reversed the goals of typical history courses in which students learn primarily from documented sources and lecture in order to prepare their own documents that do not have a public audience.

Practice Makes (Nearly) Perfect in this Reciprocal Relationship

For the success of the project, it was imperative that student interviewers understood how an oral history interview deviates from other kinds of interviews, that they practiced interviewing techniques, and that the students considered their obligations to the community partner and to the interviewees. Most students believe that an oral history interview is either an extension of ordinary conversation or a journalistic interview. Approaching an oral history interview as a conversation will nearly always result in a failed interview. Oral historians for the most part are silent, only breaking that silence to ask open-ended and informed questions that illicit information as a way to guide the interview. This is markedly different from conversations where many of us frequently interrupt and/or prepare our response while our partner is speaking. Students also often have television or radio news shows as their model for how interviews are conducted, one in which a public figure interviews another public figure often under adversarial conditions. These interviews are designed to be entertaining first and informative only secondarily. While they have their place in popular media, these interview practices could be considered unethical when employed by oral historians. A student must practice technique so that the instructor is confident that the student is sufficiently skilled to properly conduct a professional interview. Training includes conducting practice interviews, receiving critiques from faculty and other students, and role-playing poor interview techniques. Students draft and re-draft questions to learn what open-ended non-judgmental language looks like and conduct research related to the person they will interview. When faced with an actual interview that will be recorded and archived, students are prepared to perform professional-level work.

The Effort is Worth It

Although collecting oral histories with students is an intensive, high-stakes endeavor, these projects are worthwhile: they enlarge the historical record, strengthen community-university partnerships, and make students better historians. Histories that would not otherwise have been documented find their place in historical archives.

Due to its reputation for student-conducted professional quality work, numerous organizations search out Portland State University when considering oral history projects. Many of these partnerships have grown beyond a single course. The early success of the conscientious objector project developed into a collaborative book project based upon the interviews. The Forest Service partner hired students for further work and, in return, students got real-world work experience. This fosters the reciprocal relationship that service-learning and CBR practices strive to engender. PSU students learn civic skills when they make a public difference by influencing the historic record.

Such projects also transform student perceptions about history. Most history courses focus on information mastery but oral history courses require more. Students experience the links between a historian's work and the community and consider ethical issues that are part of the history discipline. Finally, the students who interviewed World War 11 conscientious objectors discovered alternative narratives of patriotism and forms of citizenship that will shape their interactions with local, regional, and national communities.

Time for training students in oral history technique can only be reserved by significantly adjusting the curriculum. Faculty must prioritize the skills-based training necessary for student success in civic learning projects if students are to produce professional quality work that is useful to the community partner and ultimately to the general public. Reserving time for training should not be viewed solely as a retreat from historical content. The time taken for training can enhance traditional curricula by allowing student to put academic content into a working context.

Conclusion

15 years ago, when the service-learning field was becoming established in mainstream K-12 and higher education settings, it was more likely to be considered sufficient to simply place students in communities, focus on affective learning outcomes, and provide a service that was hopefully beneficial to a community partner. Today, due to the maturation of service-learning and CBR practices, coupled with an alarming rate of youth disengagement from political processes, educators are called to intentionally consider civic or public dimensions of the curriculum. As the cases in this article demonstrate, civic learning cannot be left up to chance.

Educators must seriously consider the end uses of the data that their CBR students generate and ensure that they are near professional quality. Student-generated data must have consistent utility and withstand the scrutiny of public forums. If there is a long-standing perception (as in the biomonitoring case discussed) or a new perception (i.e., the Peterson trial example) that student-generated data cannot be trusted or have limited civic utility, then educators who have students gather such data may unwittingly reinforce negative beliefs about students' abilities to affect civic change. Unless educators are willing to seriously reconsider and modify their curricula to intentionally provide substantial amounts of in-class time to prepare students for community-based data-gathering and other techniques (for example, the PSU oral history case), then we believe it is best not to incorporate service- or civic-learning outcomes into the course. As the service- and civic-learning stakes get raised, such as with murder trials and public policies affecting communities, so too must educators' and students' attention be raised in equal measure to meet, and exceed, public expectations.

Endnotes

[1] The ways in which student-conducted oral history work enriches curriculum is well documented. For example, the Oral History Review dedicated a full volume to the teaching oral history in 1998. Less attention has been paid to the intersections of oral history work in the classroom and service-learning. An exception is A. Glen Crothers" article, "'Bringing History to Life': Oral History, Community Research, and Multiple Levels of Learning" The Journal of American History 88.4 (2002): 15 pars. 11 March 2004 http://www.historycooperative.org/ journals/jah/88.4/crothers.html. In his recently published book Better Together: Restoring the American Community (New York: Simon & Schuster, 2003), Robert Putnam identifies storytelling generally (a category under which oral history projects would fall) as one of the characteristics of community engagement.

References

Battistoni, R. M. (2002) Civic Engagement Across the Curriculum. Providence, RI. Campus Compact.

Edwards, P. (2005), Aquatic Macroinvertebrate Field Guide and Biomonitoring Reference Manual for the Willamette Valley. Center for Science Education, Portland State University.

"Fallout Continues From Survey Scandal in Peterson Case". (January 13, 2004) Modesto Bee, http://www.modbee.com

Gottlieb, K., Robinson, G., (2002), A Practical Guide for Integrating Civic Responsibility into the Curriculum. Washington D.C. Community College Press.

Howard, J. (2001), "Service-Learning Course Design Workbook". Michigan Journal of Community Service Learning. Ann Arbor, MI. OCSL Press.

Kahne, J., Westheimer, J. (2003), "Political Choices and Educational Goals". Campus Compact Reader. Providence, RI. Campus Compact.

Polanyi, M., Cockburn, L. (2003), "Opportunities and Pitfalls of Community-Based Research: A Case Study". Michigan Journal of Community Service Learning. Ann Arbor, MI. OCSL Press

Strand, K., Marullo, S., Cutforth, N., Stoecker, R., Donohue, P. (2003) Community-Based Research and Higher Education. San Francisco, CA. Jossey-Bass

"Students Faked Peterson Trial Survey". (January 13, 2004). Associated Press. http://www.cnn.com/2004/LAW/01/13/laci.peterson.survey.ap/

"Trial Moved: Survey Flawed?" (January 9, 2004) Modesto Bee, http://www.modbee.com

Willis, J., Peresie, J., Waldref, V., Stockmann, D., (2003), "The Undergraduate Perspective on Community-Based Research". Michigan Journal of Community Service Learning. Ann Arbor, M1. OCSL Press.

Kevin J. Kecskes, Portland State University

Katrine E. Barber, Portland State University

Patrick Edwards, Portland State University

Kecskes, Ph.D. candidate, is Director for Community-University Partnerships, Barber, Ph.D., is Assistant Professor of History, and Edwards, MS., is an Instructor in the Center for Science Education.