Peer debriefing: who, what, when, why, how.

Abstract

The research methods literature recommends peer debriefing as a process to enhance the credibility of qualitative research. However, few details about how to plan, implement, and report this process are provided. This article delineates specific issues to consider: whom to select, what to do, when to meet, how to conduct, and how to report the process. Students may use these guidelines to assist in designing, executing and evaluating qualitative research studies. Incorporating these considerations may result in more effective implementation of peer debriefing methods and more credible reports of qualitative research.

**********

The purpose of this article is to discuss peer debriefing in qualitative research--why it is important, who should do it, how to conduct it, and how to report it. Research methods texts advocate peer debriefing as a process to enhance the credibility or validity of qualitative research (Creswell, 1998; Ely, Anzul, Friedman, Garner, & Steinmetz, 1991; Lincoln & Guba, 1985; Maxwell, 1996; Merriam, 1998). Despite widespread recommendation of this technique, the methodological literature provides few specific details about how to implement and evaluate this process. This article provides a set of decision-making considerations for researchers who are designing and conducting peer debriefing methods. While this paper is directed primarily to doctoral students conducting qualitative research studies, this article may also be of interest to faculty who teach research methods and to experienced researchers.

Why?

In peer debriefing, researchers meet with one or more impartial colleagues in order to critically review the implementation and evolution of their research methods. The role of the peer debriefer is to facilitate the researcher's consideration of methodological activities and provide feedback concerning the accuracy and completeness of the researcher's data collection and data analysis procedures.

Works by Lincoln and Guba (1985) and Spall (1998) are useful sources for novice researchers contemplating peer debriefing. Lincoln and Guba provided an influential discussion of criteria to evaluate the trustworthiness of qualitative research and proposed a comprehensive list of techniques, including peer debriefing, to address these standards. Spall reported research on operational models used for peer debriefing by dissertation students. Her findings indicated the importance of trust in choosing debriefers, the commitment of all parties to cultivate a high-quality product, and the developmental nature of the process. While Spall articulated research findings rather than methodological advice, her article nonetheless contains detailed information that stimulates additional ideas for implementation.

The ultimate purpose of peer debriefing, contend Lincoln and Guba (1985), is to enhance the credibility, or truth value, of a qualitative study, by providing "an external check on the inquiry process" (p. 301). Peer debriefring is particularly advisable because of a distinctive characteristic of qualitative research--the researcher-as-instrument. Individual researchers are the primary means for data collection and analysis. Each investigator brings a different combination of subjective knowledge, skills, and values to the research endeavor. As a result, researchers must come to know their unique characteristics as research tools and to understand how their subjectivity affects the conduct and results of their research. Researcher subjectivity is often equated with bias and seen as something to be avoided. In contrast, Peshkin (1988) asserts that subjectivity "can be seen as virtuous, for it is the basis of researchers' making a distinctive contribution, one that results from the unique configuration of their personal qualities joined to the data they have collected" (p. 18). Peer debriefers can be especially helpful to assist students to become aware of what they bring to the research, whether virtues or pitfalls. The next section discusses issues to consider in: a) selecting peer debriefers, b) scheduling the meetings, c) planning activities, d) conducting the sessions, and e) reporting the outcomes of the peer debriefing process.

Who?

What characteristics should student researchers consider in peer debriefers? Perhaps the most important qualification is methodological training and experience in qualitative research methods. Debriefers need to understand the epistemology of qualitative research and know the standards of quality appropriate to this paradigm. Some experience with conducting at least one qualitative research project, no matter its scope, enables debriefers to appreciate that things rarely go "by the book" when in the field. The debriefer will be prepared to assist the student to adapt to the unanticipated circumstances of the research context. The study conducted by Spall (1998) highlighted trust as essential to develop an effective peer debriefing relationship. Student researchers are expected to reveal their uncertainties and must feel confident that these disclosures will be met with discretion and diplomacy. The debriefing environment should encourage discussion and disagreement in which all parties feel that their views are respected. When students feel safe from judgments about their abilities, they feel ready to take the intellectual risks necessary in the inquiry process.

Student researchers should also consider where debriefers fall on the continuum of insider to outsider perspectives. An insider refers to someone who has prior understanding or experience with the topic or setting under study, while an outsider is unfamiliar with these. Debriefers may be insiders or outsiders to the discipline, topic or context for the research. Each viewpoint offers advantages and disadvantages. The insider as debriefer may comprehend what the student and the research participants are saying without elaborate explanations and may be able to connect data to conceptual ideas in the field. However, the insider may be subject to the same unquestioned or habitual ways of thinking as the student researcher. The outsider as debriefer may notice more readily the student's taken-for-granted beliefs and thus stimulate the student's self-examination. On the other hand, the outsider may not understand the area of research well enough to provide meaningful assistance to the student researcher. Maxwell (1996) recommends seeking "feedback from a variety of people, both those familiar with the phenomena or settings you're studying and those who are strangers to this situation. They will give you different sorts of comments, but both are valuable" (p. 94).

Dissertation students might also look for other characteristics of debriefers, such as methodological, philosophical, or theoretical perspectives that can contribute to the study. These deliberations should be designed to use the strengths of peer debriefers to compensate for any deficiencies of the student researcher. Novice researchers might ask themselves: How can one or more peer debriefers balance what I bring to this research? The choice of debriefers may vary from study to study depending on the strengths or weaknesses of the researcher in relation to the topic or methods.

Decisions about the number of debriefers are also important. An ongoing group with a mix of debriefers with diverse characteristics may be ideal, although not always possible. Alternatively, the student might seek input successively from two or three different peer debriefers, each offering a different perspective or expertise. Spall (1998) reported an arrangement in which students conducted concurrent projects and acted as debriefers for each other, both giving and receiving consultation. This approach was most satisfying for students since it resulted in a synergistic interplay of ideas and experiences. In some instances, only one debriefer may be feasible due to practical considerations such as availability, commitment, and dependability.

When?

When should peer debriefing start? Peshkin (1988) notes, "Subjectivity operates during the entire research process" (p. 17). Thus, some doctoral students may prefer to begin with the inception and planning of their study. At the design stage, there are numerous decisions to be made: developing research questions, defining criteria to select the research context or participants, and choosing data collection and data analysis techniques. Debriefers may also assist student researchers to clarify the need for the study and to articulate the initial assumptions they bring to the project. Other students may choose to begin the debriefing sessions as they implement the study. Since many validity threats occur during data collection and data analysis, the most pressing need for debriefing will ordinarily occur at this stage.

How often should debriefing meetings occur and how many should be held? One approach is to set up a regular schedule or standing meeting, continuing until the final report is produced. This alternative allows student researchers and debriefers to work through the inquiry process step-by-step as it emerges. Others may elect to adopt a task-oriented strategy, meeting only when the researcher identifies specific tasks, such as a coding check, to be accomplished. A single meeting with a debriefer, perhaps at the data analysis stage, provides less compelling evidence to argue for the validity of the researcher's methods.

What?

What are some specific activities that can be done in peer debriefing? All of the standards for good practice in qualitative research can be reviewed and evaluated. For example, during data collection, students and debriefers might undertake a methodological critique of an interview transcript. They could search for moments when the student researcher overlooked the participant's perspective, failed to elicit more concrete descriptions, or did not manage the interview well. They could then discuss how to improve these areas as the research proceeds. During data analysis, students typically request a check of their coding process. Debriefers can ascertain if initial categories stay close to the data and if summaries of data accurately reflect the informant's perspective. Data displays and other analytical techniques can be explored and tentative conclusions can be discussed and debated by debriefers. Writing up results in qualitative research allows room for literary and creative expression. Debriefers might, for instance, encourage students to experiment with alternative forms of representation.

Lincoln & Guba (1985) recommend that peer debriefers "provide a sympathetic listening point for personal catharsis" (p. 283) and "assist the inquirer to devise coping strategies" (p. 308). Extending this idea, Ely et al. (1991) suggest forming "peer support groups" that provide continuous affirmation, encouragement, and commitment to members' success. Indeed, peer support groups are valuable at every phase of the dissertation process, offering students an opportunity for companionship on their research journeys. However, while peer support is desirable, this purpose does not necessarily enhance the credibility of the research results. If a psychological emphasis enables the student researcher to be more open to evaluation, more clear-headed, or more aware of personal assumptions, then this aspect contributes to the critical review process. Students should reconsider their priorities if emotional support becomes the overriding purpose for the debriefing process.

How to Conduct?

How might peer debriefers provide feedback? How do debriefers help researchers to become aware of their biases? Students often ask debriefers to review specific aspects of their work output or to review methodological decisions. Debriefers should attend to how they offer feedback to the student researcher. Feedback is helpful when it is clear, concrete, and constructive. Debriefers should not forget to point out strengths as well as areas for improvement in the work. Feedback will be unwelcome from debriefers who are condescending or overly critical. Researchers may equally discount debriefers who beat around the bush or simply concur with every opinion. Those who seek out debriefers typically look forward to the opportunity to broaden their perspectives and consider ideas from alternative points of view.

Although debriefers express their own viewpoints, they must also elicit the student researcher's thought process. Debriefers can promote a reflective dialogue that challenges students to clarify their views and uncover the ways in which their beliefs and values play out in the research. Such thought-provoking questions or comments might be: What do you mean by ...? What is important (or not) about this to you? Let's brainstorm some alternatives. Why do you think this is true? What were you thinking at that point? What would happen if ...? What are the benefits or risks of that approach? How does that relate to ...? What areas do you feel uncertain about? Realizations as a result of this process are sometimes surprising, disconcerting, or even unwelcome. Lincoln and Guba (1985) advise that the debriefer should continue to play "the devil's advocate even when it becomes apparent that to do so produces pain for the inquirer" (p. 309).

How to Report?

Peer debriefing is a means to an end. Its purpose is to enhance the credibility of the research results. However, Maxwell (1996) explained, "The validity of your results is not guaranteed by following some prescribed procedure.... Validity threats are made implausible by evidence, not methods; methods are only a way of getting evidence that can help you rule out these threats" [emphasis in original] (p. 86). Thus, simply engaging in the debriefing process does not automatically result in a more valid study. Peer debriefing may be done well or poorly, effectively or ineffectively, depending on the particular research context. Dissertation students need to use the circumstances and outcomes of the debriefing process as evidence to argue for the validity of their results.

Student researchers must also report how their particular perspectives or subjective values shaped the process and outcomes of the study. In qualitative research, eliminating the researcher's influence is not possible or even desirable. Instead, the goal is to trace the researcher's effect on the study (Maxwell, 1996).

During meetings, the student researcher needs to take detailed notes. After meetings, the student should document any modifications to the inquiry. The final report about peer debriefing might address the following questions:

Who: Who were the peer debriefers? What kind of perspectives or expertise did they bring to the research? How did the characteristics of the debriefers balance the skills or values of the student researcher?

When: How many peer debriefing sessions were held? Over what time span? How long was each session? At what stage in the research process did the debriefings begin? What and How: What specific aspects of the research process (e.g., sampling) were discussed? What concrete products (e.g., analytical matrices) did the debriefers review? In what manner did debriefers dialogue with dissertation students about how their subjectivity affected their research?

Outcomes: What did the researcher change or confirm as a result of the debriefing process? Were there any emergent methodological decisions (e.g., when to stop collecting data) and how were these reached? What was the nature of the relationship between the debriefer and the student researcher? In what ways did debriefing activities enable students to become aware of and explain their effects on the research? In this section of the report, it is particularly effective to walk readers through an example or two of an important decision, change, or realization resulting from the debriefing sessions.

Conclusion

The preceding sections described specific criteria to consider when conducting peer debriefing in qualitative research. However, decisions about whom to select, when to schedule meetings, and what activities to review depend on specific research circumstances. Validity threats will vary with each research study because the researcher's knowledge, skills, and attitudes in relation to the topic will vary as well. Researchers need to be aware of and respond to each unique research context.

Thus, there are no right or wrong ways to conduct peer debriefing, but rather choices that are more or less effective to address the validity threats that exist under particular research conditions. The student researcher's methodological choices, whatever they may be, should be described and justified in sufficient detail so that readers of the research are well informed. This paper provides a framework for planning, executing, and reporting the method of peer debriefing in qualitative research. Careful consideration to the issues raised here may contribute to the persuasiveness of peer debriefing as a technique to enhance the validity of qualitative research. Additional scholarly work in this area might study the process of peer debriefing sessions, develop standards for good practice in peer debriefing, or evaluate the outcomes of specific peer debriefing activities.

References

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks: Sage.

Ely, M., Anzul, M., Friedman, T., Garner, D., & Steinmetz, A. M. (1991). Doing qualitative research: Circles within circles. London: Falmer Press.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park: Sage.

Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Thousand Oaks: Sage.

Merriam, S. B. (1998). Qualitative research and ease study applications in education. San Francisco: Jossey-Bass.

Peshkin, A. (1988). In search of subjectivity-one's own. Educational Researcher, 17(7), 17-22.

Spall, S. (1998). Peer debriefing in qualitative research: Emerging operational models. Qualitative Inquiry, 4(2), 280-292.

Marydee A. Spillett, University of New Orleans, LA

Spillett is Assistant Professor in the Department of Educational Leadership, Counseling, and Foundations, where she teaches courses in qualitative research methods.