Informing governance through evidence-based research on REBs: challenges and opportunities.
"urgent need for well-grounded research on the tensions between having standards of performance, monitoring, accreditation and processes that are sensitive to the needs, concerns and rights of research subjects and that stimulate and facilitate research. ... For too long ..., these fundamental questions have been debated in an a priori manner or simply by resort to anecdotal evidence." (3)
Scholars question whether REBs create bureaucratic impediments to the conduct of high quality and innovative research, provide researchers with value--added service and advice that might enhance the protection of participants or ensure that participants are better informed of the risks, harms, and benefits for themselves, for others and for science. (4) These sorts of questions are the kind that could and should be addressed through empirical work.
At a July 2008 retreat of Canadian Network for the Governance of Ethical Health Research Involving Humans, we sought to refine the empirical research agenda identified by Sieber (5) and others. (6) In our discussions we identified many ways in which scholars have turned their attention to the role of REBs within the research enterprise and ways in which REBs themselves are engaging in quality improvement assessments as well as action research (7) on their practices and impacts on the researchers and organizations that they serve. In this paper, we consider some of the challenges to studying REBs and address briefly five areas of interest to the Network members. These areas of interest are not intended to provide a comprehensive overview of the issues affecting the conduct of research on REBs and their processes. (8) By considering these issues, we hope to add some insights on the challenges of conducting research on REBs and urge others to empirically test our assumptions and those of others, with a view to developing robust evidence-based research that will inform and improve the ways in which REBs and research ethics administration are governed.
We have not, for example, addressed the structure and functions of REBs in developing countries (9) or the challenges faced by Canadian REBs in assessing research studies being conducted in developing countries. Nor have we considered the issue of quality assurance studies, which legitimately may be conducted by a REB or its governing organization. Other forms of evaluation, such as site visits by the National Council on Ethics in Human Research [NCEHR] or audits conducted by accreditation agencies such as the Association for Accreditation of Human Research Ethics Programs, Inc. [AAHRPP] or regulatory agencies such as Health Canada are also excluded from our consideration. The contributions of these forms of assessment to the literature on the governance of REBs are, potentially, significant and could be the topic of further research.
Much of the scholarly and applied research in which REBs are the "research subject" can be summarized as follows:
* assessments of the validity of anecdotal reports about the length of time it takes REBs to review protocols; (10)
* identifying ways to improve the quality of service to the research community; (11)
* evaluations of whether the REB process actually protects human participants through the interpretation of national policies and regulations; (12)
* assessments and the identification of the educational and professional development needs of REB chairs, members, researchers as well as administrative support staff and how such educational programs have improved the quality of the REB processes and the conduct of research; (13)
* variability across research ethics boards and evaluations of the decision-making processes of research ethics boards; (14)
* examinations of the ways in which REB members apply ethical judgment on protocols such as the proportional assessment of risks and harms; (15)
* evaluations of the impact of the REB processes on researchers and the conduct of research, (16) and, more recently;
* evaluations of the impact of accreditation on the governance of REBs. (17)
Research on research ethics boards in these few areas constitute the tip of the ice-berg--the possible topics areas for research on research ethics boards internal processes such as research ethics board's decision-making and its impact on medical, clinical (18), socio-behavioural (19), humanities and arts research and practice (20), and governance (21) are nearly endless and highly contested.
1. REBs as subjects of research--issues and potential challenges
More and more researchers, REBs and governing organizations are studying or commissioning research and quality assurance assessments of REBs, their processes and their governance structures. This trend raises the question of what it means for REBs to be arbiters of the very research that evaluates their functioning. REBs have had to evaluate how such research could be conducted while preserving the confidentiality and anonymity of key informants. Are REBs being forced to be more critically reflective of their own processes? At the same time, we argue, they must strike a reasonable balance between being open to such research while also addressing a wide range of concerns associated with being the focus of increased research attention--from protecting the confidentiality of studies they review to managing additional time commitments on an already heavy and demanding workload. More recently, in order to provide empirical evidence of REB operations, decision-making and impact on the research enterprise, empirical research on REBs has taken the form of participant-observer studies (22).
2. The Consent Process
Conducting research on REB members and administrative staff and their interactions with researchers places REBs in a new position--that of the subject of research as well as the arbiter of the ethics of research protocols. Similar to other organizations that are the subject of research, REBs need to address ethical issues of consent from a new perspective. How do REBs, as gatekeepers of research with humans, provide informed consent for research to be conducted on their deliberations, administration and impact? How does the meaning of informed consent change within this context? What are the potential conflicts of interest--can a REB review objectively a research protocol that focuses on its policies, procedures, processes, and interactions as a deliberative body and with researchers who submit protocols for review? Does this raise potential problems for the researchers who submit such protocols, and who will have to submit further protocols to the REB down the road, regardless of the findings of the REB study? What arms-length protections can be put in place to ensure integrity of the research review process? In our discussions, one member has suggested that REBs should not arbitrate a protocol in which it would be a participant. We debated whether a REB could objectively assess the quality of the research in which that particular REB, its processes (e.g., decision-making, assessment of risks and benefits, evaluation of serious adverse events, etc.) and its members are both the object and the subject of the research.
The research literature on informed consent/consent processes is immense and nuanced (23) Moreover, for many researchers who work with organizations (e.g., schools), communities and vulnerable populations, the questions of who has authority to speak on behalf of these bodies and who ought to grant access to the organizational and community members is a source of debate. This debate also applies equally to research on REBs--who may grant authority for an evaluator to conduct research on a REB? Is the permission of the Institutional Official  or the REB Chair necessary, or is it permissible for an evaluator to approach members of the REB, researchers who have interacted with the REB or REB administrative staff without their awareness? In the context of observational research, are REB members and administrators also expected to provide consent? Can administrative staff members that provide support to the REB decline to participate in a study that is investigating REB-researcher interaction once the Chair of the REB and the 10 have approved the research? What safeguards from reprisals or reprimand need to be in place to protect the administrative staff should they be critical of the operations of the REB, the timeliness, comprehensiveness and appropriateness of REB members' review of protocols, and/or the ways in which researchers interact with both the REB and its administrative support?
These questions highlight the tension and challenge of conducting research on the primary research review/ oversight mechanism, and suggest the need for a mechanism to provide checks and balances for research on REBs. It seems entirely appropriate for the 10 or REB to legitimately turn down a research protocol that aims to unduly undermine the legitimacy of the REB, however, there must be a mechanism by which those who wish to evaluate the REB can examine the governance process and policies of REBs, and the ways in which REBs shape the contours of research with humans.
Within the context of informed consent, there is a need to ensure that the wishes of the researcher undergoing REB review are respected. Take the case where an evaluator is studying the REB by sitting in on REB meetings. Suppose that the REB invites researchers to the meeting to field questions about their protocols. Does the evaluator need the consent of the researcher whose protocol is being adjudicated by the REB? The researcher and her/his research protocol is not the unit of analysis, but the REB process, such as the assessment of risks and benefits or the consent process, is the unit of analysis for this study, not the researcher.
3. Challenges with respect to confidentiality
REBs differ in the openness of their deliberations. In particular, the issue of confidentiality of protocol review poses difficulties for an evaluator wishing to examine how REBs assess the scientific merit of protocols, balance risks, benefits, and evaluate Serious Adverse Events (SAE) reports or protocol amendments and revisions. Depending on the REB, there may be concerns about confidentiality related to the specific protocols under review or the identity of individuals conducting the reviews. Evaluators, of course, can assess these interpersonal and subjective assessments through surveys or interviews with the "REB participants". Yet, having access to an REB as an "embedded" observer provides rich data and analysis that could lead to recommendations for ways to improve the governance of REBs and the human research protection program [HRPP] of organizations. One solution would be for evaluators who wish to gain access to REB deliberations to sign letters of confidentiality to protect the intellectual property rights [IPR] of sponsors or researchers and the identities of the reviewers who made specific comments. This must be done in such a way as to ensure that the outcomes of the research are available to not only inform the operations of the REB and its administrative support systems, but also to contribute to the body of knowledge on the governance of REBs. However, since evaluators are investigating only the ways in which the REB and its administrative support structures function, the permission of researchers and sponsors might not be required since the evaluators would not have access to the studies and the researcher is not the unit of analysis (24).
4. Appropriate methods?
Methods that could provide the greatest insights and outcomes into REB processes and the ways in which REBs operate include: participant-observation, interviews, surveys and participatory-action research (25). Each method has advantages and disadvantages and their use depends on the types of questions being asked and the data necessary to answer the questions. Surveys run the risk of low participation rates and sampling issues, yet may provide larger data sets from which researchers can generalize findings. Participant-observation techniques provide rich qualitative data, but may be more limited in the generalizabilty of the findings. A retrospective study examining REB decisions in documents might provide limited insight into how risk/benefit was assessed, but is unlikely to provide us with insights into the dynamics of the discussion that ensued amongst the REB members. A prospective study where the researcher sits in and observes and records the REB in action may skew authenticity, as REB members may act differently because they know they are being watched.
REBs, in their assessment of research protocols that focus on REB functions, decision-making and governance, may vary substantially in their opinion as to what methods should be used by evaluators to document their interactions and decision-making. For example, an REB Chair may enthusiastically accept an invitation to participate in an interview and be very forthright in providing answers but only on condition of not being audio recorded. Taking notes might be agreeable, but actual transcripts of the conversation might be shunned as they could undermine a frank discussion. Some chairs and members might prefer to provide answers in writing. Others might insist on viewing the interview guide in advance, prior to agreeing to be participants in the study. There is no one correct method to conduct research with REBs, each brings its own advantages and disadvantages and the researcher must be able to discuss and be willing to explain to the REBs which method will provide the best data for her particular project and objectives.
5. What impact will such research have on governance of REBs?
Priority setting for research on REBs should reflect issues identified by REBs, their organizations, researchers, sponsors, and subjects of the research protocols. We anticipate that a robust research agenda could better our understanding of the functioning of REBs, contribute to the resolution of issues outstanding between REBs and researchers and sponsors, provide approaches to improved review of protocols, and identify ways to increase the protections afforded to individuals and communities participating in research studies.
The impact of research on the functions of REBs is, potentially, profound. One outcome could be a contextualized understanding of the ways in which REBs operate, instead of the oppositional critiques of REBs that dominate the literature. Evidence-based insights into how REBs operate may lead to quality improvement in the administration and governance of REBs, permitting REBs to allocate their time proportionately to higher risk protocols and to evaluating whether participants will understand the nature of the research study, their rights, and their role within the study. Such research may provide REBs with the tools to assess their own competencies and methods or point to more effective modes of education for REB members and researchers. Policy focused studies could assess whether increased oversight of REBs, quality improvement approaches to governance, or accreditation will enhance the ways in which REBs are governed.
In the end, it is anticipated, although not certain, that open, transparent and critical research on REBs, whether sponsored by REBs and their governing organizations, national agencies or scholars, will contribute to the improvement of the governance of research involving humans in Canada by providing insights that will inform policy development and implementation organizationally and nationally.
(1) Joan E. Sieber, "Empirical Research on Research Ethics" (2004) 14:4 Ethics & Behavior 397.
(2) Annemarie Stroustrup, Susan Kornetsky, & Steven Joffe, "Knowledge of Regulations Governing Pediatric Research: A Pilot Study" (2008) 30:5 IRB: Ethics and Human Research 1.
(3) Michael McDonald, "Section F: Conclusions and Recommendations," The Governance of Health Research Involving Human Subjects (HRIHS) (Ottawa: Law Reform Commission of Canada, 2000), online: Library and Archives Canada <http://epe.lac-bac.gc.ca/100/200/301/Icc-cdc/governance_health_res-e/index.html> ["Section F"].
(4) American Association of University Professors, Institutional Review Boards and Social Science Research (2000), online: American Association of University Professors <http://www.aaup.org/AAUP/comm/rep/A/protecting.htm> Gary Allen, "Getting Beyond Form Filling: The Role of Institutional Governance in Human Research Ethics" (2008) 6 Journal of Academic Ethics 105; Scott Burris & Katherine Moss, "US Health Researchers Review their Ethics Review Boards: A Qualitative Study" (2006) 1:2 Journal of Empirical Research on Human Research Ethics 39; Constance F. Citro, Daniel R. Ilgen & Cora B. Marrett, eds., Protecting Participants and Facilitating Social and Behavioral Sciences Research (Washington, D.C.: National Academies Press, 2003); J. Paul Grayson, "How Ethics Committees Are Killing Survey Research on Canadian Students" University Affairs (January 10, 2004) online: University Affairs <http://www.universityaffairs.ca/article.aspx?id=1874> J. Paul Grayson & Richard Myles, "How Research Ethics Boards Are Undermining Survey Research on Canadian University Students" (2004) 2 Journal of Academic Ethics 293; H. Knill-Griesser, "A Vision Quest of Support to Improve Student Learning: Validating My Living Standards of Practice" (2001), online: Actionresearch <http://schools. gedsb.net/ar/theses/heather/index.html>; Michael Owen, "Engaging the Humanities: Research Ethics in Canada" (2002) 33:3 Journal of Research Administration 5; Michael Owen, "Ethical Review of Social and Behavioral Science Research" in L. Chronister & E. Kulakowski, eds., Research Administration and Management (Sudbury, Mass.: Jones and Bartlett Publishers, 2006) 543; Will C. van den Hoonaard, ed., Walking the Tightrope: Ethical Issues for Qualitative Researchers (Toronto: University of Toronto Press, 2002).
(5) Joan Sieber, "Research Agendas: An Invitation to Readers" (2006) 1:1 Journal of Empirical Research on Human Research Ethics 5 .
(6) Michael McDonald, "Canadian Governance of Health Research Involving Human Subjects: Is Anybody Minding the Store?" (2001) 9 Health L.J. 1; Michael McDonald, "Dedication and Introduction" (2005) 13: 2-3 Health Law Review 5. The Canadian Network for the Governance of Ethical Health Research Involving Humans is a Canadian Institutes for Health Research (CIHR) funded network of researchers who study ethics of research involving humans. At our initial meeting in July 2008, network members held wide-ranging, yet focused discussion, on key issues in the field of governance of research ethics in Canada. Two groups formed to assess "research on research ethics". See the article by Susan Cox et al, "Ethical Challenges and Evolving Practices in Research on Research Ethics" (2009) 17:2 Health L.R. in this issue for more on this point.
(7) See Amanda L. Nolen & Jim Vander Putten, "Action Research in Education: Addressing gaps in Ethical Principles and Practices" (2007) 36 Educational Researcher 401.
(8) See Sieber, supra note 1; Cox, supra note 6; Raphael Saginur et al, "Ethics Review of Multi-Centre Trials: Where Do We Stand?" (2009) 17:2-3 Health L.R. 59; Heather Sampson et al, "Evaluating and Understanding the Need for Canadian Research Ethics Board (REB) Member Standardized Education: Governance Views from the Field" (2009) 17:2-3 Health L.R. 73; Susan A. Tilley, "A Troubled Dance: Doing the Work of Research Ethics Review" (2008) 6 Journal of Academic Ethics 91.
(9) Nancy E. Kass et al., "The Structure and Function of Research Ethics Committees in Africa: A case study" (2007) 4:1 PLoS Medicine e3.
(10) E. Cave & S. Holm, "New Governance Arrangements for Research Ethics Committees: Is Facilitating Research Achieved at the Cost of Participants' Interest" (2002) 28 Journal of Medical Ethics 318; Konrad Jamrozik, "Research Ethics Paperwork: What is the plot we seem to have lost?" (2004) 329 BMJ: British Medical Journal 286; David S. Wald, "Bureaucracy of Ethics Applications" (2004) 329 BMJ: British Medical Journal 282.
(11) J. Michael Oakes, "Risks and Wrongs in Social Science Research," 2002 26 Evaluation Review 443-479.
(12) Grayson, "How Ethics Committees Are Killing Survey Research on Canadian Students, "supra note 4; Seema Shah et al., "How do institutional review boards apply the federal risk and benefit standards for pediatric research?" (2004) 291 JAMA: The Journal of the American Medical Association 476; Carl H. Coleman & Marie-Charlotte Bouesseau, "How do we know that research ethics committees are really working? The neglected role of outcomes assessment in research ethics review", online; (2008) 9 BMC Medical Ethics 6 <http://www.biomedcentral.com> H.E.M. van Luijn et al, "Assessment of the risk/benefit ratio of phase II cancer clinical trials by Institutional Review Board (IRB) members" (2002) 13 Annals of Oncology 1307.
(13) Ademola J. Ajuwon & Nancy Kass, "Outcome of a research ethics training workshop among clinicians and scientists in a Nigerian university", online: (2008) 9 BMC Medical Ethics 1 <http://www.biomedcentral.com> Adrian Bardon, "Ethics education and value prioritization among members of U.S. hospital ethics committees" (2004) 14:4 Kennedy Institute of Ethics Journal 395; Charles N. Bertolami, "Why our ethics curricula don't work" (2004) 68:4 Journal of Dental Education 414; Glen Greiner, "Emphasis on Social Behavioural Research" (Address presented to the FOCUS Conference 2006: An International Conference on Learning to Conduct Ethical Research in Humans: Educational Needs and Models, Washington D.C., 1 June 2006) [unpublished J; Michael Owen & Shawna McCambridge (October 15-16, 2007), "Ensuring Effective Ethical Review: Educating Institutional Review Boards and Researchers" (Poster presentation to the Society of Research Administrators International, Nashville, Tenn.; Sampson et al, supra note 8; Tilley, supra note 8.
(14) K.G.M.M. Alberti, "Multicentre Research Ethics Committees: Has the Cure Been Worse than the Disease?" (2000) 320 BMJ: British Medical Journal 1157; J. de Champlain & J. Patenaude, "Review of a mock research protocol in functional neuroimaging by Canadian research ethics boards" (2006) 32 Journal of Medical Ethics 530; C. Foster, "Why Do Research Ethics Committees Disagree with Each Other?" (1995) 29:4 Journal of the Royal College of Physicians of London 315; J.T. Hester Ward et al, "Obstacles to Conducting Epidemiological Research in the UK General Population" (2004) 329 BMJ: British Medical Journal 277; Georgine S. Burke, "Looking into the Institutional Review Board: Observations From Both Sides of the Table" (2005) 135 Journal of Nutrition 921; Julian Savulescu, Iain Chalmers & Jennifer Blunt, "Are research ethics committees behaving ethically? Some suggestions for improving performance and accountability" (1996) 313 BMJ: British Medical Journal 1390.
(15) John H. Mueller & John L. Furedym, "Reviewing for Risk: What's the Evidence That It Works?" (2001) 14:7 Association for Psychological Science Observer.
(16) Knill-Griesser, supra note 4; Grayson, "How Ethics Committees Are Killing Survey Research on Canadian Students," supra note 4; Interagency Advisory Panel on Research Ethics, "NSERC community whose research involves human subjects: Results of a survey about their needs and supports in research ethics", (2003) online: Interagency Advisory Panel on Research Ethics <http://pre.ethics.gc.ca/english/publicationsandreports/publicationsandreports/nsercsr2003_a.cfm> and Interagency Advisory Panel on Research Ethics, "Response to the Recommendations in the Report of the survey of the Natural Sciences and Engineering Community whose research involves human subjects", (2003), online: Interagency Advisory Panel on Research Ethics <http://pre.ethics.gc.ca/english/publicationsandreports/publicationsandreports/nsercsr2003_c.cfm>Interagency Advisory Panel on Research Ethics, Social Sciences and Humanities Research Ethics Special Working Committee, Giving Voice to the Spectrum (Ottawa: Interagency Advisory Panel and Secretariat on Research Ethics, 2004); Wald, supra note 10; Alison E. While, "Ethics Committees: Impediments to Research or Guardians of Ethical Standards?" (1995) 311 BMJ: British Medical Journal 661.
(17) Association for the Accreditation of Human Research Protection Programs AAHRPP Advance Newsletter, online: AAHRPP <http://www.aahrpp.org/www.aspx?PageID=142$7>.
(18) See D.J. Lawrence & M.A. Honras, "Do Chiropractic College Faculty Understand Informed Consent: A Pilot Study", online: (2006) 14 Chiropractic & Osteopathy 27 <http://www.chiroandosteo.com/content/14/1/27>.
(19) See Citro et al., supra note 4; Brad Olsen, Stephen Soldz & Martha Davis, "The Ethics of Interrogation and the American Psychological Association: A Critique of Policy and Process", online: (2008) 3 Philosophy, Ethics and Humanities in Medicine 3 < http://www.peh-med.com/content/3/1/3>.
(20) Social Sciences and Humanities Research Ethics Special Working Committee, Extending the Spectrum: The TCPS and research involving creative practice: A Report with Questions for Furthering the Dialogue. (Ottawa: Interagency Advisory Panel on Research Ethics, 2007); supra note 4.
(21) McDonald, "Section F", supra note 3.
(22) Tilley, supra note 8.
(23) J. Sugarman, D.C. McCrory, D. Powell, A. Krasny, B. Adams, E. Ball and C. Cassells (1999), "Empirical research on informed consent: An annotated bibliography," Hastings Center Report, 29:1 (January-February), pp. SI-42, identified nearly 400 publications related to informed consent; "Informed Consent Bibliography," United States Department of Veterans Health Administration, <http://www.research.va.gov/resources/pubs/informed_consent/> (retrieved March 3, 2009), identified an additional 60 publications by 2001.
(24) See Cox et al., supra note 6.
(25) See Cox et al., supra note 6; Tilley, supra note 8.
Michael Owen is Vice-President Research & Graduate Studies and Professor of Liberal Studies, Ontario College of Art & Design.
Claudia Emerson is a Postdoctoral Fellow in the Program on Ethics and Commercialization at the McLaughlin-Rotman Centre for Global Health, University Health Network and University of Toronto.
Pam Kolopack is a PhD Student at the Dalla Lana School of Public Health, Joint Centre for Bioethics, University of Toronto, Toronto, Ontario.
Nina Preto is a doctoral student at the W. M. Young Centre for Applied Ethics at the University of British Columbia.
Heather Sampson, RN, CCRP, MHSc, Toronto East General Hospital, Joint Centre for Bioethics, University of Toronto.
Anne Townsend is a Post-doctoral fellow at the W. M. Young Centre for Applied Ethics, University of British Columbia.
Donald Willison is an Associate Professor, Clinical Epidemiology & Biostatistics at McMaster University.
Roberta L. Woodgate is an Associate Professor, Faculty of Nursing, University of Manitoba.
Corresponding author: Michael Owen, Vice-President Research & Graduate Studies, Ontario College of Art & Design, 100 McCaul Street, Toronto, ON, Canada, M5T1W1.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Research Ethics Boards|
|Author:||Owen, Michael; Emerson, Claudia; Kolopack, Pam; Preto, Nina; Sampson, Heather; Townsend, Anne; Willi|
|Publication:||Health Law Review|
|Date:||Mar 22, 2009|
|Previous Article:||Ethical challenges and evolving practices in research on ethics in health research.|
|Next Article:||Challenges to ethics review in health research.|