Ethical considerations when establishing survey standards.
My first assumption is that survey performance standards are largely irrelevant to ethical issues. This is not to denigrate the importance of quality performance when evaluating a survey's professional credentials. One hallmark of professionals is that they can, and do, meet performance standards. My point is that it is possible to meet performance standards regarding such matters as completion rates and statistical tests of significance and still fall ethically. I further believe that a rigid adherence to absolute performance standards does not necessarily protect, and in some circumstances may even weaken, professional ethics.
The irrelevance of performance standards to professional ethics is obvious with respect to surveys conducted in countries, or parts of countries, with limited or inadequate survey resources or with life style characteristics that, for example, may make for low completion rates. Under conditions that prevail in such cases, it may prove to be very costly (even perhaps impossible) to implement satisfactorily a probability sample in conformity with state-of-the-art standards that would apply in the USA or Western Europe. Insistence on conformity to performance standards in such cases could result in cancelling a study, whereas some flexibility in the application of such standards might make it possible to conduct a survey whose results might still be usable. For this reason, even though I unequivocally advocate the theoretical superiority of probability sampling, I also assume that its standards should be adjustable to meet the realities of specific times and places. The implication of this assumption will become obvious in comments I will make later in this paper.
My second assumption is that the purpose of a survey must not be morally objectionable to a professional survey researcher. This is not to say that every survey must contribute to some high moral purpose but only that there must not be a moral reason for not conducting it. This assumption follows the old medical precept, `Do no harm'. Thus, although others might disagree with me, I personally find it morally unacceptable to conduct a survey for a tobacco company, a right-wing racist political party, or a military dictatorship. My point is that considerations of morality go far beyond professional ethics per se.
Even as I make this last point, I want to disavow any facile self-righteous connotations that it may have. I fully recognize that in totalitarian and authoritarian countries survey researchers do not have the luxury of deciding when not to conduct a survey. But, I do think it is necessary to recognize that important moral questions are not always addressed by standards of professional ethics.
PROFESSIONALIZATION OF SURVEY RESEARCH
in accord with precedents set by law and medicine, developing a code of standards has long been central to the professionalization of any occupation. Following this tradition, at its birth in 1947, the American Association for Public Opinion Research immediately took up the question of whether it should adopt a professional code and what should be in that code. Much of the ensuing debate had to do with whether the code should resolve the controversy over probability versus quote sampling. Integral to that controversy was whether performance standards should be part of the code. The code that was adopted emphasized ethical standards, making no mention of performance standards (Sheatsley and Mitofsky 1992). However, the debate as to whether the AAPOR code should be expanded to include performance standards still continues. WAPOR must also deal with this issue.
There can be no argument that professionalism has both technical and ethical dimensions. Therefore, before focusing our attention on ethics, I want to offer a general comment regarding technical competence and the professional quality of surveys. In the twentieth century, the legal recognition of an occupation's professional status has rested primarily on its requiring documentary evidence of the technical competence of accepted individuals. This evidence might include holding an advanced degree, passing a special examination, and/or having a specified number of years experience. Nonetheless, the fact that a survey is designed and implemented by an individual who meets the legal criteria of technical achievement in no way guarantees that it will meet mandated performance standards. Thus, whether a professional code of survey research should include performance standards and whether survey research should be granted legal professional status based on demonstrated technical competence are separate questions.
Having taken notice of the existence, distinctiveness, and importance of performance standards, I shall focus my remaining comments on the ethical dimension of survey research as a profession.
THE PROFESSIONAL OBLIGATIONS OF SURVEY RESEARCHERS
Professional obligations relate to three types of relationships--to one's clients, to others whose interests may be affected by the conduct of specific survey tasks, and thirdly to the profession as such and, consequently, to fellow surveyers. (There is another relationship, that of survey research to society at large, that must also be acknowledged. As we shall see later, this fourth relationship provides the context for the first three.) While these three types of obligations are interlocking, because they are also distinctive they can sometimes be difficult to meet concurrently. Among other things, as we shall see, to some degree they may be in potential conflict with each other. It is also pertinent to observe that most survey researchers believe that meeting their professional obligations earns them the right to expect that political authorities will, in turn, respect and support these obligations.
OBLIGATIONS TO CLIENTS
Obligations to our clients are in many ways obvious and clear-cut. Above all, we must act as something more than businessmen selling our services. Instead, we must be guided by a commitment to serving the client's interests as if we were the client her- or himself. Thus, we are obligated to develop and implement survey designs that will meet client objectives at a reasonable cost. At the minimum, this means that we must not restrict ourselves to promoting only profitable, proprietary research designs. Nor should we insist on using sophisticated state-of-the-art methods when simpler, less expensive designs are all that is needed. We are also obligated to inform clients when survey research is not appropriate to their objectives. Accordingly, we should inform clients of the extent to which their objectives can or cannot be met using available knowledge and resources, and within available budget. We should also be prepared to explain our judgments in this regard, for example, by citing previous experience--without, however, compromising our obligations of confidentiality to other clients.
In that spirit, our client obligations also include the responsibility of informing them about the research design specifications we intend to use, for example, sample size and design, questionnaire design, interviewing methods, analysis plans, and applicable performance standards--and, also, whether the survey will be conducted as part of a larger or omnibus survey. Any failure to meet stated specifications should be acknowledged. And, we should be prepared to document claims we make concerning trade secrets and proprietary methods. A related cost consideration is that while we are free to develop new methods by applying what we have learned from a particular study, we cannot expect our clients to pay for development costs unless they think it is worth their while to do so.
These last comments have implications with respect to performance standards. When it comes to fulfilling the obligation to develop a research design when it will meet a client's objectives, satisfying performance standards are often no more than the starting point. In fact, it is frequently the case that introducing design elements whose absence would in no way violate accepted performance standards more than compensates for falling short on any one standard. In election research, for example, in some cases weighting by regional turnout rates may contribute far more than improving moderately satisfactory completion rates.
Additionally, what should one do if a client's budget is too small to conduct the number of call-backs needed to achieve a satisfactory completion rate without reducing the sample size? The text-book answer is to recommend not conducting the study unless the budget is increased. However, in such circumstances there are at least two additional options that are commonly used but fall short of established performance standards--conducting something less than a full-scale probability sample or conducting focus groups. I suggest that the professional solution is to present all three options, making sure that the client fully understands the limitations of each. For example,
If a survey is not conducted, clients will have to rely instead on whatever limited information may already be available and/or whatever prior experience they may have.
If a non-probability sample design is used, clients must be ready to accept its potential bias and limited projectability.
If focus groups are conducted, the study objective must be revised to deal with qualitative issues only, giving up any pretence of measurements.
In other words, I maintain that professional ethics require us to make clear to clients the implications of adopting a particular course of action but not necessarily to insist upon a rigid conformity to performance standards.
OBLIGATIONS TO OTHERS
As professionals we must also consider how a study may affect the well-being of others who are not clients, especially and most particularly respondents. To some degree, there is an element of self-interest in our obligations to respondents. For example, if we do not respect the privacy and confidentiality of what respondents tell us, eventually we will lose the public's cooperation. Beyond self-interest, this obligation also rests on the proposition that we do not have the right to trick anyone into acting against his or her self-interest. Analogously a medical researcher does not ask potential subjects to take a newly formulated drug that contains a known toxic substance without informing them of the risk.
In surveys, questions about illegal behavior, such as drug use, clearly place respondents at risk. But answers to seemingly innocuous questions may also harm respondent interests. For example, if not kept confidential, answers to standard income questions may threaten the interests of someone in financial litigation or who is applying to the government for financial assistance. By making it a general obligation to protect the confidentiality of all answers to a survey, we act to protect respondents from a host of potential threats, including those that we do not anticipate. And, of course, there is always the principle of protecting individual privacy.
Meeting obligations to respondents is complicated by the fact that respondent and client interests may be in conflict. How much can we tell respondents about study objectives and methods without infringing on client rights or affecting the integrity of the research design? And how far can other interested parties go in claiming that a survey should not be conducted, at least as designed, because their interests would be harmed?
Promises, explicit or implicit, that we make to respondents to persuade them to be interviewed also raises ethical questions. We typically tell respondents that the results of a marketing survey will be used to improve a product or a service when in reality its purpose may be to find a way of reducing quality without customers finding this out, to deceive consumers about the quantity contained in a package, or to raise prices. Similarly, the results of political surveys may be used to undermine the interests of population segments included in the survey sample. And there is always the problem of being honest with respondents concerning interview length.
There arc no easy answers to any of these issues concerning obligations to respondents. Nonetheless, I believe there is reason to believe that if we try hard we can often find ways of fulfilling these obligations without reducing the quality of our research and perhaps even improving it. Two examples illustrate this point:
(1) A common practice in multiple-wave surveys which require linking responses from successive waves used to be using devices, such as secret codes and even invisible ink, to disguise the fact that anonymity was not being maintained. Every so often, someone would detect the device, resulting in unfavorable publicity, demands for government regulation, and reduced cooperation. Today, overt links such as matching identity codes are commonly used, usually highlighted with explanations of why this is being done and how confidentiality is still protected. Completion rates have not suffered from this change.
(2) One-way mirrors were initially used in focus group research in order to encourage open discussion and to `protect' the research from being `contaminated' by external influences. In reaction to criticism of the presumed dishonesty of this practice, standard procedure today is to inform the group that there are people behind the mirror. Doing so has not reduced the productivity of the group discussion. Rather, my experience is that it adds to confidence in the researcher's honesty. Furthermore, it makes it possible for messages to be sent to the group moderator about additional topics to raise and questions to ask.
I conclude that when there is conflict between client and respondent interests our professional obligation is to make every effort to develop ways of reconciling that conflict and not to concentrate on inventing ingenious subterfuges.
OBLIGATIONS TO THE PROFESSION
In addition to client and respondent obligations, we must also consider our obligations, individual and collective, to survey research as a profession and to our professional colleagues. To the extent that we do not meet those obligations, we undermine the professional status of survey research. That status derives from a tacit `contract' between society at large and occupations with professional aspirations. That contract states, as it were, that in exchange for an occupation's adherence to ethical standards that are not required of others, society confers special privileges and entitlements upon its practitioners. Therefore, to justify receiving the benefits we receive personally as professionals, we owe it to the profession to live up to its side of the `contract'.
OBLIGATIONS TO SOCIETY
All this may sound abstract and ephemeral, easy to relegate to the back burner when under pressure to complete a study when time and/or budget are limited. But, the fact is that if we succumb to practical reality and ignore ethical principles of professionalism, there is no doubt that in a short time survey research will become nothing more than another technical trade. The result will not only be our individual loss of privilege and entitlement but, more importantly, the loss to society of surveys conducted in a manner that contributes to the general good.
Collectively as a profession, and individually as practitioners, we are obliged not to misrepresent the field to others, not to block either its technical or ethical development, and to contribute insofar as possible to its continued technical and ethical development. This principle is most clear, and usually most easily enforced, within the academic sector of survey research. But if commercial practitioners want to be treated as professionals, they too must accept it.
At a minimum, both academic and commercial survey researchers are obliged to describe their research designs, and to present their results, as fully and openly as possible so that competent others can evaluate their projects. Unfortunately, there will be times when this obligation will conflict with the confidentiality of client interests, placing the researcher in a very difficult position. I think it is safe to say that no one expects survey researchers to violate the confidentiality of client interests. On the other hand, my impression over the years is that the most serious problems arise under the following two conditions: (a) when clients seek to publicize selected findings in a way that distorts the true picture and (b) when survey practitioners seek to disguise what they do, even from their clients.
I do not have any hard-and-fast rules to suggest that will tell us specifically what to do in individual cases of either type. Nonetheless, under both conditions, the ethical course of action is to me indisputable. When clients decide to publicize survey results, they give up any right to confidentiality. And while practitioners are entitled to protect proprietary rights, they are not entitled to do so in a way that misleads their clients. Thus, a principle of openess is common to our obligation to inform clients as fully as practical about our research designs, procedures, and results and to our obligations to the profession.
A common theme running through all my remarks is that, as professionals, survey researchers have the obligation to provide as much information as possible about the work they do-to their clients, to respondents and others whose interests may be affected by the research, and, especially but not only when the research is in the public domain, to the general public. From a practical point of view, private interests, including the researcher's, the client's, and even of the respondents, will always place limits on this principle of openness. Nonetheless, I have no doubt that over the long run, and often even in the short run, the quality of the work we do is enhanced by being as open as possible.
In conclusion, I have not attempted to provide a list of criteria which should be part of any professional code of standards for survey research. Instead, I have tried to define a valid and practical perspective for thinking about what should be in such a code.
This article continues the series of contributions on quality criteria in survey research, based on papers given at the WAPOR Regional Seminar `Quality Criteria in Survey Research', held in Cadenabbia, Italy, in June 1996.
Correspondence should be addressed to Irving Crespi, 9 Orchard Circle, Princeton, NJ 08540, USA.
Paul B. Sheatsley and Warren J. Mitofsky (eds.) (1992): A Meeting Place: A History of the American Association for Public Opinion Research, AAPOR
Irving Crespi, now retired, has over 40 years experience in public opinion and marketing research, mostly in the USA but also including international studies.
|Printer friendly Cite/link Email Feedback|
|Publication:||International Journal of Public Opinion Research|
|Date:||Mar 22, 1998|
|Previous Article:||Media priming effects: accessibility, association, and activation.|
|Next Article:||The Public Opinion Process: How the People Speak.|