Who Wants to Try a Questionnaire?

This new column will appear in every other issue of the Journal. Written alternately by Drs. Robert Goldbort and Ray Dolle, it represents a resurrection of the "Technical Writer" column the Journal published during the early 1990s. Effective writing is a skill that nearly everyone can benefit from, and we hope you both learn form and enjoy "Writing Professionally."

If you are planning to distribute a questionnaire, what should you do first?

a) Phone a friend.

b) Dress like Regis Philbin.

c) Ask the audience.

d) Read this column.

Introduction

Creating, administering, and tabulating a questionnaire takes expertise. A researcher who wants to use a formal survey to gather valid, reliable scientific data must follow established testing and measurement procedures to ensure that the questions really ask for the information sought; experts in statistics use special methods to test the validity and reliability of questions. Market-research firms pilot-test surveys on small sample populations before mass distribution to a statistically derived random sample of respondents. Collation and analysis of the responses may require computer scoring sheets and statistical software. The entire process is time-consuming and complex.

Nevertheless, researchers use questionnaires because these instruments can be valuable in determining public attitudes, observations, opinions, habits, and concerns related to environmental health and-public safety. According to Briley, Fowler, and Teel, whose column discussed the assessment of health-department performance in the December 1999 issue of this Journal, "In some instances, the best tool for gauging the relative success of a program is a survey. Citizens, consumers, and the regulated community should be surveyed at regular intervals" (p. 36). Although we cannot promise to make you an instant expert on surveying techniques, we can offer some basic advice about developing a simple, informal questionnaire of limited scope. So, if you answered "d" to the question above, you got it right!

Fastest-Finger Questions

Considering the problems involved in using a questionnaire--and the imposition of asking somebody to fill one out--you should first be sure that a questionnaire is really necessary. You must decide beforehand whether you can get the desired information from a survey and whether the information has already been published. Perhaps it can be obtained from sources like interviews, site visits, field observations, courthouse records, government reports, library files, newspapers, journal articles, or the World Wide Web; if so, you can often save time, money, and effort by avoiding the questionnaire process. Even if your preliminary research does not yield what you need, your findings will help you to plan and focus survey questions. The fewer and more specific the questions, the more likely recipients are to respond.

If you decide that a questionnaire is the best way to obtain some of the needed information, you must remember that recipients decide within a few seconds to either discard or reply to unsolicited requests. The instrument must catch the reader's attention and give him or her an incentive to fill it out. Either in the initial instructions or in a cover letter, you should indicate the purpose of the survey and its relevance to the reader. Do not just say, "We would appreciate your answering the following questions." You might try something like this: "As warm weather approaches and you start enjoying outdoor activities, you might be worrying about environmental health concerns. We have sent you this survey so that you can help us to ..." Once you have your reader's interest, however, you may not be able to hold it very long, so you must keep the questionnaire brief, design a layout with fonts and white space that are inviting, and ask easy-to-answer questions that follow logically from general impressions to specific details or from objective observations to subjective opinions. If you must make the survey long, you should divide it into sections (e.g., Part 1, Part 2), each section focusing on a particular aspect or subset of the topic.

Hot-Seat Questions

Deciding what questions to ask and how to ask them is more difficult than writing the questions for Who Wants to Be a Millionaire. Different types of questions elicit different kinds of data. Statistics and surveying textbooks describe dozens of question types, ranging from dichotomous (yes/no, true/false) to multiple choice to rank order to Likert scale (least-to-most). You also can ask open questions, such as fill-in-the-blank and even essay questions, although instruments of this type are difficult to tabulate and entail other problems, such as illegible handwriting.

Short, simple, objective questions provide the most reliable and valid results. The most obvious thing you can do to increase reliability and validity is to formulate questions in a way that ensures a correct interpretation (i.e., an interpretation that matches your intention).

* When possible, you should use familiar, commonplace words, especially in surveys of the general public, and avoid words with vague or multiple meanings. The question "Do you regularly receive verbal warnings?" would not be clear to every one. How often is regularly? Does verbal mean oral, written, either, or both? Does warning mean threat or precaution? Because the question leaves too much interpretation to the respondents, the results would not be very reliable.

* Each question should ask for only one piece of information. Compound questions, like "Are you exposed to loud noises at home or work?" should be divided into two simple questions.

* The wording of a question has to be constructed specifically to measure what you want to measure. A general question about noise exposure could not be used by itself to draw specific conclusions about loud machinery, ambient noise, or traffic emissions as risk factors.

* Questions must be unbiased--that is, worded neutrally so that they do not imply a "correct" answer. You should not load a question with words that have emotional connotations. Here is an example of a leading question: "Do the organic contaminants polluting our local groundwater make it taste bad?" Well, of course! A less leading way to phrase the question might be, "Is the taste of the local groundwater affected by organic contaminants?"

* Similarly, the range of suggested answers must be inclusive enough to avoid prejudicing the results. You must give the respondents a wider scale of choices than just, say, "good, very good, outstanding." If possible, the range of possible answers should provide quantitative rather than qualitative data. For instance, "monthly" might be used instead of "seldom," "weekly" instead of "sometimes," and "daily" instead of "often."

Typical Questions

A dichotomous question offers the respondent a choice between two answers. This approach can be useful when black-and-white facts are at issue, but experts disagree on the value and use of the dichotomous question. Some say it eliminates the possibility of a noncommittal middle answer. Others say a third choice (e.g., "maybe," "no informed opinion") should be offered, or the results will be skewed.

Multiple-choice questions ask the respondent to select one or more answers from among several possibilities. You should offer at least three to five realistic choices. If you cannot allow for all possible responses, you can add a blanket category (e.g., "other") followed by a line on which respondents can write their answers. This approach permits unanticipated responses while providing choices sufficiently controlled for tabulation and analysis. Here is an example:

In the last year, have you experienced pain or discomfort in any of the following?

Hand(s) _____

Wrist(s) _____

Elbow(s) _____

Shoulder(s) _____

Neck _____

Back _____

Other(s) _____

Questions that use a Likert scale ask the respondent to rate an item on a least-to-most basis of some type (e.g., "never" to "always," "poor" to "excellent," "strongly agree" to "strongly disagree"). Again, you should offer a range of three to five or more choices in gradually varying degrees. Some experts recommend putting the positive option on the left side of the scale; others note that statistical tests have shown respondents to favor the left side of surveys. In any case, the direction in which the Likert scales flow should be consistent throughout your questionnaire. Here is an example:

How important are writing skills to your job performance?

Very important _____

Important _____

Somewhat Important _____

Not Important _____

Rank ordering asks the respondent to number several possibilities in order of preference, importance, frequency, or some other characteristic. Here is an example:

Using the numbers 1 (Most Often) through 5 (Least Often), rank the following chemicals according to how frequently they are used in your workplace.

Ammonia _____

Benzene _____

Cadmium _____

Carbon Tetrachloride _____

Mercury ____

No matter what types of questions you use, it is a good idea to group questions of the same type together. Once respondents get into a certain mode, they like to stay in it and not have to figure out a new format for every other question. You can even arrange a series of questions in columns or on a grid to facilitate responses. Figure 1 and Figure 2 give examples.

Questionnaires often end with an essay question. The question is usually somewhat general and open-ended, asking for additional comments, overall opinions, suggestions, explanations, reasons why or why not. The obvious advantage is the freedom a question of this type allows the respondent in composing a detailed answer. The replies may contain points that you have not considered or have not addressed in the other questions. There are drawbacks, however. Besides the practical problems of deciphering handwriting and interpreting idiosyncratic responses, essay questions pose an open invitation for irrelevant comments, complaints, testimonials, and so forth. Usually, only those respondents who feel very strongly about the topic will take the time to write a personal statement.

Final Answer

Despite the time and trouble involved, a questionnaire is sometimes the answer. For an informal, in-house survey, the advice in this column should be enough. For a formal scientific study, you can consult one of the books listed below You can also go to Frederic D'Astous's "Guide to Questionnaires and Surveys" at [less than]http://members.tripod.com/[sim]frede_dast/conseill_a.html[gre ater than].

That's my final answer. Suggested Reading

Berdie, Douglas R. (1986). Questionnaires: Design and use. Metuchen, NJ: Scarecrow Press.

Converse, Jean M. (1986). Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage.

Foddy, William. (1993). Constructing questions for interviews and questionnaires: Theory and practice in social research. Cambridge, UK; New York: Cambridge University Press.

Labaw, Patricia J. (1980). Advanced questionnaire design. Cambridge, MA: Abt Books.

Patten, Mildred L. (1998). Questionnaire research: a practical guide. Los Angeles, CA: Pyrczak Publishing.

Survey Questions Arranged in columns
Circle your response below.        Strongly Agreely (5)  Agree (4)
Safety meetings result in
improved health and safety.                 5                4
Safety meeting minutes should be
posted for all employees to read.           5                4
Safety meetings should be
attended by all employees.                  5                4
Circle your response below.        No Opinion (3)  Disagree (2)
Safety meetings result in
improved health and safety.              3              2
Safety meeting minutes should be
posted for all employees to read.        3              2
Safety meetings should be
attended by all employees.               3              2
Circle your response below.        Strongly Disagree (1)
Safety meetings result in
improved health and safety.                  1
Safety meeting minutes should be
posted for all employees to read.            1
Safety meetings should be
attended by all employees.                   1