Completion of Self-Administered Questionnaires in a Sex Survey.

Sell-administered questionnaires (SAQs) are a commonly used method of eliciting information on sensitive topics in personal visit or face-to-face surveys. SAQs have been used in studies ranging from sexual behavior (e.g., Catania, Gibson, Chitwood, & Coates, 1990; Catania, McDermott, & Pollack, 1986; Johnson & DeLamater, 1976; Smith, 1992; Tourangeau, Rasinski, Jobe, Smith, & Pratt, 1997; Tourangeau & Smith, 1996) to drug use (e.g., Aquilino, 1992, 1994; Turner, Lessler, & Gfroerer, 1992) and abortion reporting (e.g., London & Williams, 1990). SAQs are hypothesized to improve reporting of sensitive information by increasing privacy and reducing social desirability effects associated with interviewer administration (Schober, Fe Caces, Pergamit, & Branden, 1992; Turner et al., 1992). Given the widespread use of SAQs it is important to understand the conditions under which they lead to improvements in data quality, and to examine potential limitations of their use.

Much of the research on SAQs has focused on nonresponse, both at the unit level (noncompletion of the entire SAQ; see Johnson & DeLamater, 1976; Smith, 1992) and at the item level (noncompletion of some, but not all, items in the SAQ; see for example Catania et al., 1986; Wiederman, 1993). Other studies have examined measurement error issues related to SAQs (see reviews by Catania, Binson, van der Straten, & Stone, 1995; Catania et al., 1990). This paper extends the research on SAQs by examining yet another potential source of error--completion of the SAQ, but not by the respondent. Depending on how this is treated, the error could be considered measurement error or nonresponse error. In most surveys, the information on who completed the SAQ is not recorded, and it is generally assumed that the SAQ was completed as designed --by the respondent.

The National Health and Social Life Survey (NHSLS: Laumann, Gagnon, Michael, & Michaels, 1995; Michael, Gagnon, Laumann, & Kolata, 1994), conducted in 1992, provided a unique opportunity to explore this issue. Four SAQs were used in the survey, followed by an interviewer check item to record who actually completed the SAQ. We used these data to explore reasons why people may choose not to self-complete portions of the questionnaire designed for self-administration. We explored hypotheses related to capacity or ability to complete the SAQ, motivation to do so, and reasons related to the sensitivity of the content. We also examined the potential effect of this decision on the quality of the data obtained in SAQs.

A key assumption of SAQs is that data quality for sensitive items is improved under self-administration. But for such quality gains to be achieved, the SAQ must indeed be self-administered. To the extent that interviewers may assist respondents in the completion of SAQs or even treat them as interviewer-administered items for some respondents, the data quality gains may be diminished. This may be especially true if those most likely to exhibit the sensitive behavior (e.g., drug use, abortions, high-risk sexual practices) are more likely to be assisted by an interviewer, potentially negating the effects of self-completion.

BACKGROUND AND HYPOTHESES

Several studies have examined the issue of nonresponse to SAQs. For example, Johnson and DeLamater (1976) found that 8.3% of adult respondents in the Erotic Materials National Survey refused to complete the SAQ. They found that persons who did not complete the SAQ were older and less educated. Similarly, Smith (1992) reviewed the sexual behavior questions in the General Social Survey (GSS), which uses a short SAQ. He found response rates to the SAQ of 93.4% in 1988, 91.2% in 1989, and 85.5% in 1990.

A number of other studies examined item nonresponse or partial completion of SAQs. Catania et al. (1990) reported that refusal rates (item noncompletion) in SAQs ranged from 6% to 13% for items assessing frequency of vaginal intercourse, from 6.7% to 19% for masturbation items, and an average of about 6% for number of sexual partners. London and Williams (1990) included a 5-item SAQ on abortion in the 1988 National Survey of Family Growth (NSFG). They reported that 98.4% of women completed at least one of the five items, with the highest nonresponse (4.6%) for a question on age at first abortion. Wiederman (1993) examined item nonresponse using the same 3 years of GSS data as Smith (1992), and reported levels of item nonresponse higher than the overall noncompletion rate for the SAQ, suggesting that his analyses are based on those who completed at least some of the SAQ rather than the total sample. He found that nonresponders were generally older and reported less income and less education than responders.

All these studies (and others) that examined full or partial nonresponse to SAQs assumed that if an item was completed, it was done so by the respondent as intended. One exception is a study by Couper and Rowe (1996), who examined self-completion in the context of a computer assisted self-interview (CASI) with the SAQ being administered on a laptop computer. They found that 79% of respondents completed the CASI items themselves, while in 7% of the cases the interviewer assisted the respondent, and in 14% the interviewer completed the self-administered items for the respondent. They speculated that part of the reason for the low self-completion rate may be due to respondent reluctance to use the computer. We examined whether similar effects were found in paper-and-pencil SAQs.

One reason why this issue has received little attention may simply be that the information on who completed the SAQ is often not recorded or provided to analysts. The NHSLS was an exception, and we used this opportunity to examine the extent of self-completion versus interviewer-completion across four different SAQs in that study. We hypothesized that there are three types of reasons why people may choose not to self-complete an SAQ, given that they have agreed to do the interview and provide data to the interviewer. We label these capacity, motivation, and sensitivity.

Capacity

We hypothesized that some respondents may be unable to complete a self-administered questionnaire because of illiteracy or health conditions (e.g., impaired vision) that cognitively or physically constrain their ability to do the SAQ. Literacy is a necessary requirement for self-completing a questionnaire. Results from the National Adult Literacy Survey (Kirsch, Jungeblut, Jenkins, & Kostad, 1993) suggest that the ability to comprehend and respond to written questionnaires may be a serious impediment for many adults in the United States.

Smith (1992) found that cognitive ability (as measured by education, a vocabulary test, and interviewer rating of comprehension) was significantly related to the likelihood of item nonresponse on two of the SAQ questions in the GSS. Johnson and DeLamater (1976) found similar effects when using education as a proxy for literacy, as did Couper and Rowe (1996) in examining CASI self-completion. In line with these studies, we employed respondents' level of education as an indicator of literacy.

Visual impairment may also serve as a constraint on SAQ self-completion (Catania et al., 1990). Couper and Rowe (1996) found that those who reported serious vision problems in the last 12 months were much less likely to self-administer a CASI survey. They also found that persons age 65 and older were significantly less likely to complete the CASI items themselves (59.6% compared to 82.1% of those 45-64 and 64.5% for those 30-44). Smith (1992) also found age to be significantly related to item nonresponse on two SAQ items in the GSS, with 89% of those ages 18-48 completing both items, compared to 79% of those ages 49-69 and 71% of those age 70 or older. Johnson and DeLamater (1976) found nonresponders to an SAQ to be older than those who completed the SAQ (see also Catania et al., 1990). Age may serve as an indicator of possible visual limitations, but may also be an indicator of literacy. In summary, we expected those with less education and older persons to be less likely to self-complete the SAQs because of the increased difficulty of the task.

Motivation

Even if respondents have the capacity to complete an SAQ, they may be unwilling to do so. We hypothesized that the cognitive (and possibly physical) effort involved in self-completion is greater than that involved in responding to an interviewer's questions. Thus, allowing the interviewer to administer the items may be a form of satisficing (Krosnick, 1991). The task is still completed, with minimal additional effort expended. Catania et al. (1990) discuss motivation as a source of measurement error in SAQs, but we hypothesized that low motivation may also be manifested in non-self-completion.

We thus expect that respondents who are less motivated to participate in the survey initially, or who are less engaged in the interview, are also less likely to self-administer the SAQs. Indicators of initial reluctance (e.g., initial refusal, number of calls to complete interview, additional payment offered), observed prior to the start of the interview, may or may not be overcome once the interview has started. On the other hand, indicators of lack of engagement during the interview are usually assessed after the fact (by interviewer ratings of respondent cooperation). We expected the latter to be stronger indicators of non-self-completion as they are more closely related to the interviewing task. However, these post-interview ratings may be contaminated by the willingness of the respondent to complete the SAQ items, and may be an outcome rather than a cause of self-completion.

Laumann, Gagnon, Michael, and Michaels (1994) analyzed distributions on key sexual behavior items by a variety of reluctance indicators in the NHSLS data, and concluded that "reluctant and nonreluctant respondents are not very different along important dimensions of sexual life" (p. 564). However, our focus is on whether reluctant and nonreluctant respondents differ in the likelihood of completing the SAQs, and whether self-completion is related to differences in sexual behaviors and attitudes.

Sensitivity

The third factor we consider relates to the sensitivity of the SAQ items. Bradburn, Sudman, Blair, and Stocking (1978); Catania et al. (1986); and Johnson and DeLamater (1976) found sensitivity of the topic to be related to SAQ nonresponse. On the other hand, Smith (1992) found no significant relationships between a variety of sexual attitude measures and the likelihood of nonresponse to SAQ items on the number of sex partners. In looking at who completes the SAQ, we hypothesized that those who regard the SAQ items as most sensitive would be more likely to self-complete, rather than having the interviewer assist them with the task.

However, the measurement of sensitivity may be confounded with the argument that self-completion improves reporting of such sensitive information. Those who engage in sensitive behaviors might be more likely to self-administer the SAQs, but self-administration is hypothesized to increase the reporting of socially sensitive behaviors over interviewer-administration. Thus, we cannot tell whether the act of self-completion leads to more honest reporting or simply that those who have more to reveal are more likely to choose self-administration. To avoid this problem, indicators of sensitivity were developed using items in the interviewer-administered portion of the interview.

Another factor that may increase the sensitivity of the items is the presence of others during the interview. While every attempt is made in surveys such as this to ensure privacy during the interview, there may still be others present for all or part of the time (e.g., Aquilino, 1998; Reuband, 1992). We hypothesized that with others present, respondents may be more likely to self-administer the SAQs.

Another indicator of potential sensitivity may be the gender of the respondent and the interviewer. While the findings on gender matching are mixed (see, for example, Catania et al., 1996; Catania et al., 1995), we hypothesized that same-gender interviews would lead to less potential threat than different-gender interviews. Given that most interviewers in the NHSLS were women (Laumann et al., 1994), we hypothesized that female respondents talking to female interviewers would find the SAQs less sensitive than male respondents talking to female interviewers. Hence, we used gender of the respondent as a weak proxy for a (mis)match of respondent and interviewer gender.

DESIGN AND ANALYSIS

A full description of the sample, data collection, and other design features of the NHSLS appears in Laumann et al. (1994), and will not be repeated here. A few details of relevance to the present study will be mentioned. The NHSLS was based on a multistage area probability sample of households in the United States. The population of inference was all people ages 18-59 with adequate English proficiency living in households. The sample consists of two parts, a cross-sectional sample and an oversample of Blacks and Hispanics. One person was randomly selected from each household. The survey was conducted face to face in respondents' homes by trained National Opinion Research Center (NORC) interviewers using paper and pencil questionnaires. Gender or race matching of interviewers to respondents was not attempted; however, most of the interviewers used for the study were women. An overall response rate of 78.6% was achieved. The majority of the nonrespondents were refusals (16.8%), while there were 1.9% noncontacts and 2.7% other noninterviews. This yielded a total of 3,432 completed interviews, 3,159 of which were from the cross-sectional sample with the remaining 273 from the oversample.

We used the full weighted sample in our analyses. The weight used is a combination of a sampling weight (to reflect the oversample), an eligibility weight (for household size), and a poststratification weight (primarily for differential nonresponse). Tests of statistical significance and standard error estimates are based on a Taylor Series approximation, reflecting the complex sample design. The models were estimated using SUDAAN (Shah, Barnwell, & Bieler, 1996).

The NHSLS instrument contained four SAQs, interspersed throughout the interview. The content of each SAQ was, briefly, (a) SAQ1: Personal and family income; (b) SAQ2: Number of sex partners, frequency of sex, etc.; (c) SAQ3: Masturbation; and (d) SAQ4: Oral sex, anal sex, paid sex, drug use, etc. (separate male and female versions). Interviewers were instructed to hand the SAQs to respondents (Rs) at the appropriate points in the interview. On completion of each form, respondents were instructed to place it in a privacy envelope which was sealed at the end of the interview. The instruction preceding each SAQ stated "If the R cannot read and asks for your help, you may assist him/her in filling out this form" (e.g., Laumann et al., 1994, p. 668). Interviewers then completed the following check item for each SAQ:(1)

R filled out SAQ                1
Interviewer assisted with SAQ   2
R refused SAQ                   3

We use this classification for examining response to individual SAQs. We also examined the completion pattern across the four SAQs. Finding no other clear pattern across the four in preliminary analysis, we created a combined measure with the following three categories: (a) R did all SAQs, (b) R did some SAQs, and (c) R did no SAQs. Note that this last category includes both those respondents who refused all tour SAQs and those who had the interviewer complete all tour for them. In terms of the capacity hypothesis, we assumed that those respondents who completed any of the four SAQs to be capable of completing an SAQ. On the other hand, for the motivation hypothesis, we consider those who completed all four SAQs to be highly motivated. In other words, we expected the capacity measures to be more closely associated with the third category of the combined variable, and the motivation measures to be associated with the first category. For the sensitivity hypothesis, which is more likely to depend on the nature of the questions, we expected to see differential completion (some but not all SAQs). However, we also examined individual SAQs to explore the sensitivity issue.

RESULTS

We posed three general questions. First, who does and who does not self-complete SAQ items? Second, why do some self-complete and others do not? We examined these two questions together in the next section. The final question was what effect does self-completion versus interviewer completion have on the data? In other words, would different conclusions be reached if these cases were included in or excluded from the analyses?

We first examined the overall patterns of completion for the four SAQs. These are presented in Table 1. In each of the SAQs, at least 11% of respondents sought interviewer help in completing the self-administered items. Furthermore, between 4% and 16% did not complete the SAQ at all. This is consistent with the range of past findings on SAQ nonresponse. Across all four SAQs, 17.2% of respondents sought assistance from the interviewer at some point, while 3.3% of the sample provided no information on any SAQ.

Table 1. Completion of the Four Self-Administered Questionnaires

                       Who completed SAQ

       Respondent   Interviewer    Neither or    Total
         only       assistance    not reported

SAQ1     79.7          15.0           5.3        100.0%
SAQ2     82.3          13.3           4.4        100.0%
SAQ3     79.6          11.6           8.8        100.0%
SAQ4     73.4          11.0          15.6        100.0%

Note. The unweighted n for all row is 3,432.

Combining these variables into a single measure produced the following distribution: (a) 65.7% completed all SAQs without assistance, (b) 19.8% completed some SAQs without assistance, and (c) 14.6% completed no SAQ without interviewer assistance. This combined variable confounds refusals with interviewer assistance, but is sufficient for our purposes. The middle group is dominated by refusals (70% of those who did some SAQs sought no help on any SAQ), while the last group is dominated by helpseekers (77% of those who did no SAQs themselves sought interviewer assistance, while the balance refused to do any SAQs). We used this composite variable in the next few sections to explore the extent to which various types of respondents self-completed all SAQs.

Who Self-Completes and Why?

Capacity. We have only two indicators of capacity in the NHSLS data, education and age. The bivariate relationships between these two variables and respondent completion of the SAQs are presented in Table 2. Contrary to our expectation, we found a curvilinear effect of education, with the proportion self-completing all SAQs peaking at some college, then declining slightly.

Table 2. Self-Completion of SAQs by Capacity Indicators

                              All   Some   None   (unweighted n)

Education
  Less than 12th grade (%)   54.2   23.5   22.3         (530)
  High school (%)            65.0   19.9   15.1       (1,007)
  Some college (%)           69.8   19.1   11.1       (1,115)
  Completed college (%)      69.8   17.3   12.8         (541)
  Graduate school (%)        64.7   19.8   15.6         (239)

[chi square] = 36.0, d.f. = 8, p [is less than] .001

Age
  18-24 (%)                  69.7   20.7    9.6         (558)
  25-34 (%)                  65.9   18.6   15.5       (1,070)
  35-44 (%)                  65.8   20.1   14.1         (947)
  45-54 (%)                  62.9   19.8   17.3         (595)
  55-59 (%)                  61.8   20.9   17.3         (262)

[chi square] = 15.4, d.f. = 8, n.s.

Because the NHSLS limited the sample to persons between the ages of 18 and 59 years, it is not surprising that age was unrelated to completion. The bivariate results show a monotonic but nonsignificant (p [is greater than] .05) decline in self-response (the "All" category) with age.

Motivation. Table 3 presents SAQ completion by various indicators of reluctance. Those respondents who showed greater initial reluctance (i.e., the case was transferred to another interviewer, completed by telephone, required an incentive, etc.) were less likely to self-complete the SAQs.

Table 3. Self-Completion of SAQs by Indicators of Initial Reluctance

Case transferred to another
 interviewer
  Yes (%)
  No (%)

                                   [chi square] = 22.6, d.f. = 2,
                                       p [is less than] .001

Telephone conversion of case
  Yes (%)
  No (%)

                                   [chi square] = 51.7, d.f. = 2,
                                       p [is less than] .001

Fee paid to respondent
  Yes (%)
  No (%)

                                   [chi square] = 17.8, d.f. = 2,
                                       p [is less than] .001

Persuasion letter sent
  Yes (%)
  No (%)

                                   [chi square] = 1.4, d.f. = 2,
                                       n.s.

Interviewer rating of difficulty
 of gaining cooperation
  Very easy or easy (%)
  All other (%)

                                   [chi square] = 19.0, d.f. = 2,
                                       p [is less than] .001

Case number for interviewer
  Late cases (10+) (%)
  All other (%)

                                   [chi square] = 27.6, d.f. = 2,
                                       p [is less than] .001

Number of contract attempts made
  Total attempts (mean)

                                   F = 12.65, d.f. = 2,
                                       p [is less than] .01

                                   All     Some

Case transferred to another
 interviewer
  Yes (%)                          52.7    19.8
  No (%)                           68.1    19.7

Telephone conversion of case
  Yes (%)                          45.7    14.0
  No (%)                           68.0    20.4

Fee paid to respondent
  Yes (%)                          61.9    21.8
  No (%)                           71.3    16.8

Persuasion letter sent
  Yes (%)                          63.4    19.1
  No (%)                           66.1    19.9

Interviewer rating of difficulty
 of gaining cooperation
  Very easy or easy (%)            68.9    19.1
  All other (%)                    59.8    20.9

Case number for interviewer
  Late cases (10+) (%)             62.0    19.0
  All other (%)                    71.3    21.0

Number of contract attempts made
  Total attempts (mean)             6.91    7.12

                                   None    (unweighted n)

Case transferred to another
 interviewer
  Yes (%)                          27.5         (515)
  No (%)                           12.1       (2,644)

Telephone conversion of case
  Yes (%)                          40.3         (335)
  No (%)                           11.6       (3,097)

Fee paid to respondent
  Yes (%)                          16.3       (2,060)
  No (%)                           12.0       (1,372)

Persuasion letter sent
  Yes (%)                          17.5         (495)
  No (%)                           14.0       (2,664)

Interviewer rating of difficulty
 of gaining cooperation
  Very easy or easy (%)            12.0       (2,205)
  All other (%)                    19.3       (1,227)

Case number for interviewer
  Late cases (10+) (%)             19.0       (2,049)
  All other (%)                     7.8       (1,383)

Number of contract attempts made
  Total attempts (mean)            12.43      (3,432)

We found similar results for indicators of lack of engagement in the interview. The three interviewer ratings were all highly skewed, so they were collapsed into dichotomies. As shown in Table 4, all three indicators of engagement in or commitment to the interview related significantly to the likelihood of self-completing the SAQs. Specifically, the more positive the interviewer's rating of the respondent's performance (recorded after completion of the interview), the more likely the respondent was to self-complete the SAQs.

Table 4. Self-Completion of SAQs by Indicators of Lack of Engagement

                                    All    Some   None

Respondent comprehension
  Excellent, good (%)               67.4   19.4   13.2
  Other (%)                         47.3   23.5   29.2

02 = 17.1, d.f. = 2, p [is less than] .001

Respondent's cooperation
  Very cooperative (%)              68.0   19.2   12.8
  Other (%)                         53.1   22.9   24.1

02 = 33.1, d.f. = 2, p [is less than] .001

Frankness of respondent's answers
  Entirely frank (%)                68.5   18.7   12.8
  All other (%)                     58.9   22.3   18.8

02 = 15.3, d.f. = 2, p [is less than] .001

                                    (unweighted n)

Respondent comprehension
  Excellent, good (%)                  (3,148)
  Other (%)                              (248)

02 = 17.1, d.f. = 2, p [is less than] .001

Respondent's cooperation
  Very cooperative (%)                 (2,897)
  Other (%)                              (535)

02 = 33.1, d.f. = 2, p [is less than] .001

Frankness of respondent's answers
  Entirely frank (%)                   (2,417)
  All other (%)                        (1,015)

02 = 15.3, d.f. = 2, p [is less than] .001

Sensitivity. The interviewer-administered portion of the NHSLS contained a series of 14 items on the appeal of various sexual behaviors, including oral sex, anal sex, group sex, and so forth. An index of sensitivity was created by summing the number of times respondents declined to answer these questions. Given the skewness of this distribution, we collapsed this into any item not answered versus all items answered. The cross-tabulation of this variable with the combined SAQ completion indicator is presented in Table 5. The expectation that those who were unwilling to provide answers on these sensitive questions to interviewers may be more willing to self-complete the SAQs because of the sensitive nature of the information being sought was not supported. In fact, those who declined to answer one or more sensitive interviewer-administered items were less likely to self-complete all SAQs, although the effect is not statistically significant (p [is greater than] .05).

Table 5. Self-Completion of SAQs by Indicators of Sensitivity

Number of items answered
  One or more not answered (%)
  All answered (%)

                                       [chi square] = 3.84,
                                          d.f. = 2, n.s.

Lack of appeal of sexual behaviors
  Mean
  (s.e.)

                                       F = 13.09, d.f. = 2,
                                        [is less than] .01

Number of sex partners in past years
  None (%)
  One (%)
  Two or more (%)
  Missing (%)

                                       [chi square] = 40.5, d.f.
                                       = 6, p [is less than] .01

Presence of others during interview
  Yes (%)
  No (%)

                                       [chi square] = 3.28,
                                          d.f. = 2, n.s.
Respondent's gender
  Male (%)
  Female (%)

                                       [chi square] = 1.47,
                                          d.f. = 2, n.s.

                                       All      Some

Number of items answered
  One or more not answered (%)         56.3     25.0
  All answered (%)                     66.1     19.5

Lack of appeal of sexual behaviors
  Mean                                  3.08     3.07
  (s.e.)                               (0.02)   (0.02)

Number of sex partners in past years
  None (%)                             61.3     23.7
  One (%)                              69.7     18.8
  Two or more (%)                      64.7     23.8
  Missing (%)                          15.7     14.7

Presence of others during interview
  Yes (%)                              66.0     22.9
  No (%)                               65.6     19.3

Respondent's gender
  Male (%)                             66.2     18.8
  Female (%)                           65.1     20.7

                                       None     (unweighted n)

Number of items answered
  One or more not answered (%)         18.6          (120)
  All answered (%)                     14.4        (3,312)

Lack of appeal of sexual behaviors
  Mean                                  3.20       (3,432)
  (s.e.)                               (0.02)

Number of sex partners in past years
  None (%)                             15.1          (408)
  One (%)                              11.5        (2,353)
  Two or more (%)                      11.5          (517)
  Missing (%)                          69.6          (154)

Presence of others during interview
  Yes (%)                              11.1          (410)
  No (%)                               15.1        (3,022)

Respondent's gender
  Male (%)                             14.9        (1,511)
  Female (%)                           14.2        (1,921)

We also examined respondents' actual answers to the same 14 items. The assumption was that those who admitted to finding fewer "disapproved" sexual behaviors appealing may be less likely to self-complete the SAQs, as they have less to hide. On the other hand, they could be unwilling to admit engagement in such behaviors to interviewers, but may indeed be willing to acknowledge similar behaviors in an SAQ. We thus created an index of those 14 items, rescaled to the original response categories ranging from 1 (very appealing) to 4 (not at all appealing), so that a high score indicates greater sensitivity. Here we found that those who self-administered none of the SAQs reported significantly (p [is less than] .01) more behaviors to be unappealing than those who self-administered all or some of the SAQs.

A further indicator of sensitivity may be the number of sex partners reported by respondents in the interviewer-administered portion of the interview. We assumed that those in a monogamous relationship or those with no sex partners would find reporting the self-administered items to an interviewer to be less sensitive. As seen in Table 5, we found mixed results, with those reporting one partner being more likely to self-administer all SAQs than those reporting either no partners or more than one partner. However, it appears that the statistical significance of this measure is largely driven by the missing cases. Item-missing data in the interviewer-administered portion of the interview appears to be associated with non-self-completion of the SAQs, which again suggest motivational issues more so than sensitivity.

In about 12% of the interviews, another person (spouse, other adult, or child over 6) was present for all or part of the interview. We found no significant effect of the presence of others on the likelihood of self-completion (see Table 5). Finally, we also found no significant effect for respondent gender on self-completion of the SAQs.

Combined Analyses. We included all of the variables described above in a multinomial logistic regression predicting the likelihood of respondents completing all versus some and versus none of the SAQs themselves. As noted earlier, self-completion of some (but not all) SAQs may suggest greater sensitivity depending on the topic, whereas the capacity hypothesis would predict non-self-completion of all SAQs. We thus needed to examine these categories of the dependent variable separately. We also fitted a model for SAQ3 and SAQ4 (containing the most sensitive items) respectively. These results parallel those for the combined indicator for completion of all SAQs, and are not discussed further here.

The final model presented in Table 6 is the result of several model-trimming steps. We first ran separate models on the variables for each of the three sets of hypotheses (capacity, motivation, and sensitivity). Several variables remained statistically nonsignificant in both bivariate and multivariate analyses, and were dropped from the final model. In addition, variables failing to reach traditional levels of significance (p [is less than] .05) in any of the separate models tested were also dropped from the final model. The multinomial logistic regression coefficients, standard errors, and odds ratios for the remaining variables are presented in Table 6.

Table 6. Multinomial Logistic Regression of Self-Completion of SAQs

                                    None vs. All

                           coeff.         s.e.      O.R.

Constant                   -0.065        (0.72)
Education
  Less than 12th grade      0.24         (0.24)     1.27
  High school              -0.36         (0.32)     0.70
  Some college             -0.61(**)     (0.22)     0.54
  Completed college        -0.63(*)      (0.29)     0.53
  Graduate school             --
Case transferred(a)         0.92(**)     (0.23)     2.52
Phone conversion(a)         1.39(**)     (0.21)     4.01
Fee paid(a)                 0.045        (0.20)     1.05
Late case indicator(a)      0.89(**)     (0.21)     2.43
Total attempts              0.0099(**)   (0.0033)   1.01
R comprehension good or
 excellent(a)              -1.08(*)      (0.30)     0.34
Lack of appeal of sexual
 behaviors                  0.62(**)     (0.14)     1.86
Number of sex partners
  None                     -3.37(**)     (0.31)     0.034
  One                      -3.55(**)     (0.31)     0.029
  Two or more              -3.24(**)     (0.31)     0.039
  Not reported                --

                                    Some vs. All

                           coeff.         s.e.      O.R.

Constant                    0.55         (0.59)
Education
  Less than 12th grade      0.24         (0.24)     1.27
  High school              -0.022        (0.24)     0.98
  Some college             -0.12         (0.21)     0.89
  Completed college        -0.27         (0.28)     0.77
  Graduate school             --
Case transferred(a)         0.28         (0.20)     1.33
Phone conversion(a)        -0.12         (0.20)     0.89
Fee paid(a)                 0.42(**)     (0.13)     1.53
Late case indicator(a)     -0.11         (0.11)     0.90
Total attempts              0.0006       (0.0035)   1.001
R comprehension good or
 excellent(a)              -0.44(*)      (0.20)     0.64
Lack of appeal of sexual
 behaviors                 -0.089        (0.10)     0.91
Number of sex partners
  None                     -1.05(*)      (0.44)     0.35
  One                      -1.35(**)     (0.39)     0.26
  Two or more              -1.11(*)      (0.42)     0.33
  Not reported                --

(a) 1 = Yes.

(*) p < .05.

(**) p < .01.

As found in the bivariate analysis, age had no discernible effect on completion of the SAQs. This is consistent with expectation, given the truncated age range of the NHSLS sample. Education remained statistically significant (Wald F = 3.6, p [is less than] .01) in the multivariate model, suggesting capacity is a factor in the decision to self-complete.

Similarly, several of the indicators of reluctance were significant predictors of the self-completion decision, controlling for other factors. However, the lack of engagement measures (based on interviewer reports following the interview) do not appear to be reliable predictors of willingness to self-complete. The single exception is the rating of the respondent's comprehension of the survey, which may suggest a capacity measure rather than motivation.

Finally, the two measures of sensitivity remained significantly related to self-completion. While arguments could be made that capacity and motivation can be considered ignorable nonresponse--to use Rubin's (1987) terminology--if sensitivity to questions about sexual behavior, attitudes, and experiences is related to the willingness to complete SAQs on these topics, this suggests the presence of nonignorable nonresponse. In other words, the missing data on these sets of questions appear to be related to the variables of interest.

What Effect Does This Have on Estimates?

The results thus far indicate that a subset of respondents did not self-complete the SAQs. The question that then arises is whether this phenomenon affects the estimates obtained from the SAQ data. To address this question, we examined SAQ responses for both self-completers and interviewer-assisted completers to determine whether any differences exist. To elaborate, we focused on SAQ3 and SAQ4, which contained the most sensitive questions.

The third SAQ in the NHSLS contained a series of items on masturbation (Laumann et al., 1994). Table 7 presents the percentages of respondents reporting various activities on this SAQ by who completed the SAQ. The results are generally in the expected direction: Those who sought interviewer assistance reported lower levels of engagement in these sensitive behaviors than those who self-completed the SAQ.

Table 7. Responses to Masturbation Items by Who Completed SAQ, Percentages

                               Who completed SAQ

                           Respondent     Interviewer
                              only        assistance    Total

                    Frequency of masturbation

Men
  Once a week or more           26.2          11.5        24.3
  Other                         36.8          41.9        37.4
  Not at all                    37.0          46.6        38.6
Total                          100.0         100.0       100.0
(n)                           (1,173)        (164)      (1,337)

    [chi square] = 20.3, d.f. = 2, p < .01

Women
  Once a week                    6.9           4.3         6.5
  Other                         33.7          20.7        32.2
  Not at all                    59.4          74.9        61.3
Total                          100.0         100.0       100.0
(n)                           (1,521)        (215)      (1,736)

    [chi square] = 12.2, d.f. = 2, p < .01

Among those who reported
masturbating:

       Always or usually have orgasm during masturbation

Men (%)                         89.3          83.9        88.8
(n)                             (758)         (87)        (845)

    [chi square] = 1.3, d.f. = 1, n.s.

Women (%)                       75.6          63.9        74.6
(n)                             (653)         (63)        (716)

    [chi square] = 3.04, d.f. = 1, n.s.

    Always or usually have feeling guilty after masturbation

Men (%)                         60.6          28.9        57.2
(n)                             (748)         (87)        (845)

    [chi square] = 15.9, d.f. = 2, p < .01

Women (%)                       53.2          34.0        51.7
(n)                             (653)         (63)        (716)

    [chi square] = 5.0, d.f. = 1, p < .01

While there are statistically significant differences in response to several

of these items by self-versus interviewer completion, the overall effects are not large (comparing the respondent-only column to the total column). This is largely due to the relatively small proportion of those seeking interviewer assistance on these items. Thus, while those who sought interviewer assistance in the completion of this SAQ exhibit very different behaviors, overall estimates are not seriously biased.

SAQ4 contained a series of items on sexual practices (see Laumann et al., 1994). Table 8 presents these items by who completed the SAQ. These results generally mirror those found in Table 7 above. It is interesting to note that the effects appear larger for women than for men (as indicated by the percentage differences between the two groups, and the statistical significance of these relationships). This is somewhat surprising given that almost all interviewers were women.

Table 8. Percentage Reporting Selected Sexual Practices, by Who Completed SAQ

                               Who completed SAQ

                            Respondent   Interviewer
                              only       assistance    Total

Active oral sex

Men                           74.9         62.6        73.3
  [chi square] = 5.3,
  d.f.= 2, n.s.

Women                         66.9         48.0        64.5
  [chi square] = 17.3,
  d.f. = 2, p < .01

Reception oral sex

Men                           75.9         68.2        74.8
  [chi square] = 3.5,
  d.f. = 2, n.s.

Women                         71.9         54.4        69.7
  [chi square] = 9.9,
  d.f. = 2, p < .05

Active anal sex: Men          23.6         19.3        23.0
  [chi square] = 1.5,
  d.f. = 2, n.s.

Receptive anal sex: Women     19.3         10.9        18.3
  [chi square] = 8.8,
  d.f. = 2, p < .05

The percentages in Table 8 suggest differences in the reporting of these sexual practices by self-administration or interviewer-assisted completion of the SAQ. For men, none of the three differences reach statistical significance (p [is greater than] .05), while all are significant for women. The effect of interviewer assistance is to lower the estimates of engagement in the practice (comparing the "Respondent only" column to the "Total" column in Table 8). However, given that the proportion of respondents who sought interviewer assistance on this SAQ was small, the effect of the differences on the overall estimates is not large.

Incidentally, the estimates in the "Total" column in Table 8 are all lower than those reported in Laumann et al. (1994), which are based on all SAQ responses (whether self-completed or not) and on the unweighted cross-sectional sample only. For example, their estimates for active oral sex were 76.6% for men and 67.7% for women, while for receptive oral sex they were 78.7% for men and 73.1% for women. Their estimates for anal sex were 25.6% and 20.4% for men and women respectively. This suggests that the effect of excluding the oversample (as they do) is to inflate the estimates of various sexual practices, while including the self-completers (again as they do) reduces the estimates.

DISCUSSION AND CONCLUSIONS

The analyses presented here suggest that there are systematic differences in the likelihood of self-completion of SAQs by respondents. We found some support for the capacity hypothesis, suggesting that completion of a self-administered questionnaire is determined in part by the reading ability or literacy of the respondent. While the effect is not large in a national survey such as this, it may be a potential problem in surveys of certain subgroups (i.e., those with low literacy).

We found stronger support for the motivation hypothesis. Respondents who were initially reluctant appeared less likely to self-complete the SAQs. Efforts to reduce nonresponse error through increasing response rates may increase measurement error, as reluctant respondents may be less motivated than those who are initially willing to participate (see Couper, 1997). The relationship between nonresponse and measurement error deserves further research attention. The curvilinear effect of education on self-completion may also suggest that capacity constraints (i.e., literacy) operate at the lower education levels, but motivation may be a greater concern at the higher education levels.

We found little support for the sensitivity hypothesis in our analyses. This is an encouraging finding, in that respondents' decisions to self-complete do not appear to be related to the behaviors and attitudes measured in the SAQs themselves. While the content of the SAQ may affect nonresponse (refusal to do the SAQ in toto), it does not appear to affect whether the respondent chooses to self-complete or seek interviewer assistance in completing the SAQ.

Our findings parallel those reported by Couper and Rowe (1996) for a computerized self-administered instrument, suggesting that the problem of non-self-completion is common to both paper-and-pencil and computerized instruments. In the last few years, the development of audio computer-assisted self-interviewing (audio-CASI) has shown great promise for the collection of sensitive information on a variety of topics (see Tourangeau et al., 1997; Turner et al., 1998). While audio-CASI is likely to minimize the effect of literacy on the decision to self-complete, other factors, such as experience with or knowledge of computers, may be manifest. To date, most audio-CASI applications have been with younger people (those with greater experience of computers), and the issue of who completed the self-administered items has not yet been addressed. Further, the use of audio-CASI is unlikely to have much effect on the motivational factors, suggesting that the issues raised here may still be of concern in audio-CASI.

We also found evidence that self- versus interviewer completion is associated with different responses to SAQ items. This is consistent with the argument for using SAQs in the first place. While the overall effect is not large (given the relatively small proportions of those who chose not to self-complete), the impact on estimates may be more pronounced in subgroup analyses.

To summarize, there is ample empirical evidence from a number of studies that self-administration increases the overall reporting of traditionally underreported behaviors. Our results suggest it is important to explore ways to maximize self-completion by respondents in order to reap the full benefit of this mode of administration. While audio-CASI may mitigate the effect of literacy on the self-completion decision, we need to find ways to address the motivational component.

Our findings also raise questions about the relationship between nonresponse and measurement error. While those who are unable or reluctant to do the SAQs at least provide answers to the questions (i.e., they are not nonresponders), the answers they give appear to differ when they respond to interviewer administration of these items (measurement error). At the least, these findings suggest that we can identify those at risk for non-self-completion of the SAQs (e.g., those with low education levels, those who were initially reluctant to participate) early in the interview. A variety of strategies could be targeted to these respondents to increase the likelihood of self-completion. This may include a variety of motivational strategies (e.g., Cannell, Miller, & Oksenberg, 1981), as well as incentives.

Finally, our results suggest that who completes the SAQs is an important survey quality indicator, similar to item-missing data rates or nonresponse rates. The routine collection of such indicators could be used for a variety of purposes, ranging from interviewer evaluation to statistical adjustment to compensate for non-self-completion. Our work does not raise serious questions about the quality of data collected from SAQs, but certainly suggests several fruitful lines of further enquiry into this and related methods for improving reports of sensitive information.

(1) The check item variables were not included in the ICPSR release of the data. We are grateful to NORC for providing these variables to us in a form that could be merged to the substantive data from the NHSLS.

REFERENCES

Aquilino, W. S. (1992). Telephone versus face-to-face interviewing for household drug use surveys. International Journal of the Addictions, 27, 71-91.

Aquilino, W. S. (1994). Interview mode effects in surveys of drug and alcohol use: A field experiment. Public Opinion Quarterly, 58, 210-240.

Aquilino, W. S. (1998, May). Effect of third party presence on self-reported drug and alcohol use. Paper presented at the annual meeting of the American Association for Public Opinion Research, St. Louis, Mo.

Bradburn, N. M., Sudman, S., Blair, E., & Stocking, C. (1978). Question threat and response bias. Public Opinion Quarterly, 42, 221-234.

Cannell, C. F., Miller, P. V., & Oksenberg, L. (1981). Research on interviewing techniques. In S. Leinhardt (Ed.), Sociological methodology 1981. San Francisco: Jossey-Bass

Catania, J. A., Binson, D., Cantle, J., Pollack, L. M., Hauck, W., & Coates, T. J. (1996). Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Public: Opinion Quarterly 60, 345-375.

Catania, J. A., Binson, D., van der Straten, A., & Stone, V. (1995). Methodological research on sexual behavior in the AIDS era. Annual Review of Sex Research, 6, 77-125.

Catania, J. A., Gibson, D. R., Chitwood, D. D., & Coates, T. J. (1990). Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior. Psychological Bulletin, 108, 339-362.

Catania, J. A., McDermott, L. J., & Pollack, L. M. (1986). Questionnaire response bias and face-to-face interview sample bias in sexuality research. The Journal of Sex Research, 22, 52-72.

Couper, M. P. (1997). Survey introductions and data quality. Public: Opinion Quarterly, 61, 317-338.

Couper, M. P., & Rowe, B. (1996). Evaluation of a computer-assisted self-interview (CASI) component in a CAPI survey. Public Opinion Quarterly, 60, 89-105.

Johnson, W., & DeLamater, if. (1976). Response effects in sex surveys. Public Opinion Quarterly, ,10, 165-181.

Kirsch, I. W., Jungeblut, A., Jenkins, L., & Kostad, A. (1993). Adult literacy in America. Washington, DC: National Center for Education Statistics.

Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.

Laumann, E. O., Gagnon, J. H., Michael, R. T., & Michaels, S. (1994). The social organization of sexuality: Sexual practices in the United States. Chicago: University of Chicago Press.

Laumann, E. O., Gagnon, J. H., Michael, R. T., & Michaels, S. (1995). National Health and Social Life Survey, 1992] Computer file; ICPSR version]. Chicago: University of Chicago and National Opinion Research Center [Producer]. Ann Arbor, MI: Inter-University Consortium for Political and Social Research [Distributor], 1995.

London, K. A., & Williams, L. B. (1990, May). The use of a self-administered questionnaire for improving abortion reporting in the 1988 National Survey of Family Growth. Paper presented at the annual meeting of the Population Association of America, Toronto, Canada.

Michael, R. T., Gagnon, J. H., Laumann, E. O., & Kolata, G. (1994). Sex in America: A definitive study. Boston: Little, Brown.

Reuband, K.-H. (1992). On third persons in the interview situation and their impact on responses. International Journal of Public Opinion Research, 4, 269-274.

Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.

Schober, S., Fe Caces, M., Pergamit, M., & Branden, L. (1992). Effect of mode of administration on reporting of drug use in the National Longitudinal Survey. In C. F. Turner, J. T. Lessler, & J. C. Gfroerer (Eds.), Survey measurement of drug use: Methodological studies. Rockville, MD: National Institute on Drug Abuse.

Shah, B. V., Barnwell, B. G., & Bieler, G. S. (1996). SUDAAN user's manual: Release 7.0. Research Triangle Park, NC: Research Triangle Institute.

Smith, T. W. (1992). A methodological analysis of the sexual behavior questions on the General Social Survey. Journal of Official Statistics, 8, 309-325.

Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.

Tourangeau, R., Rasinski, K., Jobe, J. B., Smith, T. W., & Pratt, W. F. (1997). Sources of error in a survey on sexual behavior. Journal of Official Statistics, 13, 341-365.

Turner, C. F, Lessler, J. T., & Devore, G. (1992). Effects of mode of administration and wording on reporting of drug use. In C. F. Turner, J. T. Lessler, & J. C. Gfroerer (Eds.), Survey measurement of drug use: Methodological studies. Rockville, MD: National Institute on Drug Abuse.

Turner, C. F., Lessler, J. T, & Gfroerer, J. C. (Eds.) (1992). Survey measurement of drug use: Methodological studies. Washington, DC: U.S. Government Printing Office.

Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use and violence: Increased reporting with computer survey technology. Science, 280, 867-873.

Wiederman, M. W. (1993). Demographic and sexual characteristics of nonresponders to sexual experience items in a national survey. The Journal of Sex Research, 30, 27-35.

Manuscript accepted April 14, 1999

An earlier version of this paper was presented at the annual meeting of the American Association for Public Opinion Research, May 1997. The NHSLS data were obtained through the Inter-University Consortium for Political and Social Research. We are grateful to Stuart Michaels and Fay Booker of NORC for providing the additional data needed for the analyses reported here. The views expressed here are the sole responsibility of the authors and do not represent the official views of the Bureau of Labor Statistics. We thank the editor and reviewers for their very helpful comments on earlier drafts of this paper.

Address correspondence to Mick Couper, Survey Research Center, P.O. Box 1248, Ann Arbor, MI 48103; e-mail: [e-mail].

Mick P. Couper University of Michigan and University of Maryland

Linda L. Stinson U.S. Bureau of Labor Statistics, Washington D.C. and University of Maryland