"Talking" computers: a reliable and private method to conduct interviews on sensitive topics with children.

Children in pre- and early adolescence are increasingly becoming the focus of interventions to prevent sexually transmitted diseases (STDs) as well as unwanted pregnancy (Brooks-Gunn & Paikoff, 1993; DiClemente, 1990; Hein, 1992). Designing interventions for children requires knowledge about behaviors that jeopardize children's health and knowledge about children's beliefs and attitudes concerning those behaviors. The face-to-face (FTF) interview has been an important tool in conducting this research. However, many behaviors related to sexual risks involve sensitive topics that people find embarrassing or inappropriate to disclose in public, even if their responses are confidential (Catania, Gibson, Chitwood, & Coates, 1990; Craig, 1988; Midanik, 1988). This barrier to obtaining information is even more imposing when the respondents are children. Adults can be given self-administered questionnaires for the segments of the interview that are most sensitive (Turner, Lessler, & Devore, 1992). However, children with widely varying reading skills may require assistance in completing such instruments and may respond differently, depending on their comprehension of questions. Thus, self-administered questionnaires may not be an appropriate option for children.

One solution to the problem of conducting private interviews with children is to perform interviews over the telephone. Telephone interviews may reduce some embarrassment of public disclosure without requiring self-administration of sensitive questions (Catania et al., 1990; Lavrakas, 1993). Indeed, telephone interviews have been used to interview adolescents concerning sexual risks (Catania et al., 1992; Finkelhor & Dziuba-Leatherman, 1994; Hingson, Strunin, Berlin, & Heeren, 1990). However, telephone interviewing links the recruitment methodology (telephone subscriber ship) with the interviewing methodology. Many low-income children at high risk for STDs and pregnancy may not have telephones or may be more effectively readied in other neighborhood sites (e.g., health clinics, recreation centers, public housing, schools). By personally recruiting children where they live or spend time, interviewers may be more able to identify high-risk respondents and to assist them in completing interviews (cf. Coyle, Boruch, & Turner, 1991).

The ideal interview methodology would permit the flexibility of the FTF interview for recruiting and assisting respondents while at the same time giving respondents the privacy of a self-administered questionnaire. One solution is to administer questionnaires by computers that are programmed to present questions using the same rules that an interviewer would follow but at the same time avoiding public disclosure of the answers. Computer administration has been found to increase the reporting of socially sensitive information (Kiesler & Sproull, 1986), including adolescent sexual experience (Millstein & Irwin, 1983). However, this mode of presentation shares the same limitation as self-administered questionnaires. Respondents who differ greatly in cognitive development or cultural background or who have difficulty following written instructions may require oral presentation to reduce differences in comprehension and interpretation.

One way to overcome the limitations of computer administration is to use portable "talking" computers that can present the interview over earphones and record responses confidentially (Johnston, 1992; Romer et al., 1994). Talking computers can retain the privacy of self-administered questionnaires while not requiring respondents to read large amounts of material or to expose their answers in public. Johnston (1992) showed that talking computers are feasible for interviewing adolescents in their homes about drug use. In this report, we provide the first evidence that using talking computers to interview children across a wide developmental range elicits more reporting of sexual behavior and feelings than do FTF interviews.

Our initial purpose in this research was to develop and test an AIDS-prevention intervention for Black children living in concentrated poverty settings (e.g., public housing). Youth in these settings are at increased risk for early pregnancy (Crane, 1991; Mayer, 1991) and exposure to HIV and other STDs (CDC, 1993; Geringer, Marks, Allen, & Armstrong, 1993). To identify the prevalence and correlates of risky sexual behavior, we used talking computers to interview children across the age range of 9 to 15. We found higher rates of sexual activity than national surveys of Black youth would indicate (Romer et al., 1994), as well as relatively favorable feelings toward engaging in sex (Stanton et al., 1993). Although these findings suggested that the high-risk settings in which our participants lived were partly responsible for these elevated reports of sexual behavior, it was possible that the higher rates were also the result of our use of the more private computer interview mode.

In this research, we asked if children in the same high-risk settings would report less sexual activity and less favorable feelings toward sex if they were interviewed in FTF mode. If FTF interviewing by an adult inhibits the acknowledgment of sensitive behavior and feelings, then children should be less willing to report previous sexual activity and to acknowledge relatively favorable feelings toward engaging in sex than was found in the computer interview. To test this hypothesis, we compared answers to questions concerning sexual experience and feelings administered by computer with answers obtained from an FTF interview conducted with a second sample of children in similar public housing sites. In addition, because the FTF sample was recruited a short time following the initial computer sample, we interviewed some children who had previously completed the computer interview. This subsample enabled us to test the hypothesis in the game children. A subset of other children completed the computer interview a second time, and this subsample enabled us to see if merely reinterviewing children elicits different levels of sexual behavior and feelings.

Method

Participants

The target audience for the prevention project was children between the ages of 9 and 15 who lived in 6 large public housing sites in a large U.S. city. The housing sites contained approximately 1,600 Black children in the 9 to 15 age range. Approximately equal numbers of children appeared for the interview at each of the 6 public-housing sites, with 316 children participating 1 about 19% of the total age-appropriate population,. The interview was successfully completed by 300 (95%) of the children who volunteered to take it. Fourteen children's data had to be discarded because of various unforeseen computer malfunctions (e.g., running out of space on the hard drive). Two children failed to enter critical demographic information such as their ages at the beginning of the interview.

The FTF interview was conducted four to six weeks after the initial computer survey in three housing sites. Two sites were among the original six, whereas the third was a new site. Analyses of results from the initial six sites indicated no differences in levels of sexual experience by sites (Li et al., 1994), and so restricting the interviews to three sites should not introduce bias. However, we controlled for site differences in the current analyses.

We recruited 104 children to participate in the FTF interviews, and 96 (92%) children completed them. Eight children's partial data were discarded because the children were unable to complete the interview when it was scheduled. Approximately 72% of the completed interviews were conducted in the original two sites, with the remaining interviews conducted in the new third site.

Measures

An extensive interview containing more than 150 questions about friendship patterns, risk behaviors of friends, and personal risk behavior and attitudes was developed to assist in the planning of the AIDS-prevention intervention. We programmed the interview on a Macintosh microcomputer using Hypercard 2.1 (Claris Corp.). This software permitted us to program (a) the screen that the respondent saw, (b) questions and instructions that the respondent heard over earphones, and (c) storage for the answers that the respondent selected. The software permitted complex branching and slap patterns, depending on the answers that the respondent selected. Separate software (Macromedia Sound System Pro) was used to store a high-quality digitized recording of an interviewer's voice presenting the material. In the current interview, one male voice and one female voice were recorded to present the interview to boys and girls, respectively. Several stages of pretesting were conducted to assure that the interview was understood and could be completed by children in the target population.

The interview was divided into several sections. It began with general questions about how children spend their free time and more specific questions about the friends with whom they spend time. It then asked about various risk activities in which they and their friends may have engaged. It concluded with a set of questions concerning their knowledge of HIV-related risk activities and how to avoid those risks.

Two critical questions that were common to both interviews are the focus of this research. One question asked if the child had "ever had sex with someone." This question was preceded by two questions that asked if the respondent had ever "kissed someone" or "tongue kissed." A later question asked how the child "would feel if (she or he) were to have sex with someone in the next year." Possible responses ranged from (1) very bad, (2) bad, (3) neither good or bad, (4) good, to (5) very good.

The FTF interview contained many of the same items as the computer interview but also contained items concerning the children's personality characteristics. These personality items were administered following the relevant question concerning sexual experience and so should not have affected responses to this question. Nevertheless, the feeling question occurred after the personality items. However, considering the length of the interviews and the large number of questions contained in them, it did not seem likely that the personality items would affect the interpretation of the later sexual feeling question.

Procedure

Our original intention was to conduct FTF interviews with a random sample of children in their homes. However, previous attempts to interview residents of public housing indicated that it was difficult to gain entry to residents' homes. We became concerned that a survey conducted among families willing to allow us into their apartments could result in a sample of less risk-prone children. Therefore, an alternative interview and recruitment strategy was implemented. With this strategy, we obtained the cooperation of the resident councils to announce that we would conduct the interview in a public space (e.g., a recreation room) during specified hours. This procedure reduced the concerns of parents about entering their apartments. In addition, by administering the interview by computer over earphones, we could conduct several interviews simultaneously without sacrificing privacy.

To recruit respondents, we enlisted the assistance of several adult residents of the housing sites who informed parents of the specific times and locations in the buildings where the interview would be administered. In addition, posters and mailed notices were distributed to announce the interview times and locations. Each location was a public space (e.g., recreation room) where caretakers could feel comfortable leaving their children during the interview. We found this procedure to be quite effective in recruiting caretakers and their children to appear at the designated times. Caretakers and respondents consented to the survey after receiving a description of the project, its purpose, and its confidentiality. In addition, each child was given $5.00 for participating. The interview was conducted in spring 1992.

Researchers gave children a brief overview of how to use the computer and helped them enter some basic demographic information to acquaint them with the procedure. For most questions, children simply used a mouse to "click on" answers displayed (as icons) on the monitor (e.g., Yes; or No). For some questions, children typed their answers using the keyboard. A special "Say It Again" icon was always present to allow respondents to repeat a question if it was not fully comprehended on the first exposure. Respondents could choose not to answer questions by selecting a "Next Question" icon. Researchers were constantly present to answer questions when they arose, but otherwise they left respondents alone to complete the interview.

Children learned to use the computer with only a few minutes of instruction. Although the interviews took approximately 45 minutes to complete, respondents appeared to enjoy interacting with the computers and displayed little fatigue or lack of interest as the interview continued. Many children asked the researchers if there would be other opportunities to use the computers in the future.

The FTF interview was conducted in private by an adult researcher who read each question aloud and repeated questions when needed. From one to three children were administered the interview at a time, depending on scheduling needs. However, to increase the confidentiality of the interview, children recorded their answers on their answer sheets. They could record "don't know" if they wished not to answer a question. As with the computer interview, the gender of the interviewer matched that of the respondents.

Results

Table 1 describes the age and gender composition of the two interview samples. The samples did not differ in either age, [chi square] (6) = 2.88, p [is greater than] .25, or gender distributions, [chi square] (1) = .01, p [is greater than] .25. To study the stability of responses within and across interview modes, one subset of 34 respondents in the computer sample was reinterviewed at a later time using the same computer administration (1 to 2 weeks later). These computer tested-only children were randomly selected from the larger sample. In addition, a non-random subset of the FTF sample (n = 31) had also participated in the earlier computer interview This computer FTF group enabled us to look at stability of responding across interview modes.

Table 1 Gender and Age Distributions for the Computer and FTF Samples and the Subsamples Who Received Two Administrations of Either the Computer Tested Only or the Computer-FTF Interviews

                            Sample

           Computer   FTF       Computer-    Computer-
                                Tested Only  FTF
           (N) (%)    (N) (%)   (N) (%)      (N) (%)

Gender
  Male     140(47)    45(47)     17(50)        8(36)
  Female   160(53)    51(53)     17(50)       23(74)
Age
   9        52(17)    20(21)      5(15)       11(35)

  10        55(18)    12(13)      4(12)        4(13)
  11        48(16)    15(16)      3(9)         6(19)
  12        41(14)    15(16)      6(18)        3(10)
  13        46(15)    14(15)     10(29)        5(16)
  14        32(11)    13(14)      3(9)         2(6)
  15        26(9)      7(17)      3(9)         0(10)
Total      300(100)   96(100)    34(100)      31(100)

The computer tested-only subsample mirrored the age, [chi square] (6) = 5.96, p [is greater than] .25, and gender, [chi square](1) = .14, p [is greater than] .25, distribution of the total sample. The computer-FTF subsample was somewhat younger (mean age of 10.9 versus 12.0) than the computer sample and contained a larger proportion of girls (74% versus 50%). In our analyses, we controlled for differences in age and gender to reduce the influence of these sources of variation.

Table 2 contains the percentages Of children reporting sexual experience by the age, gender, and interview mode of respondent. Logistic regression analysis of sexual Experience was conducted using age gender, computer mode, and housing site as predictors. Although older children and boys reported more experience with sex, children interviewed in the computer mode reported more experience with sex than those in the FTF mode, [chi square](1) = 11.70, p [is less than] .01. Differences between housing sites did not contribute to reports independent of the other predictors. [chi square] 61 = 4.70. p [is greater than] .25. As seen in Figure 1. the greater reporting of sexual experience in the computer mode occurred for both boys and girls and across age groups.

Table 2
Percentage of Respondents Reporting Sexual Experience
by Age, Gender, and Mode of Interview Interview Mode

Age        Computer         FTF
         Boys   Girls    Boys   Girls
  9      25.0    3.1     14.3     0.0
  10     28.0   13.3     14.3     0.0
  11     41.7   12.5     40.0     0.0
  12     63.2   13.6     50.0     0.0
  13     76.2   48.0     50.0    25.0
  14     82.4   40.0     60.0    12.5
  15    100.0   83.3    100.0    66.7
Total    55.7   24.4     42.2     9.8
N         140    160       45      51

[Figure 1 ILLUSTRATION OMITTED]

Table 3 contains the mean ratings for reported feelings about engaging in sex in the future. A small percentage of children chose not to answer this question, but this did not vary by interview mode. Linear regression analysis indicated that reported feelings were more favorable for boys and older children. In addition, as seen in Figure 2. responses in the computer mode were more favorable toward sex than in the FTF mode, t(362) = 3.23, p [is less than] . 01. However, the difference between modes was stronger for children over age 10, t(362) = 3.01, p [is less than] .01. Because sexual experience was positively correlated with feelings toward sex (r = .48, p [is less than] .01), it is possible that the results for feelings were redundant with those for sexual experience. However, even after controlling for sexual experience, reported feelings toward sex were more favorable in the computer mode, t(36 1) = 3.07, p [is less than] .01. The interaction with age also remained significant, t(361) = 2.87, p [is less than] .01.

Table 3
Mean Ratings of Feelings Toward Sex by Age, Gender,
and Mode of Interview

                Interview Mode

Age          Computer          FTF
            Boys     Girls   Boys   Girls

   9         2.50     1.43    2.43    1.46
  10         2.52     1.90    2.43    1.25
  11         2.76     1.96    2.43    1.60
  12         3.53     3.00    2.67    2.00
  13         3.75     3.04    3.00    2.00
  14         4.00     2.60    3.60    2.00
  15         4.08     2.92    3.75    2.00
Total        3.15     2.30    2.81    1.76
N             130      153      42      49
% Missing     7.1      4.4     6.7     3.9

[Figure 2 ILLUSTRATION OMITTED]

To assess the stability of responses in our subsamples of children who received two interviews, we compared responses across interviews within each group. Table 4 contains the percentages of children reporting sexual experience across both administrations of the interview and agreement coefficients for each question- Agreement was quite stable in the computer-only condition (.88) but less stable in the computer-FTF condition (.45), although still statistically significant.

Table 4 Reports of Sexual Experience and Feelings Toward Sex and Individual Agreement Across Interviews in the Computer-Tested Only (N = 34) and Computer-FTF (N = 31) Groups

                      Interview 1   Interview 2   Agreement(a)

Computer-Only
  Sexual experience     52.9          47.1           .88(*)
  Feeling rating        2.45          3.00           .74(*)
Computer-FTF
  Sexual experience     19.4          16.1           .45(*)
  Feeling rating        2.03          1.58           .51(*)

(a) Cohen's Kappa for reports of sexual experience and Pearson's R for ratings of feelings

(*) p <. 01

Self reports of engaging in sex were quite common in the computer-tested only group (53%) but less common in the computer-FTF group (19%). This result was attributable to the higher proportion of girls and younger children in the latter group. Logistic regression analysis of sexual experience in the first interview with gender and age held constant indicated no differences between the groups, [chi square] (1) = .19, p [is greater than] .25. A similar analysis of the Second report of sexual Experience with initial report held constant also revealed no difference between groups, [chi square] (1) = .15, p [is greater than] .25.

Table 4 also lists mean reports of feelings toward sex and agreement coefficients across administrations. Agreement tended to be higher in the computer-only group (.74) than in the computer-FTF group (.51), although both were reliably greater than zero.

Figure 3 shows the rating results for each group at each interview. Regression analysis of the first rating holding gender and age constant indicated that feelings toward sex did not differ between the computer-tested only and computer-FTF groups, t(60) = 1. 17, p [is greater than] .25. However, an analysis of the second rating holding initial ratings constant indicated that feelings toward sex were less favorable when the second interview was conducted in the FTF mode, t(61) = 3.94, p [is less than] .01.

[Figure 3 ILLUSTRATION OMITTED]

Separate tests of change scores within each group indicated that feelings in the computer-tested only group were more favorable in the second interview, t(30) = 2.75, p [is less than] .01. In the computer-FTF condition, reported feelings toward sex were less favorable in the FTF interview than in the initial computer interview, t(30) 2.24, p [is less than] .05.

Discussion

The results indicate that a computerized aural interview elicits more reports of sexual experience and favorable feelings toward sex than an FTF interview. Although we did not observe this finding in a true experiment, the two administrations of the interview were conducted with similar children in the same settings. Our statistical controls for site, gender, and age differences added to our confidence that interview mode was responsible for the differences we observed. Although computer interviewing did not elicit more favorable feelings toward sex among 9- and 10-year-old children, it did appear to influence reports of sexual experience across the ages we studied.

Our analysis of children who completed both versions of the interview (computer-FTF group) indicated further support for the role of computer interviewing in reports of feelings toward sex. Children interviewed in the FTF mode reported less favorable feelings than when they were interviewed by computer. This finding stood in contrast to the children who completed the computer version on two occasions (computer-tested only group) but who did not report lower ratings of sex in the second interview. These findings suggest that it was not merely being asked about sexual feelings a second time that produced the less favorable ratings in the FTF interview.

Although the delay between interviews differed across groups, it is unlikely that the less favorable feelings toward sex observed in the second FTF interview were attributable to this difference. The trend for this rating is to increase with age rather than to decrease (Romer et al., 1994). Furthermore, in the computer-only condition, the rating actually became more favorable at the second interview.

Because feelings toward sex were still somewhat more favorable in the computer-tested only than in the computer-FTF group in the first interview, one might be concerned that the differential response in the second interview was the result of some factor related to the initial difference in ratings, rather than to the mode of administration in the second interview. However, it is not possible to account for the differential shift by assuming regression to the mean by the two groups, because this explanation would predict that their scores would converge on the second administration rather than diverge. Further more, our regression analysis controlled for differences in initial ratings (as well as gender and age differences), so that these important initial differences were unlikely to account for the shift that was observed.

We were not able to detect differences in reports of sexual experience in the children who completed both modes of the interview. However, it is also likely that the sample size and the initial level of sexual experience reports were too low in the computer-FTF group to observe a significant decline.

The rates of agreement we observed across administrations of the computer interview were quite high, and non-response rates were comparable to the FTF interview. We could not compare the reliability of the two interview modes; however, the results indicate that computer interviewing produces reliable reports of behavior and feelings (see also Stanton et al., 1993) with no appreciable difference in response rate.

The agreement coefficients for the two questions in the computer-FTF group were lower than in the computer-only group (as would be expected), but they were of similar magnitude, suggesting that item placement within the FTF interview did not dramatically influence the reliability of responses to those questions. Although the behavior question occurred before the personality items and the feeling question after those items. The pattern of reliabilities across modes suggests that item order within the FIT interview is an unlikely explanation for the differences we observed.

The results suggest that the FTF interview inhibited responses even though it was relatively private, in that children recorded their responses non-verbally on their answer sheets. Nevertheless, total privacy could not be assured because the interviewer read the questions and monitored each child's responding. Furthermore, our assertion to children that the answers to the interview were confidential may have been more persuasive when they were entered into a computer than when they were available for public viewing on an answer sheet. Thus, the results suggest that for children across a wide age range and with differing verbal skill, computer interviewing is not only reliable but also more private than FTF interviews for eliciting sensitive information concerning sex.

Our findings are also consistent with the hypothesis that FTF interviewing promotes less truthful (i.e., more Socially desirable answers than computerized interviews (Kiesler & Sproull. 1986: Millstein & Irwin, 1983). However, well as well as previous investigators) did not have a direct measure that could validate the reports of computer interviewing. For example, if we had a measure of STD incidence or other consequence of sexual activity, we would expect that reports of sexual experience in the computer interview would correlate better with STD incidence than reports in the FTF interview. Previous tests of the validity of computer and other private interview methods have been based on the assumption that greater reports of sensitive information were on their face more valid. It remains for future researchers to determine if the enhanced reporting of sensitive information in computer interviews is also more valid.

Although the rates of sexual experience we observed in the FTF interview were lower than those in the computer interview, they were not so low as to suggest that initiation of sex in our high-risk children is comparable to the levels observed in national surveys of Black youth (in FTF interviews). According to a survey of boys aged 15 to 19 conducted in 1988 (Sonenstein, Pleck, & Ku, 1989), nearly 70% of 15-year-old boys reported sexual experience in an FTF interview. However, all 15-year-old boys in our FTF sample reported sexual experience, as did the 15-year-old boys who received the computer interview.

The rates we observed for girls in our FTF interview were also higher than the rates observed by Forrest and Singh (1990) in a national survey in 1988. Black girls between the ages of 15 and 17 in that survey reported sexual experience at the rate of about 50%, whereas about two thirds of 15-year-old girls in the current study reported sexual experience in our FTF interview, and more than 80% did so in the computer interview.

These comparisons with recent national samples suggest that our samples of Black children in public housing exhibit higher rates of sexual experience than other Black children, even when interviewing is done in the FIT mode. Computer interviewing elevates these reports even more. It is likely that the high rates of sexual experience we observed in children under 15 is also the result of both the high-risk environment of our sample and the more private nature of our computer interview methodology. However, it remains for future validity studies to determine if the higher rates of sexual experience reported in computer interviewing are actually more valid or represent overreports of sexual activity. If the latter, then the computer mode may increase the efficiency of interviewing but not its accuracy.

Implications for Future Interview Methodology

The results suggest that a computerized, aural interview is a feasible method to study even young children's sexual behavior and that the method elicits greater reporting of sexually sensitive information than FTF interviews. The children in this study appeared capable of learning to use talking computers very quickly, could complete the lengthy interview without much assistance, and enjoyed the experience. As a result, it was possible to interview several children at once in a public space, thereby reducing parents' concerns about entering their homes. The added ability to maintain privacy in this interview setting made the use of talking computers especially attractive.

As children, ethnic minorities, and other difficult-to-interview persons increasingly become targets of sex education and disease-prevention programs, the need will increase for culturally and developmentally appropriate assessment tools. The talking computer appears to be a viable method for data collection that should be considered for use with any of the many populations who are difficult to study using FTF or telephone interviews or for whom a more private interview might be appropriate.

References

Brooks-Gunn, J., & Paikoff. R. L. 1993. "Sex is a gamble, kissing is a game": Adolescent sexuality and health promotion. In S. G. Millstein. A. C. Petersen. & E. O. Nightingale (Eds.), Promoting the health of adolescents: New directions for the twenty-first century (pp. 180-208). New York: Oxford.

Catania, J., Coates. T. J., Stall, R., Turner. H., Peterson, J., Hearst. N., Dolcini, M. M., Hudes, E., Gagnon, J., Wiley, J., & Groves. R. (1992). Prevalence of AIDS-related risk factors and condom use in the United States. Science, 258, 1101-1106.

Catania, J., Gibson, D. R., Chitwood. D. D., & Coates, T. J. (1990). Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior Psychological Bulletin, 108, 339-362.

Centers for Disease Control and Prevention. (1993). Update: Acquired Immunodeficiency Syndrome--United States. 1992. Morbidity and Mortality Weekly Report, 42, 547-557.

Coyle. S. L., Boruch, R. F., & Turner. C. F,. (Eds.) (1991). Evaluating AIDS prevention programs. Washington, DC: National Academy Press.

Craig, R. (1988). Diagnostic interviewing with drug abusers. Professional Psychology: Research and Practice, 19, 14-20.

Crane, J. (1991). The epidemic theory of ghettos and neighborhood effects on dropping out and teenage childbearing. American Journal of Sociology, 96, 1226-1259.

DiClemente, R. J. (1990). The emergence of adolescents as a risk group for human immunodeficiency virus infection. Journal of Adolescent Research, 5, 7-17.

Finkelhor, D., & Dziuba-Leatherman, J. (1994). Children as victims of violence: A national survey. Pediatrics, 94, 413-420.

Forrest, D., & Singh, J. D. (1990). The sexual and reproductive behavior of American women, 1982-1988. Family Planning Perspectives, 22, 206-214.

Geringer, W., Marks, S., Allen, W. J., & Armstrong, K. A. (1993). Knowledge, attitudes, and behavior related to condom use and STDs in a high risk population. The Journal of Sex Research, 30, 75-83.

Hein, K. (1992). Adolescents at risk for HIV infection. In R. J. DiClemente (Ed.), Adolescents and AIDS: A generation at risk (pp 3-16). Newbury Park, CA: Sage.

Hingson, R. W., Strunin, L., Berlin, B. M., & Heeren, T. (1990). Beliefs about AIDS, use of alcohol and drugs, and unprotected sex among Massachusetts adolescents. American Journal of Public Health, 80, 295-299.

Hypercard 2.1 for the Macintosh Computer. Claris Corp., Santa Clara, CA 95052.

Johnston, J. (1992, May). Macinterview: A technological solution to privacy and literacy concerns. Presentation at the Annual Meeting of the American Association for Public Opinion Research. St. Petersburg. FL

Kiesler, S., & Sproull. L. S. 1986). Response effects in the electronic survey. Public Opinion Quarterly, 50, 402-413.

Lavrakas. P. (1993). Telephone survey methods: Sampling, selection, and supervision. Newbury Park. CA: Sage.

Li. X., Stanton, B., Black, M. M., Romer, D., Ricardo, I., & Kaljee, L. 1994). Risk behavior and perception among youths residing in urban public housing developments. Bulletin of the New York Academy of Medicine, 71, 252-266.

MacRecorder Sound System Pro. Macromedia. San Francisco, CA 94103.

Mayer, S. E. (1991). How much does a high school's racial and socioeconomic mix affect graduation and teenage fertility rates? In C. Jencks & P. E. Peterson (Eds.), The urban underclass pp. 321-341). Washington. DC: The Brookings Institution.

Midanik. L. (1988). Validity of self-reported alcohol use: A literature review and assessment. British Journal of Addiction, 83, 1019-1029.

Millstein, S. G., & Irwin. C. E., Jr. (1983). Acceptability of computer-acquired sexual histories in adolescent girls. Journal of Pediatrics, 103, 815-819.

Romer, D., Black. M., Ricardo, I., Feigelman, S., Kaljee, L., et al. (1994). Social influences on the sexual behavior of youth at risk for HIV exposure. American Journal of Public Health, 84, 977-985.

Sonenstein, F. L., Pleck, J. H., & Ku, L. C. (1989). Sexual activity, condom use and AIDS awareness among adolescent males. Family Planning Perspectives, 21, 152-158.

Stanton, B., Romer, D., Ricardo, I., Black, M., Feigelman, S., & Galbraith, J. (1993). Early initiation of sex and its lack of association with risk behaviors among adolescent African Americans. Pediatrics, 92, 13-19.

Turner, C. F., Lessler, J. T., & Devore, J. (1992). Effects of mode of administration and wording on reporting of drug use. In C. F. Turner. J. T., Lessler, & J. C. Gfroerer (Eds.), Survey measurement of drug use: Methodological studies (pp. 177-220). Rockville, MD: National Institute of Drug Abuse.

Manuscript accepted April 11, 1996

This research was supported by a grant (#U10MH48068) from the National Institute of Mental Health to the University of Maryland at Baltimore (B. Stanton, principal investigator) and to the University of Pennsylvania (D. Romer, principal investigator). We thank Jennifer Galbraith, Linda Kaljee, and Rodney Nesbit for their many contributions to this research.

Please address correspondence to Daniel Romer, Ph.D., Center for Community Partnerships, University of Pennsylvania, 133 S. 36th Street, 5 FL, Philadelphia PA 19104-3246. Phone: [phone]. Fax: [phone]. E-mail: [e-mail].