California adolescents' use of family planning services.

In "A Comparison of Hispanic and White Adolescent Females' Use of Family Planning Services in California" [2004, 36(4):157-161], M. Rosa Solorio and colleagues conduct a secondary analysis based on the 2001 California Health Interview Survey (CHIS). The question they address regarding group-specific uses and needs for family planning services in California is certainly an important one, and we do not dispute their conclusion that "a need exists for family planning programs to target Hispanic adolescents before they become pregnant, and to send a clear message that contraceptive services are available and should be used before a woman ever has a pregnancy." However, we question whether their analyses add any support to this largely self-evident conclusion. Our main concerns relate to questionable interpretations based on very small numbers of subgroup members and the low but unreported overall adolescent response rate.

The key analyses that the authors interpret compare the use of family planning services for ever-pregnant white and Hispanic 14 17-year-olds. Although not reported in the article, a few calculations based on Table 2 indicate that within these subgroups, approximately two and five individuals, respectively, reported using family planning services. (These are approximations because one needs to apply percentages based on weighted responses to unweighted sample sizes from the table.) For example, Table 2 shows that the sample included 277 white sexually experienced females aged 14-17, that 8.3% of the 277 (approximately 23) had ever been pregnant and that 9.3% of the 23 (approximately two) reported using services in the past year. There were no significant differences in use of services between the never-pregnant white and Hispanic subgroups; therefore, the authors' main conclusions regarding white-Hispanic differences rely on the seven users of services in the ever-pregnant subgroups. This is hardly a stable foundation for analyzing between-group differences.

Subgroup memberships were employed in a logistic regression equation to predict use of family planning services. The odds ratio of 11.6 reported in Table 3 for ever-pregnant Hispanic adolescents (as compared with never-pregnant white adolescents) has a confidence interval of 1.8-76.3. The size of this confidence interval illustrates the imprecision and instability of the estimate, as well as the futility of meaningful interpretation. Some might argue that because this confidence interval does not include 1.0, a between-group difference has been statistically demonstrated. This argument cannot stand beyond the individuals in the sample subgroups compared, as it is impossible to generalize from such a small sample to all ever-pregnant adolescents in California. In other words, a change of just one or two sample respondents in either direction could drastically change these results.

Our second concern is the low and unreported overall adolescent response rate lot this survey of 24%. (1) in other words, 76% of eligible adolescents did not participate. Although a response rate at this level does not necessarily invalidate the entire survey, it raises important issues about sampling error and bias that must be considered in interpreting the results. To consider these issues requires knowledge of the overall adolescent response fate and its various components, in this case, instead of reporting the overall adolescent response rate, the authors reported only the overall adult response rate of 38% and provided a reference for further information about CHIS sampling methods. The authors neglected to report that 25% of the responding adults in households with adolescents refused to allow the adolescents to be interviewed, and that 15% of adolescents with adult permission to be interviewed did not participate in the survey. Although this information can be found at the end of a long report available on the CHIS Web site, the onus should not be on the reader to search for such critical information that might reflect on the interpretability and generalizability of the study results.

The reporting of inadequate and misleading information about the CHIS response rates is not unique to the present article. This issue needs to be better addressed in future publications that report CHIS analyses. This could be accomplished in several ways. First, the CHIS staff could make a greater effort to educate data users and consumers about the overall survey response rates and their various components. Second, authors need to ensure that they understand the response rate components and related methodological issues, and obtain the appropriate technical consultation when necessary. Finally, journals could educate their peer reviewers about response rate issues, and attempt to involve a methodologist experienced in survey sampling issues in the review process for papers that report analyses of survey data.

Norman A. Constantine

Center for Research on Adolescent Health and Development

Public Health Institute

Oakland, CA

Joel M. Moskowitz

Center for Family and Community Health

School of Public Health, University of California, Berkeley 

Berkeley, CA

(1.) UCLA Center for Health Policy Re>earth, CHI5 2001 Methodology Series: Response Rates, 2003, Table 6.7, <http://www.chis.ucla.edu/chis_methods.html>, accessed Oct. 2, 2004.

Solorio and colleagues reply:

The authors of this letter have misinterpreted our findings. As noted in Table 1 of our article, we did not report data for which the unweighted cell size was less than five or the coefficient of variation was 30% or larger. Although the relatively small size of the sexually experienced Hispanic sample was a limitation of the study, our results included only data for which unweighted cell sizes were five or greater. The analytic sample included 83 white and 22 Hispanic adolescents who had used family planning services; 12 of these white teenagers (weighted, 9% of the population) and seven Hispanic young women (42% of the population) had ever been pregnant.

We used logistic regression to examine the association between variables of interest and the outcome, controlling for various other factors. In those analyses, the variable combining adolescent ethnicity and ever having been pregnant was associated with utilization of family planning services at a statistically significant level, albeit with a wide confidence interval because the number of sexually active Hispanic adolescents was small.

The adolescent response rate for the 2001 California Health Interview Survey (CHIS) is the product of the adult response rate (38%) and the adolescent completion rate (64%), or 24%. (1) This rate, as the authors of the letter point out, reflects the substantial proportion of eligible teenagers for whom permission to participate was not granted or who declined to be interviewed.

A survey's response rate, however, is not the only, or even the best, measure of how representative it is. The key concern is how well the respondents represent the entire population being sampled, and CHIS has been shown to be generally representative of California's population. (2) CHIS, like any other telephone survey, cannot eliminate all bias, but it has employed a number of methodological and statistical techniques to minimize biases--random generation of phone numbers for the household sample, random techniques to identify respondents within households and statistical weighting of the final data. The weighted data from the 2001 survey reflect the California population based on the 2000 U.S. census for each geographic area and for the state as a whole, adjusted for a number of potential biases, such as nonresponse and households without a telephone. The CHIS sample does what it purports to do: It provides estimates that are representative of California's noninstitutionalized population living in households.

(1.) UCLA Center for Health Policy Research, California Health Interview Survey: Survey Methodology and Sample Design, 2002, <http://www.chis.ucla.edu/methods_ design.html>, accessed Apr. 29, 2004.

(2.) UCLA Center for Health Policy Research, The CHIS 2001 Sample: Response Rate and Representativeness, Los Angeles: UCLA Center for Health Policy Research, 2003.