Part-whole bias in contingent valuation: will scope effects be detected with inexpensive survey methods?

1. Introduction

There is currently much interest in measuring the benefits of environmental quality improvements for policy purposes. An ongoing controversy involves the use of the contingent valuation (CV) method to estimate the total economic value, including non-use value, for environmental resources damaged by commercial activities (Hausman 1993; Carson et al. 1994; Portney 1994; Kopp and Pease 1997). The CV method is a survey approach to the valuation of resource allocation changes. Survey respondents are directly asked about their willingness to pay for increments or to avoid decrements in the environmental resource. Nonuse value is the lost consumer surplus from economic activities (e.g., an oil spill) suffered by a household that has never consumed the resource on-site (e.g., recreation). The measurement of total economic value is problematic since a positive nonuse value does not require any associated behavior (Smith 1987). Economic values for clean environments are typically measured through observation of recreation, averting, and migration behavior (Freeman 1993). People who do not participate in recreation or other activities while enjoying the knowledge of the existence of a natural resource do not provide the behavior necessary to measure these values with conventional methods. The CV method does allow measurement of the nonuse values.

A major concern with using CV to measure total economic values that include nonuse values is whether these measures are consistent with economic theory (Diamond and Hausman 1994; Hanemann 1994). One issue that has received considerable attention is whether CV results pass a "scope test" (Arrow et al. 1993). Mitchell and Carson (1989, pp. 250-1) define part-whole bias as one type of respondent insensitivity to scope:

Part-whole biases are major amenity misspecifications, and are also a result of the tendency of respondents to respond to public goods as global symbols without paying sufficient attention to the specific description offered in a CV survey. . . . The dimensions of a good that are particularly prone to this misperception are its geographic distribution, its benefit composition, and the package of policies of which it is a part.

Through the implementation of carefully designed surveys, researchers should manage to elicit total economic value, or willingness to pay (WTP), which does not decrease with increases in the quantity or quality of the affected environmental resource. If CV-based total economic values are determined to be valid and reliable measures of economic welfare, the conduct of environmental policy analysis might undergo significant change (Portney 1994).

Some recent empirical evidence on the ability of CV to detect scope effects is negative.(1) Kahneman and Knetsch (1992) and Fischhoff et al. (1993) do not find scope effects when using open-ended WTP questions in telephone surveys. Boyle et al. (1994), using a mall-intercept survey, and McFadden (1994), using a telephone survey, do not find scope effects using open- and closed-ended WTP questions.(2) On the other hand, several studies have found that WTP is sensitive to the scope of the resource change while employing in-person interviews and mail surveys (Loomis, Lockwood, and DeLacy 1993; Carson and Mitchell 1995; Carson et al. 1996).(3) This comparison of studies suggests that it may be difficult to find significant scope results without survey methods that allow the use of visual aids.(4)

The research that has found respondent insensitivity to scope has been criticized for using relatively inexpensive survey methods, not using visual aids, and eliciting WTP with open-ended questions (Smith 1992; Carson and Mitchell 1993, 1995; Loomis, Lockwood, and DeLacy 1993). However, Schuman (1996) discusses the benefits of mail and telephone surveys, including cost, and asserts that they are useful for CV research, especially experimental treatments.(5) The results of Carson and Mitchell (1993) and Berrens, Ganderton, and Silva (1996) suggest that insensitivity to scope may not be a result of the use of inexpensive survey methods but rather induced directly by survey design or the type of environmental goods considered.

In this paper, we conduct scope tests with a split-sample survey that elicits both use and nonuse values. We employ standard telephone survey methods with no photographs or other visual aids. The WTP is elicited with closed-ended questions. Respondents are presented with short descriptions of the environmental resources, policy descriptions, payment rule, and payment vehicle. We first sketch the theory and describe the sample for our application: valuing water quality improvements in the Albemarle and Pamlico Sounds in North Carolina. We then describe the data and present the results of the scope test. Our conclusions follow.

2. Economic Theory

Suppose that survey respondents possess a utility function, u([x.sub.1], [x.sub.2], [q.sub.1], [q.sub.2], z), where [x.sub.i] is the number of recreation trips to site i (i = 1, 2), [q.sub.i] is quality at recreation site i, and z is a composite commodity of all other goods. Recreation site 1 is the Albemarle Sound, and site 2 is the Pamlico Sound. The expenditure function, e([p.sub.1], [p.sub.2], [q.sub.1], [q.sub.2], u), is obtained by minimizing expenditures,

m = [summation of] [p.sub.i][x.sub.i] where i = 1 to 2 + z,

subject to the utility constraint, where m is income, [p.sub.i] is the price to site i, and both are normalized by the price of z. Willingness to pay for an improvement in Pamlico Sound quality is

[Mathematical Expression Omitted],

where [Mathematical Expression Omitted]. Substitution of the indirect utility function evaluated at improved quality, [Mathematical Expression Omitted], into the expenditure functions yields the variation function, [s.sub.i]([center dot]):

[Mathematical Expression Omitted],

which is increasing in income, decreasing in own-price, either increasing or decreasing in cross-price, and increasing in own-quality (Whitehead 1995).

Willingness to pay for an improvement in the quality of both Albemarle and Pamlico Sound is

[Mathematical Expression Omitted],

where [Mathematical Expression Omitted], and after substitution of the indirect utility function, the variation function is

[Mathematical Expression Omitted].

The difference between willingness to pay for the Albemarle-Pamlico (A-P) and Pamlico (P) quality change is [WTP.sub.1,2] - [WTP.sub.2] = [Delta]WTP, and after simplification

[Mathematical Expression Omitted],

where [Sigma]WTP [greater than or equal to] 0 if the classical assumptions of completeness, reflexivity, transitivity, and continuity of the preference ordering hold. The null hypothesis of insensitivity to scope, AWTP [less than] 0, is tested against the one-tailed alternative hypothesis that [Delta]WTP [greater than or equal to] 0.

3. The Survey

The data for this study are from a 1995 telephone survey conducted by the East Carolina University Survey Research Laboratory. There are two versions of the telephone survey. Version [TABULAR DATA FOR TABLE 1 OMITTED] P contained a contingent market for the Pamlico Sound, and version A-P contained a contingent market for the Albemarle and Pamlico Sounds. The major difference in the two versions is the insertion of "Albemarle and" before Pamlico and making "Sound" plural in all questions. The survey used a random digit dialing sampling scheme with the sample purchased from a professional firm. The interviews were computer assisted and conducted by trained interviewers. Of the households that were contacted, 1077 respondents completed the survey for an overall response rate of 75%.

The demographic profile of the sample is similar to that of eastern North Carolina (Table 1). Except for race, none of the demographic variables are different between the P and the AP versions of the survey. The A-P sample has a higher proportion of white respondents than the P sample ([[Chi].sup.2] = 3.94[1 d.f.]). The county of residence of each respondent was recorded and distance to Pamlico Sound calculated. The TRAVCOST variable measures the own-price of a recreation trip and is the time and travel costs of a round trip to Pamlico Sound.(6)

The first group of questions in the CV section of the survey elicited information about respondents' knowledge of the Sound(s) (KNOW), current and future recreation participation, and current and future recreation intensity on the Sound(s).(7) Using chi-square tests on the frequencies of responses, there are no differences between the two survey versions on knowledge of water quality problems and past and future outdoor recreation trips. The policy scenario was established next with a series of questions in regard to respondent concern (CONCERN) about water pollution, support (SUPPORT) for tougher laws, and perceived effectiveness (EFFECT) [TABULAR DATA FOR TABLE 2 OMITTED] of these laws. These questions are designed to explain the pollution problem and the proposed policy and also to get respondents thinking about how much they value the policy. Using chi-square tests on the frequencies of responses, there are no differences between the two survey versions on concern about water quality problems and support for, and perceived effectiveness of, the policy.

The next section established the contingent market with contingent behavior, willingness to pay, and debriefing questions. The contingent behavior questions elicited information about recreation participation and intensity with improved quality. Using chi-square tests on the frequencies of responses, there are no differences between the two survey versions on future recreation participation or future trips with improved quality. The payment obligation, payment vehicle, and potential substitutes were established with statements directly preceding the willingness-to-pay questions. The price and tax amount (PT1) was randomly selected from four amounts: 100, 200, 300, and 400.(8)

4. Scope Test Results

The full sample of the responses to the valuation question is analyzed with "don't know" responses coded as "no" (YES1DK), as suggested by Schuman (1996) and Carson et al. (1995). The frequency of no responses at each price and tax amount and by split sample is presented in Table 2. We first conduct the Cochran-Mantel-Haenszel test (Snedecor and Cochran 1980) for differences in the frequency of no responses across versions of the survey while controlling for the price and tax amount. This test indicates that the frequencies are significantly different at the .05 level (Table 2).

The response proportions to each tax can be used to calculate a Turnbull distribution-free estimate of the underlying distribution of WTP and a lower-bound estimate of mean WTP. Splitting the sample between the P and the A-P versions and calculating the lower-bound estimate of mean WTP for each sub-sample allows for a split-sample distribution-free test for scope. The lower-bound estimate of mean WTP, [WTP.sub.LB], is defined as

E([WTP.sub.LB]) = [summation of][c.sub.j-1][d.sub.j] where j = 1 to M+1,

where M is the number of distinct taxes offered in the referendum, M + 1 indexes the upper bound on the range of WTP, c indexes the offered tax amounts, [d.sub.j] represents the difference in the proportion of no responses between [c.sub.j] and [c.sub.j-1], and [d.sub.M+1] = 1 - [summation of] [d.sub.M] where j = 1 to M. Intuitively, [d.sub.j] is the probability that WTP falls between two subsequently offered tax amounts. To ensure a valid distribution function, the Turnbull estimator pools responses to taxes that fail to yield a monotonic distribution function. This pooling ensures that the estimated probabilities of a no response monotonically decrease with the offered price. Details for calculating the lower-bound mean WTP values and the variance of the lower bound can be found in Haab and McConnell (1997).

For the split-sample data, the difference in lower-bound means between the P and the AP samples is $24, which is significantly different from zero at the .01 level for a one-tailed test (t = 2.00). Note that the difference in means test is more powerful than the contingency table test since it is one tailed and incorporates the magnitude of the differences in WTP.

Regression analysis is used to test the sensitivity of the scope effects while holding constant other potentially confounding variables.(9) We present our preferred regression model under the Weibull distributional assumption for WTP. The Weibull referendum model assumes that WTP is distributed according to the density function:

[Mathematical Expression Omitted],

where f([center dot]) is the density function, X represents a row vector of covariates, B is the corresponding column vector of parameters, and s (or [Sigma]) is sometimes referred to as the scale parameter. Mean WTP calculated from the Weibull distribution can be written as

E(WTP) = [e.sup.XB]G(1 + s),

where G is the gamma distribution (for more detail, see Haab and McConnell 1997).

The Weibull parameter estimates are presented in Table 3. The coefficient on the Pamlico Sound dummy variable is statistically significant at the .01 level (one-tailed test), indicating that respondents are willing to pay less for the Pamlico Sound policy relative to the Albemarle and Pamlico Sounds policy. Our choice of independent variables is based on theory (TRAVCOST and INCOME) and specification tests. Respondents are willing to pay more if they are concerned about water pollution problems and if they had prior knowledge of the environmental problem. The WTP falls with age. The income and travel cost coefficients are insignificantly different from zero. The scope effect from the Weibull model is calculated by evaluating the difference in mean WTP with the Pamlico dummy variable set equal to one and zero while holding all other covariates constant at their mean values. The scope effect is significant at the .01 level (one-tailed test).

The scope test was also conducted with different model specifications that represent decisions about various distributional assumptions, the inclusion of independent variables, and the selection of the sample. Scope tests were conducted assuming the normal, logistic, and beta distributions. These distributions for WTP were also bounded from above (WTP [less than] income) and below (WTP [greater than] 0). The sample selection rules considered include dropping respondents who protested the WTP question(10) or who answered "don't know." The statistical significance of the scope test is not influenced by the sample selection criteria or distributional assumptions about WTP. Further, the significance of the scope test results do not depend on the number and choice of independent variables included in the WTP models or whether WTP is calculated with a pooled model with a dummy variable for Pamlico or split-sample models.(11)

Table 3. Regression Results(a)

                                                  Weibull
Variable                     Mean   Coefficient             t-Value

CONSTANT                              5.64(***)               46.95
PAMLICO                      0.55    -0.12(***)               -2.60
INCOME(b)                   31.87    -0.0003                  -0.22
TRAVCOST                    87.49     0.00027                  0.59
CONCERN                      3.36     0.060(**)                2.16
KNOW                         1.88     0.056(**)                2.19
AGE                         42.81    -0.0026(**)              -2.10
SEX                          0.47     0.03                     0.69
[Sigma]                               0.42(***)               22.01
[Delta]WTP                          $34.10(***)                2.55
Log-likelihood Function                            -545.74

a Sample size = 766.

b In thousands of dollars.

** Indicates significance at the 0.05 level.

*** Indicates significance at the 0.01 level.

5. Conclusions

We find that CV-based estimates of WTP, including nonuse values, are sensitive to the scope of the policy. These results suggest that the use of inexpensive survey methods - in this case telephone surveys, which preclude presentation of visual aids to assist respondents in conceptualizing the different goods - may not be the cause of the failure to detect scope effects in some recent CV studies. Also, these results lend support to the argument that the measurement of nonuse value is not the cause of lack of sensitivity to scope.

What distinguishes between CV surveys that will elicit responses that are sensitive to scope and those that will not? Smith (1992) and Hanemann (1994) argue that survey design problems, such as vague descriptions of the environmental resource or a lack of credibility about delivery of the policy, could be the root cause of lack of sensitivity to scope. We assert that our survey did not encounter these problems since respondents were familiar with the resources, concerned about water quality problems, and perceived that government policy would be effective. Also, an environmental policy similar to the one presented in the CV scenario has been under consideration by the state, thus increasing respondent familiarity and credibility of the survey.

Although these results support the use of the CV method for environmental policy analysis, this study is limited in at least two ways. First, these tests are not conclusive. One of the independent variables we want to analyze, the telephone survey method, is not varied in our tests for scope effects. A true test of the survey method would compare telephone, mail, mall-intercept, and in-person surveys in terms of scope effects as well as the internal consistency of responses and the validity and reliability of WTP estimates. Respondents who are unfamiliar with the environmental resources and policy may require visual aids and lengthy text before they can adequately understand the issues. Future research should be directed toward this end, especially if the cost effectiveness of CV research is a serious issue.

Our results, as well as those of Carson and Mitchell (1993) and Berrens, Ganderton, and Silva (1996), suggest that inexpensive survey methods can be used to elicit WTP values that are sensitive to scope. The usefulness of inexpensive survey methods is clear. Much methodological research is needed to refine the CV method for measurement of use and nonuse values. More research can be conducted at the same cost if the cost per unit of the research is lowered. Technological improvements in telephone survey methods have lowered the costs per unit of conducting scientifically valid survey research.

Previous versions of this paper were presented at the ECU Economics Seminar, the 1996 Southern Economic Association Meetings, and the 1997 Midwest Economic Association Meetings. The authors appreciate the comments from participants in these forums, especially Donn Johnson, and two anonymous SEJ reviewers. All data used in this paper, a data appendix, survey questions, and additional empirical results are available on the Web at http://www1.ecu.edu/~ecwhiteh/data/data.htm or from the first author.

1 Some of the studies reviewed here refer to scope or part-whole effects as embedding effects. Embedding occurs when a good valued later in a valuation sequence is worth less than when valued sooner in the same sequence. Embedding is not inconsistent with demand theory and has been found in empirical demand systems for market goods (see Randall and Hoehn 1996). Our literature review is restricted to split-sample scope tests.

2 Open-ended questions ask, "How much are you willing to pay?" Closed-ended questions ask, "Are you willing to pay $A?" where A is a dollar amount that is varied across respondents. For a discussion of the merits of the two formats, see Mitchell and Carson (1989).

3 For a review of these and earlier studies, see Brown and Duffield (1995), Smith and Osborne (1996), and Carson (1997).

4 For a discussion of other survey design problems that may lead to finding no scope effects, see Mitchell and Carson (1989).

5 The issue of cost is not a trivial one. Harrison and Lesley (1996) estimate natural resource damages with student subjects, weighted to reflect a national sample, equal to the damage assessment in the Exxon Valdez study (Carson et al. 1994) at a significantly lower cost. Harrison and Lesley are subsidized by the survey development efforts of Carson et al. (1994), and the similarity of the damage assessments could be influenced by having a target to aim toward. Even so, their effort makes explicit the benefits and costs of survey method choice that seems often to be ignored.

6 The TRAVCOST variable was constructed as follows. For respondents living west of the Pamlico River, the distance was calculated as the distance from the respondent's county population center to Washington, North Carolina, on the Pamlico River. If the respondent lived north or south of the Pamlico River, the distance was calculated as the distance from the county population center to the nearest boat ramp on the sound. Distances were calculated using the Automap software package. The travel cost used is $.20 per mile, average miles per hour is 50, and the opportunity cost of travel time is valued at the wage rate (wage = INCOME/2080). We assume that all trips are day trips so that the cost of on-site time is zero.

7 The contingent market questions were revised from a previously conducted (1993) telephone survey that focused on the Pamlico Sound only (Whitehead et al. 1998). Revisions were made to the 1993 Pamlico Sound questions on the basis of empirical results with the 1993 data and then pretested with telephone subjects.

8 These amounts were chosen on the basis of the results of Whitehead et al. (1998) and the pretest.

9 The results were found using the LIMDEP econometric software (Greene 1995). None of the results are affected by the lack of randomization on the race variable. This was tested by including race as an independent variable, interacting the Pamlico Sound dummy variable with race, and weighting the data on the basis of race.

10 Protest responses were determined with the debriefing questions. For example, respondents who stated that they would not pay anything for improved water quality or that they did not trust government were flagged as protest responses.

11 Not surprisingly, the magnitude of the scope effect (and WTP values) does depend on model specification. The difference in WTP is smaller with the pooled data models relative to the split-sample models. Excluding cases with don't-know answers to the valuation questions, protest responses, and cases with missing attitude and demographic responses reduces the magnitude of the scope effect. The assumed distributions with a lower bound for WTP (WTP [greater than] 0) tend to predict smaller differences in WTP.

References

Arrow, Kenneth, Robert Solow, Edward Leamer, Paul Portney, Roy Radner, and Howard Schuman. 1993. Report of the NOAA Panel on contingent valuation. Federal Register 58:4602-14.

Berrens, Robert P., Philip Ganderton, and Carol L. Silva. 1996. Valuing the protection of minimum instream flows in New Mexico. Journal of Agricultural and Resource Economics 21:294-309.

Boyle, Kevin J, William H. Desvousges, F. Reed Johnson, Richard W. Dunford, and Sara P. Hudson. 1994. An investigation of part-whole biases in contingent valuation studies. Journal of Environmental Economics and Management 27:64-83.

Brown, Thomas C, and John W. Duffield. 1995. Testing part-whole valuation effects in contingent valuation of instream flow protection. Water Resources Research 31:2341-51.

Carson, Richard T. 1997. Contingent valuation surveys and tests of insensitivity to scope. In Determining the value of non-marketed goods, edited by Raymond J. Kopp, Werner W. Pommerehne, and Norbert Schwarz. Boston: Kluwer, pp. 128-63.

Carson, Richard T., and Robert C. Mitchell. 1993. The issue of scope in contingent valuation studies. American Journal of Agricultural Economics 75:1263-7.

Carson, Richard T., and Robert Cameron Mitchell. 1995. Sequencing and nesting in contingent valuation surveys. Journal of Environmental Economics and Management 28:155-73.

Carson, Richard T., Robert C. Mitchell, W. Michael Hanemann, Raymond J. Kopp, Stanley Presser, and Paul A. Ruud. 1994. Contingent valuation and lost passive use: Damages from the Exxon Valdez. Resources for The Future Discussion Paper 94-28, Washington, DC.

Carson, Richard T., W. Michael Hanemann, Raymond J. Kopp, Jon A. Krosnick, Robert C. Mitchell, Stanley Presser, Paul A. Ruud, and V. Kerry Smith, with Michael Conaway and Kerry Martin. 1995. Referendum design and contingent valuation: The NOAA Panel's no-vote recommendation. Resources for The Future Discussion Paper 96-05, Washington, DC.

Carson, Richard T., W. Michael Hanemann, Raymond J. Kopp, Jon A. Krosnick, Robert C. Mitchell, Stanley Presser, Paul A. Ruud, and V. Kerry Smith, with Michael Conaway and Kerry Martin. 1996. Was the NOAA Panel correct about contingent valuation? Resources for The Future Discussion Paper 96-20, Washington, DC.

Diamond, Peter A., and Jerry A. Hausman. 1994. Contingent valuation: Is some number better than no number? Journal of Economic Perspectives 8:45-64.

Fischhoff, Baruch, Marilyn Jacobs Quadrel, Mark Kamlet, George Loewenstein, Robyn Dawes, Paul Fischbeck, Steven Klepper, Jonathan LeLand, and Patrck Stroh. 1993. Embedding effects: Stimulus representation and response mode. Journal of Risk and Uncertainty 6:211-34.

Freeman, A. Myrick, III. 1993. The measurement of environmental and resource values: Theory and methods. Washington, DC: Resources for the Future.

Greene, William H. 1995. LIMDEP version 7.0: User's manual. Bellport, NY: Econometric Software.

Haab, Timothy C., and Kenneth E. McConnell. 1997. Referendum models and negative willingness to pay: Alternative solutions. Journal of Environmental Economics and Management 32:251-70.

Hanemann, W. Michael. 1994. Valuing the environment through contingent valuation. Journal of Economic Perspectives 8:19-44.

Harrison, Glenn W., and James C. Lesley. 1996. Must contingent valuation surveys cost so much? Journal of Environmental Economics and Management 31:65-78.

Hausman, Jerry. 1993. Contingent valuation: A critical assessment. Amsterdam: North-Holland.

Kahneman, Daniel, and Jack L. Knetsch. 1992. Valuing public goods: The purchase of moral satisfaction. Journal of Environmental Economics and Management 22:57-70.

Kopp, Raymond J., and Katherine A. Pease. 1997. Contingent valuation: Economics, law and politics. In Determining the value of non-marketed goods, edited by Raymond J. Kopp, Werner W. Pommerehne, and Norbert Schwarz. Boston: Kluwer, pp. 7-58.

Loomis, John, Michael Lockwood, and Terry DeLacy. 1993. Some empirical evidence on embedding effects in contingent valuation of forest protection. Journal of Environmental Economics and Management 24:45-55.

McFadden, Daniel. 1994. Contingent valuation and social choice. American Journal of Agricultural Economics 76:689-708.

Mitchell, Robert Cameron, and Richard T. Carson. 1989. Using surveys to value public goods: The contingent valuation method. Washington, DC: Resources for the Future.

Portney, Paul R. 1994. The contingent valuation debate: Why economists should care. Journal of Economic Perspectives 8:3-18.

Randall, Alan, and John P. Hoehn. 1996. Embedding in market demand systems. Journal of Environmental Economics and Management 30:369-80.

Schuman, Howard. 1996. The sensitivity of CV outcomes to CV survey methods. In The contingent valuation of environmental resources: Methodological issues and research needs, edited by David J. Bjornstad and James R. Kahn. Cheltenham, UK: Edward Elgar, pp. 75-96.

Smith, V. Kerry. 1987. Nonuse values in benefit cost analysis. Southern Economic Journal 54:19-26.

Smith, V. Kerry. 1992. Arbitrary values, good causes, and premature verdicts. Journal of Environmental Economics and Management 22:71-89.

Smith, V. Kerry, and Laura L. Osborne. 1996. Do contingent valuation estimates pass a scope test? A meta-analysis. Journal of Environmental Economics and Management 31:287-301.

Snedecor, G. W., and W. G. Cochran. 1980. Statistical methods, 7th ed., Ames: Iowa State University Press.

Whitehead, John C. 1995. Willingness to pay for quality improvements: Comparative statics and interpretation of contingent valuation results. Land Economics 71:207-15.

Whitehead, John C., Glenn C. Blomquist, Richard C. Ready, and Ju-Chin Huang. 1998. Construct validity of dichotomous and polychotomous choice contingent valuation questions. Environmental and Resource Economics 11:107-16.