Printer Friendly

Selecting field experiment locations with archival data.

The selection of field experiment locations often requires that the control location be comparable to the treatment location. Recently, a method that employs the mathematics of potential performance theory was proposed for comparing field experiment locations by parsing random versus systematic effects. Here, we apply this method to archival data (i.e., household electricity consumption). In doing so, we are able to assess the effectiveness of this method at handling real-world data, absent user-defined variables. In addition, we provide consumer researchers with a step-by-step example on how to apply this method to the problem of selecting optimal treatment and control locations for field experiments. Finally, we consider how this method could apply to policy assessment more generally.

INTRODUCTION

Field experiments have a valuable role in consumer research. They allow researchers to study consumers in their natural environment, such as gas stations, grocery stores, schools, and e-commerce sites; and their findings can inform policy on consumer affairs (e.g., Fitzpatrick 2015; Moulton et al. 2013; Yeomans and Herberich 2014). When performing field experiments, random assignment to treatment and control conditions is the preferred approach. However, while easily accomplished in the lab, random assignment is not always possible in the field. Instead, field experiments often rely on treatment and control locations in which to employ and not employ, respectively, an intervention (Harrison and List 2004). When doing so, it is important that treatment and control locations are comparable.

Employing the mathematics of potential performance theory (Trafimow and Rice 2008, 2009), Trafimow et al. (2015) proposed a methodology that enables a field researcher to evaluate and select optimal field locations by parsing their random versus systematic effects and tested the accuracy of this method using computer simulations. These simulations indicated the method was highly accurate, given an adequate number of data points and sufficiently high consistency coefficients (see Figure 1 in Trafimow et al. 2015). However, the method's perceived complexity may deter some researchers from applying it, despite its benefits. This article demystifies the use of this method by providing a step-by-step example of how to implement it on publicly available archival data.

Previous consumer researchers have obtained large amounts of field data from sources such as direct-mail catalogs, e-commerce sites, and retail scanner data (e.g., Algesheimer et al. 2010; Anderson and Simester 2001; Levav and Zhu 2009). In addition, recent advances in online data sharing have resulted in historic levels of publicly available archival data. Combined with the method described herein, these datasets can be used to determine optimal treatment and control locations for field experiments. For example, a field experiment addressing teen pregnancy among Hispanic females in California could use data from the Centers for Disease Control and Prevention (CDC) Wonder Database (http://wonder.cdc.gov/) to select counties (e.g., Los Angeles, Orange, Placer) that could serve as optimal treatment and control locations. In addition, data from the Organization for Economic Cooperation and Development (see Google's Public Data Explorer: www.google.com/publicdata/directory) could be used to determine treatment and control locations for a multinational field experiment on alcohol consumption. As another example, data from the California Department of Public Health (https://www.cdph.ca.gov/programs/immunize/) could be used to select schools for treatment and control locations for a field study on childhood immunization.

In the present research, we describe how the methodology of Trafimow et al. (2015) can be applied to archival datasets, such as those described, to select optimal treatment and control locations for field experiments. In addition to providing a step-by-step guide on using this method, our application to archival data also tests the method's feasibility when key variables (i.e., population levels and consistency coefficients) are not user defined, as was the case in the simulations used to test the accuracy of the method. Furthermore, the application of this method to archival data highlights the opportunity to use it more generally for policy assessment.

BACKGROUND

Researchers perform field experiments to study consumers in their natural environment, and the results of such studies can inform policy on consumer affairs (e.g., Collins and O'Rourke 2010; Johnson and Goldstein 2003). For instance, field experiments have been used to learn how comparative food information can affect grocery shoppers (McCracken, Boynton, and Blake 1982), how social connections can affect charitable giving (List and Price 2009), and how commitment can impact conservation efforts (Baca-Motes et al. 2013). Ideally, field experiments employ random assignment to treatment and control conditions (List, Sadoff, and Wagner 2011); however, logistical and even ethical concerns can preclude this possibility (Harrison and List 2004). When random assignment is not an option, the selection of treatment and control locations is an important consideration, since it will affect the researcher's ability to know the effectiveness of an intervention and the plausibility of alternative explanations.

For example, suppose state policy makers are concerned with increasing their constituency's enrollment in organ donation. In particular, they want to know how changing the default on their enrollment option might affect organ donor enrollments (e.g., Johnson and Goldstein 2003). Logistical constraints preclude the researchers from randomly assigning residents of the state to treatment and control conditions. In turn, the researchers select, randomly or otherwise, regions in the state to serve as treatment and control locations. In the region selected as the treatment location they implement an "opt-out' enrollment option for all residents of the region, and in the region selected as the control location they maintain their "opt-in" enrollment option for all residents of the region. After the intervention, the researchers compare enrollments between and within the treatment and control locations to assess the effectiveness of their intervention.

This approach is less than ideal, however, because it fails to account for possible differences between control and treatment locations. For instance, it could have been the case that an observed difference in enrollments between the two regions was due to factors other than the intervention (e.g., differences in the sociodemographics of the regions compared). Such concerns are often addressed by matching observed scores on potential confounding variables. This is typical of propensity score matching, for instance, in which the researcher attempts to account for all measured covariates in an effort to eliminate plausible alternative explanations (e.g., Foster 2003; Rosenbaum and Rubin 1983). This can be problematic, however, because the researcher must know all of the relevant variables on which to match. In addition, it is unlikely that treatment and control locations will match on every variable considered, forcing the researcher to control for non-matching variables using less than ideal statistical controls (e.g., partial correlations, ANCOVAS, hierarchical regression; Heckman 1998). Further, should matching on observed variables be used, there remains the problem that matching (e.g., propensity score matching) does not address the problem of regression to the mean.

To understand the problem of regression to the mean, suppose a researcher wants to know whether socioeconomic status affects the financial literacy of students, but also wants to rule out intelligence as an alternative explanation. Based on previous research (Molfese, Modglin, and Molfese 2003; Turkheimer et al. 2003), as well as archival data from Harvard Dataverse (i.e., dataverse.harvard.edu), the researcher is aware that the average IQ is higher for students with a high socioeconomic status than for those with a low socioeconomic status. Given this empirical fact, obtaining high socioeconomic status and low socioeconomic status students that match on IQ will necessitate either that the mean for the high socioeconomic status sample will be less than the mean of that population, or that the mean for the low socioeconomic status sample will exceed the mean for that population, or both. Because the populations differ, there is no way to have the sample means match without at least one or the other of the samples differing from its corresponding population. Because IQ scores, like almost all dependent variables, are influenced by random and systematic factors, these scores are subject to regression to the mean. Among other implications, it follows that if the sample were measured a second time without carry-over effects across the two test-taking occasions, the new sample mean would be closer to the corresponding population mean than the original sample mean. In Classical Test Theory (Allen and Yen 2001; Gulliksen 1987; Lord, Novick, and Birnbaum 1968) terms, the true sample mean is closer to the population mean than is the observed sample mean. Therefore, even if the experimenter matches the high and low socioeconomic status samples on observed IQ scores, the two samples will fail to be matched on true IQ scores. Consequently, despite the effort to match samples on IQ, it nevertheless remains a potential confound.

To overcome these problems, Trafimow et al. (2015) proposed a method for selecting field experiment locations that does not depend on the researcher knowing all of the relevant variables on which to match, employing problematic statistical techniques to control for field location differences, or ignoring an implication of regression to the mean--that even matching samples on observed scores may render them mismatched on true scores. This method employs the mathematics of potential performance theory (PPT; Trafimow and Rice 2008, 2009) to parse random from systematic factors and determine potential agreement (PA) between possible field experiment locations. According to this method, PA is obtained by first comparing the extent that field locations "agree" (i.e., are concordant) and "disagree" (i.e., are discordant) on an outcome measure of interest. In our previous example on organ donor enrollment, for instance, two regions in the state would be considered in agreement on any given day if their organ donor enrollments exceeded or fell below their respective average daily enrollments. On the other hand, the regions would be

considered in disagreement on any given day if enrollments in one region were above its average daily enrollments, while enrollments in the other region were below its average daily enrollments. The frequency of this agreement and disagreement between the two regions could then be used to estimate a correlation coefficient that represents the extent that these regions agreed in terms of daily organ donor enrollments.

A key insight of Trafimow et al. (2015), however, is that such a correlation is the result of both random and systematic factors. This is problematic, they argue, because the selection of optimal field experiment locations requires that locations be compared on systematic, not random, factors. To overcome this problem, their method uses mathematics from PPT to parse these random versus systematic effects. In doing so, the method allows researchers to determine "true" agreement and disagreement frequencies in the absence of randomness. These frequencies can then serve as input in calculating PA, or the agreement between locations that would result in the absence of randomness.

In regard to the selection of field experiment locations, control and treatment locations with a PA near unity indicates that systematic factors similarly influence the outcome measure of interest (e.g., organ donor enrollments) in both locations--in which case, their pairing is optimal. On the other hand, having a PA less than unity indicates a less than ideal pairing. In addition, using PA scores to select optimal treatment and control locations does not limit subsequent analyses of treatment effects such as those involving a difference in differences approach (e.g., Conley and Taber 2011). For example, optimal treatment and control locations could first be selected using PA scores; after which, pre- and post-treatment observations could be used to assess change in the outcome variable of interest (e.g., organ donor enrollments) within and between the treatment and control locations (e.g., Card and Krueger 1994).

Trafimow et al. (2015) assessed the accuracy of their method using computer simulations with 10,000 cases per simulation and found the method was highly accurate at selecting ideal treatment and control locations. This testing, however, was simulated and relied on user-defined population and consistency levels. Thus, while promising, the method has yet to be applied to actual field data where such variables are not user-defined. In what follows, we apply this method to publicly available archival field data to assess and illustrate its effectiveness for the field researcher. Doing so has several advantages. It provides a step-by-step method that future researchers can use to determine PA between possible field locations. In addition, it extends the use of archival energy consumption data to field experiments, having possible implications for policy assessment in general.

APPLICATION

How energy regulations and associated policies affect consumer behavior and demand is important to policy makers and researchers (Allcott and Mullainathan 2010; Karlin, Zinger, and Ford 2015; Roe et al. 2001). For this application, suppose a researcher wants to test the effect of a statewide intervention on household electricity consumption. Similar to past research (e.g., Ayres, Raseman, and Shih 2013; Costa and Kahn 2013), suppose the goal of the intervention is to reduce statewide household electricity consumption. This could involve, for instance, providing households in the treatment state with target information on their monthly electricity consumption while withholding such information from households in the control state. Further, suppose the researcher is capable of conducting statewide interventions in New Mexico, Alaska, and Hawaii. The problem now faced by the researcher is which state to select as the treatment location and which state to select as the control location. For reasons previously described this is not an arbitrary decision.

To address this problem, the method of Trafimow et al. (2015) can be applied to archival data to identify optimal treatment and control locations (i.e., states). The outcome measure of interest, in this example, is residential electricity consumption. Archival data on this measure is publicly available from the U.S. Energy Information Administration. (1) After downloading these data, the following procedure can be used to determine which of the locations (i.e., states) considered would make an optimal treatment and control location pairing for a statewide intervention on household electricity consumption.

Stepwise Procedure

Step 1

First, we compare whether the states agree or disagree in terms of electricity consumption during matching time periods. In this example, states agreed (disagreed) in their electricity consumption for a given month when they were both (not both) above or below their historical average monthly household electricity consumption for the 288 months considered, ranging from January 1990 to December 2013. The frequency that each pair of states agreed and disagreed can be represented using three observed frequency tables (see Table 1 for an example of one of these three tables). In Table 1, cells a and d indicate frequencies of agreement between New Mexico and Hawaii on household electricity consumption, while cells b and c indicate frequencies of disagreement.

Step 2

For each pair of states (e.g., New Mexico and Hawaii), values from their observed frequency table (e.g., Table 1) are used in equation (1) to calculate an observed correlation coefficient (see Trafimow and Rice (2008) for proofs for all equations). Where a and d denote frequencies of agreement between locations X and Y (e.g., New Mexico and Hawaii); b and c denote frequencies of disagreement between any two locations X and Y; [r.sub.1] is the sum of a and b; [r.sub.2] is the sum of c and d; [C.sub.1] is the sum of a and c; [c.sub.2] is the sum of b and d; and [r.sub.XY] is the observed correlation coefficient for locations X and Y.

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (1)

Step 3

To correct for randomness in the observed correlation coefficients found in step 2, consistency coefficients are generated for each location (i.e., New Mexico, Hawaii, and Alaska). To do so, tables of observed frequencies are created for each state by comparing agreement between neighboring months (January and February, February and March, and so on). In this example, neighboring months were considered to agree (disagree) when each (not each) was above or below its 288-month average in residential electric consumption. The frequency of this agreement and disagreement is represented, for each state, with an observed frequency table (e.g., Table 2). In Table 2, cells a and d indicate frequencies of agreement between neighboring months (labeled as "First Month" and "Second Month"), while cells b and c indicate frequencies of disagreement.

A consistency coefficient (e.g., [r.sub.xx']) for each state (e.g., New Mexico) can then be generated using equation (1) and the state's observed frequency table (e.g., Table 2). Where locations X and Y from step 2 are replaced with within-location neighboring months x and x'; a and d denote frequencies of agreement between x' and x'; b and c denote frequencies of disagreement between x and x'; [r.sub.1] is the sum of a and b; [r.sub.2] is the sum of c and d; [c.sub.1] is the sum of a and c; [c.sub.2] is the sum of b and d; and the observed correlation coefficient from step 2, [r.sub.XY], is replaced with, [r.sub.xx'], the consistency coefficient for location X (e.g., New Mexico).

Step 4

Using equation (2), a true correlation coefficient is then generated for each of the possible state pairings (New Mexico and Alaska, New Mexico and Hawaii, Hawaii and Alaska). Where [r.sub.XY] denotes the observed correlation coefficient for locations X and Y (e.g., New Mexico and Hawaii) generated in step 2; [r.sub.xx'] is the consistency coefficient generated in step 3 for location X (e.g., New Mexico); [r.sub.YY'] is the consistency coefficient generated in step 3 for location Y (e.g., Hawaii); and R is the true correlation coefficient for locations X and Y.

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (2)

Step 5

The true correlation coefficients obtained in step four are then used in equations (3)-(6) to obtain true cell frequencies--the frequency that each pair of locations would agree and disagree in the absence of randomness. Where [R.sub.1], [R.sub.2], [C.sub.1], and [C.sub.2] equate, respectively, to [r.sub.1], [r.sub.2], [c.sub.1], and [c.sub.2] from equation (1); R denotes the true correlation coefficient from equation (2); A and D denote true cell frequencies of agreement; and [BETA] and C denote true cell frequencies of disagreement.

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (3)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (4)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (5)

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (6)

Step 6

The true cell frequencies found in step 5 are represented for each pair of states (e.g., New Mexico and Hawaii) using true frequency tables (e.g., Table 3). In Table 3, for example, cells A and D indicate true cell frequencies of agreement for New Mexico and Hawaii in regards to household electricity consumption, while cells [BETA] and C indicate true cell frequencies of disagreement.

Step 7

Finally, using the estimated true cell frequencies from step 6 as input for equation (7) PA is estimated for each pair of states, where A, B, C, and D correspond to true cell frequencies estimated from equations (3)-(6); and PA denotes the estimated agreement between locations X and Y (e.g., New Mexico and Hawaii) in the absence of randomness.

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (7)

Results

Of the three state pairs considered. New Mexico and Hawaii had the highest PA with a PA of 0.92. Hawaii and Alaska had a PA of 0.59, and New Mexico and Alaska had a PA of 0.58. Based on these results, the proposed field experiment on statewide residential electricity consumption should use New Mexico and Hawaii as treatment and control locations. Relative to the other state pairings, use of these two states as treatment and control locations would help to ensure that any observance of a change in electricity consumption is the result of the intervention rather than alternative systematic factors. In addition, use of this method in the selection of field experiment locations should help to rule out alternative explanations (e.g., regression to the mean) for observed effects (Figure 1).

DISCUSSION

Following the aforementioned steps, the method of Trafimow et al. (2015) can be readily applied to archival data to identify optimal treatment and control locations for field experiments. Use of this method is particularly attractive for archival data since such data often include a large number of data points. In the present paper a relatively large number of data points were obtained from the U.S. Energy Information Administration (eia.gov); however, as previously described, data of a similar nature can be found from a number of credible online sources (e.g., data.gov, census.gov, healthdata.gov, cdc.gov, etc.). Obtaining an adequate number of data points is important when using this method since the deleterious effects of low consistency coefficients is offset by large sample sizes. As a result, using larger data sets increases the accuracy of the PA estimate calculated for each location pairing (see Figure 1 in Trafimow et al. 2015).

In addition, the application of this method to field data makes clear that it is important to obtain adequate consistency coefficients for each location considered (e.g., New Mexico, Alaska, Hawaii). This is important because higher consistency coefficients result in more accurate PA estimates for each location pairing, especially when sample size is low (e.g., [NU] < 60). This was demonstrated in the simulations conducted by Trafimow et al. (2015); however, the consistency coefficient values (i.e., 0.7, 0.8, and 0.9) used in these simulations for each hypothetical location were user-defined and did not come from actual field data. While this was needed to standardize the testing of the method, it left open the question as to how to obtain consistency coefficients when working with actual field data. In the present example, consistency coefficients for each location were generated by assessing the extent that neighboring months (e.g., January and February, February and March, and so on) agreed in their residential electricity consumption. Neighboring months agreed when they were both above or both below their monthly average in electricity consumption. However, depending on the nature of the data (e.g., matching repeated measures), alternative methods may generate higher consistency coefficients (for instance, see Trafimow, MacDonald, and Rice 2012). Comparing alternative methods for estimating consistency coefficients is beyond the scope of the present work; however, doing so would broaden the applicability of the current method.

[FIGURE 1 OMITTED]

Another potential problem not solved by the present research is how to best determine agreement and disagreement between data points from each location. We addressed this issue by comparing the average electricity consumption of a given month to average monthly electricity consumption. However, the median, rather than the mean, could have been used with potentially similar results. At present, there is a lack of empirical testing across different types of data (e.g., hourly, daily, monthly) to indicate the ideal approach. Using diverse datasets and outcome measures, future research could work toward identifying best practices and boundary conditions for the method proposed.

POLICY ASSESSMENT

In addition to selecting optimal field experiment locations, the current method may also aide in the challenging and oft debated task of policy assessment. For instance, over the last four decades U.S. environmental policy has focused on regulating energy consumption at the state and national levels. From the late 1970s, California has mandated building codes meant to reduce the energy used in typical buildings by as much as 80% (Geller et al. 2006). More recently, the Obama Administration has promoted energy regulations to reduce greenhouse emissions to 17% below 2005 levels by 2020. Proponents of such legislature encourage states to follow California's lead and adopt stricter energy regulations, arguing that such regulations have resulted in California using less electricity per capita relative to other U.S. states. However, critics argue that California's success can be attributed to coincidental factors such as population, climate, and demographics (Levinson 2014). Such disagreement over the efficacy of environmental policies at the state level highlights the need to further develop the assessment of statewide energy regulations.

Traditionally, a top-down or bottom-up approach has been used when assessing the effectiveness of energy regulations. The bottom-up approach relies on engineering estimates regarding the efficiency gains resulting from improved products (e.g., appliances). For instance, this approach would assume that mandating appliances to be 20% more efficient would result in consumers using 20% less energy when using these appliances. However, such estimates are often optimistic and fail to account for how consumption demands may change as a result of using more efficient products (e.g., Jevons paradox; York 2006). The top-down approach, on the other hand, attempts to control for differences between comparison groups using a regression framework. For instance, models used to compare electricity consumption between states may attempt to control for factors such as main heating fuel (e.g., electricity, natural gas), cooling equipment (e.g., central air conditioning, window/wall units), and housing type (e.g., apartments, mobile homes, single-family homes; U.S. Energy Information Administration 2015).

However, as in field experimentation, regression-based approaches for policy assessment are problematic for the same reasons described earlier. The current method could overcome many of the methodological challenges facing policy assessment. The stepwise procedure used to determine optimal control and treatment locations for field experiments could be employed to compare states with and without a given policy. In particular, PA estimates between a policy-present state and policy-absent states could be used to determine which policy-absent state the policy-present state should be compared on an outcome variable of interest (e.g., residential household electricity consumption). In this way, policy could be assessed by identifying the optimal location (e.g., policy-absent state) to serve as a comparison.

REFERENCES

Allcott. Hunt and Sendhil Mullainathan. 2010. Behavior and Energy Policy. Science. 327 (5970): 1204-1205.

Algesheimer, Rene. Sharad Borle, Utpal M. Dholakia, and Siddarth S. Singh. 2010. The Impact of Customer Community Participation on Customer Behaviors: An Empirical Investigation. Marketing Science, 29 (4): 756-769.

Allen. Mary J. and Wendy Yen. 2001. Introduction to Measurement Theory. Long Grove, IL: Waveland Press.

Anderson, Eric T. and Duncan I. Simester. 2001. Are Sale Signs Less Effective When More Products Have Them? Marketing Science, 20 (2): 121-142.

Ayres, Ian. Sophie Raseman, and Alice Shih. 2013. Evidence from Two Large Field Experiments that Peer Comparison Feedback can Reduce Residential Energy Usage. Journal of Law, Economics, and Organization, 29 (5): 992-1022.

Baca-Motes, Katie, Amber Brown. Ayelet Gneezy, Elizabeth A. Keenan, and Leif D. Nelson. 2013. Commitment and Behavior Change: Evidence from the Field. Journal of Consumer Research, 39 (5): 1070-1084.

Card, David and Alan B. Krueger. 1994. Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania. The American Economic Review, 84 (4): 772-793.

Collins, Michael and Collin M. O'Rourke. 2010. Financial Education and Counseling--Still Holding Promise. Journal of Consumer Affairs, 44 (3): 483-498.

Conley. Timothy and Christopher Taber. 2011. Inference with "Difference in Differences" with a Small Number of Policy Changes. The Review of Economics and Statistics. 93 (1): 113-125.

Costa, Dora L. and Mathew E. Kahn. 2013. Energy Conservation "Nudges" and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment. Journal of the European Economic Association. 11 (3): 680-702.

Fitzpatrick. Katie. 2015. Does "Banking the Unbanked" Help Families to Save? Evidence from the United Kingdom. Journal of Consumer Affairs. 49 (1): 223-249.

Foster, Michael. 2003. Propensity Score Matching: An Illustrative Analysis of Dose Response. Medical Care.41 (10): 1183-1192.

Geller, Howard. Philip Harrington. Arthur H. Rosenfeld. Satoshi Tanishima. and Fridtjof Unander. 2006. Polices for Increasing Energy Efficiency: Thirty Years of Experience in OECD Countries. Energy Policy, 34 (5): 556-573.

Gulliksen, Harold. 1987. Theory of Mental Tests. Hillsdale. NJ: Lawrence Erlbaum Associates Publishers.

Harrison, Glenn W. and John A. List. 2004. Field Experiments. Journal of Economic Literature, 42 (4): 1009-1055.

Heckman. James J. 1998. Detecting Discrimination. The Journal of Economic Perspectives, 12 (2): 101-116.

Johnson, Eric J. and Daniel Goldstein. 2003. Do Defaults Save Lives? Science, 302 (5649): 1338-1339.

Karlin. Beth. Joanne F. Zinger. and Rebecca Ford. 2015. The Effects of Feedback on Energy Conservation: A Meta-Analysis. Psychological Bulletin, 141 (6): 1205-1227.

Levav, Jonathan and Rui J. Zhu. 2009. Seeking Freedom through Variety. Journal of Consumer Research, 36 (4): 600-610.

Levinson, Arik. 2014. California Energy Efficiency: Lessons for the Rest of the World, or Not? Journal of Economic Behavior and Organization, 107: 269-289.

List, John A. and Michael K. Price. 2009. The Role of Social Connections in Charitable Fundraising: Evidence from a Natural Field Experiment. Journal of Economic Behavior & Organization, 69 (2): 160-169.

List, John A., Sally Sadoff. and Mathis Wagner. 2011. So You Want to Run an Experiment, Now What? Some Simple Rules of Thumb for Optimal Experimental Design. Experimental Economics, 14 (4): 439-457.

Lord, Frederic M.. Melvin R. Novick. and Allan Birnbaum. 1968. Statistical Theories of Mental Test Scores. Boston. MA: Addison-Wesley.

McCracken. Vicki [ALPHA].. Robert D. Boynton. and Brian F Blake. 1982. The Impact of Comparative Food Price Information on Consumers and Grocery Retailers: Some Preliminary Findings of a Field Experiment. Journal of Consumer Affairs, 16 (2): 224-240.

Molfese, Victoria J.. Arlene Modglin. and Dennis Molfese. 2003. The Role of Environment in the Development of Reading Skills: A Longitudinal Study of Pre-School and School-Age Measures. Journal of Learning Disabilities, 36 ( 1): 59-67.

Moulton, Stephanie. Cazilia Loibl. Anya Samak. and Michael Collins. 2013. Borrowing Capacity and Financial Decisions of Low-to-Moderate Income First-Time Homebuyers. Journal of Consumer Affairs, 47(3): 375-403.

Roe, Brian. Mario F. Teisl, Huaping Rong, and Alan S. Levy. 2001. Characteristics of Consumer-Preferred Labeling Policies: Experimental Evidence from Price and Environmental Disclosure for Deregulated Electricity Services. Journal of Consumer Affairs, 35 (1): 1-26.

Rosenbaum. Paul R. and Donald B. Rubin. 1983. The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika, 70 (1): 41 -55.

Trafimow, David, James M. Leonhardt. Mihai Niculescu, and Collin Payne. 2015. A Method for Evaluating and Selecting Field Experiment Locations. Marketing Letters: 1-11. DOI: 10. 1007/sl 1002-014-9345-7

Trafimow, David. Justine A. MacDonald, and Stephen Rice. 2012. Using PPT to Account for Randomness in Perception. Attention, Perception, and Psychophysics, 74 (6): 1355-1365.

Trafimow. David and Stephen Rice. 2008. Potential Performance Theory (PPT): A General Theory of Task Performance Applied to Morality. Psychological Review, 115 (2): 447-462.

Trafimow, David and Stephen Rice. 2009. Potential Performance Theory (PPT): Describing a Methodology for Analyzing Task Performance. Behavior Research Methods, 41 (2): 359-371.

Turkheimer, Eric. Andreana Haley. Mary Waldron. Brian D'Onofrio. and Irving Gottesman. 2003. Socioeconomic Status Modifies Heritability of IQ in Young Children. Psychological Science, 14 (6): 623-628.

U.S. Energy Information Administration. 2015. Drivers of U.S. Household Energy Consumption, 1980-2009. Washington, DC: U.S. Department of Energy, http://www.eia.gov/analysis/studies/buildings/households/.

Yeomans, Mike and David Herberich. 2014. An Experimental Test of the Effect of Negative Social Norms on Energy-Efficient Investments. Journal of Economic Behavior and Organization, 108: 187-197.

York. Richard. 2006. Ecological Paradoxes: William Stanley Jevons and the Paperless Office. Human Ecology Review, 13(2): 143.

JAMES M. LEONHARDT, DAVID TRAFIMOW, AND MIHAI NICULESCU

James M. Leonhardt (jleonhardt@unr.edu) is an Assistant Professor of Marketing at University of Nevada, Reno. David Trafimow (dtrafimo@nmsu.edu) is a Professor of Psychology and Mihai Niculescu (niculem@nmsu.edu) is an Associate Professor of Marketing, both at New Mexico State University.

(1.) U.S. Energy Information Administration. Independent Statistics and Analysis. Retrieved February 10. 2015 from www.eia.gov/electricity/data.cfm#sales
TABLE 1
Observed Frequency of Agreement (cells a and d) and Disagreement (cells
b and c) for New Mexico and Hawaii

                                        New Mexico
                            Above (column 1)   Below (column 2)

Hawaii      Above (row 1)            a = 104            b = 26
            Below (row 2)            c = 31             d = 127
                             [c.sub.1] = 135    [c.sub.2] = 153

Hawaii      [r.sub.1] = 130
            [r.sub.2] = 158

TABLE 2
Observed Frequency of Agreement (cells a and d) and Disagreement (cells
b and c) between Neighboring Months for New Mexico

                                             First Month
                                  Above (column 1)   Below (column 2)

Second Month      Above (row 1)            a = 110            b = 24
                  Below (row 2)            c = 13             d = 128
                                   [c.sub.1] = 123    [c.sub.2] = 152

Second Month      [r.sub.1] = 134
                  [r.sub.2] = 141


TABLE 3
True Frequencies of Agreement (cells A and D) and Disagreement (cells B
and C) for New Mexico and Hawaii

                                        New Mexico
                            Above (column 1)   Below (column 2)

Hawaii      Above (row 1)            A = 121.48         B = 8.52
            Below (row 2)            C = 13.52          D = 144.48
                             [C.sub.1] = 135    [C.sub.2] = 153

Hawaii      [R.sub.1] = 130
            [R.sub.2] = 158
COPYRIGHT 2017 American Council on Consumer Interests
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Trends and Applications
Author:Leonhardt, James M.; Trafimow, David; Niculescu, Mihai
Publication:Journal of Consumer Affairs
Geographic Code:1USA
Date:Jun 22, 2017
Words:5451
Previous Article:Accounting for complex sample designs in analyses of the Survey of Consumer Finances.
Next Article:The road to unintended consequences is paved with motivational apps.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters