Nonparametric analysis of matched pairs data in business research.

ABSTRACT

Nonparametric methods of statistical analysis frequently utilized as alternatives to traditional statistical methods based on normal theory assumptions. Benefits of the use of nonparametric methods include wider applicability in terms of level of measurement required and less stringent distributional assumptions as well as the opportunity for increased statistical power. In this paper, we demonstrate an important fundamental weakness of normal theory based methods for certain data structures. This weakness is illustrated in the context of the analysis of the results of a matched pairs experiment. Additionally, we apply the use of nonparametric analysis of data on the change in dividend yield subsequent to tax policy changes for a study of tax policy effects in four countries. Among our conclusions is that the combined use of normal theory and nonparametric methods can uncover effects not identified by either single approach.

Keywords: Nonparametric statistical methods, dividend tax policy, matched pairs study

1. INTRODUCTION

Nonparametric methods of statistical analysis are frequently presented as alternatives to traditional statistical methods based on normal theory assumptions. Common reasons given for their use include the level of measurement of the data, and the validity of such methods under less stringent distributional assumptions. For example, nonparametric tests such as the Wilcoxon signed rank test, Mann-Whitney test, and the Kruskal-Wallis test are based only on some form of ranking of the variable of interest and hence applicable in situations where traditional t and F tests are not. Likewise, such tests do not require normally distributed data but only less restrictive conditions such as symmetry of distribution for the Wilcoxon signed rank test and consistent distributional shape across populations for the Mann-Whitney and Kruskal-Wallis tests.

Another important reason for employing nonparametric methods, often given less emphasis in applied statistics textbooks designed for business researchers, is the potential for increased power of statistical tests under certain distributional structures. For example, the Wilcoxon signed rank test is more powerful than the t-test when the data are generated from non-normal distributions with so called heavy tails (Hollander and Wolfe, 1999). While such possibilities are well known, the practical implications of such results may not be as well understood by business researchers.

Additionally, when data are from asymmetric distributions, investigations concerning location parameters will depend on the parameter used as a measure of location. For example, means and medians will differ under such distributional structures. Correspondingly, traditional methods of inference may be valid for problems formulated using means, while nonparametric methods of inference provide appropriate analyses for problems involving medians. For multigroup problems in which the researcher is investigating possible differences among groups, both types of inference may be needed to more completely identify possible differences among groups.

In this paper, we provide some analysis that focuses on the potential advantage of nonparametric methods of analysis. We begin with a hypothetical data set which illustrates the practical implications of such potential advantages as well as illustrating a disturbing feature of parametric tests when applied to data including outliers. The remainder of the paper discusses the analysis of the results of a study involving an evaluation of the relationship between tax policy and corporate dividend policy for each of four countries. Again, by virtue of the analysis of these data sets, we demonstrate the practical importance to researchers of the decision to utilize nonparametric analysis. Our analysis and discussion provide evidence that neglecting to apply nonparametric methods may result in the failure to identify important practical findings in at least one instance as well as illustrating some possible limitations and differences between the use of parametric and nonparametric methods in other instances. We conclude with a discussion of some possible advantages in using both approaches before determining any final conclusions.

2. A DEFECT OF MOST PARAMETRIC TESTS.

Consider a hypothetical experiment in which a characteristic of interest is measured for a sample of cases both before and after some event takes place. For example, the event might be the changeover to a new management team and a job satisfaction measure might be obtained for a sample of employees both before and after the transition to the new team. Data resulting from such an experiment are commonly referred to as matched pair data. Standard methods of analysis of matched pair data involve computing the differences between the before and after measurements and then analyzing this sample of differences. Both parametric methods, such as one-sample t-tests and confidence intervals, as well as nonparametric tests and intervals could be applied to such data.

Suppose that a study is conducted in which the dividend yields of a sample of companies are compared before and after a change in tax policy affecting corporations. The results of such a study are described and analyzed in the next section. For now, we wish to consider some hypothetical data that might result from such a study. Suppose that a sample of nine companies is reviewed and it is found that the changes in dividend yields after the tax law change are as follows: 2,5,6,-6,5,3,1,10,8. (Note that these values are percentages) Here, positive values indicate an increase in dividend yield while negative values indicate a decrease.

Statistical testing could be applied to this data to attempt to confirm that the tax policy change resulted in increased dividend yields, i.e., the mean change in dividend yield is greater than zero. To test this hypothesis, each of the t-test (T), sign test(S), and the Wilcoxon signed rank test (W) were applied to this sample. The resulting P-values were .020, .0195, and .033, respectively. Thus each test criterion indicated substantial evidence to support the theory that the dividend yields have increased and each resulted in statistical significance at the 0.05 level.

Now suppose that it is discovered that there was a tenth company that was analyzed and that for this company the dividend yield increased by 50. Intuitively, this new information would appear to provide substantial additional support for the theory that the tax policy is associated with the higher dividend yields and it would be anticipated that formal methods of data analysis would confirm this expectation. Unfortunately, this does not happen with the T test. In fact, when the data based on all ten companies are reanalyzed, the P-values for the three tests are .058, .011, and .018, for the T, S, and W tests, respectively. Thus, adding the information from the tenth company results in the T test indicating there is LESS evidence supporting the theory of increased dividend yields and, in fact, causes the test to no longer be significant at the 0.05 level. It would be very difficult to explain the logic of this change in conclusion to the researcher conducting such a study. Notice that the nonparametric tests, S and W, result in lower P-values with this new information, as would be expected. It is not difficult to confirm that, in general, tests such as these will always result in lower P-values if additional data that strongly supports the alternative hypothesis is added to the previously analyzed data set.

To see why the T test behaves in this disturbing manner, it is instructive to consider the corresponding 95% confidence intervals based on the t-distribution. For the original nine companies, this interval is (0.22,7.34) centered at 3.78, while after adding the tenth company the interval becomes (-2.5,19.3) centered at 8.4. Thus, even though the estimated mean increases from 3.78 to 8.4 with the addition of the tenth company, the increased variability results in confidence limits that are so much wider that the lower limit is negative. Thus, the increased sample variability results in so much uncertainty that the T test is no longer significant at the 0.05 level. This uncertainty is related to a low power for the T test in the presence of outliers such as company ten. This occurs, of course, because the T test and interval are based on the theory that the sample is generated from a normal distribution and so an outlier such as the tenth company is not anticipated.

In summary, we believe this example demonstrates that the T test can result in results completely inconsistent with what a researcher might refer to as "common sense". Such a result is most likely to result from small samples with a small number of outliers. Other parametric tests for comparing means (e.g. t and F tests) are equally vulnerable to this problem. Nonparametric tests which are based on ranks, due to their insensitivity to outliers, are not susceptible to this disadvantage.

3. ANALYSIS OF DIVIDEND YIELD DATA

In this section, we further contrast the use of parametric and nonparametric analysis of data from matched pairs experiments by considering the results of research conducted by Nweke (Nweke, 1994). As part of this study, Nweke considered the effects of a change in tax policy for dividends on the dividend yield of a corporation. This was done for each of four countries for which such a tax policy change was identified, namely, Australia, France, Germany, and Japan. For each of the four countries, dividend yields were obtained both before and after the tax policy change. The number of corporations in the samples was 37, 107, 80, and 251, respectively, for Australia, France, Germany, and Japan. Some statistical summaries of the difference in dividend yield, computed as before minus after, for corporations within each country are presented in Table 1 along with a set of side by side boxplots of the data for the four countries. For more details on the timing and type of tax policy change, selection of corporations for the samples, and methods for obtaining dividend yield data, see (Nweke, 1994). Here, we limit discussion to appropriate analysis of the data just described for the purpose of attempting to identify any changes in dividend yield policy subsequent to the tax law changes.

The lower portion of Table 1 provides a summary of relevant statistics generated from the data for each of the four countries. P-test of normality refers to the P-value from a statistical test of normality. The test used here is the Shapiro-Wilk test (Shapiro and Wilk, 1965). Next, we present the two-tailed P-values associated with the t-test, Sign(S) test, and Wilcoxon signed rank test (W). These will be labeled P-T, P-S, and P-W, respectively, in what follows. Lastly, we include corresponding confidence interval estimates generated from the T, S, and W statistics. These are labeled TINT, SINT, and WINT, respectively. For more information on the procedures for generating SINT, and WINT, see (Hollander and Wolfe, 1999).

We begin with analysis of the data for Germany. Notice that the values of the kurtosis and skewness are close to zero and that the test of normality indicates this sample is consistent with the normality assumption. Accordingly, we can anticipate general consistency in the P-values associated with the T, S, and W tests and, in fact, that is what appears in Table 1. None of these tests indicate much evidence of a change in mean dividend yield with P-values ranging from .626 to .734. For this data, use of the nonparametric analysis leads to the same general conclusions as the parametric analysis.

Next, we discuss the data for France and Australia since these data sets have a number of common features. First, observe that both data sets exhibit kurtosis values substantially larger than zero as well as skewness values indicating some asymmetry. In view of these observations, it should not be surprising that the tests of normality indicate clear evidence of non-normality with a P-value of .000 for each of the two countries. This evidence of non-normality means that the T test results must be interpreted with caution. Notice that the P-values associated with the T test are much smaller than the corresponding P-values for the other two tests and that this is true for each of the two countries. Next, we discuss why this large discrepancy in P-values occurs.

Since the data is not normally distributed, one possible explanation for the observed discrepancy in P-values is that the T test is sensitive to the non-normality and hence the actual significance level is much higher than the stated level leading to a tendency to obtain false significant results. However, (Chaffin and Rhiel, 1993) show that the level of the T test is robust to changes in skewness and kurtosis of the magnitude encountered here so this explanation is unlikely.

On the other hand, it must be remembered that, for asymmetric data, the T test tests hypotheses concerning change in MEAN while the S and W tests refer to changes in MEDIAN. Accordingly, a more probable explanation of the observed discrepancy is that there is some evidence (P - T = .113 for Australia and P - T = .0564 for France) of a shift in mean dividend yield after the tax change but less conclusive evidence (from P-S and P-W) of any shift in median. This explanation is enhanced by viewing the confidence intervals in Table 1, TINT, SINT, and WINT. For example, for France, TINT is (-0.11,8.27) with a mean estimate of 4.08 while SINT is (-3.42,2.72) and WINT is (-1.98,4.63) with median estimates of -1.18 and .96, respectively. Again, these intervals and estimates indicate some evidence of a shift in mean but little evidence of any shift in median. Similar remarks apply to the Australia data though the evidence of the mean shift is not as strong. Notice that the application of both parametric and nonparametric methods of analysis permits the possibility of identifying possible changes in either or both of these characteristics of the distribution of differences.

The contrast in results between the parametric and nonparametric analyses suggests that the parametric analysis may have been strongly influenced by the presence of a few extreme values. To investigate this possibility, we reanalyzed the data for Australia without the three most extreme differences. The resulting P-values for the T and W test were .78 and .77, respectively. Thus, without the extreme values, the two tests provide comparable results.

Next, we discuss analysis of the data for Japan. Again the test for normality along with the skewness and especially the large, positive kurtosis provide strong evidence of non-normality. In addition, the large kurtosis indicated a situation in which the S and W tests are likely to be substantially more powerful than the T test. This higher power means that the nonparametric tests are more likely to uncover any significant changes in dividend yields if they have occurred. Upon observing the P-values from the T, S, and W tests, we see evidence of precisely this phenomenon. An analysis based only on the T test would fail to find significant changes at the 0.05 level (P - T = .079). However, analysis based on the S or W test indicated very strong evidence of a change in dividend yields (P - S = P - W = .000). Thus, failure to apply a nonparametric analysis would lead to some doubt about the existence of any effects when it appears from the nonparametric analysis that such effects maybe substantial. Further evidence for this conclusion is seen in the corresponding interval based on T. This is also consistent with the theory comparing these methods for distributions with large positive values of kurtosis. In summary, the nonparametric analysis indicates clear evidence of a median shift while the parametric analysis is inconclusive.

4. CONCLUSION

The examples presented in this paper have illustrated a number of differences between parametric and nonparametric analysis for matched pairs data. Both the hypothetical data set and the Japan data demonstrate the importance of considering nonparametric methods in order to effectively identify an existing treatment effect. Data for Australia, France, and, to a lesser degree, Japan, all show differences in conclusions from the two analysis approaches which can at least partially be attributed to the presence of some skewness and hence to a difference in parameter characteristics which the methods are designed to test or estimate.

These results suggest that it may be prudent for the data analyst to apply both methods. If the results concur, as with the Germany data, then no new difficulties arise. If the results differ, then our examples indicate that a more careful inquiry into the reasons for any such differences is appropriate. Of course, in reporting results subsequent to such analyses, it would be improper to report only the results of the most significant test. Instead, all test results conducted should be reported. In doing so, the interpretation of the significance level for the complete analysis becomes somewhat clouded due to the multiple tests conducted. Another alternative is to use data characteristics to decode which form of analysis to conduct. Formal, systematic approaches to incorporating this idea are known as adaptive tests. See (Randles and Wolfe, 1979), p. 386, for details and reference on adaptive tests for analysis of matched pairs data.

In conclusion, we believe that the examples presented suggest the need for researchers to include nonparametric methods of analysis when evaluating the results of matched pair experiments and that the combination of both parametric and nonparametric analyses offer some advantages over the traditional approach of routinely applying only parametric methods unless some form of "red flag" suggests problems might occur.

REFERENCES:

Chaffin, W .W. and Rhiel, G.S. ,"The Effect of Skewness and Kurtosis on the One-Sample T-test and the Impact of knowledge of the Population Standard Deviation", Journal of Statistical Computation Simulation, Vol. 46, 1993, 79-90.

Hollander, M. and Wolfe, D. A., Nonparametric Statistical Methods, John Wiley Publishing, New York, 1999.

Nweke, C.E. Taxes and Dividend Policy: An International Study, D.B.A. dissertation, Old Dominion University, 1994.

Randles, R. H. and Wolfe, D. A., Introduction to the Theory of Nonparametric Statistics, John Wiley Publishing, New York, 1979.

Shapiro, S.S. and Wilk, M.B., "An Analysis of Variance Test for Normality", Biometrika, Vol. 52, 1965, 591-611.

Carol A. Markowski, Old Dominion University, Norfolk, Virginia, USA

Edward P. Markowski, Old Dominion University, Norfolk, Virginia, USA

Dr. Carol A. Markowski is Professor of Decision Sciences at Old Dominion University. She received her Ph.D. in 1980 in Operations Research and Industrial Engineering from The Pennsylvania State University. Her research interests focus on the application of statistical and management science theory to the solution of business problems. She has published in a variety of journals such as Decision Sciences and the European Journal of Operational Research.

Dr. Edward P. Markowski is University Professor of Decision Sciences at Old Dominion University. He received his Ph.D. in 1980 in Statistics from The Pennsylvania State University. His research interests include robust statistical methods, improving methods of teaching statistics, and application of structural equation modeling to business research. He has published in a variety of journals such as Decision Sciences, the Journal of the American Statistical Association, and the Journal of Services Marketing.

TABLE 1
SUMMARY STATISTICS FOR DIVIDEND YIELD CHANGE

[ILLUSTRATION OMITTED]

                             German         Australia
Count                            80                37

Mean                          1.339            -7.501
Median                        0.600            -2.320
Standard Deviation           24.506            28.083
Kurtosis                      0.698             3.935
Skewness                      0.251            -1.756
P-test of Normality           0.665             0.000

Pvalue, T test                0.626             0.113
Pvalue, Sign test             0.734             0.743
Pvalue, W test                0.682             0.287

                            German         Australia

TINT                  (-4.12, 6.79)   (-16.87, 1.860)
SINT                  (-2.65, 4.44)   (-15.20, 6.37)
WINT                  (-3.93, 5.67)   (-11.00, 3.40)

                             France           Japan
Count                           107             251

Mean                          4.078           2.386
Median                       -1.180           4.000
Standard Deviation           21.864          21.435
Kurtosis                      2.899           5.391
Skewness                      1.368          -1.535
P-test of Normality           0.000           0.000

Pvalue, T test                0.056           0.079
Pvalue, Sign test             0.694           0.000
Pvalue, W test                0.494           0.000

                             France           Japan

TINT                  (-0.11, 8.27)   (-0.31, 5.03)
SINT                  (-3.47, 2.72)   (2.99, 6.02)
WINT                  (-1.98, 4.63)   (2.50, 6.04)