# Is it content or style? An evaluation of two competitive measurement models applied to a balanced set of ethnocentrism items (1).

1. IntroductionMany attitude surveys use sets of items with identical response scales in order to create attitude constructs. Survey questions are often formulated using Likert scales. Respondents are asked to indicate how strongly they agree or disagree with each attitude statement. There is considerable proof that such a response format can be susceptible to an "agreeing-response bias" called acquiescence (Billiet and McClendon 2000). Previous research has shown that it is possible to single out this response bias in a structural equation model (Billiet and McClendon 2000) and that this leads to better fitting factorial invariant models for cross-cultural surveys (Billiet et al. 1999). The style factor that is used to model acquiescence in the structural equation model correlates very strongly (.90) with a variable that counts the number of agreements to a balanced set of items. Others who tried to model acquiescence with the structural equations approach were confronted with the phenomenon that models with a positive and a negative factor appeared to be as likely as models with a bipolar factor and style (acquiescence) factor (Mirowsky and Ross 1991; McClendon 1992). This is a well-known and often debated phenomenon in psychometry and sociometry: oppositely worded indicators that were constructed to operationalize one (bipolar) concept are often found to represent two different factors (Marsh 1996:810). Some researchers favor a substantive explanation of this phenomenon, arguing that this result (two different factors) is a correct. Others, like us, promote a methodological explanation (Marsh 1996; Saris 1988a, 1988b; van Schuur and Kiers 1994).

Using cross-cultural data, we will evaluate these two competitive measurement models, applied to a balanced set of ethnocentrism items, using a structural modeling approach. We will argue that controlling for acquiescence in balanced scales not only leads to a better fitting model in bi-cultural research, but in multi-cultural research as well. Three arguments will be studied. After evaluating the fit of the model with one (bipolar) factor and a style factor and the model with two-content factors, we will discuss which of these two approaches is best for obtaining construct equivalence in cross-cultural research. Construct equivalence is achieved if the instrument measures the same latent trait in all of the cultural groups under investigation (van de Vijver and Leung 1997). Rensvold and Cheung (1998) define construct equivalence operationally as factorial invariance. This means that a construct is equivalent across two cultural groups if the loading of a certain indicator on that construct in the first grou p can be set equal to the corresponding loading in the second group and for all indicators of the construct. For the evaluation of construct equivalence in multi-group settings, see Welkenhuysen-Gybels et al. (2000), Billiet (2000) and Billiet et al. (1999).

The emphasis of this paper will be on the third way of evaluation: the study of substantial differences between the two models. A close look will be taken at the relationships between the substantive construct(s) and relevant background variables for both types of measurement approaches. We will argue that in view of a decision about the competing measurement models, they should be evaluated in the context of a theoretical meaningful nomological network. When the relationships of the two-content factors with other relevant variables remain invariant, then the one-content and style-factor model is preferred. The two-content factor model is preferred when substantial and theoretically meaningful differences are observed in the relationships between the substantive concepts and the background variables.

2. Previous Research

2.1. Bi-cultural Research

Here, we will briefly present our previous research concerning the two alternative measurement models for specifying the empirical relations within a balanced set of items intended to measure the same concept: the two-content factor model versus the one-content factor and one-style factor model.

In Figure 1 the two alternative models are applied to a hypothetical example of a balanced set of six items, measuring the attitude towards ethnic minorities (OUTGROUP). In the two-content factors model, POSOUT consists of the positively worded items, and NEGOUT of the negatively worded items.

Both models were empirically tested for two balanced sets of items in two different Belgian populations, Flanders (Dutch speaking) and Wallonia (French speaking). The construct equivalence of the two-content factors and the style factor was also evaluated. Both the corresponding factor loadings ([LAMBDA]1=[LAMBDA]2) in the two cultural groups and the corresponding error (co)variances [epsilon]1 = [epsilon]2 are equal. Moreover, in the model with a style factor, the factor loadings of the indicator variables on the style factor are all identical because it is assumed that all indicators are equally susceptible to acquiescence.

The content factors are not allowed to correlate with the style factor, but in the base model, the variances of the other latent content factors are not constrained ([PSI]1i (1) [PSI]2i for i = Style) to be equal over the groups. Although a model with completely constrained latent factor invariance fits the data as well.

Since we are dealing with ordinal-scaled variables, the procedure proposed by Joreskog (1990) is used. (2) For this illustration, a Flemish (N=1,020) and a Walloon (N=1,012) random sample of nearly equal size from the 1995 Belgian General Election Survey were used.

The two balanced sets of indicators measure the concept "feeling threatened by immigrants" (an aspect of ethnocentrism, 6 items, 3 positive and 3 negative) and the concept "distrust in politics" (4 items, 2 positive and 2 negative). The two alternative models, one with two-content factors and the other with two-content factors and an additional method (or style) factor, fit the data very well. The fit-indices are reported in Table 1. The value of the RMSEA is lower than .05 and the values of NFL and GFI are close to 1. This indicates that a model with factorial invariance can be retained. Which of the two models should be preferred? The slightly better values of the fit-indices are not decisive. The difference in the chi-square value, however, clearly favours Model 2. The Chi-square value decreases 43.22 units for only two degrees of freedom, which is statistically significant (p<.000). (3)

A closer look at the parameters reveals that the variance of the style factor is considerably smaller than the variance of the content factors, but it is still significantly different from zero. The factor loadings of the style factor are also smaller than the loadings of the content factors. This is what we expected since the indicators are not intended to measure acquiescence, only feelings of threat towards immigrants or political distrust. The variances of the two latent variables (THREAT and DISTRUST) are nearly the same as in Model 1, and the correlations do not change much (.50 in the Flemish sample and .42 in the Walloon sample) across the two models.

2.2. Multi-Group Setting

Table 1 indicated that both models, the model with a style factor and the model without, yield factorial invariance. What can then be a reason for including a style factor? The answer to this question will become more obvious in an example with a larger number of cultural groups that not only differ in language, but also in institutional context and in the practical realisation of the data collection. In such a case, it is possible that error covariance can no longer be set equal to zero, which means that other unidentified sources of common variance are present. The introduction of a style or method factor will unify many of these disparate sources of common non-random variance in the indicators, and identify them as a well documented method or style effect, in this case the agreeing-response bias. We are quasi sure that it is indeed this bias since in the two samples, a variable "sum of agreements" (4) correlates more than .90 with the latent style factor (see Billiet and McClendon 2000).

In this example, the construct equivalence of the indicators for ethnocentrism in the 1995 samples of the International Survey Program (ISSP) will be evaluated. Nine Western European countries are selected for this illustration. Nine items indicate the OUTGROUP dimension of ethnocentrism, four of which are positively worded and four are negatively worded items (one item was neutral formulated).

In the models, all error co-variances are constrained to zero, and all the loadings on the style factor were set equal (except the loading of the neutral indicator because that item cannot be susceptible to an acquiescent response bias). On the basis of the last column in Table 2, we can conclude that the specification of a style factor that is significantly different from zero always results in a very significant drop in the Chi-square statistic. Hence, we may conclude that in all countries, the Liked items are susceptible to an agreeing-response bias and that models that specify the method factor perform much better than models without a method factor. The method factor behaves in precisely the same manner as was found in the Belgian data in the previous example. After the response scales were inverted in order to assign the maximum score (5) to the answering category "completely agree," all the factor loadings on the method factor have small, significant, positive, and equal loadings (except for the neutra l item) in nearly all countries. The covariance between the substantive factor (F) and the style factor (S) was fixed to zero ([[[PSI].sup.g.sub.F,S] = 0) in each country (g [member of] {1,...9}) because it is very unlikely that the agreeing-response bias is related to the substantive latent variable. The variance of the method factor ([[[PSI].sup.g.sub.S,S]) is always substantially lower than the variance of the content factor ([[[PSI].sup.g.sub.F,F]). Finally, the style factor correlates very strongly with a variable "sum of agreements." Hence, we may conclude that the method factor is the tendency to agree with Likert items.

3. Data and Methods: The RAMP Research

The analyses will be performed on (part of) the data of the 1999 RAMP (Religious and Moral Pluralism) dataset. (5) However, not all 11 countries in which the survey was conducted will be taken up in the analysis. (6) Only the Western European countries will be taken into account, namely: Belgium, Denmark, Finland, United Kingdom, Italy, The Netherlands, Norway, Portugal, and Sweden. The data for Belgium will be split up according to whether the respondents are French speaking or Dutch speaking. Hence 10 groups will be included in the analysis. The questionnaire contained (a translation of) a number of items in Likert format about the respondent's feeling of threat towards immigrants, foreigners, or ethnic minorities. The english version of these items is shown in Table 3. As required for the detection of acquiescence, the scale is balanced: it contains as many positively as negatively worded items.

To avoid loss of data due to missing values, an imputation technique was used. The imputation was conducted separately for the negatively and the positively worded items, and for the other items in this scale. (7) For the positively worded items, missing values were only imputed if the respondent had one missing value on these three items. In this case the missing value was replaced by the mean value of the responses on the other items. Respondents with more than one missing value were excluded from the analysis. An analogous method was used for the negatively worded items and for the INGROUP items. Hence, at most 3 values were imputed per observation.

Some methodological remarks can be made with respect to the RAMP data.

(1) The question wording of the positively worded items is often quite poor. In standardized questionnaires the use of double negatives is strongly prohibited (Converse and Presser 1986:13). For example, the item "immigrants are no less intelligent than the British," can easily cause respondents to misplace themselves on the Likert scale. In some countries the question wording was corrected towards "immigrants are as intelligent as [X]." This means that the question was not exactly the same in all countries. The rather low factor loadings of the positively formulated items on the substantive factor could be caused by this problem of question wording.

(2) Exploratory factor analysis on the ethnocentrism items showed large differences between countries. In some countries, three factors could be distinguished: negative attitude towards the OUTGROUP (negative items), feelings of threat (positive items; though strongly correlated with the first factor), and the attitude towards the own group (INGROUP). In other countries, a two-factor solution was found: this is one factor for the INGROUP and one factor for the OUTGROUP (positive and negative items together). Insufficient information about the context of the questionnaire (sampling procedure, interviewers, nonresponse, translation, and so on) makes it unclear whether those differences (both in factors as in factor loadings) are caused by methodological reasons or by cultural differences.

4. Evaluation of Two Competitive Measurement Models

The two competitive measurement models (two-content factors versus one-content and one-style factor) will be compared with respect to three criteria. First, we will take a close look at the fit of the two models for each country. Secondly, we will evaluate whether sets of equivalent countries obtained under the one-content and style-factor model include more countries than the sets obtained under the two-content factors model. And thirdly, the relationship of the two substantive factors of both models to some relevant background variables will be studied.

4.1 Different Measurement Models for Each Country

In order to evaluate the measurement models, a first step is to take a closer look at the fit of the different models. Three models can be distinguished: in the first model, six items load on one factor (OUTGROUP); in the second model a style factor is included (OUGROUP + STYLE). A third model contains two-content factors (POSOUT and NEGOUT) but no style factor.

Using structural equation modelling (LISREL 8.30), these three models were tested for every group in the analysis. The summary statistics for the different models can be found in Table 4.

Because all error-covariances were constrained to zero, the criterium that the Chisquare may not be larger than 3 times the degrees of freedom (Bollen 1989:278; Carmines and McIver 1981) is not always attained. The model with two-content factors and, especially, the model with a style factor, can be improved by allowing some error-covariances to differ from zero. The model with only one-content factor (OUTGROUP) never fits the data. The models with one-content and one-style factor and with two-content factors yield a much better fit than the latter model in every country. In the Scandinavian countries (Denmark, Finland, Norway and Sweden) the two-content factor model yields a lower [X.sup.2]-value, while in the other six countries the model with one-content and one-style factor is preferred. This is especially the case for Italy, The Netherlands, and Portugal.

Except for Portugal the variance of the style factor is smaller than the variance of the content factor. In each of the countries this variance is still significantly different from zero. The factor loadings of the style factor are smaller than the loadings of the content factor. The correlation between the style factor and "yessay" (the sum of agreements) is .6 or higher. The correlation between the two content factors varies from .23 (Italy) to .79 (Denmark). In the Scandinavian countries, this correlation is at least .62 (Norway).

4.2 Construct Equivalence for Ten Western European Countries

A second step in the evaluation of the two competitive models pertains to the evaluation of construct equivalence across countries. Do both measurement models lead to the same groups of equivalent countries?

When looking for groups of countries with equivalent constructs within a population of 10 countries, it is important to know where to start. In other words, it is important to know which countries should be combined first and where to go from there. Some researchers propose a top-to-bottom procedure, while others suggest a bottom-to-top way. In a recent paper, Welkenhuysen-Gybels et al. (2000) proposed a procedure in which all possible subsets of groups are explored. They developed a computer program, named TCEL (Testing Construct Equivalence in Lisrel), (8) which tests all possible subsets of groups in a dataset for equivalence with respect to their factor loadings. The results for this test for the RAMP dataset can be found in Tables 5 and 6.

These tables show that under the content/style factor model, construct equivalence can be obtained for more countries than under the two-content factor model. This is, however; not surprising: compared to the model with one-content factor (and a style factor), the two-content factor model has equality constraints on more factor loadings. In the one-content and one-style factor model, the equality constraints only pertain to the loadings of the content factor (6 loadings), where in the two-content model constraints are placed on all 12 loadings (no cross loadings were allowed). Steenkamp and Baumgartner (1998) define this as "configural invariance" (p.80). In essence, this principle states that the pattern of salient (nonzero) and nonsalient (zero or near zero) loadings should be the same across different countries.

According to the results of the TCEL analysis for the one-content and one-style model, the 10 Western European countries can be divided into two mutually exclusive equivalent groups. One equivalent group of countries consists of Flanders, Wallonia, U.K., Portugal, and Finland. The other group contains Denmark, The Netherlands, Norway, and Sweden. Hence, all countries (except Italy) can be ascribed to one of the two groups.

In the two-content factors model, a group of four countries (Denmark, The Netherlands, Sweden, and Norway or Portugal) can be combined with a group of three countries (Flanders, Wallonia, U.K. or Flanders, U.K., and Finland). In this model, not only Italy, but also two other countries cannot be attributed to one of the two groups.

4.3 The Relationship between the Two Models with other Relevant Variables

As a result of the above findings, one can conclude that in some countries the model with one-content factor and a style factor performs better, while in other (Scandinavian) countries the two-content factors model is better, although the correlation between the two-content factors is very high (.62). Achieving construct equivalence for the two-content factors model is more difficult than for the one-content and one-style factor model.

An important argument in the comparison of the two competitive measurement models will be the relationship of the ethnocentrism scale with some relevant background variables. The two models should be evaluated in the context of a theoretically meaningful network. If, in the two-content factor model, the relationship of each of the content factors with other relevant variables is similar, this implies that these factors function equivalently (Hui and Triandis 1985). The two-content factors can then be interpreted as "opposite sides of the same coin" and, therefore, should be merged. In other words, the one-content and style-factor model should be preferred over the two-content factor model. If, on the other hand, substantial and theoretically meaningful differences in the relationships between each concept and other variables are observed, the concepts do not function equivalently. In this case, the two-content factor model is the preferred model.

To examine this, a regression analysis was performed for four different countries: Flanders, Wallonia, The Netherlands, and Norway. The first three countries are selected because the model with one-content factor and a style factor performs (slightly) better than the model with two-content factors (which is still acceptable). Furthermore, some knowledge about the relationships between ethnocentrism and relevant background variables is available (see Billiet et al. 1996). Norway is selected because in this country, the two-content factors model performs best and the variable "education" is comparable with Belgium and the Netherlands. The selected background variables are gender, age, education, and church involvement. The variable age was centered around the mean. Church involvement was constructed on the basis of two variables: v98 (belonging to a church) and v87 (church attendance). The reference category consists of those who don't belong to any church (no religion, v98=2). Core church members are those wh o go to church at least once a week. Modal church members attend religious services monthly or yearly, while marginal church members rarely or never visit church. The factor scores of the respective models serve as the dependent variable.

All models were tested for normality, multicollinearity, heteroscedasticity, and linearity of the effect of the continuous dependent variables. The variable age is the only continuous variable in the model. In Flanders, Wallonia and Norway, the effect of age on the dependent variables was linear. The Dutch sample, however, shows a nonlinear effect of age. This effect should be interpreted as follows: the first coefficient for age is the effect of age for respondents that are less than 41.3 years old (2 years younger than the mean age, which is 43.3). The second coefficient for age is the additional effect of age for respondents older than 41.3 years. Hence, for the respondents who are more than 41.3 years old, the effect of age is equal to the sum of the two coefficients for age. A graphical account for this effect is shown in Figure 2. This additional effect was exactly the same for the three factors. Tables 7 to 10 present the results for each of the countries.

In Flanders (Table 7), there are only minor differences between the three regression models. Gender and church involvement are not significant (which is strange, since other research found a direct and indirect effect of church involvement on ethnocentrism, see Billiet et al. 1996). Age and (some categories of) education have a significant effect on ethnocentrism. One can conclude that the effect of age is exactly the same for the three factors: the level of significance (p<.01) and the estimates (.119 versus .108 and .119) do not vary at all.

Concerning the level of education, a small but significant effect for NEGOUT exists for those who completed secondary school (-.106). This effect does not occur for POSOUT, nor for OUTGROUP. The difference is negligible and the overall conclusion is that in Flanders the relationship between the two-content factors and the background variables is invariant. Hence in this country, the one-content and one-style factor model is preferred.

In Wallonia (Table 8), some minor differences occur. Again, gender is not a relevant variable, but the other variables show some significant effects. The effect of age is statistically signficant (p<.001) for POSOUT, but not for NEGOUT. The effect for OUTGROUP lies in between and is borderline significant. For modal church members, the estimates reflect significant effects for NEGOUT (and OUTGROUP), but not for POSOUT. The same conclusion goes for respondents with incomplete and complete secondary education. Do these figures indicate a substantial difference in the relationship between the two-content factors and the background variables? In other words, do we have to prefer the two-content factor model over the one-content and one-style factor model? Not in our opinion. Not only are the extra effects for NEGOUT barely significant (except for age), the estimates of all variables all point in the same direction. Furthermore, we already know that the question wording for the POSOUT items is methodologically inc orrect and that other research never found major differences in the attitude towards foreigners between the two communities of Belgium (Billiet et al. 1990; Dekker and Van Praag 1990; Delooz and Kerkhofs 1992; these studies are based on three different data sets): at least on the issue of foreigners, Belgians are remarkably unanimous. The small differences between the two-content factors in Wallonia, and the intermediate results for the model with a style factor, lead us to conclude that the OUTGROUP (and style) model is a fair reflection of the Wallonian opinion.

In The Netherlands (Table 9), there are also some small differences. The main effect of age is signficant (p<.05) for NEGOUT, but not for POSOUT (or OUTGROUP). For the additional effect for every age higher then 41.3, all factors show a similar significant effect. More relevant differences occur for church involvement: the significant effects for core church members and modal church members for POSOUT are not found for NEGOUT. The effect of having a university degree is similar for all factors, but again, the effect for secondary education is not present for NEGOUT. On the whole, however, all effects point in the same direction. Further research has to reveal whether or not these minor differences in relationships are substantial, before deciding that the two-content factor model has to be chosen.

Earlier in this paper, Norway was found to be a country in which the two-content factor model performed better than the one-content and one-style factor model. But the latter model could fit the data as well. So it is important to look for substantial differences between the POSOUT and NEGOUT factor and some background variables.

In Norway (Table 10), gender is a relevant variable, although the effect for NEGOUT is less significant. The effect of age is only significant for POSOUT. Church involvement is not relevant in Norway to explain the attitude towards foreigners. The effect of education is significant for respondents with a university degree, but the estimates for NEGOUT are much stronger (-.293/-.267) than the effects for POSOUT (-.128/-.155). The models for NEGOUT and POSOUT can be used, although the model with one-content factor (and a style factor) performs as good as the model with two-content factors (except for age).

5. Conclusion

It is a familiar finding that indicators which were meant to operationalize one (bipolar) concept, appear to form two correlated factors: one for the positively worded and one for the negatively worded items. In this paper, it is argued that this is due to an acquiescent response bias which occurs on items with a Likert format, and that the inclusion of a style factor instead of two-content factors leads at least to an equally good solution as the two-content factor model.

We tried to evaluate those two competitive measurement models, using the cross-cultural RAMP data for ten Western European countries. The evaluation is based on three criteria. First, a close inspection of the different models for each country was conducted. We could conclude that in some countries the model with one-content factor and a style factor yielded a better fit, while in the Scandinavian countries the two-content factors model performed better. In these countries, however, the correlation between these two factors was high (.62). A second method of evaluation pertained to construct equivalence across countries. Using a new developed computer program, named TCEL, it was found that larger sets of equivalent countries could be obtained for the one-content and one-style factor model. Nine of the ten Western European countries could be divided into two equivalent groups. In the two-content factor model, this was not possible. A third evaluation concerned the context of the factors. It was argued that if the relationships of the two-content factors with other relevant (background) variables were invariant, the one-content factor and one-style factor model was preferred. The two-content factor model was preferred when one observes substantial and theoretically meaningful differences in the relationships between each concept and other variables. This was tested for four countries. In Flanders, the two-content factors were invariant, so the earlier finding of a better fit for the one-content and one-style factor model was confirmed. In Wallonia, some minor differences exist, but due to some methodological reasons (question wording for the POSOUT items) and previous research (no differences between Flanders and Wallonia), the content and style-factor model was still preferred. In the Netherlands, all effects point in the same direction, but the effect of church involvement (and to a certain degree also age and education) is different for both content factors. In Norway, the two-content factor model performed best , and some very small differences between these two factors were found (age, gender, and education), so the two-content factor model should be preferred (although the content/style model fits the data as well and reflects the same effects for background variables).

Further research is necessary to explore the relationships of the competitive models to relevant background variables more thoroughly. For now, the conclusion is that a model with a content factor and a style factor fits the data at least equally good in most countries, leads more easily to construct equivalence and shows a fair reflection of the effects of relevant background variables. If the two-content factors showed different relationships to other variables, these differences were always quite small. The effects for all variables pointed in the same direction.

[FIGURE 2 OMITTED]

Table 1 Comparison of the (Factorial Invariant) Models with and without Style Factor Models Chi-square df RMSEA p-value of NFI GFI N close fit Model 1: no style factor 197.85 85 .036 1.0 .992 .996 1,020 Model 2: with style factor 154.63 83 .029 1.0 .994 .996 1,012 Table 2 Summary Statistics for Models with One-Content Factor and Models with a Content Factor and a Style Factor (ISSP 1995) * Country Model 1. Only content factor Model 2. Content and Style factor Chi-square df RMSEA Chi-square Austria 244.290 27 .100 108.689 (West) Germany 307.109 27 .101 172.143 Ireland 215.085 27 .091 113.331 Italy 159.076 27 .071 94.490 Netherlands 129.885 27 .065 47.808 Norway 152.066 27 .059 71.000 Spain 121.989 27 .058 80.744 Sweden 149.301 27 .065 124.344 United Kingdom 195.652 27 .85 85.668 Country Model 2. Content and Diff. In Chi-Square for Idf Style factor df RMSEA Austria 26 .059 -135.601 (West) Germany 26 .072 -134.966 Ireland 26 .060 -101.754 Italy 26 .050 -64.586 Netherlands 26 .029 -82.005 Norway 26 .037 -81.066 Spain 26 .044 -41.245 Sweden 26 .060 -24.957 United Kingdom 26 .048 -109.984 * The sample sizes vary from 926 (Austria) to 1,143 (Sweden). Table 3 Question Wording (British version of the Questionnaire) Item Question wording v60(+) Immigrants are no less (more) intelligent than the [British] v63(+) Immigrants are no less (more) trustworthy than the [British] v64(-) The arrival of immigrants in [Britain] poses threat to our own way of life. v66(-) Immigrants are getting jobs at the expense of the [British]. v67(+) Immigrants are no less (more) hard-working than the [British] v68(-) On the whole. [Britain] has suffered from the arrival of immigrants. Table 4 Summary Statistics for the Models with One-Content Factor (OUTGROUP), with One-Content Factor (OUTGROUP) and a Style Factor and with Two-Content Factors (POSOUT and NEGOUT) Country Model Chi- Df RMSEA p-value of square close fit Flanders outgr 96.6 9 0.113 <.001 outgr+style 16.1 8 0.0339 0.852 posout + negout 18.028 8 0.0375 0.790 Wallonia outgr 115.9 9 0.113 <.001 outgr+style 32.5 8 0.0649 0.124 posout + negout 36.3 8 0.0685 0.784 Denmark outgr 79.2 9 0.1210 <.001 outgr+style 31.9 8 0.0710 0.078 posout + negout 19.5 8 0.0490 0.476 Finland outgr 199.9 9 0.1860 <.001 outgr+style 30.2 8 0.0638 0.140 posout + negout 21.2 8 0.0487 0.491 United outgr 424.6 9 0.2110 <.001 Kingdom outgr+style 12.5 8 0.0201 0.993 posout + negout 14.3 8 0.0248 0.983 Italy outgr 1848.7 9 0.3240 <.001 outgr+style 52.0 8 0.0501 0.468 posout + negout 101.9 8 0.0744 0.0008 Netherlands outgr 110.3 9 0.1140 <.001 outgr+style 23.0 8 0.0436 0.658 posout + negout 39.5 8 0.0631 0.122 Norway outgr 168.1 9 0.2020 <.001 outgr+style 53.6 8 0.1070 0.0003 posout + negout 39.2 8 0.0885 0.107 Portugal outgr 266.7 9 0.1780 <.001 outgr+style 64.4 8 0.0860 0.001 posout + negout 91.6 8 0.1070 <.001 Sweden outgr 150.7 9 0.1350 <.001 outgr+style 23.4 8 0.0443 0.641 posout + negout 10.7 8 0.0194 0.981 Country Var (F) (t-ratio) Flanders 0.717 (5.741) 0.848 (6.358) / 0.221 (7.497) 1.015 (6.464) / 1.953 (9.807) Wallonia 0.342 (4.140) 0.496 (5.046) / 0.259 (7.668) 0.601 (5.162) / 1.679 (8.996) Denmark 0.525 (6.201) 0.521 (6.203) / 0.140 (5.812) 0.572 (6.424) / 2.111 (8.865) Finland 1.198 (9.561) 1.179 (9.428) / 0.252 (9.599) 1.272 (9.770) / 1.527 (9.690) United 0.817 (10.169) Kingdom 0.922 (11.172) / 0.306 (14,216) 1.194 (12.382) / 1.666 (12.157) Italy 1.675 (16.034) 0.901 (11.334) / 0.895 (24.876) 1.723 (16.185) / 2.184 (16.782) Netherlands 0.194 (3.937) 0.263 (4.683) / 0.173 (7.814) 1.350 (5.076) / 2.000 (11.295) Norway 0.486 (5.219) 0.493 (5.258) / 0.252 (7.864) 0.573 (5.689) / 1.584 (7.052) Portugal 0.049 (1.563) 0.262 (3.698) / 0.446 (11.208) 0.890 (6.004) / 1.605 (8.748) Sweden 0.214 (6.076) 0.239 (6.441) / 0.153 (9.003) 0.284 (6.869) / 1.855 (11.820) Table 5 Testing Construct Equivalence in LISREL: One-Content Factor (OUTGROUP) and One-Style Factor (TCEL, version 1.0) Nr Equivalent groups df Chi-square Fit-ratio RMSEA 5 Flanders, Wallonia, Great- 58 173.495 2.991 .046 Britain, Portugal, Finland 5 Flanders, Wallonia, Norway, 54 149.808 2.774 .049 Portugal, Finland 5 Flanders, U.K., Norway, 56 156.448 2.794 .045 Portugal, Finland 5 Denmark, Netherlands, 57 167.863 2.945 .049 Norway, Portugal, Sweden 4 Flanders, Wallonia, U.K., 45 120.904 2.687 .042 Portugal 4 Flanders, Wallonia, U.K., 46 133.622 2.905 .046 Finland 4 Flanders, Wallonia, Norway, 42 116.680 2.778 .051 Finland 4 Flanders, Wallonia, Portugal, 44 127.502 2.898 .048 Finland 4 Flanders, Denmark, 45 133.429 2.965 .048 Netherlands, Portugal 4 Flanders, U.K., Portugal, 46 135.097 2.937 .044 Finland 4 Wallonia, U.K., Portugal, 44 124.124 2.821 .044 Finland 4 Wallonia, Norway, Portugal, 41 116.861 2.850 .051 Finland 4 Denmark, Netherlands, Norway, 44 131.343 2.985 .051 Portugal 4 Denmark, Netherlands, Norway, 45 127.482 2.833 .049 Sweden 4 Denmark, Netherlands, 45 120.487 2.677 .043 Portugal, Sweden 4 Denmark, Norway, Portugal, 43 121.493 2.825 .049 Sweden 4 Netherlands, Norway, 44 125.190 2.845 .046 Portugal, Sweden p-value of Nr close fit 5 1.0 5 1.0 5 1.0 5 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 4 1.0 Table 6 Testing Construct Equivalence in LISREL: Two-Content Factors (POSOUT and NEGOUT) (TCEL, version 1.0) Nr Equivalent groups df Chi-square Fit-ratio RMSEA p-value of close fit 4 Denmark, Netherlands, 43 113.901 2.649 .047 1.0 Norway, Sweden 4 Denmark, Netherlands, 42 125.607 2.991 .047 1.0 Portugal, Sweden 3 Flanders, Wallonia, U.K. 31 89.715 2.894 .045 1.0 3 Flanders, Denmark, 31 89.191 2.877 .048 .999 Netherlands 3 Flanders, U.K., Finland 32 81.96 2.561 .040 1.0 3 Denmark, Netherlands, 30 78.425 2.614 .049 .997 Norway 3 Denmark, Netherlands, 32 83.259 2.602 .043 1.0 Sweden 3 Denmark, Norway, Sweden 32 92.795 2.900 .053 .994 3 Netherlands, Norway, 31 84.755 2.734 .047 .999 Sweden Table 7 Regression Model for the Latent Variable Scores from a Structural Equation Model in Flanders (N=862). Factor Scores for the Model with One-Content and One-Style Factor (OUTGROUP) and for the Model with Two-Content Factors (POSOUT and NEGOUT) One-Content Factor Two-Content Factor OUTGROUP (+ STYLE) POSOUT Predictors Parameter Standardized Parameter estimates estimates estimates Intercept .126 0 -1.357 *** Gender Women -.019 -.012 -.080 Reference:Man Age .006 ** .119 .006 ** Reference: mean age Church involvement Core members -0.13 -0.06 .006 Modal members .008 .005 .041 Marginal members .087 .036 .104 Reference: no religion Education Some Secondary .029 .015 .072 Finish Secondary -.151 -.086 -.089 Some University -.382 *** -.184 -.304 ** Finish University -.581 *** -.230 -.550 *** [R.sup.2] .092 .079 Adj [R.sup.2] .082 .069 Two-Content Two-Content Factor POSOUT factors (NEGOUT) Predictors Standardized Parameter Standardized estimates estimates estimates Intercept 0 1.849 *** 0 Gender Women -.047 .034 .014 Reference:Man Age .108 .009 ** .119 Reference: mean age Church involvement Core members .003 -0.41 -0.12 Modal members .023 -.021 -.008 Marginal members .042 .110 .031 Reference: no religion Education Some Secondary .036 -.005 -.002 Finish Secondary -.049 -.277 * -.106 Some University -.142 -.617 *** -.200 Finish University -.211 -.849 *** -.225 [R.sup.2] .090 Adj [R.sup.2] .081 * p<.05 ** p<.01 *** p<.001 Table 8 Regression Model for the Latent Variable Scores from a Structural Equation Model in Wallonia (N=727). Factor Scores for the Model with One-Content and One-Style Factor (OUTGROUP) and for the Model with Two-Content Factors (POSOUT and NEGOUT) One-Content Factor Two-Content Factors OUTGROUP (+ POSOUT STYLE) Predictors Parameter Standardized Parameter estimates estimates estimates Intercept .317 *** 0 -1.139 Gender Women .004 .003 -.029 Reference: Man Age .004 * .097 .006 *** Reference: mean age Church involvement Core members -.282 *** -.152 -.245 ** Modal members -.134 * -.094 -.100 Marginal members -.055 -.033 -.019 Reference: no religion Education Some Secondary -.126 -.077 -.040 Finish Secondary -.161* -.119 -.055 Some University -.384 *** -.250 -.249 ** Finish University -.552 *** -.314 -.391 *** [R.sup.2] .109 .081 Adj [R.sup.2] .098 .069 Two-Content Two-Content Factors POSOUT factors (NEGOUT) Predictors Standardized Parameter Standardized estimates estimates estimates Intercept 0 2.258 *** 0 Gender Women -.022 .047 .020 Reference: Man Age .146 .005 .062 Reference: mean age Church involvement Core members -.129 -.541 *** -.159 Modal members -.068 -.261 * -.099 Marginal members -.011 -.117 -.038 Reference: no religion Education Some Secondary -.024 -.317 * -.105 Finish Secondary -.039 -.388 * -.156 Some University -.158 -.805 *** -.286 Finish University -.216 -1.110 *** -.344 [R.sup.2] .114 Adj [R.sup.2] .104 * p<.05 ** p<.01 *** p<.001 Table 9 Regression Model for the Latent Variable Scores from a Structural Equation Model in The Netherlands (N=977). Factor Scores for the Model with One-Content and One-Style Factor (OUTGROUP) and for the Model with Two-Content Factors (POSOUT and NEGOUT) One-Content Factor Two-Content Factors OUTGROUP (+ POSOUT STYLE) Predictors Parameter Standardized Parameter estimates estimates estimates Intercept -.126 0 -.979 *** Gender Women .010 .011 -.019 Reference: Man Age -.005 -.147 -.005 Age > 41.3 .015 ** .307 .014 *** Reference: mean age Church involvement Core members .067 .045 .125 * Modal members .075 * .072 .103 ** Marginal members .078 .045 .094 Reference: no religion Education Finish Secondary -.155 * -.164 -.220 ** Some University -.391 *** -.372 -.452 *** Finish University -.409 *** -.253 -.464 *** [R.sup.2] .138 .126 Adj [R.sup.2] .130 .118 Two-Content Two-Content Factors POSOUT factors (NEGOUT) Predictors Standardized Parameter Standardized estimates estimates estimates Intercept 0 1.210 0 Gender Women -.018 .089 .035 Reference: Man Age -.124 -.015 * -.161 Age > 41.3 .255 .044 .330 Reference: mean age Church involvement Core members .078 .070 .017 Modal members .090 .156 .055 Marginal members .049 .186 .039 Reference: no religion Education Finish Secondary -.214 -.314 -.122 Some University -.392 -.966 *** -.335 Finish University -.263 -1.019 *** -.231 [R.sup.2] .132 Adj [R.sup.2] .124 * p<.05 ** p<.01 *** p<.001 Table 10 Regression Model for the Latent Variable Scores from a Structural Equation Model in Norway (N=476). Factor Scores for the Model with One-Content and One-Style Factor (OUTGROUP) and for the Model with Two-Content Factors (POSOUT and NEGOUT) One-Content Factor Two-Content Factors OUTGROUP (+ POSOUT STYLE) Predictors Parameter Standardized Parameter estimates estimates estimates Intercept -.651 *** 0 -2.112 *** Gender Women -.193 *** -.151 -.234 *** Reference:Man Age .003 .076 .004 * Reference: mean age Church involvement Core members .041 .014 -.016 Modal members .238 .185 .211 Marginal members .179 .139 .144 Reference: no religion Education Some Secondary .086 .035 .197 Finish Secondary -.047 -.036 .050 Some University -.376 *** -.238 -.214 *** Finish University -.486 *** -.237 -.334 *** [R.sup.2] .134 .110 Adj [R.sup.2] .117 .093 Two-Content Two-Content Factors POSOUT factors (NEGOUT) Predictors Standardized Parameter Standardized estimates estimates estimates Intercept 0 1.255 *** 0 Gender Women -.174 -.239 * -.107 Reference:Man Age .100 .003 .047 Reference: mean age Church involvement Core members -.005 -.126 .024 Modal members .157 .401 .178 Marginal members .106 .322 .143 Reference: no religion Education Some Secondary .077 .001 .001 Finish Secondary .037 -.188 -.083 Some University -.128 -.811 *** -.293 Finish University -.155 -.959 *** -.267 [R.sup.2] .129 Adj [R.sup.2] .112 * p<.05 ** p<.01 *** p<.001

NOTES

(1.) This project is sponsored by the Fund for Scientific Research, Flanders. Grant No. G0125/98. Scientific Research Network: "Methodology of Longitudinal and Comparative Research into Social and Cultural Change."

(2.) According to Joreskog (1990) a polychoric correlation matrix, the assymptotic (co)variance matrix and a weighted least s quare estimation (WLS) procedure should be used. Polychoric correlations are based on the continuous variables that are assumed to underlie the observed variables. In multi-group comparisons, the thresholds are computed for all groups together and then fixed in each sample.

(3.) This comparison is legitimate since Model 1 is nested under Model 2. Model 1 is actually a model with a style factor with equal variance.

(4.) This is called "scoring for acquiescence."

(5.) It concerns a cross-cultural survey research on Religious and Moral Pluralism, conducted by academics in 11 European countries and coordinated by the Central Archive for Empirical Research at the University of Cologne (GESIS 2000).

(6.) Because this research is part of a broader research on ethnocentrism in Western Europe, only these countries will be taken into account.

(7.) The scale in the questionnaire contained also 3 items that represent the respondent's feelings towards one's own national group (INGROUP). For reasons mentioned in Note 4, the imputation was conducted on these items as well.

(8.) The program is written on the PYTHON interpreter, and uses LISREL 8.30 to test the models. The TCEL scripts are available at www.kuleuven.ac.be/facdep/social/soc/software.htm.

REFERENCES

BILLIET, J., B. Cambre, and J. Welkenhuysen-Gybels.

1999. "Equivalence of Measurement Instruments for Balanced Sets of Items in Cross-cultural Surveys." Presented at the International Conference on Large Scale Facilities, May 25-29, Eurolab, Cologne.

BILLIET, J., A. Carton, and R. Huys.

1990. Onbekend of onbemind? Een sociologisch onderzoek naar de houding van de Belgen tegenover migranten. Leuven: KULeven, Department Sociologie.

BILLIET, J., R. Eisinga, and P. Scheepers.

1996. "Ethnocentrism in the Low Countries: A Comparative Perspective." New Community 22(3):401-416.

BILLIET, J. and J. McKee McClendon.

2000. "Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items." Structural Equation Modeling. An Interdisciplinary Journal 7(4):608-629.

BOLLEN, K.A.

1989. Structural Equations with Latent Variables. New York: Wiley.

CARMINES, E.G. and J.P. McIver.

1981. "Analyzing Model with Unobserved Variables: Analysis of Covariance Structures." Pp. 65-115 in Social Measurement: Current Issues, edited by G.W. Bohrnstedt and E.F. Borgatta. Beverly Hills: Sage.

CONVERSE, J.M. and S. Presser.

1986. Survey Question. Handcrafting the Standardized Questionnaire. Beverly Hills: Sage.

DEKKER, P. and C. Van Praag.

1990. "Xenofobie in West-Europa." Migrantenstudies 4:37-57.

DELOOZ, P. and J. Kerkhofs.

1992. "Ethiek, op zoek naar een haalbaar evenwicht." Pp. 221-272 in De versneide ommekeer. De waarden van vlamingen, Walen en Brusselaars in de jarennegentig, edited by J. Kerkhofs et al. Tielt: Lannoo.

HUI, C.H. and H.C. Triandis.

1985. "Measurement in Cross-Cultural Psychology: A Review and Comparison of Strategies." Journal of Cross-Cultural Psychology 16:131-152.

JORESKOG K.G.

1990. "New Developments in LISREL. Analysis of Ordinal Variables Using Polychoric Correlations and Weighted Least Squares." Quality and Quantity 24:387-404.

JORESKOG, K.G and K. Sorbom.

1993. LISREL 8 Users Reference Guide. Chicago: Scientific Software International.

MARSH, H.W.

1996. "Positive and Negative Global Self-Esteem: A Substantively Meaningful Distinction of Artifactors?" Journal of Personality and Social Psychology 70:810-819.

MCCLENDON, J. Mc Kee.

1992. "On the Measurement and Control of Acquiescence." Presented at the 1992 Meeting of the American Sociological Association, August 20-26, Pittsburgh, USA.

MIROWSKY, J. and C.E. Ross.

1991. "Eliminating Defense and Agreement Bias from Measures of the Sense of Control: A 2 x 2 Index." Social Psychology Quarterly 54:127-145.

RENSVOLD, RB. and G.W. Cheung.

1998. "Testing Measurement Models for Factorial Invariance: A Systematic Approach." Educational and Psychological Measurement 58:1017-1034.

SARIS, W.

1988a. Individual Response Functions and Correlations Between Judgements. Sociometric Research. Vol 1, Data collection and Scaling, edited by W. Saris and I.N. Gallhofer. London: Macmillan.

SARIS, W.

1988b. Variations in Response Functions. A Source of Measurement Error in Attitude Research. Amsterdam: Sociometric Research Foundation.

STEENKAMP, J.-B. and H. Baumgartner.

1998. "Assessing Measurement Invariance in Cross-National Consumer Research." Journal of Consumer Research 25:78-90.

VAN DE VIJVER, F. and K. Leung.

1997. Methods and Data Analysis for Cross-cultural Research. London: Sage.

VAN SCHUUR, W.H. and H.A.L. Kiers.

1994. "Why Factor Analysis Often is the Incorrect Model for Analyzing Bipolar Concepts, and What Model to Use Instead." Applied Psychological Measurement 18:97-110.

WELKENHUYSEN-GYBELS, J., I. Hajnal, and J. Billiet.

2000. "On the Evaluation of Construct Equivalence in a Multigroup Setting." Presented at the 22nd Biennial Conference of the Society for Multivariate Analysis in the Behavioural Sciences, July 17-19, London.

Jaak Billiet *

* Catholic University Leuven, Centre for Data Collection and Analysis, Department of Sociology, E. Van Evenstraat 2B, B-3000 Leuven, Belgium.

Printer friendly Cite/link Email Feedback | |

Author: | Cambre, Bart; Welkenhuysen-Gybels, Jerry; Billiet, Jaak |
---|---|

Publication: | International Journal of Comparative Sociology |

Article Type: | Statistical Data Included |

Geographic Code: | 4E |

Date: | Mar 1, 2002 |

Words: | 7655 |

Previous Article: | A Cultural History of Civil Examinations in Late Imperial China. |

Next Article: | The logic of nazification: the case of the International Criminal Police Commission ("Interpol") **. |

Topics: |