Printer Friendly

Is it content or style? An evaluation of two competitive measurement models applied to a balanced set of ethnocentrism items (1).

1. Introduction

Many attitude surveys use sets of items with identical response scales in order to create attitude constructs. Survey questions are often formulated using Likert scales. Respondents are asked to indicate how strongly they agree or disagree with each attitude statement. There is considerable proof that such a response format can be susceptible to an "agreeing-response bias" called acquiescence (Billiet and McClendon 2000). Previous research has shown that it is possible to single out this response bias in a structural equation model (Billiet and McClendon 2000) and that this leads to better fitting factorial invariant models for cross-cultural surveys (Billiet et al. 1999). The style factor that is used to model acquiescence in the structural equation model correlates very strongly (.90) with a variable that counts the number of agreements to a balanced set of items. Others who tried to model acquiescence with the structural equations approach were confronted with the phenomenon that models with a positive and a negative factor appeared to be as likely as models with a bipolar factor and style (acquiescence) factor (Mirowsky and Ross 1991; McClendon 1992). This is a well-known and often debated phenomenon in psychometry and sociometry: oppositely worded indicators that were constructed to operationalize one (bipolar) concept are often found to represent two different factors (Marsh 1996:810). Some researchers favor a substantive explanation of this phenomenon, arguing that this result (two different factors) is a correct. Others, like us, promote a methodological explanation (Marsh 1996; Saris 1988a, 1988b; van Schuur and Kiers 1994).

Using cross-cultural data, we will evaluate these two competitive measurement models, applied to a balanced set of ethnocentrism items, using a structural modeling approach. We will argue that controlling for acquiescence in balanced scales not only leads to a better fitting model in bi-cultural research, but in multi-cultural research as well. Three arguments will be studied. After evaluating the fit of the model with one (bipolar) factor and a style factor and the model with two-content factors, we will discuss which of these two approaches is best for obtaining construct equivalence in cross-cultural research. Construct equivalence is achieved if the instrument measures the same latent trait in all of the cultural groups under investigation (van de Vijver and Leung 1997). Rensvold and Cheung (1998) define construct equivalence operationally as factorial invariance. This means that a construct is equivalent across two cultural groups if the loading of a certain indicator on that construct in the first grou p can be set equal to the corresponding loading in the second group and for all indicators of the construct. For the evaluation of construct equivalence in multi-group settings, see Welkenhuysen-Gybels et al. (2000), Billiet (2000) and Billiet et al. (1999).

The emphasis of this paper will be on the third way of evaluation: the study of substantial differences between the two models. A close look will be taken at the relationships between the substantive construct(s) and relevant background variables for both types of measurement approaches. We will argue that in view of a decision about the competing measurement models, they should be evaluated in the context of a theoretical meaningful nomological network. When the relationships of the two-content factors with other relevant variables remain invariant, then the one-content and style-factor model is preferred. The two-content factor model is preferred when substantial and theoretically meaningful differences are observed in the relationships between the substantive concepts and the background variables.

2. Previous Research

2.1. Bi-cultural Research

Here, we will briefly present our previous research concerning the two alternative measurement models for specifying the empirical relations within a balanced set of items intended to measure the same concept: the two-content factor model versus the one-content factor and one-style factor model.

In Figure 1 the two alternative models are applied to a hypothetical example of a balanced set of six items, measuring the attitude towards ethnic minorities (OUTGROUP). In the two-content factors model, POSOUT consists of the positively worded items, and NEGOUT of the negatively worded items.

Both models were empirically tested for two balanced sets of items in two different Belgian populations, Flanders (Dutch speaking) and Wallonia (French speaking). The construct equivalence of the two-content factors and the style factor was also evaluated. Both the corresponding factor loadings ([LAMBDA]1=[LAMBDA]2) in the two cultural groups and the corresponding error (co)variances [epsilon]1 = [epsilon]2 are equal. Moreover, in the model with a style factor, the factor loadings of the indicator variables on the style factor are all identical because it is assumed that all indicators are equally susceptible to acquiescence.

The content factors are not allowed to correlate with the style factor, but in the base model, the variances of the other latent content factors are not constrained ([PSI]1i (1) [PSI]2i for i = Style) to be equal over the groups. Although a model with completely constrained latent factor invariance fits the data as well.

Since we are dealing with ordinal-scaled variables, the procedure proposed by Joreskog (1990) is used. (2) For this illustration, a Flemish (N=1,020) and a Walloon (N=1,012) random sample of nearly equal size from the 1995 Belgian General Election Survey were used.

The two balanced sets of indicators measure the concept "feeling threatened by immigrants" (an aspect of ethnocentrism, 6 items, 3 positive and 3 negative) and the concept "distrust in politics" (4 items, 2 positive and 2 negative). The two alternative models, one with two-content factors and the other with two-content factors and an additional method (or style) factor, fit the data very well. The fit-indices are reported in Table 1. The value of the RMSEA is lower than .05 and the values of NFL and GFI are close to 1. This indicates that a model with factorial invariance can be retained. Which of the two models should be preferred? The slightly better values of the fit-indices are not decisive. The difference in the chi-square value, however, clearly favours Model 2. The Chi-square value decreases 43.22 units for only two degrees of freedom, which is statistically significant (p<.000). (3)

A closer look at the parameters reveals that the variance of the style factor is considerably smaller than the variance of the content factors, but it is still significantly different from zero. The factor loadings of the style factor are also smaller than the loadings of the content factors. This is what we expected since the indicators are not intended to measure acquiescence, only feelings of threat towards immigrants or political distrust. The variances of the two latent variables (THREAT and DISTRUST) are nearly the same as in Model 1, and the correlations do not change much (.50 in the Flemish sample and .42 in the Walloon sample) across the two models.

2.2. Multi-Group Setting

Table 1 indicated that both models, the model with a style factor and the model without, yield factorial invariance. What can then be a reason for including a style factor? The answer to this question will become more obvious in an example with a larger number of cultural groups that not only differ in language, but also in institutional context and in the practical realisation of the data collection. In such a case, it is possible that error covariance can no longer be set equal to zero, which means that other unidentified sources of common variance are present. The introduction of a style or method factor will unify many of these disparate sources of common non-random variance in the indicators, and identify them as a well documented method or style effect, in this case the agreeing-response bias. We are quasi sure that it is indeed this bias since in the two samples, a variable "sum of agreements" (4) correlates more than .90 with the latent style factor (see Billiet and McClendon 2000).

In this example, the construct equivalence of the indicators for ethnocentrism in the 1995 samples of the International Survey Program (ISSP) will be evaluated. Nine Western European countries are selected for this illustration. Nine items indicate the OUTGROUP dimension of ethnocentrism, four of which are positively worded and four are negatively worded items (one item was neutral formulated).

In the models, all error co-variances are constrained to zero, and all the loadings on the style factor were set equal (except the loading of the neutral indicator because that item cannot be susceptible to an acquiescent response bias). On the basis of the last column in Table 2, we can conclude that the specification of a style factor that is significantly different from zero always results in a very significant drop in the Chi-square statistic. Hence, we may conclude that in all countries, the Liked items are susceptible to an agreeing-response bias and that models that specify the method factor perform much better than models without a method factor. The method factor behaves in precisely the same manner as was found in the Belgian data in the previous example. After the response scales were inverted in order to assign the maximum score (5) to the answering category "completely agree," all the factor loadings on the method factor have small, significant, positive, and equal loadings (except for the neutra l item) in nearly all countries. The covariance between the substantive factor (F) and the style factor (S) was fixed to zero ([[[PSI].sup.g.sub.F,S] = 0) in each country (g [member of] {1,...9}) because it is very unlikely that the agreeing-response bias is related to the substantive latent variable. The variance of the method factor ([[[PSI].sup.g.sub.S,S]) is always substantially lower than the variance of the content factor ([[[PSI].sup.g.sub.F,F]). Finally, the style factor correlates very strongly with a variable "sum of agreements." Hence, we may conclude that the method factor is the tendency to agree with Likert items.

3. Data and Methods: The RAMP Research

The analyses will be performed on (part of) the data of the 1999 RAMP (Religious and Moral Pluralism) dataset. (5) However, not all 11 countries in which the survey was conducted will be taken up in the analysis. (6) Only the Western European countries will be taken into account, namely: Belgium, Denmark, Finland, United Kingdom, Italy, The Netherlands, Norway, Portugal, and Sweden. The data for Belgium will be split up according to whether the respondents are French speaking or Dutch speaking. Hence 10 groups will be included in the analysis. The questionnaire contained (a translation of) a number of items in Likert format about the respondent's feeling of threat towards immigrants, foreigners, or ethnic minorities. The english version of these items is shown in Table 3. As required for the detection of acquiescence, the scale is balanced: it contains as many positively as negatively worded items.

To avoid loss of data due to missing values, an imputation technique was used. The imputation was conducted separately for the negatively and the positively worded items, and for the other items in this scale. (7) For the positively worded items, missing values were only imputed if the respondent had one missing value on these three items. In this case the missing value was replaced by the mean value of the responses on the other items. Respondents with more than one missing value were excluded from the analysis. An analogous method was used for the negatively worded items and for the INGROUP items. Hence, at most 3 values were imputed per observation.

Some methodological remarks can be made with respect to the RAMP data.

(1) The question wording of the positively worded items is often quite poor. In standardized questionnaires the use of double negatives is strongly prohibited (Converse and Presser 1986:13). For example, the item "immigrants are no less intelligent than the British," can easily cause respondents to misplace themselves on the Likert scale. In some countries the question wording was corrected towards "immigrants are as intelligent as [X]." This means that the question was not exactly the same in all countries. The rather low factor loadings of the positively formulated items on the substantive factor could be caused by this problem of question wording.

(2) Exploratory factor analysis on the ethnocentrism items showed large differences between countries. In some countries, three factors could be distinguished: negative attitude towards the OUTGROUP (negative items), feelings of threat (positive items; though strongly correlated with the first factor), and the attitude towards the own group (INGROUP). In other countries, a two-factor solution was found: this is one factor for the INGROUP and one factor for the OUTGROUP (positive and negative items together). Insufficient information about the context of the questionnaire (sampling procedure, interviewers, nonresponse, translation, and so on) makes it unclear whether those differences (both in factors as in factor loadings) are caused by methodological reasons or by cultural differences.

4. Evaluation of Two Competitive Measurement Models

The two competitive measurement models (two-content factors versus one-content and one-style factor) will be compared with respect to three criteria. First, we will take a close look at the fit of the two models for each country. Secondly, we will evaluate whether sets of equivalent countries obtained under the one-content and style-factor model include more countries than the sets obtained under the two-content factors model. And thirdly, the relationship of the two substantive factors of both models to some relevant background variables will be studied.

4.1 Different Measurement Models for Each Country

In order to evaluate the measurement models, a first step is to take a closer look at the fit of the different models. Three models can be distinguished: in the first model, six items load on one factor (OUTGROUP); in the second model a style factor is included (OUGROUP + STYLE). A third model contains two-content factors (POSOUT and NEGOUT) but no style factor.

Using structural equation modelling (LISREL 8.30), these three models were tested for every group in the analysis. The summary statistics for the different models can be found in Table 4.

Because all error-covariances were constrained to zero, the criterium that the Chisquare may not be larger than 3 times the degrees of freedom (Bollen 1989:278; Carmines and McIver 1981) is not always attained. The model with two-content factors and, especially, the model with a style factor, can be improved by allowing some error-covariances to differ from zero. The model with only one-content factor (OUTGROUP) never fits the data. The models with one-content and one-style factor and with two-content factors yield a much better fit than the latter model in every country. In the Scandinavian countries (Denmark, Finland, Norway and Sweden) the two-content factor model yields a lower [X.sup.2]-value, while in the other six countries the model with one-content and one-style factor is preferred. This is especially the case for Italy, The Netherlands, and Portugal.

Except for Portugal the variance of the style factor is smaller than the variance of the content factor. In each of the countries this variance is still significantly different from zero. The factor loadings of the style factor are smaller than the loadings of the content factor. The correlation between the style factor and "yessay" (the sum of agreements) is .6 or higher. The correlation between the two content factors varies from .23 (Italy) to .79 (Denmark). In the Scandinavian countries, this correlation is at least .62 (Norway).

4.2 Construct Equivalence for Ten Western European Countries

A second step in the evaluation of the two competitive models pertains to the evaluation of construct equivalence across countries. Do both measurement models lead to the same groups of equivalent countries?

When looking for groups of countries with equivalent constructs within a population of 10 countries, it is important to know where to start. In other words, it is important to know which countries should be combined first and where to go from there. Some researchers propose a top-to-bottom procedure, while others suggest a bottom-to-top way. In a recent paper, Welkenhuysen-Gybels et al. (2000) proposed a procedure in which all possible subsets of groups are explored. They developed a computer program, named TCEL (Testing Construct Equivalence in Lisrel), (8) which tests all possible subsets of groups in a dataset for equivalence with respect to their factor loadings. The results for this test for the RAMP dataset can be found in Tables 5 and 6.

These tables show that under the content/style factor model, construct equivalence can be obtained for more countries than under the two-content factor model. This is, however; not surprising: compared to the model with one-content factor (and a style factor), the two-content factor model has equality constraints on more factor loadings. In the one-content and one-style factor model, the equality constraints only pertain to the loadings of the content factor (6 loadings), where in the two-content model constraints are placed on all 12 loadings (no cross loadings were allowed). Steenkamp and Baumgartner (1998) define this as "configural invariance" (p.80). In essence, this principle states that the pattern of salient (nonzero) and nonsalient (zero or near zero) loadings should be the same across different countries.

According to the results of the TCEL analysis for the one-content and one-style model, the 10 Western European countries can be divided into two mutually exclusive equivalent groups. One equivalent group of countries consists of Flanders, Wallonia, U.K., Portugal, and Finland. The other group contains Denmark, The Netherlands, Norway, and Sweden. Hence, all countries (except Italy) can be ascribed to one of the two groups.

In the two-content factors model, a group of four countries (Denmark, The Netherlands, Sweden, and Norway or Portugal) can be combined with a group of three countries (Flanders, Wallonia, U.K. or Flanders, U.K., and Finland). In this model, not only Italy, but also two other countries cannot be attributed to one of the two groups.

4.3 The Relationship between the Two Models with other Relevant Variables

As a result of the above findings, one can conclude that in some countries the model with one-content factor and a style factor performs better, while in other (Scandinavian) countries the two-content factors model is better, although the correlation between the two-content factors is very high (.62). Achieving construct equivalence for the two-content factors model is more difficult than for the one-content and one-style factor model.

An important argument in the comparison of the two competitive measurement models will be the relationship of the ethnocentrism scale with some relevant background variables. The two models should be evaluated in the context of a theoretically meaningful network. If, in the two-content factor model, the relationship of each of the content factors with other relevant variables is similar, this implies that these factors function equivalently (Hui and Triandis 1985). The two-content factors can then be interpreted as "opposite sides of the same coin" and, therefore, should be merged. In other words, the one-content and style-factor model should be preferred over the two-content factor model. If, on the other hand, substantial and theoretically meaningful differences in the relationships between each concept and other variables are observed, the concepts do not function equivalently. In this case, the two-content factor model is the preferred model.

To examine this, a regression analysis was performed for four different countries: Flanders, Wallonia, The Netherlands, and Norway. The first three countries are selected because the model with one-content factor and a style factor performs (slightly) better than the model with two-content factors (which is still acceptable). Furthermore, some knowledge about the relationships between ethnocentrism and relevant background variables is available (see Billiet et al. 1996). Norway is selected because in this country, the two-content factors model performs best and the variable "education" is comparable with Belgium and the Netherlands. The selected background variables are gender, age, education, and church involvement. The variable age was centered around the mean. Church involvement was constructed on the basis of two variables: v98 (belonging to a church) and v87 (church attendance). The reference category consists of those who don't belong to any church (no religion, v98=2). Core church members are those wh o go to church at least once a week. Modal church members attend religious services monthly or yearly, while marginal church members rarely or never visit church. The factor scores of the respective models serve as the dependent variable.

All models were tested for normality, multicollinearity, heteroscedasticity, and linearity of the effect of the continuous dependent variables. The variable age is the only continuous variable in the model. In Flanders, Wallonia and Norway, the effect of age on the dependent variables was linear. The Dutch sample, however, shows a nonlinear effect of age. This effect should be interpreted as follows: the first coefficient for age is the effect of age for respondents that are less than 41.3 years old (2 years younger than the mean age, which is 43.3). The second coefficient for age is the additional effect of age for respondents older than 41.3 years. Hence, for the respondents who are more than 41.3 years old, the effect of age is equal to the sum of the two coefficients for age. A graphical account for this effect is shown in Figure 2. This additional effect was exactly the same for the three factors. Tables 7 to 10 present the results for each of the countries.

In Flanders (Table 7), there are only minor differences between the three regression models. Gender and church involvement are not significant (which is strange, since other research found a direct and indirect effect of church involvement on ethnocentrism, see Billiet et al. 1996). Age and (some categories of) education have a significant effect on ethnocentrism. One can conclude that the effect of age is exactly the same for the three factors: the level of significance (p<.01) and the estimates (.119 versus .108 and .119) do not vary at all.

Concerning the level of education, a small but significant effect for NEGOUT exists for those who completed secondary school (-.106). This effect does not occur for POSOUT, nor for OUTGROUP. The difference is negligible and the overall conclusion is that in Flanders the relationship between the two-content factors and the background variables is invariant. Hence in this country, the one-content and one-style factor model is preferred.

In Wallonia (Table 8), some minor differences occur. Again, gender is not a relevant variable, but the other variables show some significant effects. The effect of age is statistically signficant (p<.001) for POSOUT, but not for NEGOUT. The effect for OUTGROUP lies in between and is borderline significant. For modal church members, the estimates reflect significant effects for NEGOUT (and OUTGROUP), but not for POSOUT. The same conclusion goes for respondents with incomplete and complete secondary education. Do these figures indicate a substantial difference in the relationship between the two-content factors and the background variables? In other words, do we have to prefer the two-content factor model over the one-content and one-style factor model? Not in our opinion. Not only are the extra effects for NEGOUT barely significant (except for age), the estimates of all variables all point in the same direction. Furthermore, we already know that the question wording for the POSOUT items is methodologically inc orrect and that other research never found major differences in the attitude towards foreigners between the two communities of Belgium (Billiet et al. 1990; Dekker and Van Praag 1990; Delooz and Kerkhofs 1992; these studies are based on three different data sets): at least on the issue of foreigners, Belgians are remarkably unanimous. The small differences between the two-content factors in Wallonia, and the intermediate results for the model with a style factor, lead us to conclude that the OUTGROUP (and style) model is a fair reflection of the Wallonian opinion.

In The Netherlands (Table 9), there are also some small differences. The main effect of age is signficant (p<.05) for NEGOUT, but not for POSOUT (or OUTGROUP). For the additional effect for every age higher then 41.3, all factors show a similar significant effect. More relevant differences occur for church involvement: the significant effects for core church members and modal church members for POSOUT are not found for NEGOUT. The effect of having a university degree is similar for all factors, but again, the effect for secondary education is not present for NEGOUT. On the whole, however, all effects point in the same direction. Further research has to reveal whether or not these minor differences in relationships are substantial, before deciding that the two-content factor model has to be chosen.

Earlier in this paper, Norway was found to be a country in which the two-content factor model performed better than the one-content and one-style factor model. But the latter model could fit the data as well. So it is important to look for substantial differences between the POSOUT and NEGOUT factor and some background variables.

In Norway (Table 10), gender is a relevant variable, although the effect for NEGOUT is less significant. The effect of age is only significant for POSOUT. Church involvement is not relevant in Norway to explain the attitude towards foreigners. The effect of education is significant for respondents with a university degree, but the estimates for NEGOUT are much stronger (-.293/-.267) than the effects for POSOUT (-.128/-.155). The models for NEGOUT and POSOUT can be used, although the model with one-content factor (and a style factor) performs as good as the model with two-content factors (except for age).

5. Conclusion

It is a familiar finding that indicators which were meant to operationalize one (bipolar) concept, appear to form two correlated factors: one for the positively worded and one for the negatively worded items. In this paper, it is argued that this is due to an acquiescent response bias which occurs on items with a Likert format, and that the inclusion of a style factor instead of two-content factors leads at least to an equally good solution as the two-content factor model.

We tried to evaluate those two competitive measurement models, using the cross-cultural RAMP data for ten Western European countries. The evaluation is based on three criteria. First, a close inspection of the different models for each country was conducted. We could conclude that in some countries the model with one-content factor and a style factor yielded a better fit, while in the Scandinavian countries the two-content factors model performed better. In these countries, however, the correlation between these two factors was high (.62). A second method of evaluation pertained to construct equivalence across countries. Using a new developed computer program, named TCEL, it was found that larger sets of equivalent countries could be obtained for the one-content and one-style factor model. Nine of the ten Western European countries could be divided into two equivalent groups. In the two-content factor model, this was not possible. A third evaluation concerned the context of the factors. It was argued that if the relationships of the two-content factors with other relevant (background) variables were invariant, the one-content factor and one-style factor model was preferred. The two-content factor model was preferred when one observes substantial and theoretically meaningful differences in the relationships between each concept and other variables. This was tested for four countries. In Flanders, the two-content factors were invariant, so the earlier finding of a better fit for the one-content and one-style factor model was confirmed. In Wallonia, some minor differences exist, but due to some methodological reasons (question wording for the POSOUT items) and previous research (no differences between Flanders and Wallonia), the content and style-factor model was still preferred. In the Netherlands, all effects point in the same direction, but the effect of church involvement (and to a certain degree also age and education) is different for both content factors. In Norway, the two-content factor model performed best , and some very small differences between these two factors were found (age, gender, and education), so the two-content factor model should be preferred (although the content/style model fits the data as well and reflects the same effects for background variables).

Further research is necessary to explore the relationships of the competitive models to relevant background variables more thoroughly. For now, the conclusion is that a model with a content factor and a style factor fits the data at least equally good in most countries, leads more easily to construct equivalence and shows a fair reflection of the effects of relevant background variables. If the two-content factors showed different relationships to other variables, these differences were always quite small. The effects for all variables pointed in the same direction.

[FIGURE 2 OMITTED]
Table 1

Comparison of the (Factorial Invariant) Models with and without Style
Factor

Models             Chi-square  df  RMSEA  p-value of  NFI   GFI   N
                                          close fit

Model 1:
no style factor    197.85      85  .036   1.0         .992  .996  1,020

Model 2:
with style factor  154.63      83  .029   1.0         .994  .996  1,012
Table 2

Summary Statistics for Models with One-Content Factor and Models with a
Content Factor and a Style Factor (ISSP 1995) *

Country          Model 1. Only content factor       Model 2. Content and
                                                        Style factor

             Chi-square       df        RMSEA     Chi-square

Austria      244.290          27        .100       108.689
(West)
Germany      307.109          27        .101       172.143
Ireland      215.085          27        .091       113.331
Italy        159.076          27        .071        94.490
Netherlands  129.885          27        .065        47.808
Norway       152.066          27        .059        71.000
Spain        121.989          27        .058        80.744
Sweden       149.301          27        .065       124.344
United
Kingdom      195.652          27        .85         85.668

Country       Model 2. Content and   Diff. In Chi-Square for Idf
                  Style factor

             df          RMSEA

Austria      26          .059               -135.601
(West)
Germany      26          .072               -134.966
Ireland      26          .060               -101.754
Italy        26          .050                -64.586
Netherlands  26          .029                -82.005
Norway       26          .037                -81.066
Spain        26          .044                -41.245
Sweden       26          .060                -24.957
United
Kingdom      26          .048               -109.984

* The sample sizes vary from 926 (Austria) to 1,143 (Sweden).
Table 3

Question Wording (British version of the Questionnaire)

Item    Question wording

v60(+)  Immigrants are no less (more)
        intelligent than the [British]
v63(+)  Immigrants are no less (more)
        trustworthy than the [British]
v64(-)  The arrival of immigrants in [Britain]
        poses threat to our own way of life.
v66(-)  Immigrants are getting jobs at the
        expense of the [British].
v67(+)  Immigrants are no less (more)
        hard-working than the [British]
v68(-)  On the whole. [Britain] has suffered
        from the arrival of immigrants.
Table 4

Summary Statistics for the Models with One-Content Factor (OUTGROUP),
with One-Content Factor (OUTGROUP) and a Style Factor and with
Two-Content Factors (POSOUT and NEGOUT)

Country      Model            Chi-    Df  RMSEA   p-value of
                              square              close fit

Flanders     outgr            96.6    9   0.113   <.001
             outgr+style      16.1    8   0.0339  0.852
             posout + negout  18.028  8   0.0375  0.790

Wallonia     outgr            115.9   9   0.113   <.001
             outgr+style      32.5    8   0.0649  0.124
             posout + negout  36.3    8   0.0685  0.784

Denmark      outgr            79.2    9   0.1210  <.001
             outgr+style      31.9    8   0.0710  0.078
             posout + negout  19.5    8   0.0490  0.476

Finland      outgr            199.9   9   0.1860  <.001
             outgr+style      30.2    8   0.0638  0.140
             posout + negout  21.2    8   0.0487  0.491

United       outgr            424.6   9   0.2110  <.001
Kingdom      outgr+style      12.5    8   0.0201  0.993

             posout + negout  14.3    8   0.0248  0.983


Italy        outgr            1848.7  9   0.3240  <.001
             outgr+style      52.0    8   0.0501  0.468

             posout + negout  101.9   8   0.0744  0.0008


Netherlands  outgr            110.3   9   0.1140  <.001
             outgr+style      23.0    8   0.0436  0.658
             posout + negout  39.5    8   0.0631  0.122

Norway       outgr            168.1   9   0.2020  <.001
             outgr+style      53.6    8   0.1070  0.0003
             posout + negout  39.2    8   0.0885  0.107

Portugal     outgr            266.7   9   0.1780  <.001
             outgr+style      64.4    8   0.0860  0.001
             posout + negout  91.6    8   0.1070  <.001

Sweden       outgr            150.7   9   0.1350  <.001
             outgr+style      23.4    8   0.0443  0.641
             posout + negout  10.7    8   0.0194  0.981

Country                             Var (F)
                                  (t-ratio)

Flanders                      0.717 (5.741)
              0.848 (6.358) / 0.221 (7.497)
              1.015 (6.464) / 1.953 (9.807)

Wallonia                      0.342 (4.140)
              0.496 (5.046) / 0.259 (7.668)
              0.601 (5.162) / 1.679 (8.996)

Denmark                       0.525 (6.201)
              0.521 (6.203) / 0.140 (5.812)
              0.572 (6.424) / 2.111 (8.865)

Finland                       1.198 (9.561)
              1.179 (9.428) / 0.252 (9.599)
              1.272 (9.770) / 1.527 (9.690)

United                       0.817 (10.169)
Kingdom              0.922 (11.172) / 0.306
                                   (14,216)
                     1.194 (12.382) / 1.666
                                   (12.157)

Italy                        1.675 (16.034)
                     0.901 (11.334) / 0.895
                                   (24.876)
                     1.723 (16.185) / 2.184
                                   (16.782)

Netherlands                   0.194 (3.937)
              0.263 (4.683) / 0.173 (7.814)
             1.350 (5.076) / 2.000 (11.295)

Norway                        0.486 (5.219)
              0.493 (5.258) / 0.252 (7.864)
              0.573 (5.689) / 1.584 (7.052)

Portugal                      0.049 (1.563)
             0.262 (3.698) / 0.446 (11.208)
              0.890 (6.004) / 1.605 (8.748)

Sweden                        0.214 (6.076)
              0.239 (6.441) / 0.153 (9.003)
             0.284 (6.869) / 1.855 (11.820)
Table 5

Testing Construct Equivalence in LISREL: One-Content Factor (OUTGROUP)
and One-Style Factor (TCEL, version 1.0)


Nr  Equivalent groups              df  Chi-square  Fit-ratio  RMSEA

5   Flanders, Wallonia, Great-     58  173.495     2.991      .046
    Britain, Portugal, Finland
5   Flanders, Wallonia, Norway,    54  149.808     2.774      .049
    Portugal, Finland
5   Flanders, U.K., Norway,        56  156.448     2.794      .045
    Portugal, Finland
5   Denmark, Netherlands,          57  167.863     2.945      .049
    Norway, Portugal, Sweden
4   Flanders, Wallonia, U.K.,      45  120.904     2.687      .042
    Portugal
4   Flanders, Wallonia, U.K.,      46  133.622     2.905      .046
    Finland
4   Flanders, Wallonia, Norway,    42  116.680     2.778      .051
    Finland
4   Flanders, Wallonia, Portugal,  44  127.502     2.898      .048
    Finland
4   Flanders, Denmark,             45  133.429     2.965      .048
    Netherlands, Portugal
4   Flanders, U.K., Portugal,      46  135.097     2.937      .044
    Finland
4   Wallonia, U.K., Portugal,      44  124.124     2.821      .044
    Finland
4   Wallonia, Norway, Portugal,    41  116.861     2.850      .051
    Finland
4   Denmark, Netherlands, Norway,  44  131.343     2.985      .051
    Portugal
4   Denmark, Netherlands, Norway,  45  127.482     2.833      .049
    Sweden
4   Denmark, Netherlands,          45  120.487     2.677      .043
    Portugal, Sweden
4   Denmark, Norway, Portugal,     43  121.493     2.825      .049
    Sweden
4   Netherlands, Norway,           44  125.190     2.845      .046
    Portugal, Sweden

    p-value of
Nr  close fit

5   1.0

5   1.0

5   1.0

5   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0

4   1.0
Table 6

Testing Construct Equivalence in LISREL: Two-Content Factors (POSOUT and
NEGOUT) (TCEL, version 1.0)

Nr  Equivalent groups         df  Chi-square  Fit-ratio  RMSEA   p-value
                                                                of close
                                                                     fit

4   Denmark, Netherlands,     43  113.901     2.649      .047      1.0
    Norway, Sweden
4   Denmark, Netherlands,     42  125.607     2.991      .047      1.0
    Portugal, Sweden
3   Flanders, Wallonia, U.K.  31  89.715      2.894      .045      1.0
3   Flanders, Denmark,        31  89.191      2.877      .048       .999
    Netherlands
3   Flanders, U.K., Finland   32  81.96       2.561      .040      1.0
3   Denmark, Netherlands,     30  78.425      2.614      .049       .997
    Norway
3   Denmark, Netherlands,     32  83.259      2.602      .043      1.0
    Sweden
3   Denmark, Norway, Sweden   32  92.795      2.900      .053       .994
3   Netherlands, Norway,      31  84.755      2.734      .047       .999
    Sweden
Table 7

Regression Model for the Latent Variable Scores from a Structural
Equation Model in Flanders (N=862). Factor Scores for the Model with
One-Content and One-Style Factor (OUTGROUP) and for the Model with
Two-Content Factors (POSOUT and NEGOUT)

                             One-Content Factor        Two-Content
                                                          Factor
                             OUTGROUP (+ STYLE)           POSOUT

Predictors               Parameter    Standardized   Parameter
                         estimates     estimates     estimates

Intercept                   .126           0          -1.357 ***

Gender
Women                      -.019         -.012         -.080
Reference:Man

Age                         .006 **       .119          .006 **
Reference: mean age

Church involvement
Core members              -0.13         -0.06           .006
Modal members               .008          .005          .041
Marginal members            .087          .036          .104
Reference: no religion

Education
Some Secondary              .029          .015          .072
Finish Secondary           -.151         -.086         -.089
Some University            -.382 ***     -.184         -.304 **
Finish University          -.581 ***     -.230         -.550 ***

[R.sup.2]                   .092                        .079
Adj [R.sup.2]               .082                        .069


                         Two-Content            Two-Content
                            Factor
                            POSOUT            factors (NEGOUT)

Predictors              Standardized   Parameter    Standardized
                         estimates     estimates     estimates

Intercept                    0           1.849 ***        0

Gender
Women                      -.047          .034           .014
Reference:Man

Age                         .108          .009 **        .119
Reference: mean age

Church involvement
Core members                .003        -0.41          -0.12
Modal members               .023         -.021          -.008
Marginal members            .042          .110           .031
Reference: no religion

Education
Some Secondary              .036         -.005          -.002
Finish Secondary           -.049         -.277 *        -.106
Some University            -.142         -.617 ***      -.200
Finish University          -.211         -.849 ***      -.225

[R.sup.2]                                 .090
Adj [R.sup.2]                             .081


* p<.05

** p<.01

*** p<.001
Table 8

Regression Model for the Latent Variable Scores from a Structural
Equation Model in Wallonia (N=727). Factor Scores for the Model with
One-Content and One-Style Factor (OUTGROUP) and for the Model with
Two-Content Factors (POSOUT and NEGOUT)

                             One-Content Factor        Two-Content
                                                         Factors
                                OUTGROUP (+               POSOUT
                                   STYLE)

Predictors              Parameter     Standardized  Parameter
                        estimates      estimates    estimates

Intercept                  .317 ***        0         -1.139

Gender
 Women                     .004           .003        -.029
Reference: Man

Age                        .004 *         .097         .006 ***
Reference: mean age

Church involvement

 Core members             -.282 ***      -.152        -.245 **
 Modal members            -.134 *        -.094        -.100
 Marginal members         -.055          -.033        -.019
Reference: no religion

Education

 Some Secondary           -.126          -.077        -.040
 Finish Secondary         -.161*         -.119        -.055
 Some University          -.384 ***      -.250        -.249 **
 Finish University        -.552 ***      -.314        -.391 ***

[R.sup.2]                  .109                        .081
Adj [R.sup.2]              .098                        .069

                         Two-Content            Two-Content
                           Factors
                            POSOUT            factors (NEGOUT)


Predictors              Standardized  Parameter     Standardized
                         estimates    estimates      estimates

Intercept                    0           2.258 ***       0

Gender
 Women                     -.022         .047           .020
Reference: Man

Age                         .146         .005           .062
Reference: mean age

Church involvement

 Core members              -.129        -.541 ***      -.159
 Modal members             -.068        -.261 *        -.099
 Marginal members          -.011        -.117          -.038
Reference: no religion

Education

 Some Secondary            -.024        -.317 *        -.105
 Finish Secondary          -.039        -.388 *        -.156
 Some University           -.158        -.805 ***      -.286
 Finish University         -.216        -1.110 ***     -.344

[R.sup.2]                                 .114
Adj [R.sup.2]                             .104

* p<.05

** p<.01

*** p<.001
Table 9

Regression Model for the Latent Variable Scores from a Structural
Equation Model in The Netherlands (N=977). Factor Scores for the Model
with One-Content and One-Style Factor (OUTGROUP) and for the Model with
Two-Content Factors (POSOUT and NEGOUT)

                             One-Content Factor        Two-Content
                                                         Factors
                                OUTGROUP (+               POSOUT
                                   STYLE)

Predictors              Parameter     Standardized  Parameter
                        estimates      estimates    estimates

Intercept                 -.126            0          -.979 ***

Gender

 Women                    .010            .011        -.019
Reference: Man

Age                       -.005          -.147        -.005
 Age > 41.3                .015 **        .307         .014 ***
Reference: mean age

Church involvement

 Core members              .067           .045         .125 *
 Modal members             .075 *         .072         .103 **
 Marginal members          .078           .045         .094
Reference: no religion

Education

 Finish Secondary         -.155 *        -.164        -.220 **
 Some University          -.391 ***      -.372        -.452 ***
 Finish University        -.409 ***      -.253        -.464 ***

[R.sup.2]                  .138                        .126
Adj [R.sup.2]              .130                        .118

                         Two-Content            Two-Content
                           Factors
                            POSOUT            factors (NEGOUT)


Predictors              Standardized  Parameter     Standardized
                         estimates    estimates      estimates

Intercept                    0          1.210            0

Gender

 Women                     -.018         .089           .035
Reference: Man

Age                        -.124        -.015 *        -.161
 Age > 41.3                 .255         .044           .330
Reference: mean age

Church involvement

 Core members               .078         .070           .017
 Modal members              .090         .156           .055
 Marginal members           .049         .186           .039
Reference: no religion

Education

 Finish Secondary          -.214        -.314          -.122
 Some University           -.392        -.966 ***      -.335
 Finish University         -.263       -1.019 ***      -.231

[R.sup.2]                                .132
Adj [R.sup.2]                            .124

* p<.05

** p<.01

*** p<.001
Table 10

Regression Model for the Latent Variable Scores from a Structural
Equation Model in Norway (N=476). Factor Scores for the Model with
One-Content and One-Style Factor (OUTGROUP) and for the Model with
Two-Content Factors (POSOUT and NEGOUT)

                              One-Content Factor        Two-Content
                                                          Factors
                                 OUTGROUP (+               POSOUT
                                    STYLE)

Predictors              Parameter      Standardized   Parameter
                        estimates        estimates    estimates

Intercept                   -.651 ***        0        -2.112 ***

Gender
 Women                      -.193 ***      -.151       -.234 ***
Reference:Man

Age                          .003           .076        .004 *
Reference: mean age

Church involvement
 Core members                .041           .014       -.016
 Modal members               .238           .185        .211
 Marginal members            .179           .139        .144
Reference: no religion

Education
 Some Secondary              .086           .035        .197
 Finish Secondary           -.047          -.036        .050
 Some University            -.376 ***      -.238       -.214 ***
 Finish University          -.486 ***      -.237       -.334 ***

[R.sup.2]                    .134                       .110
Adj [R.sup.2]                .117                       .093

                          Two-Content            Two-Content
                            Factors
                             POSOUT           factors (NEGOUT)


Predictors               Standardized   Parameter  Standardized
                          estimates     estimates   estimates

Intercept                     0         1.255 ***       0

Gender
 Women                      -.174       -.239 *       -.107
Reference:Man

Age                          .100        .003          .047
Reference: mean age

Church involvement
 Core members               -.005       -.126          .024
 Modal members               .157        .401          .178
 Marginal members            .106        .322          .143
Reference: no religion

Education
 Some Secondary              .077        .001          .001
 Finish Secondary            .037       -.188         -.083
 Some University            -.128       -.811 ***     -.293
 Finish University          -.155       -.959 ***     -.267

[R.sup.2]                                .129
Adj [R.sup.2]                            .112

* p<.05

** p<.01

*** p<.001


NOTES

(1.) This project is sponsored by the Fund for Scientific Research, Flanders. Grant No. G0125/98. Scientific Research Network: "Methodology of Longitudinal and Comparative Research into Social and Cultural Change."

(2.) According to Joreskog (1990) a polychoric correlation matrix, the assymptotic (co)variance matrix and a weighted least s quare estimation (WLS) procedure should be used. Polychoric correlations are based on the continuous variables that are assumed to underlie the observed variables. In multi-group comparisons, the thresholds are computed for all groups together and then fixed in each sample.

(3.) This comparison is legitimate since Model 1 is nested under Model 2. Model 1 is actually a model with a style factor with equal variance.

(4.) This is called "scoring for acquiescence."

(5.) It concerns a cross-cultural survey research on Religious and Moral Pluralism, conducted by academics in 11 European countries and coordinated by the Central Archive for Empirical Research at the University of Cologne (GESIS 2000).

(6.) Because this research is part of a broader research on ethnocentrism in Western Europe, only these countries will be taken into account.

(7.) The scale in the questionnaire contained also 3 items that represent the respondent's feelings towards one's own national group (INGROUP). For reasons mentioned in Note 4, the imputation was conducted on these items as well.

(8.) The program is written on the PYTHON interpreter, and uses LISREL 8.30 to test the models. The TCEL scripts are available at www.kuleuven.ac.be/facdep/social/soc/software.htm.

REFERENCES

BILLIET, J., B. Cambre, and J. Welkenhuysen-Gybels.

1999. "Equivalence of Measurement Instruments for Balanced Sets of Items in Cross-cultural Surveys." Presented at the International Conference on Large Scale Facilities, May 25-29, Eurolab, Cologne.

BILLIET, J., A. Carton, and R. Huys.

1990. Onbekend of onbemind? Een sociologisch onderzoek naar de houding van de Belgen tegenover migranten. Leuven: KULeven, Department Sociologie.

BILLIET, J., R. Eisinga, and P. Scheepers.

1996. "Ethnocentrism in the Low Countries: A Comparative Perspective." New Community 22(3):401-416.

BILLIET, J. and J. McKee McClendon.

2000. "Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items." Structural Equation Modeling. An Interdisciplinary Journal 7(4):608-629.

BOLLEN, K.A.

1989. Structural Equations with Latent Variables. New York: Wiley.

CARMINES, E.G. and J.P. McIver.

1981. "Analyzing Model with Unobserved Variables: Analysis of Covariance Structures." Pp. 65-115 in Social Measurement: Current Issues, edited by G.W. Bohrnstedt and E.F. Borgatta. Beverly Hills: Sage.

CONVERSE, J.M. and S. Presser.

1986. Survey Question. Handcrafting the Standardized Questionnaire. Beverly Hills: Sage.

DEKKER, P. and C. Van Praag.

1990. "Xenofobie in West-Europa." Migrantenstudies 4:37-57.

DELOOZ, P. and J. Kerkhofs.

1992. "Ethiek, op zoek naar een haalbaar evenwicht." Pp. 221-272 in De versneide ommekeer. De waarden van vlamingen, Walen en Brusselaars in de jarennegentig, edited by J. Kerkhofs et al. Tielt: Lannoo.

HUI, C.H. and H.C. Triandis.

1985. "Measurement in Cross-Cultural Psychology: A Review and Comparison of Strategies." Journal of Cross-Cultural Psychology 16:131-152.

JORESKOG K.G.

1990. "New Developments in LISREL. Analysis of Ordinal Variables Using Polychoric Correlations and Weighted Least Squares." Quality and Quantity 24:387-404.

JORESKOG, K.G and K. Sorbom.

1993. LISREL 8 Users Reference Guide. Chicago: Scientific Software International.

MARSH, H.W.

1996. "Positive and Negative Global Self-Esteem: A Substantively Meaningful Distinction of Artifactors?" Journal of Personality and Social Psychology 70:810-819.

MCCLENDON, J. Mc Kee.

1992. "On the Measurement and Control of Acquiescence." Presented at the 1992 Meeting of the American Sociological Association, August 20-26, Pittsburgh, USA.

MIROWSKY, J. and C.E. Ross.

1991. "Eliminating Defense and Agreement Bias from Measures of the Sense of Control: A 2 x 2 Index." Social Psychology Quarterly 54:127-145.

RENSVOLD, RB. and G.W. Cheung.

1998. "Testing Measurement Models for Factorial Invariance: A Systematic Approach." Educational and Psychological Measurement 58:1017-1034.

SARIS, W.

1988a. Individual Response Functions and Correlations Between Judgements. Sociometric Research. Vol 1, Data collection and Scaling, edited by W. Saris and I.N. Gallhofer. London: Macmillan.

SARIS, W.

1988b. Variations in Response Functions. A Source of Measurement Error in Attitude Research. Amsterdam: Sociometric Research Foundation.

STEENKAMP, J.-B. and H. Baumgartner.

1998. "Assessing Measurement Invariance in Cross-National Consumer Research." Journal of Consumer Research 25:78-90.

VAN DE VIJVER, F. and K. Leung.

1997. Methods and Data Analysis for Cross-cultural Research. London: Sage.

VAN SCHUUR, W.H. and H.A.L. Kiers.

1994. "Why Factor Analysis Often is the Incorrect Model for Analyzing Bipolar Concepts, and What Model to Use Instead." Applied Psychological Measurement 18:97-110.

WELKENHUYSEN-GYBELS, J., I. Hajnal, and J. Billiet.

2000. "On the Evaluation of Construct Equivalence in a Multigroup Setting." Presented at the 22nd Biennial Conference of the Society for Multivariate Analysis in the Behavioural Sciences, July 17-19, London.

Jaak Billiet *

* Catholic University Leuven, Centre for Data Collection and Analysis, Department of Sociology, E. Van Evenstraat 2B, B-3000 Leuven, Belgium.
COPYRIGHT 2002 Sage Publications, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2002 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Cambre, Bart; Welkenhuysen-Gybels, Jerry; Billiet, Jaak
Publication:International Journal of Comparative Sociology
Article Type:Statistical Data Included
Geographic Code:4E
Date:Mar 1, 2002
Words:7655
Previous Article:A Cultural History of Civil Examinations in Late Imperial China.
Next Article:The logic of nazification: the case of the International Criminal Police Commission ("Interpol") **.
Topics:

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters