Consistency Criteria and Unidimensionality: an Attempt At Clarification

Citation:

Richard P. Bagozzi and Claes Fornell (1989) ,"Consistency Criteria and Unidimensionality: an Attempt At Clarification", in NA - Advances in Consumer Research Volume 16, eds. Thomas K. Srull, Provo, UT : Association for Consumer Research, Pages: 321-325.

Advances in Consumer Research Volume 16, 1989      Pages 321-325

CONSISTENCY CRITERIA AND UNIDIMENSIONALITY: AN ATTEMPT AT CLARIFICATION

Richard P. Bagozzi, The University of Michigan

Claes Fornell, The University of Michigan

There has been much debate in marketing recently about so called consistency criteria and unidimensional measurement. In trying to clarify the issues this paper argues that these criteria, as specified in confirmatory factor analysis, are of very limited value.

INTRODUCTION

Consistency criteria have been proposed in various guises a number of times in the past 60-70 years beginning with Spearman and Holzinger (1924) in psychology, Costner (1969) and Blalock (1971) in sociology and now recently in marketing (Kumar and Dillon, 1987a,b; Anderson and Gerbing 1982; Anderson, Gerbing and Hunter, 1987). However, the treatment of these criteria in marketing has ignored similar work in other disciplines and produced conclusions that are either incomplete or questionable.

First, contrary to what is claimed by Anderson, Gerbing and Hunter (1987), Anderson and Gerbing (1982) were not the first to introduce consistency criteria in marketing. For example, Bagozzi (1980) and Fornell and Larcker (1981) both discussed these criteria as "vanishing tetrads," whose application have a long history and are new neither to psychologists nor marketers. It is also well known that they are not sufficient tests of either unidimensional measurement or correct model specification unless all tetrad equations (i.e., internal and external consistency and consistency in solutions of correlations between factors) are examined. Kumar and Dillon (1987a, 1987b) reiterate these points, whereas Anderson, Gerbing and Hunter (1987) maintain that consistency criteria can be readily applied via confirmatory factor analysis to assess unidimensionality. This paper will make the following arguments:

(1) Consistency criteria are nothing but special cases of vanishing tetrads.

(2) Vanishing tetrads can be examined via MIMIC models and tested via standard statistics in LISREL, EQS, and similar programs.

(3) There are cases in which internal and external consistency criteria are met even though all measures are not unidimensional.

(4) Unidimensionality as evaluated via confirmatory factor analysis is a joint property of the data and the model specification. It is not a property of the measures that can be generalized to other specifications.

Most of these arguments are not new. Anderson and Gerbing (1982) acknowledge the sources of the consistency criteria as well as vanishing tetrads. In a later paper with Hunter (1987), they also acknowledge that internal consistency can be viewed as a special case of external consistency but that they had been less than complete in formulating this criterion (this was also a major point in the criticism by Kumar and Dillon, 1987a).

The second argument that tetrads can be examined directly in LISREL with MIMIC specifications is straightforward as we will show this.

The third argument that the satisfaction of consistency criteria in terms of the fit of a unidimensional confirmatory factor analysis model does not imply unidimensionality is a point made by Kumar and Dillon (1987a). This is correct and also shown in texts such as Kenny (1979) and Blalock (e.g., Blalock, 1971) and others. We will present a proof.

The final point is probably the most important: Unidimensionality is a property of both model specification and data. If one is changed, unidimensionality is no longer assured. The implication of this is that confirmatory factor analysis is of very limited applicability in examining unidimensionality.

CONSISTENCY CRITERIA AS MIMIC MODELS

The overidentifying restriction in a confirmatory factor analysis with, say, four variables is that of a vanishing tetrad

R₁₃    =   R₂₃

R₁₄         R₂₄     (1)

where R is a correlation coefficient. The test of a vanishing tetrad was originally suggested by Spearman and Holzinger (1924).

We will first show that vanishing tetrads can also be tested via MIMIC models in traditional covariance structure analysis. If such tests are programmed for the computer, they can possibly give the analyst useful information as to model respecification. However, contrary to the suggestion by Anderson, Gerbing and Hunter (1987), they cannot always be used for the evaluation of unidimensionality. Rewriting equation (1) as

(R₁₃R₂₄) - (R₁₄R₂₃) = 0     (2)

It becomes obvious that the correlation matrix

EQUATION     (3)

must have rank 1. This is equivalent to a second canonical correlation coefficient of zero if

h = p₁x₁ + p₂x₂     (4)

x=p^*₁x₃+p^*₂x₄     (5)

where h and x are canonical variates and P and P* are canonical weight vectors. Further, as shown by Bagozzi, Fornell and Larcker (1981), a MIMIC model of the following type (in LISREL notation)

h=Gx     (6)

where G is a coefficient vector and,

EQUATION      (7)

EQUATION      (8)

with no restrictions on the errors (i.e., the variance-covariance matrix of e is not constrained to be diagonal) can be used to statistically test if equation 1 is true (using the likelihood ratio chi-square test). Nevertheless, without a computerized program to estimate all possible MIMIC model combinations, any type of tetrad analysis quickly becomes very tedious as the number of variables increases. For larger models, the number of tetrads to examine grows exponentially (see for example, Kenny, 1979). There is however, a computer program available for the researcher who wishes to examine tetrads in order to improve a specific model or to find alternative models. The TETRAD program (Glymour and Scheines, 1986) computes all tetrads implied by the model in question and produces diagnostics for model revision.

CONSISTENCY CRITERIA AND UNIDIMENSIONALITY

Even though vanishing tetrads can be used to examine model misspecification, they are not always sufficient tests for unidimensionality. In order to see this, we begin with a very simple model with one overidentifying restriction (depicted in Figure 1A).

A Simple Model

Given the standard error assumptions and standardizing all variables to a mean of zero and unit variance, we can write the parameter matrix of the model in Figure 1A in LISREL notation as:

EQUATION      (9)

Proposition 1: The overidentifying restriction for the two-construct/two-indicator model implied by equation 9 is identical to the vanishing tetrad of equation 1.

Proof: Consider the submatrix

EQUATION      (10)

It is easy to show that |S^*₁|=0 since

(l_x1fl_x3)(l_x2fl_x4)=(l_x1fl_x4)(l_x2fl_x3)       (11)

simplifies to

l_x1l_x2fl_x3l_x4-l_x1l_x2fl_x3l_x4=0

Thus, in order for S₁ = S₁, the elements in S (the correlations) must satisfy the same constraint. That is,

EQUATION (12)

and |S^*₁| = 0. Thus, we have the vanishing tetrad:

FIGURE 1A

AN EXAMPLE FOR A UNIDIMENSIONAL STRUCTURE

EQUATION 13

(R_x1x3R_x2x4)-(R_x1x4R_x2x3)=0      Q.E.D.

Let us now alter the model in Figure 1A so that it no longer applies to the unidimensional case. This is illustrated in Figure 1B.

Given the same assumptions as in the model of Figure 1A, we now have: (EQUATION 13)

Proposition 2: The restriction for the two-dimensional two-construct/two-indicator model implied by equation 13 is identical to that for a corresponding unidimensional model (i.e., equations 1 or 9).

Proof: Consider the submatrix

EQUATION      (14)

Despite the fact that we are now dealing with a two-dimensional model, it is easy to show that we still have |S^*₂| = 0. Since

(l_x1fl_x4+l_x1l_x3)(l_x2fl_x5) = (l_x2fl_x4 + l_x2l_x3)(l_x1fl_x5) (15)

simplifies to

l_x1l_x2f²l_x4l_x5-l_x1l_x2f²l_x4l_x5 = 0.

FIGURE 1B

AN EXAMPLE FOR A TWO DIMENSIONAL STRUCTURE

Consequently, we end up with exactly the same constraint as in the unidimensional case.    Q.E.D.

In other words, for this particular model, confirmatory factor analysis fails as a test for unidimensionality. Note that this is true even though some parameters of the model are not identified. It should also be pointed out that models with fewer than four indicators would obviously rely solely on tetrads involving cross-construct correlations. Anderson and Gerbing (1982) suggest that one must also examine tetrads within constructs (internal consistency in their terminology). However, using the same type of analysis as we have done here, it is equally simple (but very tedious due to the large number of tetrad ) to show that joint application of internal and external consistency criteria (as defined by Anderson and Gerbing, 1982) also fail to detect multi-dimensionality. [This is corrected in Anderson, Gerbing and Hunter (1987) where external consistency is extended to cover what Costner (1969) calls an additional consistency criterion and what Kenny (1979) refers to as consistency of epistemic correlations.] For example, a model such as the one in Figure 1B with two added indicators for each construct (each loading on one construct only) would satisfy these criteria as mentioned by Kumar and Dillon (1987b) in their reply to Anderson, Gerbing and Hunter (1987) and earlier demonstrated in various textbooks (Kenny, 1979; Blalock, 1971).

There are, however, some tetrads in such an extended model that will not vanish. They may, however, easily be obscured in empirical analysis if the others do vanish.

Conclusion

Unidimensionality is a concept that is not easily established empirically. It is clearly not enough to have diagonal error matrices in confirmatory factor analysis. First, there may be other factors (e.g., methods factors) that produce off diagonal terms. Second, the number and type of overriding restrictions go beyond parameter identification in order for all relevant tetrads to be examined. Third, even if one overcomes the first two hurdles--i.e., rules out (or in) methods factors and sets up a model with enough overidentifying restrictions, the support (or rejection) of unidimensionality cannot be generalized beyond one's particular model specification. The addition or deletion of a variable or factor, or the specification of a restricted structural model (rather than the saturated confirmatory factor analysis model), may well alter error matrices in such a way that what was previously found to be unidimensional is now multidimensional and vice versa.

REFERENCES

Anderson, J. C. and Gerbing, D. W. (1982). "Some Methods for Respecifying Measurement Models to Obtain Unidimensional Construct Measurement," Journal of Marketing Research, 19, 453-460.

Anderson, J. C., Gerbing, D. W., and Hunter, J. E. (1987). "On the Assessment of Unidimensional Measurement: Internal and External Consistency, and Overall Consistency Criteria," Journal of Marketing Research, 24, 432-437.

Bagozzi, R. P. (1980). Causal Models in Marketing, New York: Wiley.

Bagozzi, R. P., Fornell, C., and Larcker, D. F. (1981). "Canonical Correlation Analysis as a Special Case of a Structural Relations Model," Multivariate Behavioral Research, 16, 437-454.

Blalock, H. M., Jr. (1971) (Ed.). Causal Models in the Social Sciences, Chicago: Aldine.

Costner, H. L. (1969). "Theory, Deduction, and Rules of Correspondence," American Journal of Sociology, 75, 245-263.

Fornell, C. and Larcker, D. F. (1981). "Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics," Journal of Marketing Research, 1 Co 382-388

Glymour, C. and Scheines R. (1986). "Causal Modeling with the Tetrad Program," Synthese, 68, 37-63.

Kenny, D. A. (1979). Correlation and Causality. New York: Wiley.

Kumar, A. and Dillon, W. R. (1987a). "The Interaction of Measurement and Structure in Simultaneous Equation Models with Unobservable Variables," Journal of Marketing Research, 24, 98-105.

Kumar, A. and Dillon, W. R. (1987b). "Some Further Comments on Measurement/Structure Interaction and the Unidimensionality of Constructs," Journal of Marketing Research, 24, 438 111.

Spearman, C. and Holzinger, K. (1924). "The Sampling Error in the Theory of Two Factors," --British Journal of Psychology, 15, 17-19.

----------------------------------------