Structural Modelling: an Application For Testing Attitude Models and Convergent Validity

The general approach used by both sets of authors to estimate parameters of a model of this form is maximum likelihood. Conceptually this is done by trying to find that set of parameters which best reproduces the covariance (or correlation) matrix of the observed data. To get a feel of how this is done, let us address our attention to a few elements in the covariance matrix of the y's assuming that the measurement process is modeled as stated in Equation 1. For example, the variance of the first observed measure for can be expressed in terms construct 1, i.e., Var(y₁₁)parameters of the model's as follows:

Likewise the covariance between y₁₁ and the first measure of the second construct y₂₁ is

while the covariance between y₁₁ and the second measure of the first construct y₁₂ is

In an analogous manner each element of the observed variable covariance matrix can be represented in terms of the unobserved model parameters. It is often the case in analyses of this type that the number of possible model parameters obtained in the above fashion exceeds the number of observations. Consequently, it is not possible to uniquely identify the parameter without limiting the parameter space by making some assumptions about the permissible values for certain of these parameters. For example, the assumptions used in the B-B paper are as follows.

Applying the above four assumptions to Equations 2-4 yields equations of the following form:

Although it should go without saying, it should be noted that the above methodology can not be used to prove a particular theory is correct. As is true with any hypothesis testing, statistical tests of hypotheses are capable only of rejecting a given theory (subject to a type-I error) or conversely saying that the observed data is consistent with the postulated theory. Thus even though the final models postulated by B-B and J-R are not rejected, it is not proper to say that these models are correct. In other words, B-B and J-R do not show that the measures used have convergent validity or that attitude is two dimensional, they only show that they can not reject these statements.

In a somewhat similar vein it must be recognized that the structural modeling methodology does not prove that the model as postulated is valid (i.e., does not prove construct validity). However, the approach is consistent with the philosophy of science notion of testing a theory in that a) the structural equations explicitly state the theory to be tested and b) the methodology allows the theory to be rejected.

RESULTS

The B-B and J-R papers postulated a number of structural models of attitude and the measurement process. The simplest of these models is stated in Figure 1 of B-B. Using a data base collected by Fishbein and Ajzen (1974) they estimated the parameters of the model using LISREL. The parameter estimates and relevant test statistics for this model are given in Table 1 of their paper. They conclude, based on the X² value for the two attitude factor model, that this model can not be rejected and thus "convergent validity is established for the two-factor affective/cognitive model of attitudes." As previously mentioned, this is too strong a statement since the X² test only indicated that the data were consistent with their model.

FIGURE 2

SCHEMATIC REPRESENTATION OF THE JOINT INFLUENCES OF METHOD AND ATTITUDE ON OBSERVED RESPONSES

The approach of J-R was similar to B-B in that they also postulated a model which stated that attitude was a multi-dimensional concept and that the measures used were fallible. However, their model of the measurement process reflected the fact that the observed responses could also be affected by the method used to obtain the responses. More specifically, they postulated that the observed measures y_ijk are influenced not only by the level of the particular construct (A_ij) they were intended to measure, but also that they were influenced by the method (M_ik) used to obtain the measure. This conceptualization is schematically displayed in Figure 2 where the three A's represent three components of attitude and the three M's the three different methods used to obtain the nine y responses. Thus (using their notation) the observed response for person i for measure l on trait j using method k, is

where the covariance matrix of the unobserved constructs (these being the A's and the M's) are allowed to be correlated.

The second major observation about Equation 8 is more technical, but is probably more important. Although I haven't been able to prove it, I don't think their general model (i.e., Model 1 of their paper) is identified. This means that the estimates they report are not unique. My general approach to determine if the model was identified was to write out the covariance matrix of the y's in terms of the model parameters (in a manner similar to my Equations 2 through 4). Then after reducing the parameter space by making the assumptions similar to those made by B-B, I attempted to uniquely determine the parameters in terms of the observed y's. To the best of my ability I was not able to do this. Moreover, I I don't think it is possible to uniquely identify all of the parameters. I suspect that the authors did not attempt this analysis either but instead assumed their model was identified simply because the number of parameters estimated was less than the number of observations.

The problem of identification is always difficult in models as complex as that postulated by J-R. One of the standard ways of handling the identification problem is to finesse the type of analysis I attempted and instead input the specified model into the LISREL program and let the program "determine" identification via the information matrix. For identified models the information matrix will almost certainly be positive definite. Conversely, if the information matrix is not positive definite, the specified model is almost certainly not identified (J÷reskog and S÷rbom, 1977). Since positive definite matrices can be inverted while non positive definite matrices can not, inversion of the information matrix implies that the information matrix is positive definite. Thus, inversion is taken as "proof" that the model is identified. Unfortunately, inversion of the information matrix is only a necessary condition (in contrast to a sufficient condition) for identification. Thus even though I understand why J-R might have resorted to the above "rule of thumb," this rule may have misled them into believing that their general model is identified.

As further "proof" that this model is not identified, note that when the authors further restricted the general model (yielding their Models 2, 3 and 4) the program failed to converge. Lack of convergence is normally a sign that the model is not identified. Since it is impossible for a subset of a model not to be identified when a more general model is identified, my priors are that the general model is also not identified. Thus, their Model 1 results displayed in Figure 3 of their paper are meaningless.

The net result of the above discussion is that I believe it is impossible to identify (solve) for a general three-trait, three-method model where the methods and attitudes are allowed to be correlated using the J-R approach. However, I was able to develop a model which is identified (in fact many of the parameters are over-identified) that assumes that the observed score is affected by both the respondent's attitude and the type of measurement method used. This model is presented next.

ALTERNATIVE APPROACH TO J-R MODEL

Assume the data of Ostrom (1969) (i.e., the data used by J-R) where three measures are obtained for each of three different attitude components. Let y_ijk represent the observed measure for person i on attitude component j using measure k. Let A_ij be person i's attitude score on component j and let M_ik be person i's measurement score on measure k. Next, postulate the model

y_ijk = A_ij + M_ik ,  l = 1,2,3;   j = 1,2,3;  k = (l-1)*3 + j,   (9)

As with previously discussed models, this specification makes a number of assumptions. First, it assumes that the person's measurement error is not affected by the level of a person's attitude. Thus, for example, a person with a favorable attitude is no more likely to have a high (low) measurement error than one with an unfavorable attitude. Second, it assumes that measurement errors associated with different methods are independent. Finally, it assumes that the attitude score and measurement error are additive. Although I offer no proof as to the veracity of these assumptions they do not seem to strain one's credibility.

It should be noted that the model is stated in terms of covariance versus correlations. Moreover, it is not possible to arbitrarily rescale the model to get the y's to have unit variances without imposing restrictions on the magnitude of the variances of the unobserved variables. Thus, if the correlation matrix were used instead of the covariance matrix, the parameter space would he further restricted so that

Clearly this is not a desired model property since there is no reason to believe that all the measurement errors using the same method have the same variance.

The above indicates that it is improper to use the correlation matrix as input. Unfortunately, Ostrom reported only the correlation matrix. Although there is some indication that he tried to insure that the variances for the observed scores for a given method are equal, use of the correlation matrix to estimate model parameters is incorrect. However, given the alternatives of doing nothing or doing something, I elected for the latter since it appeared that the variances of the y's for a given method were approximately equal. Consequently, I did not feel that there was too much error in this case in using the correlation matrix to estimate the model parameters. These estimated model parameter values are reported in Table 2.

TABLE 1

COVARIANCE MATRIX IN TERMS OF MODEL PARAMETERS

The results indicate that it is not possible to reject the hypothesis that three components of attitude exist and that measurement process is correct as modeled. The estimated correlations between the three attitude components indicate a high degree of commonality between the element (i.e., between .79 and .91), although the estimated correlations are lower than those reported by J-R. In general, the Likert method seemed to produce the smallest error variance, with Thurstone being next best, followed by the Guttman method. Interestingly, the correlation between the scores on measures using the same method were usually very small. The only method to exhibit significant (managerial and statistical) correlations was the Likert method.

SUMMARY OF CONVERGENT VALIDITY PAPERS

The above discussion is intended to supplement B-B's and J-R's presentation of the structural equation approach used by both sets of authors to test for convergent and discriminant validity. One of the major themes of this discussion is that if researchers are to use this approach they must first carefully think out all the assumptions which are implicitly or explicitly tied to the technique. In my opinion the J-R paper does not reflect this careful thinking in that they present a model which has no theoretical justification nor is it identified.

A second aim of my discussion is to put forward a model which simultaneously takes into account the influences on the observed responses of the constructs being tested, any relationships between the constructs and the methods used to obtain these measures. This structural model is an outgrowth of a model suggested by Bagozzi (1978) and implemented by J-R. Unfortunately the B-B paper does not use either the model proposed by Bagozzi or a variation of this model when testing the validity of the two component models of attitude even though the measures used to test the theory were obtained using the same method (see Figure 3 of B-B paper). Consequently, it is not possible for them to test if the estimated relationship between constructs A₂(affect) and B (behavioral intentions) is due to the measurement technique or the basic underlying relationship between the two variables. Hopefully, future work in this area will control for such confounding influences by incorporating my approach or some improved variation.

PREDICTIVE VALIDITY

The emphasis of the Cattin paper is on obtaining a model which has high predictive power. This is in contrast with the two previously discussed papers which were conceived with testing the validity of measures used to test a theory. Thus the only real commonality between Cattin's paper and the papers of B-B and J-R is the word "validity."

The Cattin paper first reviewed six formulae which estimate the correlation between an observation and the prediction of that observation (based on the regression model). It then presented an example which indicates for the data set selected that it is better to use an estimate of the population cross validation correlation than the more commonly used sample correlation estimate when selecting between regression models. Cattin offered no proof as to the generalizability of these results although he did review a series of Monte Carlo studies aimed at determining the properties of each estimator.

My major concern with the Cattin paper is the emphasis on predictive power versus theory as the criterion for selecting a model. Thus I don't believe a model normally should be selected just because it has a high population cross validation correlation. Instead, in most consumer behavior research, a researcher should start with a theory. This theory will determine the predictor variables and the functional form of the model. If the researcher wants to conduct an F test to determine if a subset of the model adequately represents the data, this is certainly permissible. However, selecting a model just because it has the highest predictive power is atheoretical and should be avoided in most consumer research settings. Thus, I am not in disagreement with Cattin over any technical issue. Instead, I question the value of the cross validation correlation coefficient in most consumer research situations. The only time this statistic would be of value is when a researcher wanted to develop a model that predicts well for different data sets. Even then, I would not just use the cross validation correlation coefficient in selecting the model, but also blend in my opinions as to the soundness of the alternative models.

TABLE 2

ESTIMATED PARAMETERS FOR OSTROM DATA

REFERENCES

R. P. Bagozzi, "The Construct Validity of the Affective, Behavioral and Cognitive Components of Attitude by Analysis of Covariance Structures," Multivariate Behavioral Research, 13 (January, 1978), 9-31.

M. Fishbein and I. Ajzen, "Attitude Towards Objects as Predictors of Single and Multiple Behavioral Criteria," Psychological Review, 81, (1974), 59-74.

K. G. J÷reskog and D. S÷rbom, "Statistical Models and Methods for Analysis of Longitudinal Data." In Latent Variables in Socioeconomic Models, ed. by D. J. Aigner and A. S. Goldberger, North Holland, Amsterdam 1977.

K. G. Joreskog and M. van Thillo, LISREL: A general computer program for estimating a linear structural equation system involving multiple indicators of unmeasured variables. Educational Testing Service Research Bulletin, (1972), 72-156.

T. M Ostrom, "The Relationship Between the Affective, Behavioral, and Cognitive Components of Attitudes," Journal of Experimental Social Psychology, 5 (1969), 12-30.

----------------------------------------