Construct Validation in Marketing: a Comparison of Methods in Assessing the Validity of the Affective, Conative, and Cognitive Components of Attitudes

George John, (student), Northwestern University (student), Northwestern University
Torger Reve,
ABSTRACT - The assessment of the validity of constructs in marketing are examined. An empirical comparison of different methods of establishing convergent and discriminant validity within the framework of multitrait-multimethod data analysis is made in the case of the affective, conative and cognitive components of attitude. Ostrom's (1969) data regarding attitudes toward the church was examined and revealed a lack of convergent and discriminant validity of the three components. The relative advantages and disadvantages of the different methods of analysis are compared and contrasted.
[ to cite ]:
George John and Torger Reve (1979) ,"Construct Validation in Marketing: a Comparison of Methods in Assessing the Validity of the Affective, Conative, and Cognitive Components of Attitudes", in NA - Advances in Consumer Research Volume 06, eds. William L. Wilkie, Ann Abor, MI : Association for Consumer Research, Pages: 288-294.

Advances in Consumer Research Volume 6, 1979      Pages 288-294

CONSTRUCT VALIDATION IN MARKETING: A COMPARISON OF METHODS IN ASSESSING THE VALIDITY OF THE AFFECTIVE, CONATIVE, AND COGNITIVE COMPONENTS OF ATTITUDES

George John (student), Northwestern University

Torger Reve (student), Northwestern University

ABSTRACT -

The assessment of the validity of constructs in marketing are examined. An empirical comparison of different methods of establishing convergent and discriminant validity within the framework of multitrait-multimethod data analysis is made in the case of the affective, conative and cognitive components of attitude. Ostrom's (1969) data regarding attitudes toward the church was examined and revealed a lack of convergent and discriminant validity of the three components. The relative advantages and disadvantages of the different methods of analysis are compared and contrasted.

DIMENSIONS OF CONSTRUCT VALIDITY

One of the most basic issues in empirical, behavioral research in marketing, as well as in the other social sciences, is the notion of construct validity, i.e., "the extent to which an operationalization measures the concept which it purports to measure" (Zaltman et al., 1973, p. 44). In spite of the importance of construct validity from both a scientific and an applied point of view, construct validation has been largely ignored by most researchers in marketing and consumer research. Most studies in marketing tend to be single-measure, one-shot affairs with little or no attention to validity. Cook and Campbell (1976), in their extensive discussion of the various threats to validity confronting experimental and quasi-experimental research in field settings, equate construct validity with what experimental psychologists mean by confounding, i.e., "the possibility that the operational definition of a cause or effect can be construed in terms of more than one construct, all of which are stated at the same level of reduction" (1976, p. 238). Confounding in this context means that "what one investigator might interpret as a causal relationship between A and B, another investigator might interpret as a causal relationship between A and Y or between X and B or even between X and Y, and later experiments might support one or the other of these reinterpretations" (ibid., p. 238).

This leads to a consideration of construct validity in terms of three empirical dimensions (Campbell and Fiske, 1959; Campbell, 1960; Kerlinger, 1973; Cook and Campbell, 1976, Bagozzi, 1977): (1) Convergent validity, i.e., the degree to which two or more attempts to measure the same concept through maximally different methods are in agreement, that is, intermethod convergence. (2) Discriminant validity, i.e., the degree to which a concept differs from other concepts. (3) Nomological validity, i.e., the degree to which predictions from formal theoretical networks containing the concept in question are confirmed. Bagozzi (1978) specifies two additional criteria which need to be met in order to demonstrate construct validity: (4) Semantic criteria, i.e., theoretical and observational meaningfulness. (5) Operational empirical criteria, i.e., internal consistency and reliability. The theoretical meaningfulness of a construct refers to the nature and internal consistency of the language used to represent the construct, while operational meaningfulness refers to the conceptual relationships between a theoretical variable and its operationalizations(s).

Comparing this full set of five criteria for achieving construct validity with the type of criteria usually found in published marketing studies clearly shows the current insufficiencies. At best, three types of criteria are applied in evaluating the measures used in marketing studies (c.f., Heeler and Ray, 1972): (1) Face or consensus validity, i.e., the degree to which a measure "looks as if" it should indicate a particular variable or concept. An example would be the prevalent use of recall measures as an indicator of advertising exposure and reception. (2) Predictive or concurrent validity, i.e., the extent to which a particular measure predicts other criterion measures (Zaltman et al., 1973, p. 44). An example would be the degree of correlation between post ad recall and attitude change. Another interpretation of the criterion is that the measure has been used extensively by others, which thus may lead to further strengthening of the "conventional wisdom." (3) Reliability, i.e., the degree of agreement between two efforts to measure the same concept through maximally similar methods (Campbell and Fiske, 1959), that is, intramethod convergence. Testing for reliability shows the extent to which scores on specific measures may be caused by irrelevant factors.

Most marketing studies seem to be lacking in terms of demonstrating either convergent or discriminant validity of the concepts used. Even when such validation is attempted, the usual practice is to use unstructured or ad hoc procedures, which are generally factor analytic or correlational in nature. A discussion of some of these factor analytic approaches follows in the next section.

AD HOC PROCEDURES FOR ASSESSING CONSTRUCT VALIDITY

The various empirical procedures for assessing construct validity which will be dealt with below, are basically confined to an analysis of convergent and discriminant validity.

Factor Analysis for Assessing Convergent Validity

The context for using a simple factor analytic method for assessing convergent validity is where a construct has been operationalized in terms of multiple measures, e.g., when several items in a questionnaire are taken to represent the same variable. The obtained measures for each variable can be factor analyzed to determine whether all items merit inclusion in the variable. Loosely, the criterion for excluding a particular item is that it exhibits little loading on the first factor extracted. If on the other hand, all measures load heavily on the first factor extracted, one can usually conclude that the items are all dimensions of the same underlying factor, that is, the variable that one intended to measure. An extension of this approach is to use the obtained factor scores for each subject in order to weight the items empirically when computing the composite score for the variable.

Factor Analysis for Assessing Discriminant Validity

The context for using a simple factor analytic method for assessing discriminant validity is where one is to show that two or more theoretical constructs are distinct and separate. A case in question would be where several constructs are included in a study, and each construct is operationalized by multiple measures. Evidence for discriminant validity is obtained if the measures for separate constructs do not load on the same factors. An example from marketing of this method for establishing discriminant validity is a study of distribution channels by Lusch (1976). Trying to distinguish between coercive and noncoercive sources of power, he included 16 sources which were thought to be noncoercive and 6 sources which were thought to be coercive and subjected his data obtained to common factor analysis. Of the five factors extracted, the first was the coercive factor, while the other four factors were noncoercive. In no cases did coercive and noncoercive sources load on the same factor, which was then taken as evidence for discriminant validity of the two constructs. A possible confounding in this particular case, however, was that different types of questions and different types of scales were used for coercive and noncoercive sources of power. Although the differences in scales are probably partly resolved through normalization of the data, the differences in types of questions may partially account for the loading on separate factors, that is, the extraction of a method factor rather than actual discriminant validity of the two constructs.

Factor Analysis for Assessing Both Convergent and Discriminant Validity

The two previous factor analytic procedures could easily be combined in the case where both multiple constructs and multiple measures of each construct are utilized. In terms of the example discussed above (Lusch, 1976), a simultaneous demonstration of discriminant and convergent validity could have been made if the five factors extracted corresponded to one coercive power factor, and to four distinct subcategories of noncoercive power (French and Raven, 1959). The above methods for assessing convergent and discriminant validity are ad hoc approaches in that there is no structured procedure for disentangling the method and trait variance. A solution to such problems may be found when turning to multitrait-multimethod procedures for assessing construct validity as first suggested by Campbell and Fiske (1959). Some recent approaches for assessing construct validity using multitrait-multi-method data will then be presented and applied to the attitude construct.

MULTITRAIT-MULTIMETHOD PROCEDURES FOR ASSESSING CONSTRUCT VALIDITY

Campbell and Fiske (1959), introduced the multitrait-multimethod matrix (MTMM-matrix) as a tool for assessing convergent and discriminant validity. The first reported attempt to utilize the MTMM methodology in marketing were made by Myers (1968) and Ray (1968). In spite of the large number of present and potential applications of MTMM analysis in marketing suggested by Heeler and Ray (1972), this methodology for assessing construct validity has not yet been widely used within marketing and consumer research. Some of the possible extensions of the MTMM approach to construct validation as suggested by several marketing studies include multi-respondent-multimethod matrices (Davis, 1971), multi-trait-multi-context matrices (Robertson and Myers, 1969; Myers and Robertson, 1972), and even multi-item-multi-product matrices (Sears, 1969). Davis (1971) in his study of family decision making measured purchase influence of husbands and wives using four different measures, while Myers and Robertson (1972) developed the data for opinion leadership traits across a large number of contexts, but without putting the data into a full matrix format. Several methods for analyzing multitrait-multimethod data exist (Schmitt et al., 1977).

In this paper three different methods will be outlined: (1) Campbell and Fiske counting procedure, (2) Joreskog Restricted Maximum Likelihood Factor Analysis, (3) Jackson's Two Stage Principal Component Analysis.

Campbell and Fiske Analysis

The multitrait-multimethod approach tries to determine convergent and discriminant validity through an analysis of the pattern of correlations among two or more traits as measured by two or more methods. An example of a multitrait-multimethod matrix where three traits (A, B, C) are measured by three methods (1,2,3) is shown in Figure 1. The MTMM matrix contains the intercorrelations among the observed measures which comprise the available observed data. Let r(jk)(j'k') represent the correlation between the measure jk and the measure j'k' where j designates trait and k designates method. The MTMM matrix can then be described by the following partitions: 1. The monotrait-monomethod values are composed of the elements r(jk)(j'k') where j = j' and k = k'. These main diagonal elements of the MTMM matrix represent the reliabilities of the measures, either in terms of stability or internal consistency. If these correlations are found to be sufficiently high, further validation can be undertaken. 2. The heterotrait-heteromethod triangles are composed of the elements r(jk)(j'k') where j j' and k k'. 3. The validity diagonals or monotrait-heteromethod values are composed of the elements r(jk)(j'k') where j = j' and k k'. 4. The heterotrait-monomethod triangles are composed of the elements r(jk)(j'k') where j j' and k = k'. To assess convergent and discriminant validity in terms of the pattern of correlations as indicated by the MTMM matrix as described above, Campbell and Fiske (1959, pp. 82-83) suggested four types of comparisons of correlation coefficients, sometimes referred to as the counting method: 1. Evidence of convergent validity is established by examining the correlations between different measures of the same traits. This means that the elements in the validity diagonal value should be higher than the values lying in its column and row in the heterotrait-heteromethod triangles. For a given variable, its validity diagonal value should be higher than its corresponding value in the heterotrait-monomethod triangles. The same pattern of trait interrelationships should be shown in all of the heterotrait triangles of both monomethod and heteromethod blocks. As can be shown from the simple case of three traits and three methods, a full utilization of the Campbell and Fiske criteria will lead to a large number of comparisons of correlations in order to establish both convergent and discriminant validity. Furthermore the criteria become ambiguous when only some of the comparisons fulfill the criteria while others do not. A simple counting procedure of the number of comparisons that satisfy each criterion to the number of comparisons that do not (e.g., Kothanadapani, 1971) may lead to false conclusions as to convergent and discriminant validity (Bagozzi, 1978).

The Campbell and Fiske counting criteria further assume that (1) the traits and methods are uncorrelated, (2) the methods are only minimally intercorrelated (Campbell and Fiske, 1959; Alwin, 1974; Kalleberg and Kluegel, 1975). In practice, however, this is rarely true (c.f., Jackson, 1969; Alwin, 1974). Consider for instance, the violation of assumption (2). One would obtain spuriously high correlations in the monotrait-heteromethod cases, thus, leading to a misleading conclusion of high convergent validity. Consequently, various analytic schemes have been proposed as alternatives to the original Campbell and Fiske criteria.

Joreskog Restricted Maximum Likelihood Factor Analysis

One class of models is the restricted maximum-likelihood factor analysis models. Boruch and Wolins (1970) propose the model

Yijk = Bjk Xij + Cjk Xik + Eijk  (1)

where Yijk = observation on ith subject on hypthetical factor associated with trait j  (2)

Xik = score of ith subject on hypothetical  (3)

Eijk = error associated with ith subject, trait, j. method k  (4)

The covariance matrix of the observations can be written as S = LfL' + y where f is the factor intercorrelation matrix and y is the covariance matrix of the error terms. If the data is to exhibit evidence of convergent and discriminant validity then it should be possible to fit such a model to the data which is also a unique solution in that it cannot be rotated as usual factor analysis results can be treated. We obtain such a unique solution by specifying certain restrictions on the parameters of the various matrices. Consider the loading matrix in Figure 2.

FIGURE 1

MULTITRAIT-MULTIMETHOD MATRIX OF CORRELATIONS FROM OSTROM (1969)

Each column represents a trait or a method factor and the rows represent the different trait-method measures. Now if a particular trait is present in a measure, then the loading of that measure on its trait factor is left unrestricted. The same logic is applied to the methods in each measure. However, all other loadings are restricted to be zero. This is the model which would hold if the traits did indeed possess discriminant and convergent validity. The f matrix is left unrestricted. However, we may restrict the parameters of f such that the trait and method factors are uncorrelated if we feel that such an assumption is warranted in a particular case.

A unique solution (or identifiability) may not always be possible and the necessary and sufficient conditions for such a solution existing have not been worked out for the general case of this class of models. As for significance tests, a X2 statistic based on a likelihood ratio is available which compares the estimated correlation matrix produced by the model with the actual correlation matrix. This statistic can also be used to test particular parameters in a model by estimating one model as a restricted subset of another model (J÷reskog, 1971). In essence, then, in this approach, we are hypothesizing a particular factor structure and testing it. J÷reskog (1967, 1971) has developed an efficient estimation procedure using maximum-likelihood procedures for this kind of factor analytic model (i.e., the RMLFA model).

The analysis strategy, then, is to attempt to search for a reasonable model of the sort described which fits the data to an acceptable degree thus enabling us to conclude that discriminant validity and convergent validity have been achieved.

Jackson's Two Stage Principal Components Method

Jackson (1969) proposed an alternate approach to the analysis of multimethod multitrait data. The previous model (S = LfL' + y) assumes that the data is comprised of a set of trait and method factors. If this is indeed the case, a principal axes analysis of the correlation should yield these trait and method factors. However, a straightforward application of a principal components analysis would yield mixed method and trait factors because of the trait-method factor intercorrelations. In order to derive estimates of the variance accounted for by the trait and method factors, Jackson (1975) proposed the following procedure. Essentially, the procedure is a two-stage factor analysis where the first stage consists of an orthogonalization of the mono-method heterotrait submatrices via a full principal components analysis. Then, a cross-products matrix of factor scores is constructed and subjected to a further principal components analysis. In this second stage, the factor loading matrix that is obtained should exhibit "clean" trait factors if convergent and discriminant validity exists. For the technical details of the method see Jackson (1975). The justification of this procedure follows from the fact that the restriction of orthogonality we imposed via a principal components analysis on the individual traits within a method from being reflected in the constructed matrix of component scores. Thus, if we define method variance as variance unique to a particular method of measurement, then such method variance will not be present in common factors derived from the intercorrelation matrix of factor scores. In a similar fashion, trait variance can be eliminated and method factors can be extracted by rearranging the measures so that all methods of measuring each trait are grouped together in the diagonal submatrices of the MTMM matrix.

FIGURE 2

A MATRIX

The Tripartite Classification of the Attitude Construct

The MTMM matrix analyzed in this study attempts to establish the convergent and discriminant validity of the tripartite classification of the attitude construct.

The attitude construct itself has been the center of considerable controversy especially in its relation to behavior. At one extreme, it has been argued (e.g., Doob, 1947) that attitudes are merely overt behavioral responses and that its existence separate from behavior cannot be justified. On the other hand, the evidence from attitude-behavior studies seems to indicate a lack of consistent, strong relationships between attitude a and behavior (Wicker, 1969). Generally, though, it is held (Fishbein and Ajzen, 1975; Greenwald, 1968) that attitudes are causally linked to behavior. If we are to attempt to test this linkage, it is imperative to define the attitude construct and measure its components. Otherwise, our conclusions regarding attitude-behavior consistency will remain equivocal as it becomes difficult, if not impossible, to pin down the locus of the cause of the observed relationship (or lack thereof). There are two broad classes of models of the attitude construct. The first of these is the expectancy value model (e.g., Fishbein and Ajzen, 1975) which holds that an attitude toward an object or act consists of two components, viz. cognitions of values of outcomes or goals associated with the object and the beliefs that the object will help or hinder the attainment of these outcomes or goals. Then, these cognitions and beliefs are combined in some fashion (usually a linear additive model is used) to determine one's attitude.

The major competing model is the tripartite classification scheme (Rosenberg, 1960; Insko and Scholpler, 1967). 1967). Here, three distinguishable classes of concepts regarding an object are held to exist. These are the cognitive, affective and conative (or behavioral intention) components. Briefly, the cognitions are the statements of beliefs or facts regarding the object, while the affective component is the emotional content of thoughts regarding the object. The affective component is also regarded as being evaluative/directional with respect to the object. The conative component is the action tendency one has toward the object. No hierarchical ordering of these components is presumed about the linkages between these categories. Each component can affect behavior separately. More importantly different antecedent processes are associated with each of them (e.g., Greenwald, 1968).

There have been two major attempts at validating the scheme by collecting data within an MTMM format (Ostrom, 1969; Kothanadapani, 1971). Some evidence for convergent and discriminant validity of the components were found by both authors but the results must be regarded as rather tentative because of the limitations of the analysis used in the two studies. In this reanalysis, the data from Ostrom's study is investigated for convergent and discriminant validity.

Data Collection

Ostrom (1969) collected data about attitudes toward the church. From his sample of 189 undergraduate students, he constructed four measures of each attitude component. We shall examine the MTMM matrix formed from three of the methods (viz. a Thurstone scale, a Likert scale and a Guttman scale). A complete description of the actual scale items, the procedures used in scale construction and the collection of data is given in Ostrom (1969). Figure 1 shows the MTMM matrix.

EMPIRICAL RESULTS

Campbell-Fiske Counting Method: Ostrom applied the counting criteria to his MTMM matrix and concluded that it did provide evidence of convergent and discriminant validity of the components. We did not duplicate his analysis but shall note that this result can be compared to those obtained from the other two methods.

Joreskog's Restricted Maximum Likelihood Factor Analysis: The parameters of the various hypothesized models were estimated by the COFAMM computer program developed by J÷reskog and S÷rbom (1976). The first model that was estimated was a six factor model (three trait and three method factors). Figure 3 shows the estimated L, f, and y matrices. The goodness-of-fit statistic shows an excellent fit (p = .98). However, an examination of the parameters reveals certain problems. For instance, two factor loading parameters are greater than unity while one specific variance estimate is negative. These boundary problems indicate that the model is an unacceptable solution and can at best be used to give clues towards modifying the model. Specifically, the high factor intercorrelations suggested that a more parsimonious solution may be possible. Table 1 summarizes the various models and their solutions. It should be noted that boundary problems were present for almost every single model although the X2 statistic indicated an acceptable fit in almost every case. The two models that were characterized by acceptable parameter estimates are shown in Figure 4 and Figure 5. Figure 4 displays a three trait two method model with restrictions in the matrix such that the trait and method factors are uncorrelated. The degree of fit is acceptable (p = .72), and one would assume that convergent and discriminant validity had been established. However, an examination of the trait factor intercorrelations reveals extremely high values. Specifically, w13 = .935, w13 = .935, w23 = .901 and w12 = .857. This means that the trait factors are highly collinear and suggests that the traits (i.e., the attitude components) are virtually indistinguishable from each other. Consequently, two of the components (affect and cognition) were collapsed into a single factor and the resulting model was estimated. Figure 5 shows that the fit is not unacceptable (p = .58) and that this model cannot be rejected as an adequate representation of the data.

This final model does not support a conclusion of three distinct, discriminable traits. Convergent validity, however, is present when considering the magnitudes of the loadings in the model estimates in Figure 4 and Figure 5. It should be noted that the X2 tests are not independent of each other, and, ideally, we would like to have a hold-out sample for confirmation of the final model.

FIGURE 3

MODEL 1 IN TABLE 1

FIGURE 4

MODEL 6 IN TABLE 1

TABLE 1

RESTRICTED MAXIMUM LIKELIHOOD FACTOR ANALYSIS MODELS

FIGURE 5

MODEL 7 IN TABLE 1: X2 (16) = 14 (p = .58)

Jackson's Two Stage Principal Components Analysis

The present authors applied the two stage technique to this data and the loading matrix (after Varimax rotation) of the second stage is shown in Figure 6. The three factor solution in Figure 6 was chosen on the basis of an eigenvalue cutoff rule (i.e., less than unity). Consider the first factor, A1, B3 and C2 all load heavily on this factor. Since these are measures of affect, conation and cognition respectively, it is clear that we do not have a clean 'trait' factor. In the second factor, B1, A2, A3 are the measures that load heavily on it. Again, the presence of the conation measure, B1, prevents us from labeling it as an 'affect' factor. C1, B2 and C3 contribute heavily to the third factor and again, we do not seem to have a clean trait factor.

The conclusion to be drawn from this analysis is that we cannot conclude that the traits or components are sufficiently different and distinguishable from each other to say that convergent and discriminant validity have been established.

FIGURE 6

LOADING MATRIX OF THE SOLUTION OF THE 2-STAGE PRINCIPAL COMPONENTS ANALYSIS. VARIMAX ROTATED.

DISCUSSION

The two factor analytic methods concur in that there is a lack of convergent and discriminant validity. The counting procedure, on the other hand, finds support for both types of validity. These apparent contradictions can be traced back to the assumptions of each analysis.

In the counting procedure, the assumptions that the trait and method factors are uncorrelated is probably untenable for this data. Figure 3 shows the estimated correlations of the factors in the matrix and it can be seen that they are very high. In fact, two of the method factors were collapsed into a single factor in the later models. These results mean that the conclusions drawn from the counting procedure may be spurious.

The restricted maximum likelihood approach is an elegant analytic scheme, but it suffers from a number of operational and computational drawbacks. As we indicated previously, a large number of the fitted models have parameter estimates that are clearly meaningless (e.g., negative specific variances). These problems leave the status of these models unsettled. This is also true of those models which failed to converge to a solution. The result of these problems is that the final fitted model is the result of considerable iterative model fitting. This can result in different interpretations being drawn from the same data. Bagozzi (1978) analyzed this data and estimated a model that is virtually identical to Figure 6 and concluded that convergent and discriminant validity had been established. However, as we saw previously, we did not accept this model as the final model because of the unacceptably high intercorrelation between the trait factors.

The Jackson procedure, in contrast, is computationally more appealing and is subject to fewer problems of the type mentioned above. However, at the theoretical level, it suffers from some potentially serious drawbacks. For instance, in the first stage, we orthogonalize the diagonal submatrices to extract trait factors. However, we are not disentangling the variance contribution from the trait and method sources; i.e., there is a possibility of confounding these sources of variation. Further, the principal components model does not allow for specific or residual variance. Thus, if we are dealing with measures with relatively large amounts of specific variance, this could well obscure the convergence and discriminability of the traits.

Table 2 summarizes the relative advantages and disadvantages of the methods. The best strategy to follow in the face of these various drawbacks of these different techniques is to proceed to use them simultaneously. In this way, the credibility of the findings is enhanced if different techniques permit one to draw the same conclusion. In the present study, both factor analytic techniques arrive at the same conclusion that it is not possible to establish the validity of the tripartite classification scheme for this data. We should mention, however, that this was a very stringent test of the scheme as we were dealing with attitudes toward an object where we would expect a great deal of consistency and collinearity between the traits. This is because attitudes towards the church are formed over a long period of time and is therefore likely to exhibit a great deal of stability and consistency. In spite of this, we did establish the presence of two separate traits on the J÷reskog models; this indicates that it is possible that the three components will be discriminable in other less stringent situations.

TABLE 2

COMPARISON OF METHODS OF ANALYZING MTMM DATA

REFERENCES

D. Alwin, "An Analytic Comparison of Four Approaches to the Interpretation of Relationships in the Multi-trait-Multimethod Matrix," in H. Costner (ed.), Sociological Methodology, 1973-1974 (San Francisco: Jossey-Bass, 1974)

R. P. Bagozzi, "The Construct Validity of the Affective, Behavioral and Cognitive Components of Attitude by Analysis of Covariance Structures," Multivariate Behavioral Research, 13 (January, 1978), 9-31.

R. F. Boruch and L. Wolins, "A Procedure for Estimation of Trait, Method, and Error Variance Attributable to a Measure," Educational and Psychological Measurement, 30 (1970), 547-574.

D. T. Campbell, "Recommendations for APA Test Standards Regarding Construct, Trait, or Discriminant Validity," American Psychologist, 15(1960), 546-553.

D. T. Campbell and D. W. Fiske, "Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin, 56(March, 1959), 81-105.

T. D. Cook and D. T. Campbell, "The Design and Conduct of Quasi-Experiments and True Experiments in Field Settings," in M. D. Dunnette (ed.), Handbook of Industrial and Organizational Psychology (Chicago: Rand McNally, 1976), 223-326.

H.J. Davis, "Measurement of Husband-Wife Influence in Consumer Purchase Decisions," Journal of Marketing Research, 8(August, 1971), 305-312.

L. W. Doob, "The Behavior of Attitudes," Psychological Review, 54(1947), 135-156.

M. Fishbein and I. Ajzen, Belief, Attitude, Intention and Behavior (Reading, Mass.: Addison-Wesley, 1975)

J. R. D. French and B. Raven, "The Bases of Social Power," in D. Cartwright (ed.), Studies in Social Power (Ann Arbor, Institute for Social Research, University of Michigan, 1959), 150-167.

A. G. Greenwald, "On Defining Attitude and Attitude Theory," in A. G. Greenwald, T. C. Brock and T. M. Ostrom (eds.), Psychological Foundations of Attitudes (New York: Academic Press, 1968), 361-388.

R. M. Heeler and M. L. Ray, "Measure Validation in Marketing,'' Journal of Marketing Research, 9(November, 1972), 361-370.

R. M. Heeler, T. W. Whipple and T. P. Hustad, "Maximum Likelihood Factor Analysis of Attitude Data," Journal of Marketing Research, 14(February, 1977), 42-51.

C.A. Insko and J. Schopler, "Triadic Consistency: A Statement of Affective-Cognitive Consistency," Psychological Review, 74(1967), 361-376.

D. N. Jackson, "Multimethod Factor Analysis in the Evaluation of Convergent and Discriminant Validity," Psychological Bulletin, 72(July, 1969), 30-49.

D. N. Jackson, "Multimethod Factor Analysis," Multivariate Behavioral Research, 10 (July, 1975).

K. G. Joreskog, "Some Contributions to Maximum Likelihood Factor Analysis," Psychometrika, 32(December, 1967), 443-482.

K. G. Joreskog, "Statistical Analysis of Sets of Congeneric Tests," Psychometrika, 36(June, 1971), 109-133.

K. G. Joreskog and D. S÷rbom, COFAMM: A Fortran IV Program (Chicago: National Resources, Inc., 1976).

A. J. Kalleberg and J. R. Kluegel, "Analysis of the Multi-trait-Multi-Method Matrix: Some Limitations and an Alternative,'' Journal of Applied Psychology, 60(1975), 1-9.

F. N. Kerlinger, Foundations of Behavioral Research, 2nd Edition (New York: Holt, Rinehart and Winston, 1973).

V. Kothanadapani, "Validation of Feeling, Belief, and Intention to Act as Three Components of Attitude and Their Contribution to Prediction of Contraceptive Behavior,'' Journal of Personality and Social Psychology, 19 (September, 1971), 321-333.

R. F. Lusch, "Sources of Power: Their Impact on Intrachannel Conflict," Journal of Marketing Research, 13 (November, 1976), 382-390.

J. G. Myers, Consumer Image and Attitude (Berkeley: Institute of Business and Economic Research, University of California at Berkeley, 1968)

J. G. Myers and T. S. Robertson, "Dimensions of Opinion Leadership," Journal of Marketing Research, 9(1972).

T. M. Ostrom, "The Relationship Between the Affective, Behavioral, and Cognitive Components of Attitudes," Journal of Experimental Social Psychology, 5(1969), 12-30.

M. L. Ray, "Neglected Problems (opportunities) in Research: The Development of Multiple and Unobtrusive Measures," Fall Conference Proceedings (Chicago: American Marketing Association, 1968), 176-183.

T. S. Robertson and J. G. Meyers, "Personality Correlates of Opinion Leadership and Innovative Buying Behavior," Journal of Marketing Research, 6(May, 1969), 164-168.

M. J. Rosenberg, C.I. Hovland, W. J. McGuire, R. P. Abelson and W. J. Brehm, Attitude Organization and Change (New Haven, Conn.: Yale University Press, 1960)

D. O. Sears, "Political Behavior," in G. Linzey and E. Aronson (eds.), Handbook of Social Psychology, V (Reading, Mass.: Addison-Wesley, 1969), 315-457.

N. Schmitt, B. W. Coyle and B. B. Saari, "A Review and Critique of Analyses of Multitrait-Multimethod Matrices" Multivariate Behavioral Research, 12(December, 1977), 447-478.

A. Wicker, "Attitudes Versus Actions: The Relationship of Verbal and Overt Behavioral Response to Attitude Objects,'' Journal of Social Issues, 25(1969), 41-47.

G. Zaltman, C. R. A. Angelmar and R. Angelmar, Metatheory and Consumer Research (New York: Holt, Rinehart and Winston, 1973).

----------------------------------------