Proceedings of the Third Annual Conference of the Association for Consumer Research, 1972 Pages 586-593
AN EXAMINATION OF CONCEPT VALIDITY
Reinhard Angelmar, Northwestern University
Gerald Zaltman, Northwestern University
Christian Pinson, Northwestern University
[Gerald Zaltman is Associate Professor of Behavioral Science, Director of Research, Graduate School of Management and Faculty Associate, the Center for the Interdisciplinary Study of Science and Technology, Northwestern University. Reinhard Angelmar and Christian Pinson are currently completing their doctoral requirements in the Marketing Department, Graduate School of Management, Northwestern University.]
This paper will examine various types of concept validity of relevance to the study of consumer behavior. The issue of concept validity may well be the Achilles heel in the study of consumer behavior. This statement itself has some consensual validity. For example, in an article devoted to the problems in consumer behavior research, Kollat, et. al. (1970:328) pointed out that "future progress in consumer behavior research will depend on overcoming several problems with commonly used variables and constructs." Kassarjian (1971), in his review of personality concepts in consumer behavior, has similarly emphasized the crucial role of having valid concepts.
Any discussion of the validity of concepts involves metatheoretical considerations. Metatheory is the investigation, analysis, and the description of (1) the technology of building theory, (2) the theory itself, and (3) the utilization of theory. Concepts are the essential building blocks of theory and a theory can be no better than its concepts. Especially relevant*is the issue of the validity of concepts and hence the validity of theories. [For a more complete treatment of the various dimensions of concepts, see Chapter 2 in Gerald Zaltman, Christian Pinson and Reinhard Angelmar, Metatheory and Consumer Research (New York: Holt, Rinehart & Winston, 1973).] The goal of this paper is to contribute to the solution of the present conceptual problems by reviewing several commonly used but rarely made explicit types and criteria of concept validity.
TYPES OF CONCEPT VALIDITY
Seven types of concept validity will be discussed below. These are shown in summary form in Table 1. The first and most traditional approach considers only observational concepts to be valid. In its extreme form this approach, which has been called operationism, requires the exhaustive reducibility of all concepts to observations (Nagel, 1961). Hempel (1966:88) notes that the "central idea of operationism is that the meaning of every scientific term must be specifiable by indicating a definite testing operation that provides the criterion for its application." Concepts that are provided with such criteria are said to be operationally defined. Bridgman (1927) adds a further specification to this point of view, stipulating that different operations characterize different concepts which ideally should be designated by different terms. This version of operationism has also been called "definitional" operationism (Campbell, 1969).
TYPES OF CONCEPT VALIDITY
The basic motivation underlying operationism is to "emancipate science from any dependency on unverifiable 'metaphysical' commitments" (Nagel, 1961:119) This view has been criticized on several counts. One counterargument notes that
"if explicit definitions of all theoretical terms by means of observables could be carried out, theories would be incapable of growth and therefore useless" (Hesse, 1967:406-407). Hempel argues that the definitional operationist maxim would lead to a proliferation of concepts "that would not only be practically unmanageable but theoretically endless, and this would defeat one of the principal purposes of science; namely, the attainment of a simple, systematically unified account of empirical phenomena" (Hempel, 1966:94). Campbell also advances a theoretical and a practical argument against definitional operationism. The theoretical argument says that any specific measurement reflects not a single parameter of a scientific theory but is "a joint function of many scientific laws." The practical argument says that the doctrine does not take into account the oncoming effort to improve measurement devices.
A less extreme form of operationism has been advanced by Carnap (1956). His requirement involves only partial reducibility of concepts to observations. If a concept is introduced into some scientific system one must be able to construct some proposition containing this new concept which, together with one or several other propositions containing only already tested terms, entails observation statements whose truth can be directly tested. This approach avoids the disadvantages of the extreme operationist position while still guaranteeing the empirical significance of concepts.
Most consumer behavior researchers seem to be well aware of this type of concept validity. In fact, one sometimes gets the impression that the observational validity of concepts is emphasized too much, and at the expense of other validity-types (such as construct validity).
The next three types of concept validity to be dealt with have received substantial attention by psychologists. Here, we refer to content, criterion-related, and construct-validity.
Content validity refers to "the degree that the score or scale being used represents the concept about which generalizations are to be made" (Bohrnstedt, 1970:91). In order to assess content validity, it is necessary to define the universe. Only if this is done can the representativeness of the measure be evaluated. Consider the example of the concept of opinion leadership. In order to determine whether a certain opinion leadership measure has content validity, the classes of behavior to which the concept refers have to be defined. Following this, it is possible to determine whether the observations which the measure implies are representative of this universe.
Criterion-related validity is concerned with how well the concept enables one to predict the value of some other concept which constitutes the criterion. It is also called empirical or practical validity (Campbell, 1960). Criterion-validity consists of two subtypes, predictive and concurrent validity. In predictive validity, the criterion measure is separated in time from the predictor concept, while for concurrent validity both concepts are measured at the same time.
The distinctive characteristic of criterion-related validity is that, due to the "socially institutionalized and valued nature of the 'criterion,' it is taken as an immutable given" (Campbell, 1960:547). Purchase behavior, which fits Campbell's characterization as well as any other variable, is a frequent criterion in consumer behavior research.
A study by Axelrod (1968) provides an example of Predictive validation. Axelrod was interested in finding a "measure that not only reflects the immediate effect of a stimulus on a consumer but also predicts his subsequent purchase behavior." For this purpose he developed ten measures, tested each one and concluded that two of the measures had the highest predictive validity as far as short-term trends in purchase behavior are concerned. The predictive validity of each measure was determined as the percentage of obtained market as compared to predicted market.
Many studies in consumer behavior research consist of the simultaneous collection of measures of many consumer characteristics. One or several of these characteristics--usually those related to purchase behavior--are then taken to be the criterion, and their presence or absence (or their value if the criterion is quantitative) is "predicted" with the help of the remaining variables. Such studies are typical examples of concurrent validation. The better a concept "predicts" the criterion, the greater its concurrent validity.
A study by Robertson and Kennedy (1969) illustrates concurrent validation. Data on a number of consumers were collected. One of the characteristics measured was possession of a small home appliance. This was taken to be the criterion. The remaining variables were used to predict the possession of the appliance. The main result of the study was that, from among the variables considered, venturesomeness and social mobility had the highest concurrent validity
Construct validity refers to the extent to which an operationalization measures the concept which it purports to measure. Following Campbell (1960), three types of construct validity can be distinguished: convergent, discriminant, and nomological validity. The first two types can be considered together under the label "trait validity."
The distinguishing characteristic of trait validity is that, in contrast to criterion-related validity, there is "no a priori defining criterion . . . available as a perfect measure or defining operation" (Campbell, 1960) against which to check a new measure. Instead, all of the measures are considered to be fallible.
Convergent validity refers to the degree to which two attempts to measure the same concept through maximally different methods are convergent. Discriminant validity refers to the extent to which the measure of a concept is related to measures of other concepts from which it is supposed to differ (Campbell ant Fiske, 1959).
A recent study by Jacoby (1972) illustrates convergent and discriminant validation procedures. The concepts investigated were opinion leadership for clothing, for alcoholic beverages, and for LP records. While these three concepts are supposed to be different. they are also supposed to be related. This makes the establishment of discriminant validity somewhat more difficult than if the concepts chosen had been independent from one another. The methods consisted of self-designation, sociometric choice, and the key-informant technique.
Convergent validity of each concept was measured by the correlation between the results of the three methods to measure the same concept. For example, the correlation between the results of the key-informant method and the sociometric method for measuring opinion leadership for LP records was calculated. All of these correlations turned out to be quite high.
In order to establish discriminant validity three conditions have to be satisfied (Campbell and Fiske, 1959): (1) The convergent validity for any concept should be higher than the correlation between any measure of that concept and a different concept measured by a different method. For example, the correlation between opinion leadership for clothing as measured by the key informant technique and by the sociometric technique, should be higher than the correlation between opinion leadership for clothing as measured by self-designation and opinion leadership for alcoholic beverages as measured by sociometric choice. (2) The convergent validity for any concept should be higher than the correlation of a concept with another concept, when both are measured by the same method. For example, the correlation between opinion leadership for alcoholic beverages measured by self-designation and as measured by the key-informant technique, should be higher than the correlation between the value type of opinion leadership and opinion leadership for clothing, with both concepts being measured by the sociometric technique. (3) The same pattern of interrelationships ought to obtain between the correlations of different concepts measured by the same method and as measured by different methods. For example, the ranking by magnitude of the correlations between opinion leadership for clothing, alcoholic beverages, and LP records, measured by the key-in format technique, should be the same as the ranking of the correlations between these concepts where each is measured by different methods. In Jacoby's study all of the three conditions of discriminant validity were fairly well satisfied.
Nomological validity refers to the extent to which predictions based on the concept which an instrument purports to measure are confirmed (Cronbach and Meehl, 1955). Another study of opinion leadership illustrates this type of validation procedure. Corey was interested in determining whether a particular type of technique was a valid measure of opinion leadership (Corey, 1971). At the time of the study, a body of knowledge concerning certain characteristics of opinion leaders had been accumulated. Corey reasoned that if his measure were in fact a valid measure of opinion leadership, the people classified as opinion leaders by it ought to have the characteristics indicated by the literature. This turned out to be true. Hence, he concluded that his instrument was a valid measure of opinion leadership.
The next approach to concept validity is concerned with a concept's systemic validity. This refers to the extent to which a concept can "establish relations among concepts and contribute thereby to systemicity (theoretical fertility) " (Bunge, 1957:133). For example, the concept of reciprocity can be used to explain the adoption or purchase of a standardized product in an interpersonal selling situation. The concept of reciprocity refers to feelings of obligation experienced by one party (e.g., a potential buyer) as he perceives another party (e.g., a salesman) investing or expending scarce resources on his behalf.
Another dimension of concept validity refers to the extent to which a concept has a uniform semantic usage (Marx, 1963). This can be called semantic validity. Kollat et. al., in their article earlier, mention several concepts whose semantic validity is notoriously low, among them brand loyalty, innovation, culture, and motive. This dimension of concept validity is of particular importance for the comparison, accumulation, and synthesis of findings, activities all of which are basic to paradigmatic research (Kuhn, 1962).
The last but not least important dimension to be considered here is a concept's control validity. This refers to the extent to which a concept is manipulatable and capable of influencing other variables of interest. The concepts which one finds in consumer behavior research range all the way from directly manipulatable concepts such as price to indirectly manipulatable concepts such as attitude to not at all manipulatable concepts such as birth-order (Kirchner, 1971). From the point of view of control validity concepts such as birth order have little validity. Intermediate concepts such as attitude have control validity to the extent to which antecedents of attitudes can be manipulated, and to the extent to which there is a substantial relationship between attitudes and purchase behavior. For example, Bauer (1966:8) has asserted that "the validity of our assessments of attitudes is solely the utility of the inferred concept for understanding, predicting and influencing the behavior of individuals."
This paper has noted rather briefly several types of concept validity which are relevant for the study of consumer behavior. Greater sensitivity to these validity criteria should produce more concrete and more useful concepts in marketing research. This in turn should lead to stronger theories in consumer behavior contexts. It is felt that the validity of concepts in current use in marketing and particularly in consumer behavior, leaves much to be desired. Evaluating concepts in terms of the types of concept validity presented here should strengthen this present state somewhat. Attention should be given to other possible types of concept validity.
Axelrod, J. N. Attitude measures that predict purchase. Journal of Marketing Research, 1968, 8, 3-18.
Bauer, R. Attitudes, verbal behavior and other behavior. In Adler, L. and Crespi, I. (eds.), Attitude research at sea. Chicago: The American Marketing Association, 1966, 3-14.
Bohrnstedt, G. W. Reliability and validity assessment in attitude measurement. In Summers, G. F. (ed.), Attitude measurement. Chicago: Rand-McNally, 1970.
Bridgeman, P. W. The logic of modern physics. New York: Macmillan, 1927.
Bunge, M. Scientific research, I and II. New York: Springer-Verlag, Inc., 1967.
Campbell, D. T. Definitional versus multiple operationalism. In Et. A1., 9969, 2, 14-17.
Campbell, D; T. Prospective: artifact and control. In Rosenthal, R. and Rosnow, R. L. (eds.), Artifact in behavioral research. New York: Academic Press. 1969. 351-382.
Campbell, D. T. Recommendations for APA test standards regarding construct, trait, or discriminant validity. American Psychologist, 1960, 15, 546-553.
Campbell, D. T. and Fiske, D. W. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 1959, 56, 81-105.
Carnap, R. The methodological character of theoretical concepts. Minnesota Studies in the Philosophy of Science, 1956, I.
Corey, L. G. People who claim to be opinion leaders: identifying their characteristics by self-report. Journal of Marketing. October, 1971, 35, 48-53.
Cronbach, L. J. and Meehl, P. E. Construct validity in psychological tests. Psychological Bulletin, 1955, 52, 281-302.
Helmstadter, G. C. Principles of Psychological measurement. New York: Appleton-Century-Crofts, 1964.
Hempel, C. G. Philosophy of natural science. Englewood Cliffs, N.J.: Prentice-Hall, 1966.
Hesse, M. Laws and theories. In Edwards, P. (ed.), EncycloPedia of philosophy. New York: Free Press, 1967, 406-407.
Jacoby, J. Establishing the construct validity of opinion leadership. Purdue papers in Consumer Psychology, 1972, 121.
Kassarjian, H. H. Personality and consumer behavior: a review. Journal of Marketing, 1971,8,No.4, 409-419.
Kirchner, D. F. Personal influence, ordinal position and purchasing behavior. ACR-Proceedings, 1971, 82-98.
Kollat, D. T., Engel, J. F., and Blackwell, R. D. Current problems in consumer behavior research. Journal of]Marketing Research, 1970, 7, 327-332.
Kuhn, T.S. The structure of scientific revolutions, 2nd ed. Chicago: University of Chicago Press, 1962.
Marx, M H. The dimension of operational clarity. In Marx, M H. (ed.), Theories in contemporary Psychology. New York: Macmillan, 1963, 187-202.
Nagel, E. The structure of science. New York: Harcourt, Brace, 1961
Robertson, T. S. and Kennedy, J. N. Prediction of consumer innovators: application of multiple discriminant analysis. Journal of Marketing Research, February, 1969, 5, 64-69.