Validity Procedures in Consumer Research: a Perspective

Citation:

Albert V. Bruno (1975) ,"Validity Procedures in Consumer Research: a Perspective", in NA - Advances in Consumer Research Volume 02, eds. Mary Jane Schlinger, Ann Abor, MI : Association for Consumer Research, Pages: 757-760.

Advances in Consumer Research Volume 2, 1975      Pages 757-760

VALIDITY PROCEDURES IN CONSUMER RESEARCH: A PERSPECTIVE

Albert V. Bruno, University of Santa Clara

[Albert V. Bruno is an Associate Professor at the Graduate School of Business, University of Santa Clara.]

This session was concerned with validity procedures in consumer research. Kerlinger has described the subject as "complex, controversial, and peculiarly important^'(Kerlinger, 1964). Our session has identified additional descriptions such as elusive, expedient, and expensive.

Papers concerned directly or indirectly with validity issues are commonplace. Virtually every researcher purports to be interested in various aspects of validity.

Much of the research reported in journals relevant to our interests contains treatment of validity considerations which is vague and poorly defined, ill conceived and often constrained by the data on which the constructs are built, and not clearly identified as being cost justifiable -- validity procedures are virtually incidental to the research process.

The papers in this session represent the viewpoints and interests of a diverse set of individuals. Pessemier's paper "Incorporating Tests of Data and Model Validity in Commercial and Academic Research" discusses the need for early consideration of validity issues, incorporating validity tests in the research effort and reporting the validity of the measures and models along with other research results. Juster's paper describes "Validity Procedures at the Survey Center" as a guide for researchers. Lehmann's paper, "Validity and Goodness of Fit in Data Analysis," describes the relationship between the usefulness of a hypothesized relationship between two or more variables and the goodness of fit which results from an empirical investigation of the hypothesis. Light in "Validity -- What's It Worth" investigates the value of validate from an ad aqency's Point of view.

The aforementioned textual material and comments by the discussant, Rohloff, focuses on several important questions for researchers:

1) Validity and validity procedures. The content of the papers, the comments of the discussants, and the observation of several members of the audience demonstrated that we are not in agreement as to the nature of validity or its importance in the scientific research process. If consumer research is to rise to its appropriate position in the hierarchy of scientific inquiry, we must develop some consensus as to what validity is and why it is important.

2) Managerial vs. scientific validity. A number of researchers appear prepared to distinguish between managerial significance and significance based on statistically significant differences. If the function of our research is to aid management in making better decisions, can these distinctions exist? If there is some legitimacy in distinguishing between the two, under what circumstances and conditions can the case be made?

3) Cost/benefits of validity. Much of the treatment of validity and validity procedures has dealt with validity independent of its cost. Are all validity activities cost justifiable? Under what circumstances can some validity activities be eliminated?

RECOMMENDATIONS

The resolution of these and related issues can be achieved only if a consistent, objective methodology becomes commonplace. Toward that end, it would be useful to develop consistency in reporting of research results. The following recommendations are made:

1) Identification of "criterion" variables. Researchers-should clearly identify the nature of the "ultimate'' criterion variable(s) used for evaluating the validity of the measure under study. Conclusions and implications of research studies can not be derived unless some prior evaluation, and hopefully, as Heeler and Ray have recommended (Heeler and Ray, 1972), validation of the criterion variable has been performed. An example may illustrate this point. The Likert-scale approach to identifying the brand preference of consumers may have a conceptual foundation and substantial empirical research support; consequently, it may be well accepted as an "ultimate" criterion variable. Subsequent research which utilizes this form of brand preference (e.g., attitude/ behavioral studies) will thus be on some common footing among groups of researchers. However, if a researcher reports results which utilize a new, unsubstantiated form of brand preference (e.g., constant sum score preference) in an attitude/behavioral research project, it may be difficult to draw useful conclusions from the researcher's efforts. Some determination of the appropriateness or legitimacy of the criterion variable should be included as an integral portion of research studies, either by linkage to previous research or, at least, by intellectual exposition.

2) Distribution of responses. The mean,standard deviation, and especially, the distribution of responses of both the criterion variables and the measures of interest should be reported or at least discussed by the researchers. Researchers should take precautions to insure that no "abnormal" distributions are "creating" results. For example, if there is no spread over the response categories (i.e., low variance for the question), then high correlations can be the result of consistent responses from a small number of respondents. As Ferber has suggested, "the existence of a relationship between the attributes is in doubt until the presence of a possible fluke is investigated and further studies are made (Ferber, 1949). Although rather pedestrian, a scatter diagram would provide extremely useful information.

3) Complementary analyses. In addition to the use of product moment correlations, it is strongly recommended that other methods of validation be employed. Cross classification analysis can provide additional focus on the nature of the relationships identified by the correlations. Moreover, since predictive validity is often the ultimate goal of marketing research, the use of "hit or miss" tables and "confusion matrices" would provide strong tests of the predictive validity of the measures under consideration.

4) Application of validation procedures. There are different types of validity which have been described adequately elsewhere by a number of authors (for example, see Bruno and Pessemier, 1972; Ferber, 1949). A hierarchical ordering of validity procedures in marketing research should descend from predictive validity (at least a long as "goodness of fit" serves as the conventional measure of effectiveness), Marketing researchers who are concerned with prediction (as opposed to being concerned with theory development, for example) should be concerned with identifying measures with high predictive validity rather than with identifying measures that are reliable, have content and construct validity, but have low predictive validity. If a researcher were interested in identifying brand preference measures that predict subsequent choice of brands, then the ultimate "test" of the usefulness of the brand preference measures is their ability to correctly predict brand choices. Other types of validity are of secondary importance.

Too many marketing researchers have been guilty of utilizing less desirable forms of validity because of cost or time limitations or because their data were not in the appropriate form. Occasionally, researchers even appear to have a misconception of what is being researched. The author of a recent article entitled "Linear Attitude Models: A Study of Predictive Ability" purports to examine the question of predictive validity by correlating actual and predicted preference (Churchill, 1972).

5) Sensitivity to changes in assUmPtions. The validation procedures may be dependent to a degree on explicit or implicit assumptions by the researcher. Efforts should be made to isolate the degree of dependency and determine if it is acceptable. Perhaps a study of the dependency through some form of sensitivity analysis would be appropriate. Recent investigations on the question of the optimal number of response categories of Likert-type scales is an example of this approach (Green and Rao, 1970; Lehmann and Hulbert, 1972). Certainly, the degree of predictive validity (if correlation coefficients are utilized as criteria) of some measures is partially dependent upon the number of response categories.

6) Need for synthesis and consolidation. There is a substantial need for synthesis, consolidation, and classification as a prerequisite to the development of scientifically satisfactory and managerially useful attitudinal/behavioral methodology. The time has come for editors and reviewers to insist on more systematic. scientific. and objective investigation of measure validation.

REFERENCES

Bruno, A. V., & Pessemier, E. A. An empirical investigation of the concurrent validity of selected attitude and activity measures. In M. Venkatesan (Ed.), Proceedings of the 3rd Annual Conference of the Association for Consumer Research, University of Chicago. November. 1972.

Churchill, G. A. Linear attitude models: A study of predictive ability, Journal of Marketing Research, 1972, 9, 423-426.

Ferber, R. Statistical techniques in market research. New York: McGraw-Hill Book Company. Inc.. 1949.

Green, P. E., & Rao, V. D. Rating scales and information recovery--how many scales and response categories to use? Journal of Marketing, 1970, 34(1), 33-39.

Heeler, R. M., & Ray, M. L. Measure validation in marketing. Journal of Marketing Research, 1972, 9, 361-370.

Kerlinger, F. N. Foundations of behavioral research, New York Holt, Rinehart. and Winston. 1964.

Lehmann, D. R., & Hulbert, J. Are three-point scales always good enough? Journal of Marketing Research, 1972, 9, 444-446.

----------------------------------------