Comments on Scale Development and Testing

Citation:

Thomas J. Page (1994) ,"Comments on Scale Development and Testing", in NA - Advances in Consumer Research Volume 21, eds. Chris T. Allen and Deborah Roedder John, Provo, UT : Association for Consumer Research, Pages: 547.

Advances in Consumer Research Volume 21, 1994      Page 547

COMMENTS ON SCALE DEVELOPMENT AND TESTING

Thomas J. Page, Jr., Michigan State University

INTRODUCTION

The three papers all deal with some aspect of scale development or testing. The paper by Feltham attempts to develop a new scale for classifying stimuli into consistent categories. The papers by Malafarina & Jass and Hunt et al examine the validity of existing scales. In general, the papers represent a good combination of the different steps that need to be carried out in developing and using scales in research projects. The comments presented here will attempt to summarize the major points of the session and also make some suggestions for future research concerning the specific papers.

MAJOR POINTS OF THE SESSION

There are two points that readers of the three papers should come away with after reading them. The first is Feltham's point concerning generalizable conclusions. Her point is that we cannot make any generalizable conclusions about a theory unless we have some degree of certainty about the methods used to test it. For example, one study may call a construct humor, another may call it warmth, and still another may call it emotion, and yet they may all be talking about the same stimulus. This lack of consistency severely hinders the discipline's ability to make generalizations about a particular theory.

In addition to restricting the generalizability of results, this inconsistency is often evoked to explain unexpected results. Frequently, researchers testing a particular theory may arrive at results that are not consistent with either the theory or previous research in the area. One popular method of explaining the discrepant results is to claim that their stimuli were somehow slightly different from those used in other studies. While this may indeed be the case, it would certainly be desirable to have a method that can at least reduce some of this ambiguity. Thus, the attempt by Feltham to develop a method of dealing with this problem is certainly a step in the right direction.

The second major point of the session comes from the two papers that examine the validity of existing scales. Too often, researchers just blindly accept the validity of an existing scale without giving any thought as to how potential problems with the scale may affect their research. Obviously, problems with the dimensionality of a scale as uncovered by Malafarina and Jass, or problems with discriminant validity as found by Hunt et al, could seriously affect the results of a particular study. Just because there are a lot of published studies using a particular scale does not preclude the possibility that it will occasionally demonstrate undesirable psychometric properties. Thus, authors should, whenever possible, examine such characteristics as discriminant and convergent validity, and dimensionality of the scales they are using in their research. This does not mean that wholesale revamping of a scale should be carried out every time it used, but if a particular scale seems to have undesirable properties in a particular case, the researchers need to take into account how they may affect their results, and perhaps consider using a different scale.

COMMENTS ON INDIVIDUAL PAPERS

The paper by Feltham represents a good first step in developing a method of classifying stimuli so that generalizations can be made across studies. As the author states, though, there is still a lot of work to be done. In particular, the convergent and discriminant validity of the three categories needs to be established. Also, there is a need for some sort of independent verification that the stimuli being classified into a certain category are actually logical, emotional, or posses a particular source characteristic. Once this has been accomplished, using larger sample sizes, the classification scheme will be very useful.

Malafarina & Jass present a very good literature review of work in the area of consumer conservation. They demonstrate that some of the sub-scales of the Conservation-Oriented Consumption Scale have low coefficient alphas. They also demonstrate that the scale fails to achieve convergent validity with the Concern for Supply and Avowal of Social Norms scales. This may indeed represent a problem with the scale, but it may also be due to other causes. For example, the Avowal of Social Norms scale and the Concern for Supply Scale may not be highly correlated with each other. If this is the case, they would not demonstrate convergent validity with the Conservation-Oriented Consumption Scale. Also, if different scales were used to assess convergent validity, the Conservation-Oriented Consumption Scale might have achieved an accpetable level. Nevertheless, this paper does point out the need to critically examine a scale before using it in one's research.

Hunt et al are concerned about the transparency of the Need For Cognition Scale. In particular, the second study provided some very provocative results. Specifically, they showed that a control group's responses were not significantly different from a group that was told to deliberately attempt to display their intelligence in responding to the Need for Cognition Scale. This is taken as evidence that the scale may indeed be susceptible to social desirability effects. This is a very intriguing result. However, before condemning the Need For Cognition Scale, some more work needs to be done. The authors need to show that their manipulation was indeed manipulating social desirability and not just telling people to respond differently to the scale. This is particularly important in this case since telling someone to downplay or display their intelligence in responding to the scale is a bit transparent in itself. However, if a series of studies along these lines demonstrate consistent findings, then the Need for Cognition Scale may need to be reexamined in terms of its validity.

----------------------------------------