Stability of Self-Designated Opinion Leadership

Citation:

James H. Myers and Thomas S. Robertson (1974) ,"Stability of Self-Designated Opinion Leadership", in NA - Advances in Consumer Research Volume 01, eds. Scott Ward and Peter Wright, Ann Abor, MI : Association for Consumer Research, Pages: 417-426.

Advances in Consumer Research Volume 1, 1974    Pages 417-426

STABILITY OF SELF-DESIGNATED OPINION LEADERSHIP

James H. Myers, University of Southern California

Thomas S. Robertson, University of Pennsylvania

[The authors are indebted to Haug Associates, Inc., for financial assistance and for the use of the Haug Trend-Setters Panel. Appreciation is also extended to Professors Lee Cooper and Harold Kassarjian of UCLA and Scott Ward of Harvard for critical reviews of earlier drafts of this paper.]

[James H. Myers is DeBell Professor of Business Administration of the University of Southern California. Thomas S. Robertson is Associate Professor of Marketing at the University of Pennsylvania.]

Self-designation is frequently used as a measure of opinion leadership, despite meager evidence as to the reliability of this approach. This paper presents evidence as to the stability over time of housewives' self-designated opinion leadership for a variety of topic areas.

The quest for knowledge within the marketing discipline, and particularly within the consumer behavior area, has been rather overly concerned with generating data -- frequently to the detriment of sound measuring instruments for obtaining these data. This is very apparent in research on the phenomenon of opinion leadership.

Opinion leadership is usually viewed as the extent to which an individual provides information and advice to others. Unfortunately, this concept has been operationally defined with minimum consistency. There are presently three principal approaches to measuring opinion leadership:

1. Self-designation. The respondent gauges the extent to which he perceives himself to be an opinion leader, usually on the basis of a limited number of items forming a self-designation scale. Katz and Lazarsfeld (1955) used a two-item scale as did Montgomery and Silk (1969); Rogers and Cartano (1962) have used a six-item scale, which was then modified into a seven-item scale by King and Summers (1970); and Silk (1966) and Myers and Robertson (1972) have used a single-item scale.

2. Sociometric technique. Group members designate their information sources for particular topic areas. The classic use of this measure is that of Coleman, Katz, and Menzel (1366); it has been used in the marketing field by Myers (1966), Robertson (1968), and Stafford (1966). A modification of this method is to use "key informants" within a particular social system (Rogers and Cartano, 1962).

3. Objective technique. This involves giving products or opinions to certain individuals on a controlled basis and tracing the resulting influence within relevant interpersonal networks. A modified version of this technique is that of Myers (1966).

Our focus in this article is only upon the reliability of self-designating measures of opinion leadership, the most common form of measurement found in the marketing literature. How reliable are such measures, and can we continue their use without imposing some conditions and constraints?

RELEVANT RELIABILITY CONCEPTS

Determining the reliability of a measuring instrument is not an easy matter. Questions used to measure any construct in marketing can be framed in many ways; therefore, some process is necessary to establish the adequacy of a particular approach.

One authority (Ghiselli, 1964) has distinguished three possible interpretations of the term reliability as it applies to a particular measuring instrument:

1. Consistency--the degree to which results from one portion of the measuring instrument correlate with results from the remainder of the measuring instrument at the same point in time. For example, if an investigator establishes the opinion leadership of a respondent using 10 questions all aimed at slightly different aspects of this concept, results from any 5 of these questions should correlate highly with results from the remaining 5 questions if we are to conclude that all questions are measuring approximately the same thing -- opinion leadership. This is also known as internal consistency reliability.

2. Inter-form--the degree to which responses from one form of a measuring instrument correlate with responses from a "parallel" form, when each form consists of a completely random selection of questions drawn from the same pool of similar questions. This is simply an expanded version of internal consistency reliability.

3. Stability--the degree to which the respondent gives the same set of responses to the same set of questions at two different times. Studies have shown clearly that correlations of responses decrease quite uniformly as the time interval increases, at least for certain types of items (see Cureton, 1965). T',re stability coefficient over a given time interval is the inter-form correlation over that time interval divided by the consistency coefficient.

This paper is concerned only with the stability reliability of one particular single-item self-designated opinion leadership measure.

Opinion Leadership Reliabilities

Reliability coefficients for self-designated opinion leadership measurement are almost totally absent in marketing studies--with the exception of Kirchner's (1969) doctoral thesis. To the extent that these issues are raised by various investigators, reference is given to Rogers and Cartano (1962) as the justification for the use of self-designation measures. A summary of the results cited by Rogers and Cartano in regard to reliability and validity is provided in Table 1 (from Kirchner, 1969).

Kirchner, however, in re-testing the Rogers-Cartano six-item scale, found the index to be less reliable than previously reported. His own sample was composed of 135 undergraduate and post-graduate students. Test-retest (stability) reliability for opinion leadership scales was .79; split-half (internal consistency) reliability was only .59, and the Guttman coefficient of reproducibility was .881 which borders on the minimum acceptable level of ,90 set by Guttman (1950).

TABLE 1

RELIABILITY OF EXISTING OPINION LEADERSHIP SCALES

Importance of Reliability

It is important to note that if a measuring instrument is not reliable in its measurement of some dependent variable, then assessments of its validity become almost irrelevant. No other variable can correlate with a dependent variable higher than the square root of the reliability of the instrument used to measure that dependent variable (see Cureton, 1965). Thus, reliability of opinion leadership measurement is critical, since opinion leadership frequently is used as the dependent variable in correlation and other forms of analysis.

Suppose, for example, that opinion leadership for a given topic correlates only .50 on a test-retest basis for responses to the exact same question for the same person at two points in time. This means that an independent variable (for example, innovativeness, cosmopolitanism, or social activity) cannot possibly show a correlation of more than approximately .70 with the opinion leadership (dependent) variable. Thus, an "obtained" correlation of say .60 between innovativeness and opinion leadership for a given product category would be close to the maximum possible and yet would appear to be only a "moderate"indication of association between the variables,, using ordinary standards.

To generalize, anything less than perfect reliability in the self-designation of opinion leadership (and perfect reliability is almost never obtained in measuring constructs of this type) should cause us to reevaluate upward all correlations of other variables with opinion leadership. Of course, if the reliability of this construct is found to be too low, the proper approach would not be to make statistical allowance for this (i.e., revise upward obtained correlations), but rather to question seriously the merits of any research which is based upon self-designation as the sole means of evaluating opinion leadership.

To compound the problem there is not only some amount of unreliability in self-designations of opinion leadership, there is probably also appreciable unreliability in the measurement of nearly every other variable which investigators have related to opinion leadership (with the exception of some demographics). Thus, self-designations of innovativeness, social integration, personality, venturesomeness, cosmopolitanism, and the like all contain their own share of "error variance" (i.e., unreliability). When a researcher correlates one unreliable measure with another, the maximum possible correlation he can obtain s equal to the square root of the product of the reliability coefficients of the two measures (see Ghiselli, 1964, p. 271).

Thus, if the reliability of self-designated opinion leadership is found to be .75 and that of innovativeness is found to be .70, these two measures could not possibly correlate with one another more than .73. However, if an investigator were to obtain a correlation of, say, .60 between measurements of opinion leadership and innovativeness he would ordinarily conclude that the relationship between these constructs was only moderate, whereas, in reality it is near the maximum possible given the reliabilities of the respective measuring instruments.

An investigator wants a construct that has sufficiently high stability over time so that he is shooting at a stable rather than a moving target. Instability consists of two components: 1) low reliability of the measuring instrument over time, and 2) real changes in the phenomenon under study. Both will contribute toward lowered stability reliability as measured by re-testing at periodic intervals.

Some of the factors that would seem likely to affect the amount of real change in any construct under studs include the following:

1. The more "permanent" the phenomenon, the greater the expected test-retest reliability. For example: we should expect greater reliability of opinion leadership for art than for clothing fashions.

2. The more intense and complex the underlying value system, the greater the expected test-retest reliability. For example: we should expect greater reliability of opinion leadership for religion than for automobiles.

3. The greater the individual's involvement with the phenomenon under study, the greater the expected test-retest reliability. For example: we should generally not expect reliability in opinion leadership measures for a product such as beer among a sample population of women; most women are not particularly involved in this product area. Maccoby et al (1959) found that attitude change toward toilet training was unstable over time for women without children; their lack of involvement in the interest area suggests that the new attitude is not likely to be reinforced and. therefore. is unstable.

4. The more frequent the rate of product repurchase (or activation of the interest area), the greater the expected test-retest reliability. For example: we should expect greater reliability of opinion leadership for packaged goods than for appliances.

5. The more rigid the individual's life style, the greater the expected test-retest reliability. In this regard we should expect greater reliability of opinion leadership among individuals who are well settled into a life stage and life style than for individuals who are entering new life stages (such as marriage) or those who are socially mobile.

The above conditions are, of course, interdependent and while considerations, they do not constitute a predictive model.

The position could be taken that fairly low stability reliability in opinion leadership measures should be expected since the phenomenon is so situation- al. Previous research indicates the lack of a clear cut opinion leader profile) and opinion leadership is widely held among members of the entire population - although in varying amounts.

RESEARCH DESIGN

Our research objective was to estimate the"stability" reliability of a single-item self-designation instrument as a measure of opinion leadership by means of test-retest correlations over time. That is, respondents were asked to estimate their own degree of influence over others for each of twelve topic areas at one point in time; then they were asked the same questions about the same topic -areas nine months later. The twelve topic areas were those found to be of greatest interest to housewives and included home entertainment, cosmetics and personal care, politics, clothing and fashions, and other topics as shown in Table 2

The initial phase of this study was reported in an earlier paper (Myers & Robertson, 1972). At that time, questionnaires were sent to 400 housewives throughout the Los Angeles metropolitan area, selected by a stratified quota-type sample. All households were members of the Trend-Setters Panel of Haug Associates, Inc. Panel members were selected as to be approximately proportional to the total area population in terms of age, income, and geographical dispersion. Of the 400 questionnaires sent, 246 were returned in a form suitable for processing--a return of over 60%.

Nine months later, questionnaires were sent to the same 246 respondents who returned the original questionnaires. Replies were received from 239 of these. This high return would seem to be due to at least two factors:

1. Respondents were "pre-selected" in a sense in terms of their willingness to answer questions of the type covered in this study.

2. The second questionnaire was much briefer than the first -- one page of twelve questions versus five PaRes of ninetY-two questions.

The self-designating opinion leadership measure was a single-item, three-point scale for each of the twelve interest areas:

TABLE

This measure was considered justified on were interval-scaled using the Thurstone statements spanned a reasonable range of tabulations showed good dispersion), and to understand by mail and would make the to encourage a high rate of return.

the basis that: 1) scale statements equal interval technique (Guilford, 1965), possible responses (later marginal 3) the shortened scale would be easy to understand by mail and would make the entire task seem less formidable so as to encourage a high rate of return.

Since the identical questions on opinion leadership were measured in the two waves, comparisons were easily possible. Responses from the first administration were correlated with responses from the same respondent on the same topic at the second administration. In this manner it was possible to determine which topic areas showed the greatest stability of self-designated opinion leadership.

It should be noted, of course, that nine months is quite a long span for a reliability test and imposes a more severe test of self-designation consistency than would a time period of only a month or two. Thus, any positive findings from this study might be regarded as particularly encouraging.

RESULTS

Test-retest correlations of self-designated opinion leadership for each of the twelve topic areas covered are shown in Column 1 of Table 2. Rather low magnitudes are noted for these product moment correlations: most range between .30 and .60.

The low correlations are partially due to the small number of rating categories (three) provided in the self-designating scale. Five, seven, nine, or even more intervals, might have been used, and by doing this the correlations would have been raised to some extent. This is because correlations from data that have been grouped into a small number of intervals will be somewhat smaller than those using the same scales but with a larger number of intervals. Coarse grouping yields overestimates of standard deviations, (which constitute the denominator of the product-moment correlation formula) so that obtained correlations are reduced to some extent. A correction is available to indicate what correlations would result from using a scale of many intervals (see Ghiselli, 1964, p. 246), and the revised correlations are shown in Column 2 of Table 2.

With these adjustments, test-retest correlations (i.e., stability reliabilities) range from a high of .91 (politics) to a low of .42 (family medical care, automobiles). These levels of reliability suggest that self-designated opinion-leadership scales may have more usefulness for some topic areas than for others. Future research should always present evidence as to the reliability of a particular opinion leadership scale for a particular topic area under investigation, since this varies so widely from topic to topic (at least it did for Los Angeles housewives).

DISCUSSION

Using the above findings, let us examine the temporal reliabilities that we might have expected for these interest areas by reference to the five conditions posited earlier in the paper: (1) permanence of the phenomenon (opinion leadership), (2) intensity and complexity of the individual's value system in regard to the phenomenon, (3) the individual's involvement with the phenomenon, (4) rate of product repurchase or interest activation, and (5) rigidity of the individual's life style.

It might be helpful to arbitrarily dichotomize results in Table 2 into two categories, high and low temporal reliability. as follows:

Higher Stability Reliability

Entertaining at home

Politics

Children's behavior and upkeep

Women's clothing and fashion

Cosmetics and personal care

Cooking, recipes and new food

Low Stability Reliability

Household furnishings

Household appliances

Home cleaning and upkeep

Recreation and travel

Family medical care

Automobiles

Based on hindsight, which is indeed an inadequate form of analysis, these findings do seem to make some sense in line with the conditions posited a priori to affect reliability. The interest areas scoring higher on test-retest reliability tend to be somewhat more static phenomena; or to be more related to complex and intensely held value systems; or to be more exclusively in the domain of women and therefore of higher involvement); or to be higher on repurchase rate or interest activation.

TABLE 2

CORRELATIONS BETWEEN FIRST AND SECOND WAVE SELF-DESIGNATIONS OF OPINION LEADERSHIP

It is obvious that these findings are to be regarded as exploratory only, since definitive results cannot be expected from a single study. Further work will be needed for validation and clarification, hopefully using different forms of measuring instruments in addition to the single item self-designation used in the present study. Hopefully, this study has provided a conceptual framework for future work, in addition to presenting evidence which suggests that researchers investigating opinion leadership (as well as other constructs in consumer behavior) must be mindful of both the adequacy of the measuring instruments they use and the stability of the phenomenon under investigation.

[A distinction must be made between the terms "index" and "co-efficient" of reliability. The latter is a ratio of the variance of true scores to raw (obtained) scores and represents the proportion of the variance of the obtained scores that is accounted for by the true scores. It is estimated by correlating one form of a measuring instrument against a similar "parallel" form (consisting of the same types of items), or by correlating one portion of a measuring instrument against another portion (e.g., odd-numbered items against even-numbered items). The index of reliability is the square root of the coefficient of reliability.]

REFERENCES

Coleman, James S., Elihu Katz, and Herbert Menzel. Medical-Innovation: A Diffusion Study. Indianapolis: Bobbs-Merrill, 1966.

Cureton, Edward E. "Reliability and validity: basic assumptions and experimental designs," Educational and Psychological Measurement, 25 (1965) 327-46.

Ghiselli, Edwin E. Theory of Psychological Measurement. New York: Mcgraw-Hill, 1964.

Guilford, J. P. Fundamental Statistics in Psychology and Education. New York: McGraw-Hill, 1965.

Guttman, L. "The basis for scalogram analysis," in S. A. Stouffer, et. al., Measurement and Prediction. Princeton, N. J.: Princeton University Press, 1950, 46-90.

Katz, Elihu and Oayk F. Lazarsfeld, Personal Influence. Glencoe: The Free Press, 1955.

King, Charles W. and John 0. Summers. "Overlap of opinion leadership across consumer product categories," Journal of Marketing Research, 7 (February 1970), 43-50.

Kirchner, Don F. Personal Influence, Purchasing Behavior and Ordinal Position. An unpublished doctoral dissertation, UCLA, 1969.

Maccoby N., et. al. "'Critical periods' in seeking and accepting information," Paper read at the American Psychological Association Convention, Cincinnati. Ohio. September. 1959.

Montgomery, David B. and Alvin J. Silk, "Patterns of overlap in opinion leadership and interest for selected categories or purchasing activity," Proceedings, Marketing Involvement in Society and the Economy, American Marketing Association, Fall, 1969, 377-86.

Myers, James H. and Thomas S. Robertson. "Dimensions of opinion leadership," Journal of Marketing Research, 9, (February, 1972), 41-46.

Myers, John G. "Patterns of interpersonal influence in the adoption of new products," Proceedings, Science, Technology, and Marketing. American Marketing Association, Fall, 1966.

Robertson, Thomas S. "The effect of the informal group upon member innovative behavior," Proceedings, Marketing and the New Science of Planning, American Marketing Association, Fall 1968, 334-40.

Rogers, Everett M. and David G. Cartano. "Methods of measuring opinion leadership," Public Opinion Quarterly, 26 (Fall 1962), 435-41.

Silk, Alvin J. "Overlap among self-designated opinion leaders: a study of selected dental products and services," Journal of Marketing Research, 3 (August 1966), 255-9.

----------------------------------------