An Evaluation of Students As Surrogates in Marketing Studies

ABSTRACT - Much has been written about the use of students as surrogates for other populations in marketing studies. Overall, the findings of previous studies are inconsistent and contradictory. This paper provides a review of the relevant literature, introduces a typology for classifying this literature, draws conclusions regarding the use of student surrogates, and suggests directions for future research.

Citation:

Charles W. Lamb, Jr. and Donald E. Stem, Jr. (1980) ,"An Evaluation of Students As Surrogates in Marketing Studies", in NA - Advances in Consumer Research Volume 07, eds. Jerry C. Olson, Ann Abor, MI : Association for Consumer Research, Pages: 796-799.

Advances in Consumer Research Volume 7, 1980     Pages 796-799

AN EVALUATION OF STUDENTS AS SURROGATES IN MARKETING STUDIES

Charles W. Lamb, Jr., Texas A&M University

Donald E. Stem, Jr., Washington State University

ABSTRACT -

Much has been written about the use of students as surrogates for other populations in marketing studies. Overall, the findings of previous studies are inconsistent and contradictory. This paper provides a review of the relevant literature, introduces a typology for classifying this literature, draws conclusions regarding the use of student surrogates, and suggests directions for future research.

STUDENTS AS SURROGATES IN MARKETING STUDIES

The widespread use of students as subjects in marketing studies is well documented. Ferber (1977), in a Journal of Consumer Research editorial, noted that the use (or abuse) of convenience samples in consumer-related research is growing by leaps and bounds. Enis, Cox, and Stafford (1972) found that over half of the consumer behavior experiments reported in the first 30 issues of the Journal of Marketing Research involved student subjects. Cunningham, Anderson, and Murphy (1975) found that between 20 and 33 percent of the articles in business journals reporting consumer research findings employed student subjects. Of these, over 75 percent used convenience samples. These findings indicate that the use of students as surrogates is a significant issue. Furthermore, as increasing numbers of colleges and universities use publications as a criteria for measuring faculty performance, the use of students as surrogates for other populations in marketing studies is likely to continue and perhaps increase. Faced with increasing requirements for publications, marketing academicians will be tempted to substitute the expediency of convenience sampling for more rigorous methodologies.

It is widely recognized that improper sample selection often produces invalid research results. This may lead to incorrect inferences being drawn regarding the attitudes, beliefs, or behaviors of particular populations, and may ultimately result in the adoption of incorrect marketing strategies and/or practices. The improper use of student samples by academic researchers net only takes up publication space in our journals, but may also have a negative impact upon marketing practices and on society in general.

Should academic researchers therefore in all cases forego the advantages of convenience, speed, and compliance inherent in the use of student samples? Are there instances where student samples are appropriate? The purpose of this paper is to explore these questions.

Several studies examining the validity of using students as surrogates for other populations have appeared in the literature. The findings of these studies, summarized in Table 1, are mixed, inconsistent, and inconclusive.

REVIEW OF PAST RESEARCH

Hovland (1959), in an effort to reconcile apparent differences in results produced by experimental (using mostly students) versus survey (using mostly non-students) techniques, concluded that there were no contradictions produced by the differences between methodologies. He attributed the apparent divergence in results to personal differences between respondent groups, and to differences in situations in which the experimental and survey studies were conducted. Another study, reported by Clevenger, Lazier, and Clark (1965), found substantial similarities in the semantic differential factor patterns of undergraduate students and housewives regarding their images of two well-known companies. Sheth (1970), in a study of post-decision dissonance reduction found a remarkable degree of similarity between students and housewives in their post-decision dissonance reduction behavior. In another study, no significant differences were found between students, housewives, and experts' evaluations of carpet quality (Enis and Stafford 1969). The researchers noted, however, that students tended to give higher product ratings than housewives, and their answers had smaller variances.

TABLE 1

SUMMARY OF STUDIES REGARDING STUDENTS AS SURROGATES

Other studies contradict these findings. In a study regarding susceptibility to reference group influence, Park and Lessig (1977) found significant differences between students and housewives. Alpert's (1967) study of the attitudes of businessmen, military personnel, and students regarding a hypothetical business situation in which a manager abruptly discharged his subordinate after several months of seemingly satisfactory performance also rejected the proposition that students can be used as surrogates for businessmen. Copeland, Francia, and Strawser (1973) found that there were significant differences between the attitudes of students and other populations regarding selected financial reporting practices. Another study revealed significant differences between students and household consumers along selected socio-psychological attributes, in sources utilized in acquiring information about selected commonly purchased products, and in decision factors influencing the purchase of selected commonly purchased product classes (Cunningham, Anderson, and Murphy 1975). In a study regarding perceptions of what business is currently doing in areas of social responsibility, where business should be doing more, areas of concern to the consumer, and areas where business is trying to improve, Tankersley (1975) found significant differences between the perceptions of businessmen and business students. Vinson and Lundstrom's (1976) research revealed that "undergraduate students were not effective surrogates for their parents in evaluations of automobile attributes, in measuring the appeal of various consumer products, and more importantly, in their basic value orientations."

Additional research comparing the responses of students to housewives (Enis, Cox and Stafford 1972; Shuptrine 1975) and businessmen (Khera and Benson 1970; Roering, Schooler and Benson 1976) produced mixed or inconclusive results. Experiments regarding the influence of race of model upon consumers perceptions of advertisements and bias related to country of origin in glassware preference revealed that housewives generally rated the ads more favorable than did students. However, only 5 out of 11 comparisons were significant (Enis, Cox and Stafford 1972). Shuptrine (1975) compared the perceptions of female graduate students to housewives regarding 61 product innovations. Findings revealed a significant difference between the ratings of the two groups on 17 out of 61 new products. Students generally rated products lower than housewives and their answers had smaller variances. Khera and Benson (1970) exposed students, professional engineers, and purchasing agents to a short sales film and then asked them to express their immediate reactions on a questionnaire. There were no significant differences in the students' and the businessmen's' ratings of the sales presentation or the communicator. However, the two groups' decisions regarding whether or not they would recommend or adopt the product differed significantly. A study comparing the responses of housewives, businessmen, and students to questions regarding the extent of unethical marketing practices, their evaluation of specific marketing practices, and the responsibility for corrective action also produced mixed results (Roering, Schooler and Morgan 1976). Although the evaluations of businessmen and students were congruent, the evaluations of housewives and students were incongruent.

SUGGESTED REASONS FOR INCONSISTENCIES

The mixed and apparently contradictory findings of previous studies regarding the use of students as surrogates for other populations raise several questions. Clearly, the responses of students are different than the responses of other populations under certain circumstances. Several researchers have offered tentative explanations regarding this matter. Hovland (1959), for example, pointed out that there are personal differences between students and non-students. Students are more motivated and involved, and often in a subordinate-superior relationship with the researcher. Therefore, students are more apt to respond conscientiously to the requests of the researcher. Vinson and Lundstrom (1976) also point out that student volunteers are not passive responders and may be influenced by experimenter effects. Park and Lessig (1977) contend that there are differences in needs and motivations of students compared to other populations. These differences are due to differences in age distributions, which often accompany differences in the amount of learning, accommodation of learning, and risk handling. They also point out that the responses of students may differ from the responses of other populations because of differences in social contacts, the intensity of peer pressure on choice behavior, the visibility of social approval of the group, the rigidity of group structure, rules and regulations, the visibility of one's behavior, and the availability of frequent social contacts. Other factors that may, at least partially, account for differences between the responses of students and other populations include education, familiarity with taking tests, the campus environment and experience in situational decision making.

AN ALTERNATIVE EXPLANATION

Previous research regarding the use of students as surrogates for other populations in marketing studies can generally be classified into one or both of two categories based upon the kinds of behaviors or variables examined. Some of the previous research has focused on the measurement of subjects' mental states. These studies have examined variables such as attitudes, beliefs, interests, and opinions, and are descriptive. Other research has focused on the measurement of what might be called processes. These studies have examined variables such as psychological decision processes, dissonance reduction processes, and information handling processes, and are normative. Table 2 summarizes the research previously cited based upon whether the dependent variables examined were states, processes, or both.

TABLE 2

CLASSIFICATION OF PREVIOUS STUDIES BASED UPON KINDS OF DEPENDENT VARIABLES STUDIED

States

As Table 2 shows, 5 of the 12 studies that compared the psychological states of students with the states of other populations produced significant differences, and 4 of the 12 studies produced mixed results. It is not surprising that a number of research studies have found student subjects to hold attitudes, opinions, and interests different from those held by other populations. This finding should surprise no one and would be interesting only if it were not obtained. One might argue that virtually any two populations one could select should produce differences on variables such as attitudes, opinions, and interests. Psychological states are a function, in part, of individuals' previous experiences, their history of reinforcement and their current environment.

Processes

Although students are not generally good surrogates for other populations in studies dealing with psychological states, it may be that student subjects make good surrogates for members of other populations when psychological processes are of focal interest. One might argue than any human beings, no matter what specific characteristics they possess, engage in similar processes. In other words, it might be argued that the way human beings make decisions or process information is common. Therefore, students are appropriate subjects for research dealing with psychological processes, and findings based upon studies using convenience samples of students can be generalized to other populations. Unfortunately, the findings shown in Table 2 do not support this proposition. Two of the 3 studies that examined psychological processes found significant differences between students and other populations. These differences could possibly be attributed to age differences and/or differences in cognitive abilities.

ARE STUDENT SUBJECTS EVER APPROPRIATE?

Under what circumstances can students justifiably be used in marketing studies? Shuptrine (1975), and Khera and Benson (1970) believe that students respond similar to other populations when their knowledge, experience, and frame of reference are comparable to that of the population being studied. Enis, Cox, and Stafford (1972) have stated that students may appropriately be used in consumer behavior studies when internal validity is more important than external validity. Shuptrine (1975) favors using student subjects only for pretesting, interviewer training, etc. Ferber (1977) has stated that student subjects are appropriate for exploratory purposes, for illustrating the application of a technique, or for clinical purposes.

As has been previously noted, some studies focus on measuring the specific states or traits of populations. In this type of research, external validity is of major importance and it is critical that the sample used in the study be from the same population as that which the results are intended to be generalized. In other words, students surrogates are normally not appropriate for measuring the psychological states of other populations. Table 2 supports this conclusion.

On the other hand, some research, perhaps most academic research, is primarily concerned with examining psychological processes. In such research, internal validity is a primary consideration and external validity is of much less importance. Therefore, it might be argued that processes are processes. That is, student subjects engage in similar kinds of processes as do housewife subjects or any kind of adult human being. Although this proposition is intuitively appealing, the findings shown in Table 2 do not support it. It appears that different populations may employ psychological processes differently. Studies concerning processes have produced significantly different results as frequently as studies dealing with states. Therefore, at this point it must be concluded that students are not good surrogates for other populations even when the research is concerned with the way people make decisions or handle information.

A category or type of research has not been previously addressed is what might be called methodological research. Often researchers, particularly academic researchers, are interested in testing new data collection techniques, scaling techniques or survey methodologies. Are student subjects appropriate for testing and/or comparing research methodologies which are intended to be used on other populations? The arguments in support of using students to test methodologies are similar to the arguments in support of using students in process studies. However, there is also reason to suspect that there may be interactions between subjects types and other independent variables that produce different results for different populations. Responses of student subjects to various methodologies may differ from those of other populations not only in magnitude, but also in direction. The question of whether or not students make good surrogates for other populations in methodological studies remains to be answered. Research addressing this issue is justifiable and needed.

SUMMARY AND CONCLUSIONS

The purposes of this paper were to review the literature regarding the use of students as surrogates for other populations and to identify the conditions under which the use of students is appropriate. Previous studies were classified into two categories based upon whether the focus of the research was on psychological states or psychological processes. Findings revealed that students are generally not good surrogates for other populations regardless of whether the object of the research is to measure subjects' states or their processes.

The question was also raised as the whether or not students should even be used in methodological studies designed to evaluate questionnaire formats, scaling techniques, or new data collection methods. Although more research is needed, there is currently little empirical justification for this practice.

This conclusion does not mean that academic researchers should always forego the advantages inherent in using student samples. There are conditions which lend themselves to using student samples. Exploratory studies, preliminary pretests prior to pretests on subjects from the study population, and interviewer training are all valid applications where external validity is not an issue. In addition, students may justifiably be used for research regarding the marketing of specific products which are intended to be marketed to this segment of the population.

However, in any study where results will be generalized to other populations, certain steps are recommended. First, a thorough study of relevant research such as is presented in this paper should be undertaken to determine whether or not similar states or processes have been investigated, and if so, whether there is evidence to suggest that students react in a manner similar to the population in question. Second, it is recommended that, prior to the use of students as surrogates, a pilot study be undertaken comparing a small sample of students against a small sample from the population in question. In some cases a pilot study may show similarities on the relevant variables sufficient to justify using students as convenient, inexpensive surrogates. If significant differences are found, student samples are clearly not justified.

It is further recommended that responsible journal editors require these pilot tests prior to the acceptance of manuscripts that are based upon studies using student samples when generalizations are made to other populations.

REFERENCES

Alpert, Bernard (1967), "Non-Businessmen as Surrogates for Businessmen in Behavioral Experiments," Journal of Business, 40, 203-207.

Clevenger, Theodore, Gilbert A. Lazier, and Margaret Leitner (1955), "Measurement of Corporate Images by the Semantic Differential," Journal of Marketing Research, 2, 80-82.

Copeland, Roland M., Arthur J. Francia, and Robert H. Strawser (1973), "Students as Subjects in Behavioral Business Research," Accounting Review, 48, 365-372.

Cunningham, William H., W. Thomas Anderson, Jr., and John H. Murphy (1975), "Are Students Real People," Journal of Business, 47, 399-409.

Enis, Ben M., Keith K. Cox, and James E. Stafford (1972), "Students as Subjects in Consumer Behavior Experiments," Journal of Marketing Research, 9, 72-74.

Enis, Ben M., and James E. Stafford (1969), "Consumers' Perception of Product Quality as a Function of Various Informational Inputs," Proceedings of the American Marketing Association, 340-344.

Ferber, Robert (1977), "Research by Convenience," Journal of Consumer Research, 4, 57-58.

Hovland, Carl I. (1959), "Reconciling Conflicting Results Derived from Experimental and Survey Studies of Attitude Change," The American Psychologist, 14,8-17.

Khera, Indera P. and James D. Benson (1970), "Are Students Really Poor Substitutes for Businessmen in Behavioral Research?" Journal of Marketing Research, 7, 529-532.

Leavitt, Theodore (1965), Industrial Purchasing Behavior, Boston, MA: Harvard University.

Lundstrom, William J. and Donald Sciglimpaglia (1977), "Sex Role Portrayals in Advertising," Journal of Marketing, 41, 72-79.

Park, C. Whan and V. Parker Lessig (1977), "Students and Housewives: Differences in Susceptibility to Reference Group Influence," Journal of Consumer Research, 4, 102-110.

Roering, Kenneth J., Robert D. Schooler, and Fred W. Morgan (1976), "An Evaluation of Marketing Practices: Businessmen, Housewives and Students," Journal of Business Research, 4, 131-144.

Sheth, Jagdish N. (1970), "Are There Differences in Dissonance Reduction Behavior Between Students and Housewives?" Journal of Marketing Research, 7, 243-245.

Shuptrine, F. Kelly (1975), "On The Validity of Using Students in Consumer Behavior Investigations," Journal of Business, 48, 383-390.

Tankersley, Clint B. (1975), "A Comparison of Evaluation of 'Social Responsibility': Students vs. Businessmen," Akron Business and Economic Review, 6, 48-51.

Vinson, Donald E. and William J. Lundstrom (1976), "An Examination of the External Validity of Using Students as Surrogates in Marketing Research," Proceedings of the Southern Marketing Association, 106-108.

----------------------------------------