A Discussion of Survey Research

Citation:

James H. Barnes, Jr. (1981) ,"A Discussion of Survey Research", in NA - Advances in Consumer Research Volume 08, eds. Kent B. Monroe, Ann Abor, MI : Association for Consumer Research, Pages: 304-305.

Advances in Consumer Research Volume 8, 1981      Pages 304-305

A DISCUSSION OF SURVEY RESEARCH

James H. Barnes, Jr., The University of Georgia

INTRODUCTION

The three papers presented in this session are loosely tied together since each deals with some aspect of survey type research. The differences among them, however, make each very difficult to compare and contrast. Consequently, after presenting some concerns about each paper, I shall attempt in a separate section of this discussion to offer some observations on the state of consumer survey research which I hope will be of some use to the audience of this section. No attempt is made to evaluate the net worth nor the relative contribution of the papers since each reader is probably a better judge of that given his or her particular frame of reference.

PETERSON, LEONE AND SABERTEHRANI

The authors of this paper deal with the nagging problem of refusal of respondents to provide information relative to family income. This non-sampling error they point out is often encountered in telephone surveys. Three reasons for refusal emerged from the study. Foremost, respondents viewed the question as an invasion of privacy.. Second was the fact that they lacked knowledge of their annual household income and finally, they felt that they did not have time to calculate their total income since it was made-up of several different components. The authors also point out that the refusal rate differed across respondents.

While the paper is interesting and generally well-written, the reader is given the impression that the authors are taking an analytical technique and looking for a set of data "crunch". This of course is not necessarily bad since researchers should have an interest in research methods per se. As a methods paper, it would be interesting to see how application of the logit analysis provides greater insight into the data than, for, example, simple cross-tabs of the data set. The use of the logistic function has been found to represent the data when the dependent variable is an indicator variable such as the probability used in the present case. The application of the logit approach presented in this paper is a standard application except perhaps for the conversion of the independent variables to be all 0-1 dummy variables. The authors fail to point out the problems of collapsing the independent variables into categories. While this approach is generally motivated around maintaining cell sizes, real analysis problems can and often do arise.

A second methodological concern is one of the choice of final model arrived at by the authors. The "saturated" model with which the authors begin is one which contains all independent variables as well as all interaction terms. From this model, the technique used reduces the model on the basis of loss in explanatory power. This however, presupposes that the variables education, age, sex, and marital status were the correct variables to begin with. The reader is not informed as to why these particular respondent characteristics were chosen.

GOLDEN, ANDERSON AND SHARPE

Response rate, speed of response and quality of response have long been of vital importance to users of mail survey instruments. In this article, the authors investigate the effects of monetary incentive, salutation and habitancy on these response measures. The type of salutation is varied by three levels and a probabilistic incentive is offered to a portion of the respondents. Also, the authors evaluate the self-selection variable of habitancy and have the respondents answer four psychographic measures along with the standard demographic questions. The authors conclude that an apparent raffle effect of the monetary incentive tends to decrease response end that urbanization has an influence on response rate, speed, and quality of response.

An important task in any experiment is to control for external factors which can influence the dependent variable. In the present study, the authors use the postmark date on the respondent's return envelope as their measure of response speed (the surveys were sent from a single Texas location to four different eastern Oklahoma areas). The problem is, that this measure is probably influenced to a large extent by the time required for the Postal Service to deliver the mail to the respondent. While I am not familiar with the postal procedures of Texas-Oklahoma, I have had some experience with the variance of postel speed between and within Athens (rural) and Atlanta (metropolitan), Georgia. I suspect that a vide variance also exists in the subject study area of Oklahoma.

Response quality was measured on a one to five scale depending on how many of the four sections of the questionnaire were answered. The selection of this approach is not clear since most prior studies have selected percent of questions completed as a quality measure. This could affect the results reported in Table 4 since the mean quality varied only over the range 1.09 to 1.32. The assignment of a one to five value tends to mask small differences making them appear more significant than otherwise. For example, if a respondent completes only half of one part of the questionnaire does he get a value of four or five. The effect on the computation of the mean is obvious. Also, the authors fall to discuss what types of questions were asked in the four sections of the study. Based on the affiliation of the third author, one would suspect that the questions related to department store shopping. Could rural, urban and suburban residents have different views of department stores which would effect their interest in responding? This, of course, would have affected the measures of rate and quality. Stated differently, are these measured results questionnaire specific.

The theoretical connection between the rather global psychographic measures used and the dependent measures under study is not clear. For example, why does one expect a relationship between external locus of control and response speed? More specific measures of lifestyle would probably have been more useful.

Finally, does the procedure for entering the cash drawing force the respondent to give up anonymity? If both the questionnaire and the entry card are in the same envelope, then one would suspect that this could have some bearing on whether or not one chose to enter the drawing. Also, did many respondents attempt to enter the cash drawing without returning a questionnaire.

JACOBY AND HOYER

In this paper, the authors address the construct validity of opinion leaders. That is, they ask the question, what if so-called opinion leaders really don't know more?

Specifically, Jacoby and Hoyer test the relationship between the opinion leader construct and expertise, another construct believed logically related to opinion leadership. The authors found that as expected, opinion leadership was highly related to expertise.

The link between opinion leadership and expertise has been implied to be one of causality. That is, a person is an opinion leader because he is more of an expert. Although the. authors do not deal with the causality issue directly, one could argue that both opinion leadership and expertise are products of another variable namely involvement with the product category. Thus, correlation between opinion leader and expertise would result. Several of the variables used in the present study could be proxies for product category involvement. In fact, by checking the correlation matrix in Table 1, one can note the high correlation between the involvement proxy looking and both the expertise and opinion leader variables. Also, one can see a high correlation between opinion leader, expertise and past ownership, but, low correlation between present owner. This would also suggest involvement since past owners may be in the looking process thus more involved than present owners who are not in the search process.

One would suspect that opinion leadership is a multi-dimensioned concept. Such things as where to buy, price, and suitability for various uses and perhaps even more importantly, source credibility. The present study looks at only the link between technical expertise and opinion leaders. Future research should consider these other factors.

SOME GENERAL COMMENTS ABOUT THE AREA

More years ago then I sometimes care to remember, I was trained in the field of electronic engineering. A favorite training tool of my mentors was to give the student a black-box which contained some form of circuit. However, the student had only the opportunity to manipulate the input terminals of the box. He did not know what was inside. The approach that one must take in analyzing the box was to input various currents and voltage patterns then measure the resultant output. Based on certain principles or laws of electronic components, one could, through manipulating the input and measuring the outputs, determine, within limits, the internal make-up of the circuit components inside the box.

The above story holds, I think, a lesson for us in survey research. A very brief review of the literature reveals over 132 studies of things such as effects of cover letters, postage, incentives, offer of survey results, and other factors on response rates for mail surveys. My point is that we are really like the student above, but we have failed to accomplish the final important task. That is, we are really only looking at the output patterns of the black box as a result of various inputs (manipulations). Continued focus on only outputs-puts this research topic into the realm of "learning more and more about less and less." What is needed, is for us to take that next step and infer something about inside the black box (survey respondent). In the studies reviewed above, I questioned several authors about their choice of independent variables. In fact, neither I nor the authors here really have very firm ground for discussing which variables are or are not important. In short, we really don't have a theory on which to base our studies. Thus, my final plea to the reader is that we proceed to combine our multitude of results and determine the circuit inside survey respondents. This, of course, is no easy task, but, a necessary and much needed one at this time in our development.

----------------------------------------