Comments on the Three Survey Research Papers

Citation:

Seymour Sudman (1982) ,"Comments on the Three Survey Research Papers", in NA - Advances in Consumer Research Volume 09, eds. Andrew Mitchell, Ann Abor, MI : Association for Consumer Research, Pages: 260.

Advances in Consumer Research Volume 9, 1982      Page 260

COMMENTS ON THE THREE SURVEY RESEARCH PAPERS

Seymour Sudman, University of Illinois

The paper by Fenwick, Weisman, Becker and Heiman gives us an imaginative use of discriminant analysis to allocate undecided voters to candidates. The procedure is validated by a post-election follow-up survey that obtained reported voting behavior. As the authors point out, this new procedure has several advantages over existing voter allocation procedures, among which is the fact that it can be used on the telephone.

I believe that this new procedure should and will have substantial use in future election studies. I would suggest one extension that may be even more important, especially in primary elections--discriminating between those registered voters who will and will not vote. When asked this question, particularly in primaries, a large number of respondents are usually undecided or uncertain and a method for making a forecast about this behavior would also be very useful. It may be, however, that it is more difficult to discriminate on intention to vote than on which candidate will be selected. Still, it seems worth exploring. Finally, let me congratulate the authors on what I think is the best title at this ACR conference.

The paper by Kamakura and Srivastava on the Use of Latent Trait Theory is an example of a new research method that is sweeping through social psychology and is attracting substantial interest. It may have major importance in consumer research measurement. I would suggest that at this stage in the development of Latent Trait Theory as discussed here one is better off with a middle position. The procedure is interesting, needs to be considered and tested in real world situations, may help to solve some problems, but is not the answer to all attitude scaling problems.

Since the authors have pointed out the possibilities, let me point out some of the problems. First, as with all attitude scales, the procedure assumes that the respondent has an underlying attitude. Many people may not have well-defined, reliable attitudes on certain topics and no scale wordings or mathematical procedures can solve this issue. The don't know that reflects this lack of interest in the topic is not resolved by this or any other scaling procedure. A second problem, is whether or not there is a single universe, or whether the values of a. and b. vary among different segments of the population.l This, of course, is similar to the regression problem.

Another issue relates to the question of the sampling variances of the estimates. These are not universe values, but complex sample estimates that have variances attached. It is possible to measure this reliability by interpenetrating samples -- that is, replication. As an aside, it is not clear to me what the authors mean by their concluding claim that the procedures are independent of the sample. The method may be, but the results certainly need not be.

I am also unclear about the authors' claim that the procedure may be used to eliminate redundant or duplicative items. To the extent one wishes to do this, at the expense of reliability, much simpler methods are available. What is clearly superior in this new procedure is the ability to evaluate items at different attitude levels.

The paper by O'Connor, Sullivan and Jones addresses the issue of response quality induced by follow-ups to mail questionnaires. It has long been known that follow-ups improve cooperation, but less well known is whether there is an adverse effect on response quality. In analyses of data from personal surveys, most researchers do not find a decline in quality, so that it is comforting to see the same result in a mail survey. Only item omissions increased between the first and second wave, but even this finding is fuzzy since Wave 3's omissions were slightly lower than those from Wave 2. Given the length of the overall questionnaire and the relatively low salience as indicated by the low overall cooperation, the wave effects are fairly small on omissions.

One must ask whether it was really a good idea to send this questionnaire out by mail, since many mail surveys do very much better in overall response, as reported by Dillman in his book. It is also not clear whether the same lack of effects would occur if one were going from 50 to 75 percent cooperation instead of from 21 to 38 percent. Nevertheless, the procedure used was imaginative and is worth being used by others.

----------------------------------------