An Investigation of Response Quality and Bias Differences in Four Chinese Subpopulations

ABSTRACT - China represents an unknown consumer behavior arena with immense potential as it moves toward privatization and a market economy. This study compared price-quality responses of: (1) Chinese university students to general consumers and (2) respondents living in a rapidly advancing city to those residing in a more typical Chinese city. Contrary to expectations, no differences were found for aggregated mean responses. However, students provided fewer missing answers, and respondents residing in a more advanced consumer market displayed a no opinion-to-extreme positive response bias on the 7-point agree-disagree items. Chinese university students may be appropriate samples for initial consumer studies. Also, provisions should be made to detect and adjust for response bias in Chinese respondents living in areas of rapid development.

Citation:

Ann Veeck and Alvin C. Burns (1994) ,"An Investigation of Response Quality and Bias Differences in Four Chinese Subpopulations", in AP - Asia Pacific Advances in Consumer Research Volume 1, eds. Joseph A. Cote and Siew Meng Leong, Provo, UT : Association for Consumer Research, Pages: 135-139.

Asia Pacific Advances in Consumer Research Volume 1, 1994      Pages 135-139

AN INVESTIGATION OF RESPONSE QUALITY AND BIAS DIFFERENCES IN FOUR CHINESE SUBPOPULATIONS

Ann Veeck, Louisiana State University

Alvin C. Burns, Louisiana State University

ABSTRACT -

China represents an unknown consumer behavior arena with immense potential as it moves toward privatization and a market economy. This study compared price-quality responses of: (1) Chinese university students to general consumers and (2) respondents living in a rapidly advancing city to those residing in a more typical Chinese city. Contrary to expectations, no differences were found for aggregated mean responses. However, students provided fewer missing answers, and respondents residing in a more advanced consumer market displayed a no opinion-to-extreme positive response bias on the 7-point agree-disagree items. Chinese university students may be appropriate samples for initial consumer studies. Also, provisions should be made to detect and adjust for response bias in Chinese respondents living in areas of rapid development.

INTRODUCTION

With one-fifth of the world's population and a rapidly expanding economy, the People's Republic of China has been characterized as "the greatest experiment in consumer behavior to ever be conducted" (Belk and Zhou 1987, p.480). Since December 1978, when Deng Xiaoping introduced a historical proposal to reform China's economy and open the nation to world trade, China's GNP has grown almost 9% annually. Some projections maintain that if China's economic growth continues its pace, China will surpass the U.S. as the world's largest economy by 2010 (The Economist, 1992). While such estimates may be overly optimistic, it is certain that the recent transformation of the economy has resulted in one of the world's fastest growing markets for consumer products. It would be difficult to exaggerate how dramatically privatization has transformed the urban climate of China in the past five years. Pawn shops, real estate markets, stock markets, commodity markets, credit cards, and bank machines have emerged, seemingly overnight. Urban residents snack regularly at fast food restaurants (Lo 1993). Heels and cosmetics are now standard uniforms for many urban women. Campbell's soup, Lux soap, M&M's, and, of course, Coca-Cola are available for sale at kiosks.

This rapid metamorphosis of urban China makes it fertile ground for consumer research of all kinds. Yet, before leaping in to take part in "the greatest experiment in consumer behavior to ever be conducted," consumer researchers must be cautioned as to the ability to make generalizations concerning this vast nation of 1.2 billion people. The dramatic growth of the China's economy has led to great regional disparity in marketing efficiency. There is not only a huge gap between rural and urban areas in the marketing infrastructure, but there is also a tiered system among the urban areas. Most advanced in marketing efficiency are the four original Special Economic Zones, particularly Shenzhen, where capitalism has been given free reign. The next advanced are the larger cities along the east coast that have been the first to experience the economic reforms, including Shanghai, Guangzhou, and Tianjin, as well as the national capitol of Beijing. In the third tier are other important urban areas on the east coast, including such cities as Nanjing, Hangzhou, Suzhou, and Wuxi. In the last tier are the urban areas within the interior, which have been slower to reap the benefits of the reforms for locational and bureaucratic reasons.

The implication of China's great regional disparity is that researchers must exercise prudence when generalizing the results of local studies. Furthermore, as in all research, investigators must be careful in their selection of sample groups to represent the population as a whole. To investigate these concerns, the purpose of this study is to analyze response tendency differences among four Chinese subpopulations: (1) consumers residing in an economically advanced city, (2) consumers residing in a more typical city, (3) students studying at a university located in a economically advanced city, and (4) students studying at a university located in a more typical city. The major issues involved in our study are response quality and response bias in price-quality perceptions. We define these terms and show why they are a concern in all survey research, and particularly in cross-cultural studies. We next describe the results of a survey where the four Chinese subpopulations were sampled and systematically examined for differences in mean responses, response quality, and response bias. Last, based on our findings, we offer some tentative guidelines to researchers interested in Chinese consumer behavior.

EQUIVALENCE ISSUES IN CROSS-CULTURAL CONSUMER RESEARCH

Several thorny issues arise when Western researchers investigate the consumer behavior of cultures that are foreign to them. These concerns have been expressed in a variety of ways (see, for example, Davis, Douglas and Silk 1981; Douglas and Craig 1983; or Parameswaran and Yaprak 1987), but the essential problem is equivalence. Figure 1 lists a number of different bases of equivalence that must be addressed when research conducted in one culture is compared to findings of research on that topic which has been carried out in another culture. Obviously, a single study cannot reconcile all these concerns. A judicious approach is to take the issues individually and to examine them under unique intra- and inter-cultural contexts to understand country-specific patterns. This is our approach in the present study.

There is widespread criticism aimed at the practice of using university students to represent a general population of consumers. With cross-cultural research, university students afford a convenient sample frame, but they are representative only to the extent that the general public possesses college education. Further, university students are unrepresentative because of their youth, and their experience base with many products and services differs from the general population's experience base. The discrepancy in education is especially large in third-world countries where literacy rates are sometimes low. This gap is not true in China, where over 80 percent of the population is literate. However, only a small percent possesses a university degree, so higher education status is a salient factor.

Confounding this factor is rapid, yet uneven, privatization of the Chinese economy. As noted above, progressive Chinese city citizens have experienced a multitude of shifts in the consumer buying environment, while other areas have lagged considerably behind. These diverse circumstances suggest that some Chinese consumers have much greater experience bases and have probably acquired more consumer sophistication than their traditional counterparts and, possibly, university students. The current situation in China results in a problematic condition for consumer researchers. It is unclear how adequately a Chinese university student sample represents Chinese consumers in general. Alternatively, it is unknown whether Chinese university students represent consumers in more advanced economies better than consumers in other areas.

FIGURE 1

CATEGORIES OF EQUIVALENCE IN CROSS-NATIONAL RESEARCH

RESPONSE QUALITY AND RESPONSE BIAS

Representation has many facets. On one hand, representation may be assessed by mean differences. If a mean or a vector of means of responses for a sample comprised of one subpopulation differs from that of the sample of another subpopulation, representation is an issue unless there is good reason to expect these differences between the subgroups. At the same time, differences between samples consist of true differences and survey error. As a result, another means of understanding representation is to examine for systematic error differences. One measurable type of survey error is response quality, while another is response bias. We will describe each type of error and indicate how it can affect results.

Response quality

Response quality, or response completeness, refers to missing responses. Occasionally, a researcher will find unanswered questions on completed questionnaires. Ordinarily, these questionnaires are retained for analysis, and the unanswered responses are coded as "missing data." Missing data detract from overall response quality in two ways. First, the effective sample size is reduced when observations containing missing data are omitted from analysis. Second, some researchers opt to insert the mean of all nonmissing responses on a question for all missing responses. This practice retains the original sample size, but it effectively reduces the variance on that variable, and may adversely affect its explanatory power. Because response quality has been found to vary systematically by demographic group (see, for instance, Durand, Guffey, and Plancon 1983), imputation methods have been developed which substitute the demographic subgroup mean instead of the overall sample mean (Gilley and Leone 1991). This approach reduces response quality error; however, it does not eliminate the error.

Response bias

When a response does occur to a question, it is subject to various biases in the respondent. With scaled-response questions such as an agree-disagree scale, a "no opinion" bias exists in the tendency to indicate a neutral response category (such as, in the case of this study, "neither disagree nor disagree") rather than the (true) negative or the positive direction. "Yeasaying" is a tendency to answer on the positive extreme, while "naysaying" is the negative counterpart of yeasaying. Each of these three response biases adversely affects tabulations and analyses. No opinion bias overstates the neutral position and reduces the variance of items and their summated scales. Yeasaying causes false positive means, while naysaying begets false negative means for the items affected. Either one reduces the potential explanatory power of the questions affected by constraining the variance of responses. Last, "opinionation" is a response bias characteristic we have coined referring to the tendency to use a narrow or a wide response range around the individual's mean response. A wider range indicates more expressed opinion in both positive and negative directions across a variety of items. Alternatively, low opinionation means the respondent is using only a subrange of the response categories. Thus, the designed sensitivity of the response scale is being thwarted with low opinionation respondents.

Research Questions

As can be seen, both response quality and response bias can result in false readings of samples that exhibit these errors. The unique case of China allows the investigation of three research questions here:

1. Are there differences in aggregate means between:

a. Chinese university respondents and Chinese consumers, and/or

b. Chinese respondents in cities representing different stages of market efficiency?

2. Does response quality differ between:

a. Chinese university respondents and Chinese consumers, and/or

b. Chinese respondents in cities representing different stages of market efficiency?

3. Does response bias differ between:

a. Chinese university respondents and Chinese consumers, and/or

b. Chinese respondents in cities representing different stages of market efficiency?

METHOD

We designed a 2x2 study incorporating the four subpopulations of interest. Beijing (population of 10.8 million) was selected as a sample city, because, as the political capitol of China, it represents an area with some of the most advanced exchange infrastructure in the country, with the exception of the four Special Economic Zones. Nanjing (population of 2.8 million), although relatively advanced compared to much of China, was chosen to represent a more typical urban consumer market. Beijing and Nanjing are members of the second and third tiers, respectively, of economic reform. For university students, we used contacts at Nanjing University and Beijing Forestry University who administered our questionnaire in class settings as in common in cross-cultural research (e.g., Durvasula et al 1993; Lee and Green 1991; or Netemeyer, Durvasula and Lichtenstein 1991). Both are respected universities, with comparable curricula. Our consumer samples were gathered at the household level on a convenience basis by Chinese administrators. This is the most practical means of gathering general public data in a country where telephones are rarely owned.

The results we are reporting are part of a larger survey with 301 respondents approximately evenly distributed across our four subpopulations. The questionnaire contained 102 separate questions, and response quality was operationalized as the number of missing data points on an individual's questionnaire. A total of 48 questions was in 7-point Likert agree-disagree format asking about perceptions of price and quality of various consumer products. We posed a series of questions about price indicating quality (e.g., "The higher the price, the higher the quality") and variability of quality and price (e.g., "The quality of this item is highly variable.") for several consumer goods (e.g., color television, ball point pen, bar of soap). This approach was similar to that used by Lichtenstein and Burton (1989) and Peterson and Wilson (1985) in U.S. studies of price-quality perceptions. Although prodigious research exists on the price-quality perceptions of Americans, this research area has only begun to be explored in the burgeoning economy of China (see Doran 1994). The questionnaire design process included a focus group of Chinese students studying in the U.S. to decide the reasonable set of products. Next, the questionnaire was developed in English. It was then translated into Chinese, and backtranslation by an English-speaking Chinese assistant was used to refine the Chinese version of the questionnaire.

Operationalization of our variables was done in the following ways. An aggregate mean was calculated by computing each respondent's mean response across the 48 Likert items (Cronbach's alpha=.86). No opinion, yeasaying, and naysaying were measured as the number of neutral, extreme positive, and extreme negative responses, respectively. Opinionation was operationalized as the standard deviation of the individual's responses around the mean response for all 48 items. This approach is identical to that used by Greenleaf (1992).

FINDINGS

We began our investigation by looking at mean differences. As noted above, each respondent's mean response for the 48 Likert items was computed, and analysis of variance consistent with our experimental design was used to investigate differences among the subpopulation means. The grand mean was 4.60, and no significant differences (F(2,300)=1.514, p<.221) were observed. This result was inconsistent with our reasoning about the differences between students and consumers in countries where college education is rare, and it contradicted our rationale concerning Chinese respondents in differing market environments. That is, we did not expect to find the overall mean responses of Chinese university students identical to those of Chinese consumers, and we expected to see differences in the mean responses of residents in advanced markets when compared to those of respondents residing in more typical Chinese market environments. It is possible that no differences exist; however, it is also possible that systematic errors were responsible for these results.

The five types of potential error were measured in each of the 301 respondents and subjected to multivariate analysis of variance (MANOVA) to test for differences among the groups. We investigated a two factor model (student-versus-consumer status and more advanced-versus-less advanced city) and tested for interaction effects. Table 1 contains the relevant group means for inspection. We found the MANOVA to be significant (F(5,293)=6.113, p<.0001), and examined the various group differences with separate univariate F-tests. Here we found a significant effect of type of city with Beijing respondents higher in neutral responses (10.5 versus 7.5; F(1,297)=14.797, p<.0001) and greater in positive end responses (5.3 to 4.1; F(1,297)=4.148, p<.04) than Nanjing respondents. Status was also significant with Chinese consumers being higher in nonresponses than were students (3.3 to 1.7; F(1,297)=8.245, p<.004). The overall effect size was .10 with city type about twice that of respondents' status. Interactions were evident with Beijing students much higher on missing responses than were Nanjing students (2.8 versus .6; F(1,297)=11.71, p<.001), and Nanjing consumers higher than Beijing consumers on the same variable (4.2 versus 2.5). Beijing consumers were also higher than Nanjing consumers on neutral responses (11.7 versus 6.0, F(1,297)=12.047, p<.001). In other words, we found no significant differences between Chinese university students and Chinese consumers for: (1) no opinions, (2) yeasaying, (3) naysaying, or (4) opinionation. Similarly, we found no significant response completeness differences existing between advanced and typical Chinese respondents, nor were there differences in naysaying or opinionation between the two types of respondents.

DISCUSSION

Our study examined the differences in responses among four subpopulation samples variously representing Chinese consumers. We began the investigation fully expecting to find mean differences in responses between Chinese university students and the Chinese general public. We also anticipated finding mean differences between respondents residing in the rapidly advancing private enterprise sectors and those living in sections of China where the transition to a market economy is progressing at a slower rate. Using an aggregated mean for 48 separate 7-point agree-disagree items, we did not find significant differences. This finding contradicts the warning issued to cross-cultural consumer researchers that university student samples are not representative of a country's population as a whole. The null finding also suggests that mean differences can be comparable across Chinese cities.

Because of these unanticipated findings, we turned to various measures of response quality and bias to see if they were operating differently by respondent group. Taking the university students-versus-general population comparison first, missing responses in our Chinese consumers were twice the level found in our university students. But no differences in response bias (no opinions, yeasaying, naysaying, or opinionation) were found. Thus, it appears that for scaled agree-disagree items with a neutral category response option, the price-quality perceptions of Chinese university students and Chinese consumers are equivalent; however, the response quality (defined as missing responses) is somewhat lower for consumers.

TABLE 1

RESPONSE QUALITY AND BIAS FOR FOUR CHINESE SUBPOPULATIONS MEANS AND STANDARD DEVIATIONS

When comparing respondents (consumers and students combined) from regions of differing market advancement, no response quality differences were found. However, consumers in a more advanced urban market area were more prone to yeasaying and no opinions. These two predilections suggest a generalized right-side (neutral to extreme positive) response bias in Chinese respondents who are experiencing the rapid transition from a controlled market economy to a privatized one. Thus, the aggregated means of progressive market area respondents reflect this response bias to a greater degree than do the means of respondents who have not experienced the transition as rapidly. The effect of the distribution of means on total scores was not assessed and might be addressed in future research.

Within university students, response quality differences were apparent as well. Nanjing area students exhibited significantly less nonresponse. One might mistakenly surmise that Chinese in the more typical city have a generalized compliance trait, but these consumers in the same city were found to have the highest nonresponse rate. Our opinion is that differences in the administration of the questionnaire at the two university locations or, perhaps, teacher rapport differences underlie the low nonresponse of Nanjing university students. Indeed, we were informed that graduate assistants administered the survey at Beijing Forestry University, while faculty members administered it at Nanjing University. Within Chinese consumers, Beijing residents are more inclined to no opinion responses than are Nanjing residents. It is possible that citizens of the capitol city are more guarded in rendering their opinions in surveys or they may be expressing an inability to judge price-quality in the face of burgeoning market alternatives.

We noted earlier that several equivalence issues must be addressed by cross-cultural consumer researchers. Our study did not compare respondents across cultures. Rather, it compared four subpopulations within the greater Chinese culture that is a judicious beginning, we think, to researching Chinese consumer behavior. We offer two tentative guidelines, based on our findings. First, use of Chinese university students as a sample frame can be appropriate for initial studies of Chinese consumer behavior, and it will garner higher response quality. Precise selection of which Chinese university to use does not seem to be a major concern. Of course, we advocate subsequent general consumer samples for verification and subgroup analysis. Second, we recommend that researchers incorporate provisions for measuring response bias in all studies, but especially if the research includes respondents living in those Chinese cities that are experiencing rapid movement toward a market economy. Ideally, response bias should be adjusted out of each respondent's answers. Otherwise, the research may display a false positive consumer profile for agree-disagree items, and the researcher may be frustrated by a lack of explained variance in these measures.

REFERENCES

Beijing Review (1993), "Comfortable Life Coming Soon To Cities," 36, (3), 4.

Belk, Russel W. and Zhou Nan (1987), "Learning to Want Things," in Advances in Consumer Research, Vol.14, eds. Melanie Wallendorf and Paul Anderson, Provo, UT: Association for Consumer Research, 478-481.

Berry, J.W. (1980), "Introduction" to Handbook of Cross-Cultural Psychology: Methodology, Vol. 2, eds. Harry C. Triandis and John W. Berry, Boston: Allyn and Bacon, Inc., 1-28.

Davis, Harry L., Susan P. Douglas, and Alvin J. Silk (1981), "Measure Unreliability: A Hidden Threat to Cross-National Marketing Research," Journal of Marketing, 45 (Spring), 98-109.

Doran, Kathleen Brewer (1994), "Exploring Cultural Differences in Consumer Decision Making: Chinese Consumers in Montreal," in Advances in Consumer Research, Vol. 21, eds. Chris T. Allen and Deborah Roedder John, Provo, UT: Association for Consumer Research, 318-322.

Douglas, Susan P. and C. Samuel Craig (1983), International Marketing Research. Englewood Cliffs: Prentice Hall, Inc.

Durand, Richard M., Hugh L. Guffey, Jr. and John M. Plancon (1983), "An Examination of the Random Versus Nonrandom Nature of Item Omissions," Journal of Marketing Research, 20 (August), 305-13.

Durvasula, Srinivas, J. Craig Andrews, Steven Lysonski, and Richard G. Netemeyer (1993), "Assessing the Cross-national Applicability of Consumer Behavior Models: A Model of Attitude toward Advertising in General," Journal of Consumer Research, 19 (March), 626-636.

The Economist (1992), "When China Wakes," November 28, 2-18.

Gilley, Otis W., and Robert P. Leone (1991), "Two-Stage Imputation Procedure for Item Nonresponse in Surveys," Journal of Business Research, 22, 281-291.

Greenleaf, Eric A. (1992), "Improving Rating Scale Measures by Detecting and Correcting Bias Components in Some Response Styles," Journal of Marketing Research, 29 (May), 176-88.

Lee, Chol and Robert T. Green (1991), "Cross-Cultural Examination of the Fishbein Behavioral Intentions Model," Journal of International Business Studies, 21 (Second Quarter), 289-305.

Lichtenstein, Donald R. and Scot Burton (1989), "The Relationship Between Perceived and Objective Price-Quality," Journal of Marketing Research, 26 (November), 429-443.

Lo, Robert (1993), "American Business Doing Well in Shanghai," 36 (37), 11.

Netemeyer, Richard G., Srinivas Durvasula, and Donald R. Lichtenstein (1991), "A Cross-National Assessment of the Reliability and Validity of the CETSCALE," Journal of Marketing Research, 28 (August), 320-327.

Parameswaran, Ravi and Attila Yaprak (1987), "A Cross-National Comparison of Consumer Research Measures," Journal of International Business Studies, 17 (Spring), 35-49.

Peterson, Robert A. and William R. Wilson (1985), "Perceived Risk and Price Reliance Schema as Price-Perceived Quality Mediators," in Perceived Quality: How Consumers View Stores and Merchandise, Jacob Jacoby and Jerry C. Olson, eds., Lexington MA: D.C. Heath and Company, 247-268.

----------------------------------------