A Cross-Cultural, Between-Gender Study of Extreme Response Style

ABSTRACT - It has been noted in the literature that, when applying a Likert-type scale across cultural borders, responses vary in terms of their tendency to cluster around the mid-point or migrate to the scale extremes. This phenomenon is known as the extreme response style (ERS). This paper seeks a pattern in the ERS exhibited in responses from a sample drawn from seven disparate countries. Data is also collected about the religiosity and individualism of the respondents, and an analysis performed to investigate any relationship between these factors that might help to explain the ERS pattern in the data.

Citation:

Roger Marshall and Christina Lee (1998) ,"A Cross-Cultural, Between-Gender Study of Extreme Response Style", in E - European Advances in Consumer Research Volume 3, eds. Basil G. Englis and Anna Olofsson, Provo, UT : Association for Consumer Research, Pages: 90-95.

European Advances in Consumer Research Volume 3, 1998      Pages 90-95

A CROSS-CULTURAL, BETWEEN-GENDER STUDY OF EXTREME RESPONSE STYLE

Roger Marshall, University of Auckland, New Zealand

Christina Lee, Nanyang Technological University, Singapore

ABSTRACT -

It has been noted in the literature that, when applying a Likert-type scale across cultural borders, responses vary in terms of their tendency to cluster around the mid-point or migrate to the scale extremes. This phenomenon is known as the extreme response style (ERS). This paper seeks a pattern in the ERS exhibited in responses from a sample drawn from seven disparate countries. Data is also collected about the religiosity and individualism of the respondents, and an analysis performed to investigate any relationship between these factors that might help to explain the ERS pattern in the data.

INTRODUCTION

The central issue reported in this paper is related to the aspect of metric equivalence across cultures. It is concerned with the problem of response style, and moe specifically, with the potential influence of extreme response style in research measurements and its possible variation across cultures. Style refers to the way in which respondents react to questions as stimuli (Wells 1961); response style influences the answers respondents give because different respondents may have different ways of expressing themselves.

The notion that response style influences research results has been discussed in the social-psychological literature for some time. Early work on response style emphasized the difficulties it can cause in analysis and interpretation of research data (for example, Cronbach 1946, 1950; Lentz 1938; Rundquist 1950). Berg and Collier (1953) clearly stated that the tendency to respond in a given direction is a source of contamination which often affects test reliability and validity.

Researchers have related extreme response style (ERS) to high anxiety, low intelligence and poor adjustment. In addition, experiments have been conducted to explore the question of gender difference and age difference in regard to ERS. Despite the fact that there is a substantial amount of literature on the topic, the main themes of the literature have little to do with cross-cultural differences. Nevertheless, the identification of cross-cultural differences in ERS is of interest both as a reflection of cultural differences on substantive dimensions and for its implications for the methodology of cross-cultural research. The consequent need to thoroughly investigate cultural differences in ERS has been long perceived and suggested in previous literature (Chun & Campbell 1974; Hui & Triandis 1989).

The main aspect of interest in the present study is cross-cultural differences in ERS. An attempt to identify a cultural effect on the individual variation in the tendency to use the extremities of a bi-polar response scale is reported. The aim is to investigate response style in a variety of cultural settings in order to determine whether there is any pattern to these differences. In essence, the present research examines the interaction of culture, gender, individualism and religiosity that have been hypothesised to relate to ERS.

LITERATURE REVIEW AND HYPOTHESES FORMULATION

Response bias occurs when a response to a test item tends to be altered in such a way that it indicates something other than that which it is intended to measure (Runquist 1950). This bias may result from either response set or style.

Response bias due to the response set occurs when a respondent is motivated to answer in a particular way (Rorer 1965). For example, social desirability bias is formed when the subject intends to give socially desirable responses to a questionnaire regardless of whether the socially desirable response is true or not (Block 1965). Response style, on the other hand, may often occur unconsciously, thus revealing a person’s habitual preference for a certain response category. Acquiescent response style ("yeasaying") is one which has been long acknowledged. It refers to the phenomenon that some respondents tend to agree rather than disagree to propositions in general when in doubt (Block 1965). Likewise, extreme response style is another style frequently observed by researchers. In this case, respondents tend to answer at the end of a bi-polar scale rather than in the middle.

Response bias due to response set affect responses in relation to item content while response style bias occurs in the absence of item content. Response style bias is more likely to interact with the form of questions than the content of the questions. That is, the less explicit the content of the questions, the more likely the extent to which a style can operate.

Cultural and Gender Differences

The notion that a specific response style rflects some cultural characteristics is not new. Berg and Rapaport (1954) believe that the biased response is probably the product of cultural factors. In their study, they found that American subjects had significant preferences for the response categories such as "yes", "satisfied", "true" and "agree". This finding was believed to be consistent with traditional American cultural values. They further suggested that other response styles such as neutrality or negativism would be expected in other cultures or sub-cultures if the neutral or negative response was valued in those cultures or sub-cultures. In fact, this view is supported by an earlier study by Berg and Collier (1953), who reported that black males made more extreme responses on the Perceptual Reaction Test than white males

Other studies of cross-ethnic differences in ERS have been carried out. For example, in a study conducted in Egypt, Soueif (1958) found that minority groups would make more extreme responses than other groups. Bachman and O’Mally (1984) compared the extreme responses of American black and white high school seniors to a number of different Likert-type scales. It was reported that blacks were more likely than whites to use the extreme response categories. After they failed to explain the difference in terms of educational background and ability dimensions, the authors suggested that a possible explanation for black-white differences in response styles may lie in sub-cultural differences in language use or style.

In a similar way, Hui and Triandis (1989) conducted a survey to examine whether a cross-ethnic difference in ERS could be found between Hispanics and non-Hispanics. A significant difference was found, in that the former tended to use the extreme points of a scale more often than the latter. Triandis (1972) points out that cultures do differ in the extent to which respondents adopt an extreme checking style, and this consequently forms a critical issue in cross-cultural studies.

Zax and Takahashi (1967) reported a study on the response style difference between Japanese and American college student samples; Japanese female students, in particular, made extreme responses more often than their American counterparts. The authors indicated that there seems to be a relationship between the tendency to respond extremely and the experience of having grown up in a particular cultural setting. A similar study by Chun, Campbell and Yoo (1974) made a comparison between Korean and American samples. The evidence showed that the American sample exhibited a stronger ERS than the Korean.

Most ERS cross-cultural studies reported have been limited in scope, only comparing two or three cultures. An exception is that of Stening and Everett, 1984. They examined the response style of a total of 1647 expatriate and local managers from nine countries who were respondents to a study of stereotyping undertaken in American, British, and Japanese firms in Singapore, and in Japanese firms in Britain, Hong Kong, Indonesia, Malaysia, the Philippines, and Thailand. They reported marked differences between various nationalities in terms of their ERS.

The above studies present clear evidence of the existence of cultural differences in ERS. However, the question of why these differences exist and the consequent international pattern into which they might fall remains unanswered. Hui and Triandis (1989) argue that theoretical understanding of differences in ERS across cultures must explicate at least two aspects. One aspect is evident in various studies on human judgement and how judgement is mapped onto a rating scale. Such studies suggest, firstly, that cross-cultural difference in ERS could be, to some extent, due to the cultural differences in probabilistic thinking which have been discussed by some writers (Phillips 1977; Wright, Philips, Whalley, Choo, Ng, Tan & Wisudha 1978); secondly, they imply that the cultural difference in the way the judgement is mapped into a rating scale may lead to different ERS. This interpretation depends on the distinction between the cognitive categories a person uses to classify stimuli and the response ctegories used to communicate this judgement to others (Wyer & Carlston 1979). Hui and Triandis believe that there is a cultural difference in the way of matching cognitive categories of judgement with response categories, and they hypothesized that using a scale with more points would reduce the tendency to check the endpoints in some cultures. Their findings showed supporting evidence for their hypothesis (Hui & Triandis 1989), and thus suggest that one explanation of cultural differences in ERS could be the different way of matching cognitive categories with response categories.

A second approach to explaining cross-cultural differences in ERS which is more frequently addressed in the literature emphasizes the influence of a predominant cultural value on the respondent’s decision behaviour. For instance, in some Asian cultures it is important to be modest and to respond cautiously. Respondents in such cultures may exhibit less often than respondents from other cultures (Stening & Everett 1984).

Gender differences in ERS have been discussed in many articles, but the findings appear to be controversial. Some writers point out that there is gender difference in ERS while others contend that such differences were not found in their studies. In the cases in which gender differences in ERS were revealed, the overall results show that females make significantly more extreme responses than males (Adams & Berg 1961; Berg & Collier 1953; Borgatta & Glass 1961; Crandall 1965; Heinemann 1968; Soueif 1958; Zax & Takahashi 1967). In these studies, a variety of subject groups was included, including normal American adults, American college students, prisoners and neurotics, Egyptians, and Japanese college students.

The stimuli involved in these latter experiments were mainly affective evaluations, such as liking for abstract design or the pleasantness-unpleasantness of words. There is only one exception in which the extreme response score was a combination of evaluation, potency, and activity ratings (Zax & Takahashi 1967).

It is interesting to note that when the judgements are affective, neutral and more cognitive in nature, men do seem to use extreme responses more frequently than women (Crandall 1965; Pettigrew 1958; Wallach & Caron 1959; Wallach & Kogan 1959). These authors suggest that ERS variation across gender may originate in the gender differences in expression of affection and the way of thinking.

In addition, Crandall (1973) contends that if positive and negative response extremities are considered separately, gender difference in ERS is only explicit in positive ratings. He conducted three experiments and found that female respondents made significantly more frequent use of extreme positive ratings than male respondents, but no difference was found between genders using negative ratings.

Following from the literature then, it is reasonable to expect marked differences in ERS between cultures. Thus Hypothesis One:

H1: There are significant differences in ERS among cultures.

Furthermore, a gender difference in ERS across cultures is also believed to exist. The role differences between men and women of different cultures, both in matters of child rearing and their different social roles in various cultural environments, means that there is a possibility of an interaction between culture and gender. For example, some Asian women could have been taught to be non-assertive when children and may, then, be less likely to show ERS compared to their European counterparts. This hypothetical notion is made in the light of previous research (eg. Zax & Takahashi 1967). Hypothesis Two is designed to test the interaction between culture and gender:

H2: There is an interaction of culture and gender effects in relationship to ERS. In the Eastern countries, ERS will occur less often in the female respondents than in their male counterparts; conversely in the Western countries, the female respondents will show a stronger ERS than the male subjects.

Religion

With regard to the national pattern of extremetising, the role of religion should not be discounted. First, it has been suggested that the degree of ERS is related to the intolerance of ambiguity (Soueif 1958). That is, a person with a low tolerance of ambiguity is likely to have a strong ERS. Second, Das and Dutta (1969) discovered a positive relation between sympathy to religion and intolerance of ambiguity. In their article they cite an unpublished Master’s dissertation (from Utkal University in India) in which the degree to which an individual favours religion is found to be positively related to the extreme response set. Therefore Hypothesis Three is designed to test the link between religion and the extremising phenomenon.

H3: Respondents for whom religion is perceived as an important influence in their lives will exhibit a greater ERS than for those for whom religion is less important.

Individualism

According to Hofstede (1980), nations can be characterised by cultural dimensions. Understanding where a nation is on these cultural dimensions allows researchers to make predictions regarding the dimensions way a society operates (Shaffer & O’Hara 1995). This study is concerned with one of these cultural dimensions, that is individualism. Individualism is defined as the extent to which an individual expects personal freedom versus the acceptance of responsibility to family, tribal or national groups (i.e. Collectivism). Individualists value independence, are more self oriented, have an emphasis on ego-identity, and have a strong "I" consciousness, while collectivists value interdependence and possess a high "we" consciousness (Hofstede 1980).

TABLE 1

SAMPLE DEMOGRAPHICS

Although there is no specific research which has considered individualism and ERS, the characteristics of individualist cultures, in particular their desire to express their own independent thoughts and ego-driven nature suggest that individualists are more likely to extremitise their answers than collectivists, who are more concerned about the fellow members of their group. Hofstede (1980) notes that Asian-type cultures tend to be collectivists while Western cultures, individualists. Although not examining the relationship between individualism and ERS in particular, research by Stening and Everett (1984), and Zax and Takahashi (1967) suggest that Asian cultures tend to be more modest in their responses and tend to exhibit less ERS. Following from this thought, Hypothesis Four states:

H4: Respondents exhibiting a higher level of individualism will demonstrate higher levels of ERS, whilst low levels of individualism will correspond to lower levels of ERS.

RESEARCH METHOD

A survey was administered to students in the Business Faculties of participating universities from the seven countries named in Table 1. Although a convenience sample, at a national level this group represents a good mixture of individualism, cultural types and religions.

The questionnaire was distributed to the various lecturers administering the survey after it had been pre-tested on 10 European and 10 Chinese students who showed asimilar understanding for each of the items in the questionnaire. Translation was required for the Korean, Malaysian and Indonesian versions. This task was performed by a professional agency for the Korean version and by the author for the other two; the process of back translation was used to ensure equivalence of concepts and meanings across cultures

Using students as subjects in behavioral research has been questioned by many authors (Alpert 1967; Levitt 1965). The main argument is that using a student sample might not allow valid generalizations. Nevertheless, in the present study, it was by no means intended to claim the student samples as truly representative of their respective populations; instead, the student samples were chosen because students are a relatively comparable group across countries. As education has been shown in the past to possibly affect ERS, the selection of a student sample offers a specific, as well as general, measure of control.

Instruments

The four independent variables manipulated are culture, gender, religiosity and individualism. One independent variable, tolerance of ambiguity, was removed from the study because reliability tests on the scale produced weak results suggesting that the scale was unreliable. The dependent variable is ERS. This section describes the development and the testing of the scales used in the research.

The ERS Scale

The 14-item scale used to measure ERS is drawn from an ERS scale used by the authors (1995) in a study of ERS cross-culturally. The scale was found to be reliable and valid in both Asian and Western cultures.

The method used to compute the ERS score is based on the idea of measuring extremity in terms of deviation from the midpoint of the scale (Hamilton 1968, Neuringer 1961, Peabody 1962). This method both encapsulates the construct satisfactorily and uses all the data present. The original rating values of the data on the fourteen scale items were therefore transformed in order to represent the distance from the midpoint of the scale. The following diagram illustrates the coding principle of the transformation.

1        2          3         4          5 original value

|_____|_____|_____|_____|

2        1          0         1          2 new value

Thus each respondent’s ERS score is calculated as his or her averaged score over all fourteen items in the ERS scale. These individual mean scores form a distribution, and the mean of this distribution is taken as the ERS score for a group.

Testing the Reliability and Validity of the ERS Measure

Greenleaf (1992) notes that any valid ERS measure should consist of diverse topics or different constructs (with the intention of maintaining a relatively content-free measure) and consequently a low correlation between items in the questionnaire might be expected. Therefore conventional tests of reliability are inappropriate.

Due to these limitations the method adopted here to test reliability is derived from the concept of test-retest. Instead of conducting two tests on the same sample, the sample from each country was split into halves randomly (random split generated by SPSS). The halves were then treated as two samples for which the ERS mean scores should not be statistcally different if the scale is reliable. This is the case here, where a t-test shows no significant difference (t=1.16, p=.246).

Assessing the validity of an ERS scale poses similar problems, in that the scale should be constructed to avoid underlying constructs being present, that would elicit specific answers to clusters of items. Thus, rather than taking the typical approach of conducting factor analysis in order to understand underlying constructs, in the present instance it is more appropriate to check inter-item correlations to ensure that a uniformly low level of correlation has been achieved (Greenleaf 1992). Pearson correlation analysis between each pair of items reveals uniformly low coefficients, ranging from .01 to .19, thus supporting the validity of the instrument.

TABLE 2

MEANS OF ERS SCORES BY COUNTRY

Individualism

In addition to the 14-item ERS scale, items to measure individualism, religiosity and tolerance of ambiguity were also included. (The tolerance of ambiguity scale was later removed from the study as it is was tested unreliable.) The seven-point, Likert-type scale used to measure individualism is drawn from Marshall and Gitosudarmo (1995) and Marshall, Dong and Lee (1994), and has been tested and found to be reliable in both an Asian and European context.

Although the Individualism scale has been found reliable in previous research it did not perform well here. The six items yielded a very low Cronbach’s Alpha (.31). Therefore, a Principle Component Factor Analysis, using a Varimax rotation, was conducted. Two factors emerged (eigenvalues 1.56 and 1.25), that were formed by three items measured with positive statements and three items with negative. As these factors do not correlate well, it was decided to use only the three positive items (which correlate to each other better than the three negative). The Cronbach’s Alpha on the three-item scale rose to .51, which just meets Peter’s (1979) minimum requirement of scale reliability for basic research.

The distribution is skewed and leptokurtic - hence three unequal groups were arbitrarily determined, that seem to match natural break points in the distribution, ranging from high, medium to low on individualism.

Religiosity

Religiosity was measured using a 6-item scale developed for the purposes of this study. Both objective aspects ("I worship frequently") and the more subjective aspects ("My religion has made me what I am today") of religiosity were included. These items correlate well, displaying a Cronbach’s Alpha of 0.88. Consequently, all six items were combined into a single variable, by taking the mean for each respondent. The distribution for the religiosity scores approximates normality (mean=2.89, std.dev=.541, Kurtosis -.201, skewness=-.202). The scale was divided into three equal groups to represent high, medium and low levels of Religiosity.

RESULTS

Overall Effect of Culture and Gender

A two-way analysis of variance to investigate the overall effect of Culture and Gender on ERS was conducted. Both independent variables display a significant main effect (F_(Culture)=17.73, p<.005; F_(Gender)=3.80, p<.005), but there is no interaction between the two variables (F_{(C x G)}=.843, p=.537). Hypothesis 2 is therefore rejected as there was no interaction between culture and gender.

However, there is a significant main effect for gender, where the men seem to extremitise more than the women. There are also significant between-Country differences in ERS (reported in Table 2) supporting Hypothesis 1.

Individuaism and Religiosity

An Analysis of Variance on individualism and religiosity as independent variables and ERS as the dependent variable was used to test Hypotheses 3 and 4. There are significant main effects for both the independent variables (F_religiosity=14.03, p<.005; F_{individualism}=4.45, p=.012) but there is no statistically significant interaction effect. An inspection of the relevant means shows that the effects are all in the direction hypothesized in H3 and H4 (see Figure 1). Thus respondents exhibiting increasing levels of religious intensity, and greater levels of individualism might all be expected to answer Likert-type scales more toward the extremes that around the middle, therefore hypotheses 3 and 4 are supported.

DISCUSSION

The overall drive of this research has been to identify the existence of an extremity response bias, and to seek a pattern to that bias among the seven countries from which samples were drawn. It was expected that such a pattern might show Asian subjects displaying more ERS than European, and that women might, in general, tend to extremetise their responses more than men. Furthermore, it was hypothesized, based upon previous research, that Eastern women in general were less likely to extremetise their responses than Western. First, there were no interactions but main effects were found. Women were found to extremitise less than men (although by t-test they were no less religious (t=-1.4, p=.159), and were no different with respect to individualism (t=.13, p=.894). It is possible that the results seen here are consistent with those quoted earlier (Crandall 1965; Pettigrew 1958; Wallach & Caron 1959; Wallach & Kogan 1959) where men do seem to use extreme responses more frequently than women when the judgements are affective and neutral in nature.

A pattern of extremitising did emerge, with the Moslem, Eastern countries of Indonesia and Malaysia showing the greatest bias, and Australia showing the least. The principal, and most interesting, finding of the research is that those individuals found to be highest in terms of extremetising their responses were also significantly more religious and more individualistic.

The identification of two variables that seem to relate to the bias does offer a rule of thumb to marketers. Thus respondents from Indonesia and Malaysia are not only statistically noted to be more religious than other countries (ANOVA Country by Religiosity has a significant main effect (F=49.70, p<.001), mean value of Religiosity for Indonesia and Malaysia 3.22 and 3.38 respectively; for Australia 2.72), but religion can also subjectively be seen to have a greater impact on the daily lives of the population there.

FIGURE 1

PLOTS OF EXTREMITY BIAS OVER THREE LEVELS OF INDIVIDUALISM AND RELIGIOSITY

The final word must serve as a warning for those academics intent on pursuing an etic approach to cross-cultural research. Not only must all the difficulties of designing such studies be overcome, but the interpretation of results (at least, where they concern scaled answers) must be undertaken in the knowledge that extremity biases do vary by culture and so casual comparison between cultures is rendered hazardous. It is of some concern, too, that ERS bias has not typically been considered in a large number of published cross-cultural studies that have used Likert-type scales, which casts a shadow over the continued acceptance of their findings.

This research has demonstrated an international pattern of scale response extremetising; many replications are clearly necessary before a clearer international picture can emerge. The same can be said of the effort to identify correlates of the extremitising tendency, in order to add to our facility to predict changes in ERS over time, and even, perhaps, to shed some light on possible causation. The most immediate challenge, however, is to develop a reliable scale to measure bias, or at least some guidelines for practitioners and academics who might want to append such ascale to their cross-cultural research.

REFERENCES

Adams, H. E. and I. A. Berg (1961), "Affective tone of test option choice as a deviant response", Psychological Report, 8 79-85.

Alpert, B. (1967), "Non-businessmen as surrogates for businessmen in behavioral experiments", Journal of Business, 40 203-207.

Bachman, J. G. and P. M. O’Malley (1984), "Yea-saying, nay-saying, and going to extremes: Black-white differences in response style", Public Opinion Quarterly, 48 491-509.

Berg, I. A. and J. S. Collier (1953), "Personality and group differences in extreme response set", Educational and Psychological Measurement, 13 164-169.

Berg, I. A. and G. M. Rapaport (1954), "Response bias in an unstructured questionnaire", Journal of Psychology, 38 475-481.

Block, J. (1965), The Challenge of Response Sets. New York: Appleton-Century-Crofts.

Borgatta, E. F. and D. C. Glass (1961), "Personality concommitants of extreme response set (ERS)", Journal of Social Psychology, 55 213-221.

Chun, K. and J. B. Campbell (1974), "Extreme response style in cross-cultural research", Journal of Cross-Cultural Psychology, 5 465-479.

Crandall, J. E. (1965), "Some relationships among sex, anxiety and conservatism of judgement", Journal of Personality, 33 475-481.

Crandall, J.E. (1973), "Sex differences in extreme response style: Differences in frequency of use of extreme positive and negative ratings", Journal of Social Psychology, 89 281-293.

Cronbach, L. J. (1946), "Response sets and test validity", Educational and Psychological testing, 6 475-495.

Greenleaf, E. A. (1992), "Measuring extreme response style", Public Opinion Quarterly, 56 328-351.

Hamilton, D. L. (1968), "Personality attributes associated with extreme response style", Psychological Bulletin, 69 192-203.

Heinemann, P. O. and M. Zax (1968), "Extremeness in evaluative responses to clinical test materials", Journal of Social Psychology, 75 175-183.

Hofstede, Geert (1980), Culture’s consequences. Beverly Hills, CA: Sage.

Hui, C. H. and H. C. Triandis (1989), "Effects of culture and response format on extreme response style", Journal of Cross-Cultural Psychology, 20 296-309.

Lentz, T. F. (1938), "Acquiescence as a factor in the measurement of personality", Psychological Bulletin, 35, 659.

Levitt, T. (1965), Industrial Purchasing Behavior. Boston: Harvard University.

Marshall, Roger and Indriyo Gitosudarmo (1995), Variation in the characteristics of opinion leaders across cultural borders. Journal of International Consumer Research, 8, no. 1., 5-22.

Marshall, Roger, Dong Xu and Christina K. C. Lee (1994), The development of basic values of a sub-culture: An investigation of the changing levels of individualism exhibited by Chinese immigrants to New Zealand. In Asia Pacific Advances in Consumer Research, J. A. Cote & S. M. Leong ed., Provo, UT: Association for Consumer Research, 1, 91-96.

Neuringer, C. (1961), "Dichotomous evaluations in suicidal individuals", Journal of Consulting Psychology, 25 445-449.

Peabody, D. (1962), "Two components in ipolar scales: Direction and extremeness", Psychological Review, 69 65-73.

Peter, J. P. (1979), "Reliability: A review of psychometric basics and recent marketing practices", Journal of Marketing Research, 16(February), 6-17.

Pettigrew, T. F. (1958), "The measurement and correlates of category width as a cognitive variable", Journal of Personality, 26 532-544.

Philips, L. D. (1977). Cultural differences in viewing uncertainty and assessing probabilities. In H. Jungermann, & G. D. Zeeuw (Ed.), Decision making and Change in Human Affairs. Amsterdam: D. Reidal.

Rorer, L. G. (1965), "The great response myth", Psychological Bulletin, 63 1-125.

Rundquist, E. A. (1950), "Item and response characteristics in attitude and personality measurement", Psychological Bulletin, 66 166-177.

Shaffer, T. R. and O Hara, B. S. (1995), "The effects of country of origin on trust and ethical perceptions of legal services", Service Industries Journal, 15 (2): 162-185.

Soueif, M. I. (1958), "Extreme response sets as a measure of intolerance of ambiguity", British Journal of Psychology, 49 329-333.

Stening, B. W. and J. E. Everett (1984), "Response styles in a cross-cultural managerial study", Journal of Social Psychology, 122 151-333.

Triandis, Henry C. (1972), The Analysis of Subjective Culture. New York: John Wiley & Sons.

Wallach, M. A. and A. J. Caron (1959), "Attribute criteriality and sex-linked conservatism as determinants of psychological similarity", Journal of Abnormal and Social Psychology, 98 43-50.

Wallach, M. A. and N. Kogan (1961), "Affective tone of test choice as a deviant response", Psychological Report, 8 79-85.

Wells, W. D., (1961) "The influence of yeasaying response style", Journal of Advertising research, 1 1-12.

Wright, G. N, L. D. Philips, P. C. Whalley, G. T. Choo, K. Ng, I. Tan and A. Wisudha (1978), "Cultural differences in probabilistic thinking", Journal of Cross-Cultural Psychology, 9 285-299.

Wyer, R. S. and D. E. Carlston (1979), Social Cognition, Inference and Attribution. New Jersey: Lawrence Erlbaum and Sons.

Zax, M. and Takahashi (1967), "Cultural influences on response style: Comparison of Japanese and American college students", Journal of Social Psychology, 71 3-10.

----------------------------------------