Measuring Prior Knowledge

Catherine A. Cole, University of Iowa
Gary Gaeth, University of Iowa
Surendra N. Singh, University of Iowa
ABSTRACT - This paper assesses the convergent, divergent and criterion validity of alternate methods of measuring prior knowledge. The context used to study the validity issue is the playing of board games. Evidence is found for all three types of validity for objective test and self-reported knowledge measures of prior knowledge. Usage measures appear to be more problematic.
[ to cite ]:
Catherine A. Cole, Gary Gaeth, and Surendra N. Singh (1986) ,"Measuring Prior Knowledge", in NA - Advances in Consumer Research Volume 13, eds. Richard J. Lutz, Provo, UT : Association for Consumer Research, Pages: 64-66.

Advances in Consumer Research Volume 13, 1986      Pages 64-66


Catherine A. Cole, University of Iowa

Gary Gaeth, University of Iowa

Surendra N. Singh, University of Iowa


This paper assesses the convergent, divergent and criterion validity of alternate methods of measuring prior knowledge. The context used to study the validity issue is the playing of board games. Evidence is found for all three types of validity for objective test and self-reported knowledge measures of prior knowledge. Usage measures appear to be more problematic.


Intuitively, we know that our current behavior is influenced by our prior knowledge of a given area. If we usually buy Sears brand power tools we tend to notice and recall a newspaper advertisement for a Sears power tool sale. This proposition as well as other about the effects of prior knowledge on consumption behavior have been empirically tested. For example, there is a large body of literature examining the impact of prior knowledge on recall and use of new information about products. (&e for example Brucks forthcoming and Johnson and Russo 1984). However, one of the major difficulties with this stream of research is the lack of consensus over appropriate methods of measuring prior knowledge.

The objective of this paper is to examine the convergent, divergent and criterion validity of alternate methods of measuring prior knowledge. The context used to study this issue was the playing of board games. One reason games were chosen was that pretesting indicated that there was considerable variance among undergraduates in prior experience with the board games, Monopoly and Life. Another reason was that intrinsically motivated behavior, such as game playing, has been underinvestigated in the consumer behavior literature (Holbrook et al., 1984)


First there is a need to develop a good working definition of prior knowledge. The definition we will use is particularly appropriate for this context and has been put forth by a group of psychologists working in the area (Chiesi, Spilich and Voss 1979). They view "knowledge of a domain as an understanding of its basic contents, as well as its goals, rules and/or principles." A review of the literature suggests that there are three approaches to measuring consumers' prior knowledge of a product class: tests of knowledge, self-assessed measures of knowledge and self-reported usage history.

Objective tests of knowledge are designed to assess how much a person knows about a particular domain. A person who receives a high score on the test is considered knowledgeable. Both psychologists and consumer behavior researchers have implemented this definition by administering tests to subjects to assess their knowledge of a particular product class. (See Bruck forthcoming, Spilich, Vesonder, Chiesi dc Voss 1979, Staelin 1978). The underlying assumption is that people with equivalent scores on the knowledge test should perform in similar ways on tasks such as recall of new information.

Another method is to use a self-assessed measure of knowledge. Here subjects are asked to rate how much they know about a particular domain. For example in a recent political science study political expertise was defined using a self-reported measure of knowledge, interest and participation (Fiske, Kinder and Larter 1983). Researchers in consumer behavior have also used self-assessed measures of knowledge (Johnson dc Russo 1984). The problem with these measures is that they may reflect generalized self-confidence more than any actual state of knowledge because people who are self confident may report more knowledge than those with less confidence.

Finally, some researchers have measured knowledge by assessing usage or Purchasing behavior. Here the assumption is that if you have used, purchased or sought out information on a product class then you have an increased level of prior knowledge. For example, Bettman and Park (1980) classified subjects based on responses to questions about whether or not the respondent had ever used, ever owned or ever searched for information about a microwave. The difficulty with this approach is that different types of experience could create different types of knowledge and thus affect behavior in different ways. For example, two researchers found that some types of experience (classroom training) had no effect on while other types of experience (interactive training) had an effect on the ability of judges to ignore irrelevant information (Gaeth and Shanteau 1984).


In this study we used two different board games: Monopoly and Life. Prior knowledge of each board game was assessed in the three different ways described above. ln addition we measured the emotional responses of subjects to the experience of playing each game. The main hypotheses is that measures of knowledge for the same board game should be related (convergent validity), while measures of knowledge for different board games should be unrelated (discriminant validity). In other words, the underlying proposition is that there is a distinct prior knowledge construct for each game and that each of the three measures of knowledge is an appropriate way to assess this construct.

An additional hypothesis is that because games are part of intrinsically motivated consumer behavior, people who have acquired a great deal of knowledge of a game should report high positive affect toward the act of playing the game (criterion validity). Previous research has suggested that the PAD scale, developed by Mehrabian and Russell (1974), can be used to characterize affective responses to intrinsically motivated behavior such as leisure activities, creativity and game playing (Holbrook et. al. 1984). Consequently it is expected that prior knowledge should correlate positively with scores on the PAD scale.

The key issues in the study are summarized in the following three hypotheses:

H1: Scores on self-assessed knowledge tests, usage questions and true/false knowledge tests should be positively correlated when the tests all relate to one particular board game (convergent validity).

H2: Scores on self-assessed knowledge tests, usage questions and true/false knowledge tests should not be correlated when the tests relate to two different board games (discriminant validity).

H3: All three measures of experience should correlate positively with scores on the PAD scale (criterion validity).



Forty-five undergraduate students completed the questionnaire in sessions conducted outside the classroom in groups of 8-10. The students were given class credit for participating in the study.

Content of questionnaire.

The following information was collected in the following order: self-reported knowledge of Monopoly, self-reported knowledge of Life, usage information for Monopoly, usage information for life, PAD scale information for Monopoly, PAD scale information for Life, test Of knowledge of Monopoly and test of knowledge of Life.

Self-reported knowledge measure. Fourteen Likert statements were administered to subjects to assess self-reported measures of knowledge. A sample item is "I know most of the rules in Monopoly." Both the Monopoly and Life scale contained seven items, with coefficient alpha of .834 and .933 respectively.

Usage. Usage of the two games was determined by two open ended questions. One for Monopoly and one for Life: How many times have you played Monopoly (Life) in the last three years?

Test. Objective knowledge was assessed through a true/false test. Subjects responded to 9 true false statements about the rules of Monopoly (e g., at the start of Monopoly each player is given $1500) and 9 true/false statements about the rules of Life (e.g., in the game of Life you must land on a red pay day space to collect your salary). Response categories were definitely true, probably true, probably false and definitely false. This knowledge measure was taken last because it might bias responses to other questions.

Emotional Response (PAD scale). Subjects were told to think about the last time they played Monopoly (Life) and place a mark closest to the point which represents their feelings about the prior experience of playing Monopoly (Life). Subjects then completed the Mehrabian and Russell (1974) PAD scale. The scale consists Of 18 items, 6 measure pleasure-displeasure, 6 measure arousal-nonarousal and 6 measure dominance-submissiveness. The form of the scale is a semantic differential. The coefficient alpha's and a sample item are reported in Table 1.




First consider the evidence for convergent validity. Prior knowledge was measured in three ways for two games. The six correlations of relevance here are reported in the multi-trait, multi-method matrix (Table II). Each one has the subscript C next to it. All are non-zero, suggesting that in fact self-reported measures of knowledge, usage experience and tests of knowledge all assess the same underlying construct.



Next consider the evidence for discriminant validity. This requires considering three criteria. First, the correlations between two different measures of the same variable should be larger than the correlation between that variable and any other variable which has neither trait nor method in common (Campbell and Fiske 1959). To test this, each of the 6 entries labelled C were compared statistically to each of the 6 entries labelled D. (The r to z transformation of R. A. Risher as discussed in Sachs 1982 page 427-432 was used). Of the 36 comparisons, all were in the predicted direction and 26 were statistically different from each other (a = .05). Consequently this condition is partially satisfied.

Additional evidence of discriminant validity is garnered by examining the coefficients labeled C in relation to the correlations labeled F. The correlations in C should be higher than those labeled F because correlations within a trait measured by different method should be higher than correlations between traits which have a method in common." Of the 18 comparisons made, 10 are statistically different in the predicted direction ( X = . 05, using the r to z transformation). Six of the nonsignificant comparisons involve the correlation between usage of monopoly and usage of life which is unexpectedly large and significant. This suggests that there is some method variance present in the usage question.

Finally additional evidence of discriminant validity comes from examining the pattern of correlations labeled (F) and (D). The correlations labeled F share common methods but not traits, while the correlations labelled D share neither method nor trait in common. The rank order of correlations should be the same in both group F and group D. In this case all these correlations are close to zero, except for the usage measure. Again, using the r to z transformation, we found that none of the correlations labelled c differed from each other (a = .05) or from zero. In addition, the same pattern was observed for correlations labelled D, except for the correlations of usage for the two games. Consequently, the third criterion for discriminant validity is only partially satisfied.

A final question concerns criterion validity. Do the measures of prior knowledge predict some criterion measure? To assess criterion validity, we analyzed the correlation between prior knowledge and responses to the PAD scale. These are reported in Table III.



One thing to note is that all the correlations are in the predicted direction. That is for the correlations that are significant there is a positive relationship between prior knowledge with a particular board game and reported feelings of pleasure, arousal and dominance. Because this is a relatively unexplored area, one way of looking at the data is to compare the effects of prior knowledge on the experience of playing Monopoly with the effects of prior knowledge on the experience of playing Life. For both games self-reported knowledge correlated with self-reported feelings of pleasure and dominance. Usage of Monopoly correlated with feelings of arousal and dominance, but usage of Life correlated with feelings of pleasure and dominance. The scores on the test for Monopoly correlated positively with the reported pleasure from playing monopoly. This relationship did not hold when Life was the game in question.

Consequently, the following generalizations can be made regarding criterion validity: (1) experience with playing a game is positively related to reported feelings of affect associated with playing the game; (2) all measures of prior knowledge do not have the same effect on feelings of affect: (3) the relationship between prior knowledge and reported affect seems to be somewhat idiosyncratic to the particular game in question.


Evidence was found for convergent validity of all three measures of prior knowledge. In other words objective tests of knowledge, self-reported measures and usage measures of the same trait correlated positively.

Evidence was also found for discriminant validity. However, the measure of Monopoly usage correlated with usage of Life. This result may indicate one of two things. First it may be that respondents who play one board game also tend to play other board games This explanation, however, is weakened because the other measures of prior knowledge of Monopoly did not correlate with measures of prior knowledge of Life. An alternate explanation is that the usage measured suffered from psychometric flaws. For example, it is possible that respondents tended to overestimate or underestimate how many times they had played both games because the usage questions were asked at the same time and with the same question.

The evidence for criterion validity was less straightforward. All measures of knowledge that were significantly related to affect were in the predicted direction. In other words, when knowledge was related to a dimension of affect it tended to enhance the positive nature of the affect. However, not all measures of knowledge had the same effect on dimensions of the PAD scale. In particular for both games, self-reported knowledge is most clearly related to self-reported affect. However, objectively assessed knowledge seems to be least related to scores on the PAD scale. This pattern could arise for one of two reasons. It is possible that although the different measures of knowledge are clearly related, they do not measure identical constructs. Consequently these different measures of knowledge influence and are influenced by other constructs in unique ways. Beattie (1982), for example, suggest that the type of advertising copy one should use depends on whether the target audience has high knowledge as assessed by product usage or by product knowledge.

Alternatively, it is possible that the strong positive relationship between self-reported knowledge and self-reported affect reflects the fact that both measures are self-report scales.

There is a growing body of literature examining the impact of prior knowledge on all aspects of consumer behavior. In order to synthesize this literature, we need to know more about the psychometric properties of the alternate measures of prior knowledge. This paper represents an initial effort in that direction. One limitation of this research is that the psychometric properties of these measures were tested on just one product class. It makes intuitive sense that a person who frequently plays Monopoly will be a high scorer on a test of the rules of monopoly. However, it also makes intuitive sense that a person who frequently listens to a stereo system may not be a high scorer on a test of the technical attributes of a stereo system. Consequently, by choosing a product that one has to know a lot about in order to use, we may have obtained results that are not generalizable across product classes.

In conclusion, the data in this study suggest that one can generalize with caution across measures of knowledge. This seems to be especially true for objective test and self-reported measures of knowledge. Usage measures of knowledge need to be refined and used with care.


Beattie, A. (1983) "Product Expertise and Advertising Effectiveness" in R. Bagozzi and A. Tybout (eds), Advances in Consumer Research, Vol. X. (Ann Arbor. Association for Consumer Research) 581-584.

Bettman, J. and Park, C. W. (1980) "Effects of Prior Knowledge and Experience and Phase of Choice Process on Consumer Decision Processes A Protocol Analysis," Journal of Consumer Research, 7:234-247.

Brucks, M. "The Effects of Product Class Knowledge of Information Search Behavior," (Forthcoming, Journal of Consumer Research).

Campbell, D. T. and D. W. Fiske {1959) "Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix," Psychological Bulletin, 56:81-105.

Chiesi, H, Spilich, G. & Voss, J. T. (1979) "Acquisition of Domain Related Information in Relation to High and Low Domain Knowledge," Journal of Verbal Learning and Verbal Behavior, 18:257-273.

Churchill, G. (1979) "A Paradigm for Developing Better Measures Of Marketing Constructs," Journal of Marketing Research, 16(1):64-73.

Fiske, S., Kinder, D., and Larter, W. (1983) "The Novice and the Expert: Knowledge Based Strategies in political Cognition," Journal of Experimental Social Psychology, 19:381-400.

Gaeth, G. and Shanteau, J. (1984) "Reducing the Influence of Irrelevant Information on Experienced Decision Makers," Organizational Behavior and Human Performance, 33:263-282.

Holbrook, M., Chestnut, R., Olivia, T. and Greenleaf, E. (1984) "Play as a Consumption Experience: The Roles of Emotions, Performance and Personality in the Enjoyment of Games," Journal of Consumer Research 11(2):728-739.

Johnson, E. and Russo, J. (1984) "Product Familiarity and Learning New Information," Journal of Consumer Research 11(1):542-550.

Mehrabian, A. and J. A. Russell (1974) An Approach to Environment, Cambridge, MA, The MIT Press.

Morrison, J. E. "Methods of Empirically Deriving the Structure of Memory for Procedures," Journal of Applied Psychology, 69(3):473-481, August 1984.

Park, C. and V. Lessig (1981) "Familiarity and Its Impact on Consumer Decision Biases and Heuristics," Journal of Consumer Research, 8:223-230.

Russo, J. and Johnson, E. (1980) "What Do Consumers Know About Familiar Products," in Jerry C. Olson (Ed) Advances in Consumer Research, Vol. VII. Ann Arbor: Association for Consumer Research, 417-423.

Sach, L (1982) Applied Statistics, New York, Springer Verlag.

Spilich, Vesonder, Chiesi dc Voss (1979) "Text Processing of Domain Related Information for individuals With High and low Domain Knowledge,' Journal of Verbal Learning and Verbal Behavior 18:275-290.

Staelin, R. (1978) "The Effects of Consumer Education on Consumer Product Safety Behavior," Journal of Consumer Research, 5:30-40.