The Measurement of Declarative Knowledge

Peter A. Dacin, Univercity of Toronto
Andrew A. Mitchell, University of Toronto
ABSTRACT - Models of declarative memory and procedures for measuring their content and structure are discussed. Then, the results of a study which applies these procedures are presented and their implications discussed.
[ to cite ]:
Peter A. Dacin and Andrew A. Mitchell (1986) ,"The Measurement of Declarative Knowledge", in NA - Advances in Consumer Research Volume 13, eds. Richard J. Lutz, Provo, UT : Association for Consumer Research, Pages: 454-459.

Advances in Consumer Research Volume 13, 1986      Pages 454-459

THE MEASUREMENT OF DECLARATIVE KNOWLEDGE

Peter A. Dacin, Univercity of Toronto

Andrew A. Mitchell, University of Toronto

ABSTRACT -

Models of declarative memory and procedures for measuring their content and structure are discussed. Then, the results of a study which applies these procedures are presented and their implications discussed.

INTRODUCTION

A considerable amount of research in cognitive psychology has recently been directed at understanding the problem solving of experts within a domain (see for example; Chi, Glaser and Rees 1982; Voss, Greene, Post and Penner 1983). Initial work in the area focussed on the strategies used by experts in problem solving, however, few quantitative differences in the strategies of experts and novices were found. For instance, both chess experts and novices were found to have similar breadth and depth of search in planning chess moves (de Groot 1965). These findings indicate that it is the quality instead of the quantity of search that define expertise in decision making. This suggests that more effort should be directed at understanding what knowledge experts have within a domain.

As a result of these findings, knowledge is now recognized as one of the most important variables which affects behavior. where previously researchers directly compared the effects of various treatments on behavior, a more common approach now is to first examine the treatment's effect on memory and then to see whether the effects on memory are related to subsequent differences in behavior. This approach has resulted in a growing body of literature which indicates that knowledge has a strong effect on how individuals process new information (e.g., Graesser and Nakamura 1982) and solve problems (e.g., Chi, Glaser and Rees 1982)

In consumer behavior, a similar shift in interest has occurred. For instance, recent research examining individual level advertising effects has used different measures of memory to explain these effects (e.g., Beattie and Mitchell 1985). In addition, a number of recent studies have examined the effect of decision strategies on memory (e.g., Biehal and Chakravarti 1983) and the effects of consumer knowledge about a product category on information search (e.g. Bettman and Park 1981; Brucks 1984). However, in marketing as in cognitive psychology, most of the research examining the effect of knowledge on various information processing activities has generally viewed knowledge within a domain as unidimensional.

Knowledge within a domain, however, is most likely multidimensional. Consequently, in order to obtain a better understanding of how knowledge affects various information processing activities, we must first obtain finer grained measures of knowledge and its organization in memory. However, as we point out in this paper, these measures should be based on a theory of memory.

Given this problem, it is not surprising to find little systematic effort directed at actually measuring differences in the content and organization of information within a domain between individuals (for exceptions, see Chi and Koeske 1983; Chi, Glaser and Rees 1982; Scott, Osgood and Peterson 1979). Although the outlines of a theory of memory are available, it is still not developed well enough to provide strict guidelines for measuring knowledge. Consequently, developing systematic procedures for measuring knowledge is an arduous and time consuming task. It literally involves feeling your way in a largely unknown area.

The following paper represents an attempt to built on the existing literature in cognitive and educational psychology, social cognition, and consumer research by discussing models of memory and various procedures, based on these models, for measuring knowledge. Furthermore, we will also present and discuss the results of a study in consumer research which demonstrates the application of some of these procedures. First, however, we will discuss models of memory.

MODELS OF MEMORY

The usual distinction between long term memory (LTM) and short term memory (STM) will be used in this paper (e.g. see Horton and Mills 1984), however, we will only be concerned with measuring information in LTM. Here, LTM is defined as that subcomponent of memory which is permanent, virtually unlimited in storage capacity, and well organized. Although LTM is often represented as a complex system (Anderson and Bower 1973), a simplified representation is often sufficient for a task such as we are undertaking (e.g., Shavelson and Stanton 1975).

Also, it should be noted that knowledge within a domain can be represented in both procedural and declarative form (Anderson 1983), however, we will only be concerned with declarative knowledge here. Declarative knowledge consists primarily of the facts that are known about a particular domain while procedural knowledge represents the algorithms and heuristics that operate on these facts. A current issue in psychology is whether all our knowledge is represented in either declarative or procedural form. Here we will take the more general position that knowledge is represented in both forms

DECLARATIVE KNOWLEDGE

The most generally accepted model for representing declarative knowledge in LTM is a semantic network (Wickelgren 1981). A semantic network is a node-link structure in which concepts are represented as nodes that are linked together according to a defined set of relationships (for a discussion of nodes see Rumelhart et al. 1972: for the various relationships between nodes see Anderson 1976; Norman and Rumelhart 1975). Although, this view of LTM results in a static, though manipulable form of memory, it must be acknowledged, that for many purposes, a dynamic model may be more appropriate.

The retrieval of information from a semantic network involves a process of spreading activation (Anderson 1983). Recall of the contents of a node will occur when the amount of activation at that particular node exceeds some threshold level. Which notes then become activated depends on the strength of the association between the activated node and linked nodes. Those nodes that are most strongly associated are the most likely to be activated.

Declarative memory is, also, frequently divided into episodic (knowledge of events we have experienced) and semantic (generalized knowledge) memories (Tulving 1983). Although initially hypothesized as two separate memory systems, there exists evidence to the contrary (e.g. Anderson and Ross 1980; McCloskey and Santey 1981). In this paper, we will assume that episodic information may be linked to semantic knowledge in some cases (e.g., Reiser, Black and Abelson 1985), while in other cases it may be located in a separate memory system.

Finally it should be noted that it is also currently believed that much of our generalized knowledge is organized into packets of information. A number of different terms have been used to refer to these information packets. These include schema (Alba and Hasher 1983), scripts (Abelson 1981; Schank and Abelson 1977), frames (Minsky 1975), prototypes and exemplars (Cantor and Mischel 1979; Rosch 1978), and MOP's and TOPS's (Schank 1980. 1981).

Briefly, schema and frames are generic terms for information packets. Scripts contain our generalized knowledge about the temporal sequence of events which occur within an event (such as visiting a restaurant). A prototype is an abstraction of a particular class of objects, while an exemplar is a specific instance that is considered representative of a set of objects. MOP's (memory organizational packets), which are similar to scripts, contain generalized knowledge about general events, however, they also contain linkages to specific events when the events do not meet expectations. TOP's (thematic organizational packets) represent higher order organization of events which are used to identify instances of a general theme.

It is also important, to recognize the major competitor to the network model of memory, the feature model (Smith, Shoben and Rips 1974). Here, members of a particular category are located in a multidimensional space whose dimensions are the defining features of the objects. In this model, category members have both defining features, which are common to all the objects within the category, and characteristic features which are unique to a subset of members. Scott (1969) used a similar model to develop procedures for measuring the relationship between objects within a domain. He will refer to the latter model in this paper, however, it should be noted that although the network and feature models are conceptually different, Hollan (1975) demonstrated their mathematical equivalence.

MEASUREMENT OF DECLARATIVE KNOWLEDGE

In measuring declarative knowledge, it is important to make the distinction between structure and content (e.g., Scott, Osgood and Peterson 1979). Although a number of other dimensions of both structure and content of knowledge have been suggested (see for example, White and Gunstone 1980),the general distinction remains the most widely accepted and will be used for the purposes of our paper. Structure refers to how information within a domain is organized in memory. In a network model of memory, examples of different measures of structure might be the number of independent knowledge structures (e.g., structures with no links between them), the number of associations linked to a particular concept or the number and pattern of linkages between concepts within a structure (e.g., Chi and Koeske 1983). In addition, it may also be important to consider the pattern of linkages with other structures in memory (White and Gunstone 1980). In a feature model, examples of different measures of structure might be the number of dimensions or attributes associated with each object and the extent to which the same attributes are associated with each object.

The content of knowledge refers to the type of information stored at the nodes in memory. For instance, some individuals may have specific events associated with different brands in a product class while other individuals may have specific characteristics associated with the brands.

There are a number of different procedures that may be used for measuring either the content or the structure of knowledge within a domain. For a review, the reader is referred to Mitchell (1982) and Mitchell and Chi (1985). These procedures include paper and pencil tests, elicitation and word association, questionnaires which require the subject to evaluate objects and their attributes, tasks requiring the retrieval of information, response times, and scaling procedures. Most of these procedures, however, are capable only of measuring content or structure, but not both. One notable exception is the elicitation/word association procedure (e.g., Olson and Muderrisoglu 1979; Shavelson 1974). Under this procedure, subjects are given key terms from the domain to be tested and are asked to tell the experimenter everything that comes to mind or everything that is related to that term.

Deese (1965) was among the first to use the word association test as a mesas for examining knowledge within a domain. He defined the associations to a given stimulus probe (i.e. node in memory) as the responses obtained from that probe. He argued that although this was only a subset of all the associated concepts, it was the largest possible subset obtainable by any single technique. In addition, Deese has argued that the word association procedure comes closer than any other technique to being context free, and consequently is the most appropriate procedure for studying organization in memory. Finally, comparisons of the word association procedure with other procedures for measuring the structure and content of knowledge (e.g., paper and pencil tests) indicate substantial convergent validity between procedures (Shavelson and Stanton 1975; Preece 1976).

As mentioned previously, the major advantage of the elicitation and word association approaches is that they provide the data necessary for obtaining measures of both content and structure. Measures of the content of knowledge can be obtained by coding the elicitations into different categories, while measures of the structure of knowledge can be obtained if we assume that whenever one concept elicits or is associated with a second concept, a link between these two concepts exists in memory. Once these links are determined, we can then examine the structure in terms of pattern of links (Chi and Koeske 1983).

In addition, this procedure allows responses to be analyzed in several ways. For example, the degree of association between concepts within a domain can be calculated by the degree of overlap of the responses between concept probes (Preece 1976; Shavelson 1972).

The major problem with this approach is that we do not know exactly what probes or words to use to measure a subject's knowledge in a particular domain. Theoretically, we should probe each schema that is used to store information within that domain, however, currently our theories of memory are not precise enough to tell us which schemata individuals may have in a domain. It is also not clear how many concepts will be linked within a particular schema. If a schema contains many concepts, more probes should be used to elicit these concepts. This may be accomplished by increasing the depth of probing. For instance, after a subject has initially provided all the associations with the key word, second level probes could be used. These probes use both the key word and one of the associations given by the respondent. When subjects have complex schemas for particular concepts, these second level probes or even third level probes may be necessary to get at all this knowledge.

Second, previous research has indicated that the reliability of these procedures, while acceptable, is somewhat modest. For example, Olson and Mudderisoglu (1979) found that the same probe given at two different points in time resulted in the elicitation of 50 to 60% of the same concepts. High reliabilities for this procedure cannot be expected because according to spreading activation models, the recall of a particular node in memory is the result of a probabalistic process. However, it still needs to be determined if these reliabilities are high enough for this procedure to be useful.

Third, if the researcher is interested only in what information is stored in memory, he or she must be careful that the subject provides only the information that is activated from the memory probe and does not attempt to construct new information. In order to increase the likelihood that the construction of new information does not occur, the subject should be kept in a passive state during this procedure.

A final and related problem is that contest frequently seems to have a strong effect on information retrieval. This has been demonstrated in a number of experiments (e.g., Bower 1981). In word association techniques, however, it has been argued that as long as the major contextual variables are held constant (i.e., setting, interviewer, probes, etc.) the technique can be considered to be almost contest free (Deese 1965).

EXAMPLE

In this section we will discuss the use of both elicitation procedures (based on the semantic network model) and questionnaires (based on the feature model) to examine differences in the content and structure of declarative knowledge within a domain between high and lou knowledge individuals. The purpose of this example is to provide an illustration of the use of these procedures and an indication of the type of results that can be obtained.

Method

Domain: The domain of motorcycles was selected for a number of reasons. First, there appears to be a wide variance in the amount of knowledge that individuals have about motorcycles. Individuals seen to be either very knowledgeable about motorcycles or have very little knowledge. Second, motorcycles have distinct product types allowing for examination of these types as a basis for organizing information in memory. Third, motorcycles have a fairly large number of attributes, different manufacturers and models so that the difference in the content and structure of knowledge between high and low knowledge individuals would be evident.

Subjects: The subjects were 24 individuals associated with an eastern university in the United States. They were recruited from classrooms and from signs placed on campus. To insure that the subjects differed in their knowledge with respect to motorcycles, the subjects were first prescreened based on their self-reported knowledge about motorcycles. Only subjects that reported that they were either very knowledgeable about motorcycles or had little knowledge about motorcycles were selected. Of the 24 subjects used in this study, twenty were men and four were women. Their ages ranged from 18 to 28.

Memory probes: As mentioned earlier, in order to design the memory probes, we must first determine the different schemata that individuals have containing information about that domain. Since there has been virtually no research on how consumers organize their knowledge about a particular category, the following assumptions had to be made. First, it was assumed that there would be a schema for the product class, so our first memory probe was the word "motorcycle". Since it was believed that individuals may have very complex knowledge structures for motorcycles, second and third level memory probes were also w ed.

It was also hypothesized that individuals would probably have separate knowledge structures for the different brands, models and types of motorcycles of which they were aware. Therefore, we asked each subject to name all the different motorcycle brands, models and types that he or she could think of and then conducted separate probes for each brand, model and type that was mentioned.

Finally, we hypothesized that individuals may have separate knowledge structures for different components (e.g. motorcycle engine) and performance characteristics (e.g. handling) of motorcycles. Consequently, separate probes were conducted for different component and performance characteristics of motorcycles.

Procedure: Three different sessions were used for collecting data from each subject, however, only the first two sessions are of concern in this example. These sessions took place approximately two weeks apart. In the first session, a questionnaire containing a number of different self-report measures of knowledge that have appeared in the literature (e.g., questions concerning ownership of motorcycles), measures indicating the amount of exposure to information about motorcycles (e.g., frequency of reading motorcycle magazines) and demographic information was given to the subjects.

After completing this questionnaire, the subjects were then given a series of memory probes. Answers were recorded on a tape recorder for later transcription and analysis .

In the second session, approximately two weeks later, the subjects were given additional memory probes and a series of questionnaires to obtain the measures of memory structure proposed by Scott (1979). Except for minor modifications due to the difference in topics, these questionnaires were virtually identical to the ones in Scott, Osgood and Peterson (1979). Nest, the subjects were given a series of categorization tasks and a vocabulary quiz on a number of different motorcycle terms. Finally, a list of the five most important performance measures of motorcycles (e.g. acceleration) were given to the subject and they were asked to state which characteristics of motorcycles affected these performance attributes.

Dependent variables: One set of dependent variables that was developed included the self report measures of knowledge, number of brands, models and types mentioned, number of elicitations from each memory probe and scores on the vocabulary and the characteristic attribute tests. This set of variables was w ed to define level of expertise.

In order to measure the content of knowledge, a coding scheme with twenty sis different categories was developed for the elicited concepts These categories contain a number of broad groupings. Two of these groupings were whether the knowledge was specific to the product or related to the product. Examples of product specific knowledge include: performance (acceleration etc.), general characteristics (size, weight etc.), specific characteristics (seat, tires etc.), brands, models, types, maintenance, and quality.

The second category consists of those variables which, although not specific to a motorcycle are objects or phenomena related to motorcycles. These include events (involving strangers, read about), personal experiences, other's experiences (of known people, friends, relatives etc.), people associated with motorcycles (actors, Hell's Angels etc.), places and objects encountered while riding (bridges, potholes etc.), type of riding (racing, highway etc.), procedures involved in riding (shifting weight etc.), safety, advertisements, company or country, and motorcycle clothing.

In addition to these two categories, a third category was created which consists of personal statements. This includes feelings/emotions, evaluative statements (I like that etc.), general statements (some people enjoy it), personal thoughts/philosophy (I think it is important to, I wonder what etc.), and images in mind (I Picture ..., I can see, etc.).

Finally, a category for comparisons was developed which classified a comparison either as direct (comparing one motorcycle to another) or indirect (comparing a motorcycle to something else ... car, bicycle etc.). A preliminary measure of the reliability between coders for the coding scheme was .81.

Scott's (1969) questionnaires provide a number of different measures of the structure of knowledge. These include complexity (the number of associations an individual has about an object), evaluative centrality (the degree to which evaluative attributes are more central than attributes of neutral valence), image comparability (degree to which the attributes associated with the objects are the same) dimensionality (the number of independent dimensions or attributes required to represent the group of objects), and affective - evaluative consistency (associations between the evaluation of an object and the evaluation of the associations with that object) (Scott, Osgood and Peterson 1979). The questionnaire also provides a measure of the number of motorcycles listed.

RESULTS

An examination of the correlations between the first set of dependent variables indicates that most of these correlations were quite low (around .30), however, a number of the measures were quite highly correlated (around .75). These highly correlated measures included Rating of knowledge, No. of Motorcycles listed, and the scores on the Vocabulary Quiz. The scores for these variables were standardized over subjects and the sum of the standardized scores was used as a measure of the subject's expertise. An examination of the subjects' scores on this variable indicated that they fell into four groups of approximately equal size. These groups were labeled novices, semi-novices, semi-experts and experts.

Differences in the Content of Knowledge: In order to examine the differences between these groups on the different content categories, a one way analysis of variance was used. Both the number and percentage of statements elicited for each category were analyzed.

An analysis of the number of statements elicited indicates that, overall, significantly more statements were elicited from the experts than from the novices. Although this was expected, it provides an indication of the complexity of the knowledge structure of the experts. Significant differences were also found in the number of statements elicited for eighteen of the twenty six categories. Fourteen of these categories also displayed 8 significant linear trend. For all but one of these categories, the number of statements elicited increased with expertise. Six of the categories that increased with expertise were from the motorcycle specific grouping. These were: general characteristics, specific characteristic, brands, motels, types, and quality. The other fire categories where similar significant differences were found were company/ country, procedure, feeling/emotion, specific evaluation, and generalized statement. The only category where the number of elicitations decreased with expertise was specific evaluations. Interpretation of the category specific evaluation would indicate that novices tent to respond to a probe with an evaluation (i.e., "It's good," "I don't like it," etc.) more frequently than experts.

In a second analysis; the percentage of each category was analyzed using one-way analysis of variance. First, however, an arcsin transformation was applied to equalize the within cell variances. Significant differences in the means between groups or a significant linear trend was found for 12 of the twenty six variables developed from the elicitations. These results also indicate considerable differences in the content of knowledge between experts and novices.

Of particular interest are the differences across the groups in terms of the three categories of content variables. Again, with the exceptions of performance and maintenance, the differences between groups in the motorcycle specific category were all significant and demonstrated a linear trend increasing with expertise. These results indicate that as expertise increases, an increasing percentage of an individual's motorcycle knowledge is specific knowledge. Other variables demonstrating the same significant linear trend were company/country of origin, procedures, and direct comparisons.

Significant linear trends decreasing with expertise occurred for the following variables in the other two categories: events, riders/people associated with motorcycles, advertisements, indirect comparisons and personal thought/philosophy. This suggests that novices tend to have more knowledge based on advertisements, newspaper stories, and people they associate with motorcycles (especially Hell's Angels). Also, as a result of less motorcycle specific knowledge, novices tend, when making comparisons, to make them with things with which they are more familiar (cars, bicycles) than to other motorcycles.

Differences on measure of structure: A similar pattern emerges with the Scott measures. Experts, for instance, could name a significantly greater number of motorcycles than novices. The measure of complexity was also significantly different between groups and demonstrated a significant linear trend increasing with expertise. This indicates that experts have many more associations with the different motorcycle brands, types and models than do novices. The dimensionality index was also found to increased between novices and experts indicating that experts used more dimensions in thinking about motorcycles. Perhaps the most interesting finding is an increase in affective evaluative consistency between novices and experts. This measure is based on the correlation between a subject's evaluation of each motorcycle and the sum of the evaluations of each characteristic associated with the motorcycle. For novices, this measure was actually negative, while for experts, this measure is positive and reasonably high. This suggests that experts based their evaluations of the motorcycles on careful consideration of the characteristics of the motorcycles, while novices way have based their evaluations on external sources, such as friends or advertisements, a result which was also apparent in the analysis of elicitations.

Finally, significant differences and a significant linear trend was also found on the image comparability measure. The linear trend decreased with expertise which indicated that experts were the most likely to have unique characteristics associated with different brands and models of motorcycles.

Differences in knowledge structures: In order to examine the differences in the structure of knowledge between experts and novices, we constructed a network representation of each subject's knowledge based on the memory probes. In constructing a network, a linkage was made between the node representing the memory probe and the nodes representing the various elicitations to the probe.

For novices we generally find that all of the elicited information is interconnected. That is, responses to one probe were often found in responses to other probes. This seemed to occur because there were only a few basic ideas about motorcycles in the structures, and almost all of the probes would tap into one or more of these ideas. For example, one novice's knowledge about motorcycles consisted of the experience of riding around on the back of her boy friend's Harley Davidson, television advertisements about motorcycles, knowledge about how powerful and dangerous motorcycles are, and a memory of a high school acquaintance was badly injured in a motorcycle accident

In contrast, experts generally had very complex knowledge structures and had developed separate knowledge structures for different types of knowledge about motorcycles. For instance, the knowledge structure of experts, for the most part, revealed few if any direct linkages between the elicitations from a particular probe and all the other probes. However, there may exist superlinks between structures which were not uncovered with the particular probes which were used.

What is particularly interesting about some of the expert structures is their hierarchical organization. For example, the motorcycle engine probe given to one of the experts elicited different characteristics of motorcycle engines (e.g. 4 cylinder, fuel injected). Connected to these characteristics was information about how these characteristics affect performance.

Experts also demonstrated a tightly knit structure relating brands, models and types of motorcycles. For instance, connected to each brand were generally found the specific models for that brand and these models, in turn, were linked to the different types of motorcycles. Consequently, the expert can access specific models of motorcycles either through the brand or through types of motorcycles.

DISCUSSION

Overall, the patterns found across the two different procedures seems to indicate important differences in the content and structure of motorcycle knowledge between high and low knowledge individuals. Especially noteworthy was the finding that the proportion of motorcycle specific elicitations increased with expertise while the proportion of motorcycle related elicitations decreased. in addition, experts were found to have more associations with each brand, model or type of motorcycle, use more dimensions in thinking about motorcycles and had greater affective evaluative consistency.

Although still at preliminary stages, the procedure developed for identifying the content and structure of knowledge appears to be a useful tool for the consumer researcher. The ability to identify the concepts associated with a product and the relationships between these associations provide researchers with a clearer picture of knowledge within a particular product class across subjects. They may also be useful as dependent measures, for example, the ability to map knowledge prior to and after the acquisition of product knowledge will give the researcher a better understanding of how the subjects integrated the newly acquired knowledge into their prior knowledge structures.

We have, however, also discussed a number of problems with the approach. These include the reliability of the measures and whether or not these procedures tap all the knowledge within a product domain. It also remains to be seen whether our coding schema for the elicitations will be useful for other product classes. Future research should address these issues so that reliable and valid procedures will be available for accessing the content and structure of product class knowledge..

REFERENCES

Abelson, R.P. (1981) "Psychological Status of the Script Concept," American Psychologists 36, 715-729.

Alba, J.w. and L. Hasher (1983) "Is Memory Schematic?" Psychological Bulletin, 93, 203-31.

Anderson J.R. (1983) "A Spreading Activation Theory of Memory," Journal of Verbal Learning and Verbal Behavior, 22, 261-295.

Anderson J.R. (1976) Language, Memory and Thought, Hillsdale, NJ: Lawrence Erlbaum Associates.

Anderson, J.R. and G.R. Bower (1973) Human Associative Memory, Washington, DC: Winston.

Anderson, J.R. and B.R. Ross (1980) "Evidence Against a Semantic-Episodic Distinction," Journal of Experimental Psychology: Human Learning and Memory, 7, 66-71.

Beattie, A.E. and A. A. Mitchell (1985) "The Relationship Between Advertising Recall and Persuasion: An Experimental Investigation," in L.F. Alwitt and A.A. Mitchell (Eds.) Psychological Processes and Advertising Effects: theory, Research and Application Hillsdale, NJ: Lawrence Erlbaum Associates

Bettman, J.R. and C.W. Park (1980) "Effects of Prior Knowledge and Experience and Phase of the Choice Process on Consumer Decision Processes: A Protocol Analysis," Journal of Consumer Research, 7, 234-248.

Biehal, G.J. and Chakravarti, D. (1983) "Information Accessibility as a Moderator of Consumer Choice," Journal of Consumer Research, 10 (1). 1-14.

Bower, G.H. (1980) "Mood and Memory," American Psychologist, 36, 129-148.

Brucks, M. (1984) "The Effects of Product Class Knowledge on Information Search Behavior," Journal of Consumer Research, 12, 1-16.

Cantor, N. and W. Mischel (1979) "Prototypes in Person Perception," in L. Berkowitz (Ed.) Advances in Experimental Social Psychology, Vol. 12. New York: Academic Press.

Chi, M.T.R. and R. Koeske (1983) "Network Representation of a Child's Dinosaur Knowledge," Developmental Psychology, 19, 29-39.

Chi, M.T.H., R. Glaser and E. Rees (1982) "Expertise in Problem Solving," in R. Sternberg (Ed.) Advances in the Psychology of Human Intelligence, Vol. 1, Hillsdale, NJ, Lawrence Erlbaum Associates.

de Groot, A. (1965) Thought and Choice in Chess, The Hauge Mouton.

Deese, (1965) The Structure of Associations in Learning and Thought. Baltimore: The John Hopkins Press.

Geeslin, W.E. and R.J. Shavelson (1975) "An Exploratory Analysis of the Representation of a Mathematical Structure in Students' Cognitive Structures," American Educational Research Journal, 12, 21-39.

Graesser, A. and G.V. Nakamura (1982) "The impact of Schema on Comprehension sat Memory. n In G.R. Bower (Ed.) The Psychology of Learning and Motivation, Vol. 17, New York: Academic Press.

Hollan, J.D. (1975) "Features and Semantic Memory: Set Theoretic or Network Model," Psychological Review, 82, 154-55.

Horton, D.L. and C.B. Mills (1984) "Human Learning and Memory," Annual Review of Psychology, Palo Alto, CA: Annual Reviews.

Kintsch, W. (1972) "Notes on the Structure of Semantic Memory." In E. Tulving and R. Donaldson (Eds.) Organization of Memory, New York: Academic Press.

McCloskey, M. and J. Santee (1981) "Are Semantic Memory and Episodic Memory Distinct Systems?" Journal of Experimental Psychology: Human Learning and Memory, 7, 66-71.

Minsky, H.A. (1975) "A Framework for Representing Knowledge," in P.H. Winston (Ed.) The Psychology of Computer Vision, New York: McGraw-Hill.

Mitchell A.A. (1982) "Models of Memory: Implications for Measuring Knowledge Structures," in A. Mitchell (Ed.) Advances in Consumer Research, Vol. 9, Ann Arbor: Association for Consumer Research, 45-51.

Mitchell, A.A. and M.T.H. Chi (1985) "Measuring Knowledge within a Domain," to appear in P. Nagy (Ed.) The Representation of Cognitive Structures, Toronto: The Ontario Institute for Studies in Education.

Norman, D.A. and D.E. Rumelhart (1975) Explorations in Cognition, San Francisco: W.H. Freeman and Co.

Olson, J.C. and A. Muderrisoglu (1979) "The Stability of Responses Obtained by Free Elicitations: Implications for Measuring Attribute Salience," in W. Wilkie (Ed.) Advances in Consumer Research, Vol. 6, Ann Arbor: Association for Consumer Research, 269-275.

Preece, P.R.F. (1976) "Mapping Cognitive Structures: A Comparison of Methods," Journal of Educational Psychology, 68, 1-8.

Reiser, B.J., J.B. Black and R.P. Abelson (1985) "Knowledge Structures in the Organization and Retrieval of Autobiographical Memories," Cognitive Psychology, 17, 89-137.

Rosch, E. (1978) "Principles of Categorization," in E. Rosch and B.B. Lloyd (Eds.) Cognition and Categorization, Hillsdale, NJ: Lawrence Erlbaum Associates.

Schank, R.C. (1982) Dynamic Memory: A Theory of Reminding and Learning in Computers and People. Cambridge, MA: Cambridge University Press.

Schank, R.C. (1980) "Language and Memory," Cognitive Science, 4, 243-284.

Schank, R.C. and R.P. Abelson (1977) Scripts, Plans, Goals and Understanding. Hillsdale, NJ: Lawrence Erlbaum Associates.

Scott, W.A. (1969) "Structures of Natural Cognition, Journal of Personality and Social Psychology, 12, 261-278.

Scott, W.A. D.w. Osgood and C. Peterson (1979) Cognitive Structure: Theory and Measurement of Individual Differences. Washington, DC: V.E. Winston and Sons.

Shavelson, R.J. (1972) "Some Aspects of the Correspondence Between Content Structure and Cognitive Structure in Physics Instruction," Journal of Educational Psychology, 63, 225-234.

Shavelson, R.J. (1973) "Learning from Physics Instructions," Journal of Educational Psychology, 10, 101-111.

Shavelson, R.J. (1974) "Methods for Examining Representations of a Subject-Matter Structure in a Student's Memory," Journal of Research in Science Teaching, 11, 231-249.

Shavelson, R.J. and Stanton G.C. (1975) "Construct Validation: Methodology and Application to Three Measures of Cognitive Structure," Journal of Educational Measurement, 12 (2),67-85.

Smith, E.E., E.J. Shoben and L.J. Rips (1974) "Structures and Process in Semantic Memory: A Feature model for Semantic Memory," Psychological Review, 81, 214-41.

Tulving, E. (1983) Elements of Episodic Memory, New York: Oxford University Press.

Voss, J.F., T.R. Greene, T.A. Post, and V.C. Penner (1983) "Problem Solving Skills in the Social Sciences in G.H. Bower (Ed.) The PsYchology of Learning and Motivation, Vol. 17, New York: Academic Press.

White, R.T. and R.F. Gunstone (1980) "Converting Memory Probes into Scores on Several Dimensions," paper presented at the Annual Meeting of the Australian Association for Research in Education, Sydney

Wickelgren, W.A. (1981) "Human Learning and Memory," Annual Review of Psychology, Palo Alto, CA: Annual Reviews.

----------------------------------------