Understanding and Analyzing Contingency Data

EQUATION

In the context initially described here, judgments are assumed to be perceptual and outcome costs are very and/or roughly equal. When this is the case,

EQUATION

The subjective probabilities on the left hand side of the above equation are functions of the individual's discriminal processes.

TEST THEORY (BIRNHAUM 68, RASCH 66).

The general concepts and computation apparatus employed in theories of comparative judgments and in signal detection theory can be used in more complex categorical judgment tasks. In the case at hand, these extensions will contribute little new. Test theory, however, offers some interesting additional insight, particularly as it relates to the Law of Comparative Judgments (Brogden, 77). The Rasch and Birnhaum models express the probability that individual i will correctly respond to item (signal or stimulus) a, as:

EQUATION

where

and

exp d_a  = the stimulus (difficulty) parameter.

The model involves distributions of ability across individuals and distributions of difficulty across tests. Assuming the item's difficulty and the respondents ability all refer to the same domain, a con, non scale may be employed. Therefore, responses are the result of sampling from the two distributions and computing that p_ia as a function of the distance q_i - d_a. Note that

EQUATION

where L^-l(p_ia) is the logistic inverse, and is similar to the normal inverse that appears in the Law of Comparative Judgment.

OTHER RELATED MODELS

Closely related to the test theory model is the Guttman scale (Torgerson, 58) along which both stimuli (items) and individual respondents may be scaled. In this case, it is assumed that items can be located along a continuum according to the degree to which they possess the attribute being scaled and that individuals can also be located along the same continuum. Items above the respondent on the scale will receive one response (yes, no; agree-disagree;...) and items below the respondent will receive the other response. Since this model is deterministic, the location of the stimuli and individual forecast the individual's complete response set.

Other related procedures are various kinds of bioassay and reliability theory models where the strength of treatment or time in use is used to predict the probability of an event such as death, recovery, purchase, failure etc. Events are discrete and the treatments increase the probability that an event occurs. Individuals have different thresholds at which the treatment level or stimulus triggers the event. The use of Probit, Normit or similar form of analysis lead to a linear equation linking the amount of the treatment with the predicted proportion of the treated population that will respond, e.g., die, recover, buy the product once, or say yes. When this is the case, alterations in the physical properties of products may be used to forecast the proportion of all potential buyers who would respond that the product had a particular attribute, say sweet flavor.

Finally, multivariate versions of some of the above models may also be found. For example, the multivariate logistic distribution might be used to estimate the degree to which two or more variables influenced the proportion of the treated subjects who positively responded.

Summary

To one degree or another, the above models represent response theories linking specific types of responses, yes, no, and possibly don't know to the question, Does this stimulus object have this attribute? Each model includes specific assumptions about human perceptual judgments and responses but differences are less noticeable than similarities. All models involve an internalized value of each stimulus and one or more thresholds or standards used by subjects to make categorical responses. None of the theories directly indicate how to determine the appropriate response for an individual since the internal value of the stimulus is not observable and the consequences of Y, N, DK responses to purely perceptual judgments offers little guidance. Correctness, truthfulness, and social conformity may be significant motives but it is far from clear how to incorporate these factors in models of individual response.

INFORMATION AND DISCRIMINATION

Having devoted attention to some alternative response models for contingency data, it is useful to look at the general behavioral influences affecting the observed responses in a particular case. Three elements deserve special attention; how objects enter an individual's evoked set, the attributes that are employed in the perceptual process applied to this set and how objects are encoded into contingency data.

The Evoked Set. For any class of objects, the more similar a new object is to the objects already in an individual's evoked set, the less perceptual variety it will contribute to an evoked set. Generally, the utility of perceptual redundancy among objects is Low. It is the different object that gets attention and becomes a new addition to the set of objects about which the individual is most aware. Therefore, individual search is directed towards finding collections of objects that differ in important ways from each other and not simply adding highly similar objects.

The Attribute Set. Given the above incentives, it is plausible to believe individuals will use attributes that effectively discriminate among the objects in their evoked sets. Non-discriminating attributes do not help an individual identify genuinely new or surprising objects. Therefore, individuals are expected to store and use attribute data that help enhance the useful variety of objects in their evoked sets. Their attributes are learned from experience and usually have evaluative content.

The Stored Contingency Data. At this point it is useful to recall that the information content in a message, Object _________ has (does not have) attribute __________, depends on the probability that any object of the type in question has the attribute. The information content of an event whose probability of occurrence is x is

h(x) = -log x.

The expected information (before knowing which object is being evaluated) is

and the expected information is a maximum when x = .5 for each event (Theil, 67). Therefore, before knowing which object in a set must be identified, an individual will select independent attributes such that each one is possessed by just half the objects and there is no uncertainty (don't know judgments) about the presence or absence of the attribute.

Summary

Individuals seek evoked sets of objects (objects about which they are knowledgeable) which are varied and can be readily identified. To accomplish this purpose, the individual selects meaningful attributes across which objects can be expected to significantly vary. Finally, the selection of attributes, objects and the encoding scheme seeks to balance the presence and absence of each attribute across each object, thereby maximizing the expected information content of messages about the set's perceptual content.

INFORMATION, LEARNING AND MATURATION

When confronted with questions about the presence or absence of an attribute for a genuinely new or novel object, the candid individual must answer "don't know" to every object attribute question. As learning takes place, an assortment of determinant attributes and growing number of non-redundant objects are associated with the evoked set. In this sense, the perceptual map of the individual grows in both complexity and clarity. Early, each new stimulus tends to have a high degree of novelty and receives much attention. Later, additions to the "new" class of objects tend to be fitted into an established structure and it becomes progressively more difficult to make the individual aware of new additions. At some point, knowledge about the evoked set may become unitized in the manner described by Barbara Hayes-Roth, 77) and the objects and their attribute evaluations may become resistant to change. Ultimately, new additions will tend to displace old objects in the evoked set.

v_ijn = a_ij v_ij,n-1     informative trial

or

v_ijn = b_ij v_ij,n-1    uninformative trial

where

v_ijn = (discriminal dispersionj of i on j at trial n) ^-1

and

a_ij > 1 > b_ij.

For a simple, single threshold model, it follows that the probability of a yes on the nth trial is

EQUATION

when

B_ij = a_ij/b_ij.

The time series profile of various learning and forgetting sequences can be represented by the above model and it can be adapted to cover yes, no and don't know responses.

PRELIMINARY DATA ANALYSIS

Once a set of contingency data has been collected from a group of individuals, two questions arise; how, if at all, can the data be pooled and what, if any, preliminary transformations may be useful? The first question largely concerns the degree to which response differences are due to noise or due to heterogeneous classes of respondents. Some types of individuals may be well informed and perceptive while others may be ignorant and unperceptive. Whatever factors may be at work, an initial effort to find response typologies by clustering methods should be instructive. It may indicate the absence of important response taxonomies or it may lead to classifying the subjects according to their response profiles. In the latter case, each group's contingency data would be separately analyzed and reasons for group difference could be investigated by discriminant analysis or analysis of variance methods.

The second data question concerns potential data transformations. Several important related issues will be discussed in later sections. Here, three transformations deserve comment. If differences in the overall frequencies of yes, no and don't know responses are present, normalizing the values of the responses may be considered. Doing so changes the numerical values of yes (and possible don't know and no) responses so each subject's data sum equals the mean sum across all subjects. Another possible transformation would weight each subjects object-attribute response by the information content of the object-attribute responses for the average subject. In other words, more weight would be assigned to rare attributes, those possessed by only a few objects. Third, a priori weights could be assigned to responses for each object and/or attribute to achieve some research objective. For example, effective salience weights might be employed.

Whatever analysis sample or samples are finally chosen, and/or whatever preliminary transformation has been applied, one or more aggregate data matrices are employed in subsequent analyses. Each cell of each matrix contains the sum across subjects of the entries in an individual's object by attribute contingency matrix. Finally, if repeated measures are obtained over time, one or more aggregate matrices would represent each cross section.

DATA GENERATION MODELS - AN ALTERNATIVE VIEW

Another way to look at the yes, no, don't know response to perceptions about objects is to consider a multidimensional space spanned by components of object attributes. For an extended discussion of such spatial models, see (Green and Carroll, 77), (Lingoes 77) and (Pessemier 77), and the associated bibliographies. Consider a perceptual space (component reduced space) containing 3 objects, x, y and z, and three (unit length) attribute vectors. Here, orthogonal projections of objects on attributes defined the attribute levels of each object.

The above perceptual structure can be used to generate contingency data of the type discussed above by observing that

a) Objects with high (positive) attribute levels will be judged "attribute present" and objects with low (negative) attribute levels will be judged "attribute absent." The present-absent region may be the space falling outside a hyper-sphere with its center at the origin of the space. Alternatively, it may be the regions beyond asymmetric cut offs along each attribute.

b) DK responses will occur whenever the internal value of an object falls inside the boundaries of the present-absent region. The effect of this region on DK responses is similar to the effect of discriminal dispersion on DK responses. When discriminal dispersions are large, the don't know response rate is relatively high. This rate decreases as discriminal dispersion decreases. When the hypersphere is large, more values fall in the don't know response region and as it decreases, the rate of don't know responses declines.

In the above formulation, early in the life cycle of product or product class, products will tend to have a large diameter hypersphere and high don't know response rates. Later in the product life cycle, the diameter of the hypersphere will shrink to reflect the increasingly clear product perceptions (small discriminal dispersions of more experienced individual judges). This time-dependent change is likely to be a significant aspect of the adoption-diffusion process. Advertising and the observation of others doubtless rank high among influences on the rate at which the perceptual process clarifies.

Although the details will not be discussed here, Simulated perceptual data can be readily generated which reflect the above structural characteristics and dynamic behavior. The simulated data are initially produced in interval form and then converted to yes, no, don't know responses. Therefore, the simple recovery characteristic of various multivariate procedures can be directly examined. In this manner, changes in discrimination and recovery can be examined over time and for data that reflect various levels of perceptual clarity. Finally, the reduced perceptual spaces produced with metric data can be compared to perceptual spaces developed from the analysis of contingency data. Several approaches are discussed in the next section which assume a homogenous population.

MODELS FOR THE ANALYSIS OF CONTINGENCY DATA

The simplest approach to the analysis of contingency data would be non-dimensional. For example, the THAID algorithm could find the extent to which individuals' yes, no and don't know judgments about the presence or absence of attributes discriminate one object from another (Morgan and Messenger, 73). The result of such an analysis would be a hierarchical scheme or tree which indicated how attribute judgments could be best used to identify an object. When developed at regular time intervals, the trees could easily represent the perceptual-communications dynamics of a product class over the adoption diffusion-cycle or the product life cycle.

A second approach is dimensional. It uses object-attribute contingency data solely for the purpose of developing a proximity matrix from pooled contingent judgments. A number of proximity measures, e.g., weighted or unweighted distances and correlation indices, could be used for this purpose, Depending on the choice of proximity measure and the specific objective, a variety of metric and non-metric multidimensional scaling algorithms are available to obtain a reduced space object configuration. The inter-object distances in this space can be interpreted as object-object dissimilarities. As an aid to interpretation, attribute (vectors) may be located in the same space.

A third approach transforms proportions or probabilities. Thurstone's theories of discrimination suggest the use of response function deviates. If p_ij is the proportion of yes answers to the question, "Does object i have attribute j? and F^-1 is the inverse response function, the desired deviate (distance) measure is

y_ij =  F^-1 (p_ij).

The y_ij values would appear in each cell of the attribute by object matrix which is subjected to further analysis.

A related approach transforms the proportion or probability into the entropy of the observation (message). In this case, the cell entries in the transformed contingency matrix are

y_ij = ln(p_ij).

With respect to either method outlined above, the question of how proportions are computed becomes a question of concern. For example, it may be computed over all possible positive responses, over all possible positive responses less the don't know responses, or over only the positive responses for an attribute. The following section will illustrate some of these possibilities.

A fourth approach is also dimensional but makes direct use of the pooled contingency judgment. Ries [14] has described the basic computations required to use the approach originally suggested by Benzecri [1]. The model described below expands their work by including yes, no and don't know judgments. Subjects judge m objects, O_m, by noting the presence or absence of each of N primary attributes, A_n. The presence of an attribute is recorded by a one, the absence of an attribute is recorded by a zero and uncertainty about the presence or absence of an attribute is recorded by a blank. By using an "ambiguity" attribute to record the number of uncertain attribute judgments or blanks, the level of perceptual ambiguity surrounding each object can be easily measured.

Recording as many positive attributes as necessary to describe an object extends the perceptual domain used by (Ring 78) where only one object, the one judged to have the most (or least) of an attribute, received a score of one. Furthermore, including an ambiguity attribute and allowing uncertainty judgments insures that an explicit judgment is recorded for every attribute on every object. The judgments made by single subjects can be recorded in a matrix format. Pooled data for groups of subjects can be recorded in a similar matrix layout. If attributes are rows, for any object column, the count of 1, measures the object's attribute complexity, the count of Os measures the attribute's simplicity, and the count of blanks measures the object's ambiguity. On the other hand, attributes can be analyzed by rows where the count of is measures the attribute's rarity and the count of blanks measures the attribute's ambiguity. If the row, column and grand mean of non-zero entries in the data matrix are n_j., n_.j, and n_.. , then a matrix of "normalized deviations" has entries given by

EQUATION

The Y matrix of "normal deviations" is decomposed and plotted by the computational procedure described by (Ries, 74)and programmed for convenient application by (Jarboe 78) The output of this analysis yields a product map in which the attribute "product ambiguity" is appropriately represented and onto which the level of each product or brand can be projected.

The static characteristics of this map are of interest and can be used for a variety of analytical purposes. Of greater interest, however, are dynamic characteristics of the map, particularly during the earlier phase of a product's or product class' life cycle. During this period, the repeated use of contingency maps of the type just described can track the changing levels of primary product attribute perceptions and the changing levels of ambiguity about products and their various attributes.

SUMMARY

The practical value of contingency data has been emphasized. Investigators who need simple response modes, e.g., in telephone surveys, yes-no or yes-no-don't know responses may be essential. Also, respondents usually find these categories are natural verbal responses to a wide range of judgmental questions. In questionnaires, these formats are among the easiest for respondents to use and for investigators to code for further analysis.

The main section of the paper briefly described alternative psychological models for generating individual judgments recorded in the forms noted above. Principal attention was devoted to judgments about continuous attributes of objects. These data are needed to produce perceptual maps for various classes of choice objects such as candidates, automobiles or restaurants. Also, interpretation of the frequency of positive and don't know responses was discussed, particularly as the rate may be a function of individual information processing and maturation.

The final section dealt with various procedures which can be used to transform contingency data and produce spatial representations of objects that are interpretable in terms of determinant attributes. The direct analysis of don't know responses was also discussed. Preliminary field tests of several methods of data collection, reduction and interpretation are underway. Perhaps other investigators will test the appropriateness of the alternative psychological models which may underlie the manifest data.

REFERENCES

J. P. Benzecri, L'analyse de donnees Vol. 2, L'analyse des correspondences, Dunod, Paris, 1973.

A. Birnhaum, "Some Latent Trait Models and Their Use in Inferring an Examinee's Ability," in F. M. Lord and R. N. Novick, eds., Statistical Theories of Mental Test Scores, Addison Wesley, 1968.

R. Bock and Lyle V. Jones, The Measurement and Prediction of Judgement and Choice, Holden-Day, 1968.

H. E. Brogden, "The Rasch Model, The Law of Comparative Judgment and Additive Conjoint Measurement," Psychometrika, Vol. 42, December 1977, pp. 631-634.

Clyde H. Coombs, Robyn Dawes and Amos Tversky, Mathematical Psychology, Prentice-Hall, 1970, pp. 165-200 and 270-273.

Paul E. Green and J. Douglas Carroll, Mathematical Tools for Applied Multivariate Analysis, Academic Press, 1976.

Glen R. Jarboe, Market Map Analysis of Patronage Behavior, unpublished dissertation, Purdue University, (in preparation).

Barbara Hayes-Roth, "Evolution of Structures and Processes,'' Psychological Review, 94, May 1977, pp. 260-278.

James C. Lingoes, ed., Geometric Representations of Relational Data, Ann Arbor: Mathesis Press, 1977.

Duncan R. Luce, "Thurstone's Discriminant Process Fifty Years Later," Psychometrika, Vol. 42, December 1977, pp. 461-489.

James N. Morgan and Robert C. Messenger, THAID: A Sequential Analysis Program for the Analyses of Nominal Scale Dependent Variables, Institute for Social Research, University of Michigan, 1973.

Edgar A. Pessemier, Product Management: Strategy and Organization, Wiley/Hamilton, 1977, pp. 205-258.

G. Rasch, "An Individualistic Approach to Item Analyses," In P. L. Lazarsfeld and N. W. Henry, ed., Readings in Mathematical Social Science, Chicago: Science Research Associates, 1966.

Paul N. Ries, "Joint Space Analysis of Contingency Data," Proctor & Gamble, 1974.

Larry J. Ring and Charles W. King, "A Multiple Discriminant Analyses Approach to the Development of Retail Store Positioning," Advances in Consumer Research, H. Keith Hunt, ed., Vol. V, Ann Arbor: Association for Consumer Research, 1978, pp. 227-234.

Henri Thiel, Economics and Information Theory, North-Holland, 1967.

L. L. Thurstone, The Measurement of Value, University of Chicago Press, 1959.

Warren S. Torgerson, Theory. and Methods of Scaling, John Wiley & Sons, 1958.

John I. Yellott, Jr., "The Relationship Between Luce's Choice Axiom, Thurstone's Theory of Comparative Judgment, and the Double Exponential Distribution," Journal of Mathematical Psychology, 15, 109-144 (1977).

Forrest W. Young, "Optimal Scaling With a Variety of Models," Psychometric Laboratory, University of North Carolina, (undated).

----------------------------------------