Modeling Distance Structures: Producing Useful Knowledge For Theory and Application

Daniel R. Denison, University of Michigan
ABSTRACT - This paper presents distance structure analysis as an alternative to more sophisticated techniques such as covariance structure analysis, with particular reference to two techniques, confirmatory multidimensional scaling (CMDS) (Lingoes and Borg 1980), and LISREL (Joreskog and Sorbom 1983). The paper highlights the differences between the two types of techniques, and discusses the applications and advantages of each.
[ to cite ]:
Daniel R. Denison (1986) ,"Modeling Distance Structures: Producing Useful Knowledge For Theory and Application", in NA - Advances in Consumer Research Volume 13, eds. Richard J. Lutz, Provo, UT : Association for Consumer Research, Pages: 520-522.

Advances in Consumer Research Volume 13, 1986      Pages 520-522

MODELING DISTANCE STRUCTURES: PRODUCING USEFUL KNOWLEDGE FOR THEORY AND APPLICATION

Daniel R. Denison, University of Michigan

ABSTRACT -

This paper presents distance structure analysis as an alternative to more sophisticated techniques such as covariance structure analysis, with particular reference to two techniques, confirmatory multidimensional scaling (CMDS) (Lingoes and Borg 1980), and LISREL (Joreskog and Sorbom 1983). The paper highlights the differences between the two types of techniques, and discusses the applications and advantages of each.

INTRODUCTION

Over the past few years, developments in structural modeling techniques based upon multidimensional scaling algorithms have made the conceptual similarity between MDS techniques and more sophisticated techniques based upon covariance structure analysis much more apparent (Bloxom 1978; Lingoes and Borg 1978- Lee and Bentler 1980; Carroll, Pruzansky, and Kruskal 1980; Borg and Lingoes 1980; Lingoes and Borg 1983). This paper discusses the conceptual similarities between these two types of techniques, as well as the differences, and makes suggestions about the conditions under which the two types of techniques are most appropriate, and most useful.

Earlier work on this topic (Fornell and Denison 1981; 1982; Denison 1982; Denison and Fornell 1985) has argued that the basic principles of structural modeling logically derive from the multitrait-multimethod logic first introduced by Campbell and Fiske (1959) and that convergent, discriminant, and nomological or theory-based validity can all be expressed within the context of confirmatory multidimensional scaling, with generally similar results to covariance structure analysis.

Within the general context of a theoretical model like the one in Figure 1, a procedure has been developed for expressing the structural characteristics usually associated with a "causal model" in terms of distance structures. Figure 2 shows a set of points in a two dimensional space which we will then use as an example of the representation of a theory in terms of distance relations.

FIGURE 1

HIERARCHY OF EFFECTS MODEL OF CONSUMER BEHAVIOR

This simple example allows us to briefly illustrate our basic approach to operationalizing convergent-discriminant and nomological validity (Denison and Fornell 1985). Presentation of the specific formulae for the estimators, the procedures bar which to express a set of theoretical constraints in t@S terms, and criteria for the fit between the model and the data, are beyond the scope of this discussion, but are clearly spelled out in the original paper.

FIGURE 2

EUCLIDIAN DISTANCE ESTIMATES

Figure 2 presents a two-dimensional configuration of a three construct motel, where each construct has three indicators. XA, XB, XC, are the geometric centroids representing the "true value" of each of the constructs. A1, A2, A3, are the three indicators of construct A. Euclidian distances between the indicators and constructs are analogous to factor loadings, and may also be taken as indications of measurement error. This system allows for the operationalization of indicator-construct relationships in a way that is conceptually quite consistent with the more sophisticated approaches used in the analysis of covariance structures.

Figure 2 also includes hypothetical estimates of relationships between constructs. The distance between XA, XB, for example, is a non-metric analogue to a path coefficient, and implies that the distance between the two constructs is a representation of the relationship between variables.

Thus, in a greatly simplified form, we have discussed the relationship between a theoretical model and a set of observed variables, and the manner in which confirmatory MDS techniques have attempted to conceptualize it using distance measures. MDS techniques also typically include an estimate of fit between a theoretical model and the data, to complete the range of considerations necessary for a causal modeling system. Now we turn our attention to a brief example of the application of these two types of techniques, followed by a broader discussion of the differences between these two approaches.

A BRIEF EXAMPLE

Suppose that the same matrix represented by the two dimensional configuration in Figure 2 was analyzed via LISREL. The matrix, presented in Table 1, was designed to produce an MDS configuration which would fit a C.MDS model -- the clusters of indicators were clear, and the relationship between the clusters of indicators (and thus the underlying constructs) fit the theoretical relationships outlined in Figure 1. Will this matrix also fit the set of structural questions suggested by Figure 1?

TABLE 1

SIMULATED CORRELATION MATRIX

The answer, somewhat surprisingly, is no. Even though the parameter estimates are very strong, the LISREL fit is poor. With 25 degrees of freedom and a chi-square of 69.77 (X2/tf=2.79), the probability of these data actually fitting the theoretical model is .0000. The adjusted goodness-of-fit score is .791.

A second model helps to give further illustration of the differences between distance modeling and covariance structure analysis. Suppose that a matrix with an identical order, and an identical set of derived distances, but smaller correlations, was analyzed via LISREL. How would this affect the fit of the covariance model?

When the correlations in Table 1 were multiplied by .5, there was a substantial improvement in the fit of the LISREL model. The chi-square fit measure dropped from 69.77 to 3.32 (X2/df=.13) and the corresponding probability of fit rose from .0000 to 1.0000. The adjusted goodness-of-fit score also improved markedly from .791 to .993. Smaller correlations, with identical order and distance relations produced a markedly improved fit. Smaller correlations of course had no effect on the CMDS fit. The distance structure remained exactly the same since only the magnitude of the correlations was changed.

COMPARING THE TWO TECHNIQUES

The first point of comparison must be the general conceptual similarity and purpose of the two techniques. Both require that a set of measures of a latent construct must converge on that construct, but that the measures must still show a relation to the measures of other constructs that is in keeping with a theory about the relation between the constructs. Both techniques are designed to solve a similar problem, and in most cases, give quite similar results. The example presented here was designed to highlight the type of situation under which the techniques would show different results.

Beyond this initial similarity the differences become much more apparent. These differences are outlined below:

First, the nature of constructs is quite different in the two systems. In CMDS constructs are represented by regions, and in the system developed by Denison and Fornell (1985) the "true value" of a construct is at least indirectly observable, since it can be approximated (assuming random error) by the central point (centroid) of the component measures of any construct. A construct, in LISREL terms is represented by a dimension, not a geometric region, and is truly unobservable. In practice, the "true value" of a latent variable is highly dependent upon the other constructs and indicators which are Present in the system of equations.

Second, relationships in a CMDS model have no "functional form" per se. Proximity or similarity measures are converted to distances, but no functional form is implied by this procedure. LISREL, in contrast assumes a linear relationship, and both ordinal data and more sophisticated functional forms such as multiplicative relationships or polynomials violate the assumptions of the method.

Third, the two techniques differ enormously in the importance that they attach to inference. C>OS makes no claim to inferential power beyond that of the original proximity measures, and in general derives from a statistical tradition which places little faith in inference as a tool with which to develop theories. LISREL, in contrast, makes substantial claims to inferential power, and when the appropriate assumptions are met, allows for very powerful inference.

Fourth, one method responds primarily to ordinal relations among variables, whereas the other responds to interval relations. As the example illustrates, changing the size of the input correlations does not change the results of a CMDS analysis, unless the order relations are changed. While this insensitivity to changes in the size of input correlations is clearly one of the attractive features of this method, it can result in confusing examples. Negative correlations, extremely small or large correlations are all treated the same as long as the order relations are identical. LISREL, in contrast, is quite sensitive to changes in the size of input correlations, and relatively insensitive to changes in their order. Thus, larger correlations make a model more difficult to fit, and smaller correlations improve the fit tremendously.

This is in part due to a fifth consideration; the role of the null hypothesis. LISREL reverses the traditional role of the null hypothesis such that higher power leads to a higher (rather than lower) probability of rejecting a "true" model. Thus, a large sample size, and strong relationships among observed variables make it increasingly difficult to fit a model. By the same token, a small sample and weak relationships among observed variables make if very difficult to reject a faculty model. In contrast, CMDS takes a more traditional approach to hypothesis testing. Given a large sample, and strong relationships among observed variables, the convergence hypothesis is much more readily accepted for a given disparity between the model and the data.

GUIDELINES FOR APPLICATION

Several guidelines for the use of covariance and distance structure analysis seem to derive from the examples and discussion above. These can help to take advantage of the benefits of each of these methods, while avoiding some of the their inherent problems.

1. CMDS is probably most useful when one is developing, rather than testing theories. Although the method does include a test, and an accept/reject decision rule, it is a less-precise, ordinal test rather than a metric test. If the data meet the assumptions necessary for covariance structure analysis, CMDS is still quite useful, but primarily as an exploratory technique.

2. CMDS is quite useful for assessing convergent-discriminant validity within the context of a multiconstruct, multi-indicator model. The absence of a "true value" for a latent variable in LISREL sometimes makes this quite difficult. This guideline is related to the previous one, since convergent-discriminant validity should be resolved at the exploratory level, and should be a pre-condition for model testing.

3. Covariance structure analysis should be avoided when input correlations are either very small or very large. In these cases the outcome of the model testing is largely determined by the size of the correlations, and CMDS presents an attractive alternative. An approach based on distance structures allows one to look at the structure of the data in relationship to a theory in a way which is unaffected by the magnitude of the correlations.

4. CMDS places very little weight on probabilistic inference, while retaining a good sense of the factors associated with the "efficacy" of a causal model. This combination has a lot of intuitive appeal, but nonetheless limits powers of inference. LISREL, in contrast, places tremendous emphasis on probabilistic inference, and offers lots of inferential power, when the assumptions are met. Inference is highly important at some points in the research process, and rather irrelevant at others.

DISCUSSION

Research is basically a very disjunctive task; like a chain, a research design is only as strong as it's weakest link. The tremendous interest generated by the development of covariance structure analysis, and the relative accessibility of LISREL, has probably resulted in the overuse of this very powerful and sophisticated technique in situations where a weaker, more intuitive technique with far less demanding assumptions would be more applicable. Our work on CMDS has attempted to develop this technique to fit the need for a less sophisticated technique that can be of greater utility at the theory-building stage of research.

REFERENCES

Bloxom, B., (1978), "Constrained multidimensional scaling in N spaces", Psychometrika, 43, 397-408.

Borg, I., and J. C. Lingoes, (1980), "A model and algorithm for multidimensional scaling with external constraints on the distances", Psychometrika, 45, 25-38.

Campbell, D. B., and D. W. Fiske, (1959), "Convergent and discriminant validity by the multitrait-multimethod matrix", Psychological Bulletin, 56, 81-105.

Carroll, J. D., S. Pruzansky, and J. B. Kruskal, (1980), "CANDELINC: A general approach to multidimensional analysis of many-way arrays with linear constraints on parameters", Psychometrika, 45(1), 3-24.

Denison, D. R., (1982), "Multidimensional scaling and structural equation modeling: A comparison of multivariate techniques for theory testing". Multivariate Behavioral Research, 17, 447-470.

Fornell, C., and D. R. Denison, (1982), "A new approach to non-linear structural modeling by use of confirmatory multidimensional scaling", in C. Fornell (ed.), A Second Generation of Multivariate Analysis. New York: Praeger Publishers.

Fornell, C., and D. R. Denison, (1981), "Validity assessment via confirmatory multidimensional scaling". The Changing Marketing Environment: New Theories and Applications, Educator's Conference Proceedings, K. Bernhardt et al. (eds.), Chicago: American Marketing Association, 334-337.

Joreskog, K., and D. Sorbom, (1983), "LISREL VI: Supplement to the LISREL V Manual". Uppsala, Sweden: University of Uppsala, Department of Statistics.

Lee, S. Y., and P. H. Bentler, (1980), "Functional relations in multidimensional scaling". British Journal of Mathematical and Statistical Psychology, 33, 142-150.

Lingoes, J. C., and I. Borg, (1978), "CMDA-U Confirmatory monotone distance analysis-- Unconditional". Journal of Marketing Research, 15, 610-611.

Lingoes, J. C., and I. Borg, (1983), "A quasi-statistical model for choosing between alternative configurations derived from ordinally constrained data'.'. British Journal of Mathematical and Statistical Psychology, 36, 36-53.

Denison, D.R. and C. Fornell, (1985), "Distance structure analysis". Ann Arbor, Institute for Social Research.

----------------------------------------