The Impact of Information on Policymaker Judgments of New Technology

ABSTRACT - While consumer response to technologicaL innovation has been well documented in the marketing literature, very little is known about the "exogenous" processes influencing the technology during the early stages of its diffusion.

Citation:

Peter C. Wilton (1985) ,"The Impact of Information on Policymaker Judgments of New Technology", in NA - Advances in Consumer Research Volume 12, eds. Elizabeth C. Hirschman and Moris B. Holbrook, Provo, UT : Association for Consumer Research, Pages: 455-460.

Advances in Consumer Research Volume 12, 1985      Pages 455-460

THE IMPACT OF INFORMATION ON POLICYMAKER JUDGMENTS OF NEW TECHNOLOGY

Peter C. Wilton, University of California

[Work on this project has been supported by a grant from the National Science Foundation, Number PRA: 81-10052, entitled "Research for the Effective Utilization of Technology Assessment Information," with the author and John G. Myers as principal investigators.]

[Assistant Professor, School of Business Administration, 350 Barrows Hall, University of California, Berkeley, CA 94720. (415)642-0951]

ABSTRACT -

While consumer response to technologicaL innovation has been well documented in the marketing literature, very little is known about the "exogenous" processes influencing the technology during the early stages of its diffusion.

Rarely is the ultimate success or failure of a new technology a pure free-market process. Most new technologies will bring with them a set of consequences or impacts for segments of society. As understanding of the indirect or unanticipated consequences of the technology increases, so policymakers - both public and private - are likely to intervene to control, or mediate, the nature and extent of the technology's diffusion. To the extent these mediating effects have been ignored, estimates of the innovation's penetration will be unreliable.

It is important, therefore, to attempt to understand how policymakers react to information on the unanticipated consequences of innovations, and ultimately how decisions concerning the appropriate institutional arrangements for the technology are reached.

This paper reports partial results from a general model for investigating the mediating effects of policy formulation on innovation diffusion. The focus is upon how policymakers use and process "technology assessment" information in judgment tasks requiring ordering of alternative options for controlling the development of new technology.

INTRODUCTION

The question of consumer response to technological innovation has been well documented in the marketing literature (Robertson 1971; Sheth 1979; Tushman and Moore 1982)e Underlying most such studies is an implicit assumption that the path of diffusion for new technologies will be a direct function of consumer perceptions of the substitutability of the technology for some underlying need.

While this assumption may hold during the later stages of diffusion, it is often imprecise during the early stages of technology introduction. This is because many new technologies will have performance and/or economic characteristics which are unknown or ill-understood at the time of introduction. Further, many innovations will also be associated with a set of unanticipated or secondary consequences for society as a whole. During introduction of the technology, it is possible that some of these consequences may be overlooked or even ignored completely by potential adopters of the technology. still ether consequences, while identifiable at the time of introduction, may be impossible to assess in terms of magnitude or direction until applications of the technology have been fully developed and high penetration reached.

For innovations where the possibility of such consequences exist, the most significant factor in understanding diffusion of the innovation may well be the regulatory controls d performance standards imposed on the technology by both public and private policymakers. Recognition of the consequences of a new technology may lead to intervention aimed at stimulating, dampening, or otherwise controlling the nature and extent of diffusion of the technology.

Thus, an important question which precedes the study of consumer response to innovation is the likely form and nature of intervention by the policymaking community. In this paper, we explicitly examine the effects of information about a new technology on policymakers' preferences for control of the development and diffusion of the technology.

The study is motivated by ongoing public policy concern over the unanticipated consequences of technological innovation. This concern has (over the past decade) lead to formal mechanisms commonly known as "technology assessments" (TAs), for the early warning study of technology impacts. Since technology assessments attempt to project the consequences associated with long-run, widespread adoption of the technology, they represent a means for the policymaker to increase his/her knowledge of the technology, albeit speculatively, in a comparatively short period of time. As a result, the policymaker is presumed better able to formulate policy for control of the technology.

While considerable experience has accumulated in conducting technology assessments (see, for example, Johnson 1981), very little is known about the use of such information by the policymaking community, os its impact upon development of emerging technologies. [Possible exceptions include an important descriptive survey of TA utilization by Michaels and Berg (1978), and a study of barriers to utilization by Martino and Lentz (1977). However, these studies are purely descriptive and experiential, and as such offer only limited explanatory power.]

Of course, technology assessment information may represent only one of many possible influences on the policy formulation process. Potentially of equal importance is the organizational context in which policy choices are made (individual vs. dyadic (or group) decision processes, shared vs. independent information, etc.), and individual policymaker predispositions (knowledge of target technology, political party affiliation, etc.). In addition, formal TA reports represent only one means for the policymaker to acquire the information considered necessary to reach a policy decision. However, the objective of this research is to focus explicitly upon the impact of technology assessment information on policymaking. If information can be shown to impact policy formulation under controlled conditions, then it may subsequently be possible to modify the task environment to learn in what way factors such as organizational context alter the nature of TA utilization.

In order to examine the impact of TA information on policy formulation, measures of information processing and policy judgments can be taken under controlled laboratory conditions from a sample of (surrogate) policymakers presented with the results of a technology assessment and confronted with a specific policy formulation task. While many such judgment tasks are possible, one which bears directly upon the likely form of intervention in technology diffusion is what Michaels and Berg (1978) describe as instrumental utilization of TA information, in which the information contained in the technology assessment is expected to contribute to some decision facing the policymaker. This information may impact the decision in a number of ways. A priori, it may be expected that TA information will increase the policymaker's knowledge of the technology, first about its performance characteristics (i.e., how can the technology be described, and in what way(s) does it differ from currently available technologies?), and second about its indirect consequences. This use of TA has been supported by Michaels and Berg, who observed higher utilization of technology assessments among policymakers who felt relatively uninformed initially in the issue area, than among individuals who felt they were better informed. In general, this learning will be sequential: the policymaker will first try to understand the workings of the technology, and then shift his attention to the higher-order task of understanding the indirect impacts. This proposition has been demonstrated by Wilton (1983).

While both types of information are likely to be useful to the "naive" policymaker, they may differ in their perceived relevance to the judgment task. Increased understanding of the performance characteristics of the technology, for example, is likely to contribute less to the explanation of policymaker preferences for controlling the technology than is increased understanding of its indirect consequences, or the consequences of selected policy options. Eat is, information available in a technology assessment will differ in the degree to which it is incorporated in the final decision, with technology and policy consequences information showing the closest relation to policy judgments.

In speculating about the direction of the likely impact of TA information on policy judgments, it is necessary to consider the nature of the arguments contained in the TA, together with the policymaker's "naive (initial) policy position and subsequent pattern of information processing. If the policymaker approaches the judgment task with strong prior preferences (i.e., with high confidence in his initial position), either in favor of or against development of the technology, then his search of the TA for policy-relevant information may be only cursory. Alternatively, the policymaker may extensively search the TA for information which he expects to confirm these priors, or which he expects to conflict with them in a deliberate attempt to refute his initial position. Since higher-order consequences of a technology are by definition unexpected, the policymaker is perhaps equally likely to encounter both conflicting and confirming information during his search of the TA. The impact of these arguments on the policy judgments will then depend upon policymaker perceptions of both the credibility and relevance of the arguments; those seen as more credible being likely to have a greater influence on the policy task than those which are seen as less credible.

Given the diversity in individual initial positions and subsequent information-processing patterns, the overall impact of TA information on a sample of policymakers is certain to be bi-directional: i.e., for some policymakers, TA utilization will lead to increased preference for stimulating development of the technology, while for others utilization will lead to increased preference for dampening its development. From a macro perspective, it is this aggregate impact that is most relevant to understanding the likely path of diffusion for the technology. That is, will the prevailing regulatory and policy environment for the technology be one of "laissez-faire" or one of tight control? The study of TA should enable an answer to this question during the very early stages of technology diffusion.

METHOD

Data pertaining to these questions has been gathered as part of a larger experiment on the effects of task and information salience on information utilization. Although not germaine to the question of policy choice, this broader research effort has attempted to operationalize and measure many constructs associated with use of TA information, including multiattribute measures of information utility. A full description of this research effort is beyond the scope of this paper. [Interested readers should refer to Wilton and Myers (1984).] Instead, this paper describes only those aspects of the research that relate to judgment tasks involving choices among alternative policy options for control of a new technology (videotex and teletext in the United States).

Data for the experiment were gathered in two stages, approximately one week apart. In the first stage, subjects completed a 45-minute pencil-and-paper questionnaire designed to measure background information on a wide variety of individual predispositional factors, including attitudes towards technology, public policy, decision style, computer involvement, risk propensity, information preference and sponsor credibility.

In the second stage, subjects were asked first to complete a 30-minute series of true-false and multiple-choice items designed to measure current (pre-utilization) knowledge of videotex and teletext. They were then asked to study the available TA report with a view toward formulating policy for videotex and teletext.

Subjects were asked to act as an aide to the president of a major organization interested in participating in a series of Congressional,Hearings on the role of videotex and teletext in society. Subjects were told that the president was expected to come to the hearings prepared to make a recommendation, and that their task was to preference rank-order nine options for controlling the development of videotex and teletext, based upon an analysis of information contained in the TA. They were further instructed that they were to be able to defend their choice, and that the president did not wish to appear unknowledgeable about either the details of the technology or its likely impact on society."

Each policy option comprised a unique combination of investment support and regulatory control for the technology. The nine alternatives represent cells in a 3x3 pairwise tradeoff (conjoint) task, with three levels on each of the two attributes, investment support and regulatory control. The alternative levels of investment support included:

(1) heavy, direct investment

(2) moderate, indirect investment

(3) no investment support.

The alternative levels of regulatory control included:

(1) no regulatory control, industry self-regulation

(2) reinterpret existing laws as need arises

(3) formation of a new regulatory commission to oversee all development of videotex and teletext.

Subjects were told they would have approximately one hour in which to access the TA. Access proceeded via a main menu of information comprising the titles of the fifteen chapters of the report, and a series of fifteen submenus comprising the titles of the subsections within each chapter.

Prior to accessing any information, and after each chapter was accessed, measures were taken on each of the following variables: the preference rank-order of the nine pre-identified policy alternatives (described above), the perceived self-confidence in performing the assigned task (7-point bipolar scale), the overall (constant-sum) utility for each chapter of the report, together with measures of the expected/perceived relevance, credibility and novelty of each chapter contained in the TA (6-Point bipolar scales).

Data collection and information exposure sequences of the second stage of the research employed a computer-interactive system called STARSIS, or Strategic Technology Assessment ReSearch and Information System. STARSIS is a multi-user, interactive system in which managers can be asked questions, exposed to information, and can input responses via CRT terminals. the system represents an initial attempt to study the problems and opportunities associated with prototype online computer decision support systems for the dissemination of technology assessment information.

All sessions were administered in individual soundproof cubicles. Thus, the task environment is computer-interactive with no possibility for group interaction. [The external validity restrictions imposed by this operationalization are addressed in the discussion section of this paper.]

Subjects were 72 graduate students at a large prestigious Western university. These students were enrolled in a variety of degree programs, including Masters or PhD programs in Business Administration, Economics, Law, Public Policy, Engineering and Liberal Arts.

RESULTS

Assuming decision makers have little or no knowledge about an innovation, what regulatory or investment alternatives are they likely to adopt? In most cases, the judgments made by policymakers concerning the appropriate form and extent of control over the technology's development will require that trade-offs be made among the alternative means of control.

The two means of control employed in the current judgment task, regulation (or the passing of laws) and investment, each with three possible levels, have been represented as a 3x3 trade-off judgment task. As such, the data are suitable for analysis by any of a variety of conjoint analytic procedures (Green and Wind 1975) which provide estimates of the policymaker's utility for each means of control, together with the partworths for each level on each means. One such method is provided by LINMAP (Srinivasin and Shocker 1973).

Based upon a LINMAP analysis of the policy option rank vectors for all 72 subjects, the mean utility for (any form of) regulation as a means of control is 0.68 (standard deviation 0.26). Since the weights for the two alternative means of control are within-subject-normalized (with higher weights denoting more preferred), their sum must equal one. Thus, the mean utility for investment as a means of control for the future development of videotex and teletext is .32. Further, since for an individual the means of control receiving a weight of 0.50 or larger might be considered the most preferred control option, these results indicate a strong preference (across the sample) for regulation over investment as the most desirable way of controlling the development of the technology initially.

Table 1 shows the part-worths on regulation and investment for the aggregate sample. The numbers shown are the mean preference scores for each of the discrete levels on the two methods of control. [It should be recognized, however, that these mean scores assume homogeneity across subjects in terms of utility scale values. To the extent this homogeneity does not exist, the interpersonal utility comparisons implied by the mean scores are invalid.]

TABLE 1

PART-WORTHS FOR ALTERNATIVE LEVELS OF REGULATION AND INVESTMENT: PRE-UTILIZATION (MEAN SCORES, N=72)

As can be seen, the highest aggregate part-worth utility on the dimension of regulation is for reinterpreting existing laws-' (21.7), followed by "strict regulatory control" (-6.7). On the dimension of investment, moderate, indirect investment shows the highest partworth utility (12.9), followed by "no investment support" (0.8).

The picture that emerges is that most subjects under conditions of low prior knowledge of the technology choose the middle ground in terms of government involvement and control. A significant number, however, would support a policy of strict regulation and no government investment. For this group, one might suspect that teletext and videotex represent a potentially damaging technology that needs strict control by regulation and no attempts to stimulate diffusion via government spending.

The major research question addressed in this paper, however, concerns the impact of the utilization of technology assessment information on policy choices. A direct test of this relation is given by the magnitude and direction of changes in the ordering of the nine policy options, following utilization. [It is important to note at the outset of this analysis that not all 72 subjects participating in the exercise were exposed to the same information about the technology. As currently implemented, the STARSIS system allows free subject access to any part(s) of the complete TA report. In this sense, the study should be considered a "free choice" experiment.]

Table 2 shows the part-worth utilities associated with each level of regulation and investment, following utilization of the TA. The table provides a direct comparison to Table 1, the pre-utilization utilities.

When compared to the initial part-worths based upon limited prior knowledge of the technology, the greatest (average) shift in utilities has been for the various levels of regulation. The part-worth for all forms of strict regulation has increased from -6.7 before utilization to 7.9 following utilization. Conversely, the part-worth for no regulatory control has decreased from -15.1 to -30.1 as a result of utilization. Although shifts in the utilities for alternative forms of investment support do not appear as marked, the data indicate that, on average, information about the technology has the effect of inducing shifts towards both increased regulatory control and investment support as a means of controlling the diffusion of the technology.

TABLE 2

PART-WORTHS FOR ALTERNATIVE LEVELS OF REGULATION AND INVESTMENT: PRE-UTILIZATION (MEAN SCORES, N=72)

These average shifts in part-worths in turn imply shifts in the overall importance of each of the two means of control - regulation or investment, independent of level - indicated by the magnitude of the difference between the pre- and post-utilization weights for regulation (or investment). Across the sample, the product-moment correlation between the pre-utilization weight for regulation and the post-utilization weight is .57. The mean difference between these two weights is 0.018 (standard deviation 0.24). Although not shown due to space limitations, this represents a-distribution of differences in pre- and post-utilization weights for regulation in which the largest shift away from investment towards regulation as a means of control is 0.90, and the largest shift away from regulation towards investment is 0.47. Included in this distribution are 12 subjects (approximately 20% of the total sample) who exhibit a complete reversal of their pre-utilization preferences for control means; 8 subjects (11% of sample) switching from investment to regulation; and 4 (6%) switching from regulation to investment. This result is important, since it indicates that the effects of TA utilization on policy choice are not unidirectional. Different policymakers exposed to the same TA (though not necessarily the same information) may alter their initial policy preferences in entirely opposite directions.

The findings indicate that exposure to, and utilization of, information contained in the TA leads many subjects to shift their middle ground positions in the direction of increased regulation and, somewhat surprisingly, increased investment. The impact of utilization based on these data would appear to be a heightening of the risk factor perceived by some subjects (possibly concerning the consumer protection issues of the technology), in combination with a belief that the potential benefits of the technology and its diffusion should be accelerated by increased government spending.

Given that shifts in policymaker utilities for alternative means of control for videotex and teletext are taking place, an important question is to what extent can these shifts be explained by characteristics of the utilization process? One possible explanation for the shifts lies in the nature of the information contained in the TA. and accessed by subjects.

A test of the relationship between TA utilization and policy preferences is given in Table 3. The figures shown are the partial correlation coefficients between subjects' post-utilization weight for regulation and the intensity of processing each type of information within the TA, after controlling foe each subject's pre-utilization policy preferences. [Intensity of processing information was measured as the product of (i) the total amount of time spent reading an item, normalized within subject, and (ii) the proportion of the total number of sections accessed by the subject within the item.] Thus, the coefficients reflect only the impact of TA utilization policy preferences.

The analysis is based upon grouping of items within the TA, identified through a principal components analysis of subjects' pre-utilization scores on the expected utility of each chapter of information in the TA. As seen in Table 3, the fifteen chapters of the TA can be grouped into five main classes of information, each with a unique relation to the policy formulation task. The five classes of information are: descriptive data defining the technology and its current status, consequences information describing likely direct and indirect long-run impacts of the technology, forecasts of developments in the technology's costs and components, scenarios of future applications of the technology, and identification of key policy options for the technology.

The relationship of subjects' preferences to processing of each class of information is indicated by the partial correlation coefficients. The results are intuitively meaningful. First, subjects' preferences for regulation as a means of control are significantly and positively associated with the intensity of processing of both policy-options and technology-consequences information. Thus, as subjects learn more about the various options available for influencing the development of the technology, or about the likely impacts of the technology, so preferences for closer regulation of the technology increase. Not surprisingly, policy preferences are not influenced by the level of processing of any other type of information within the TA.

Also shown in Table 3 are the partial correlation coefficients between subjects' post-utilization utility for regulation and measures of the confirmation/disconfirmation of initial expectations of the relevance, credibility and newness of the various types of information in the TA. [For each item accessed, subjects were asked to indicate (using 6-point bipolar scales with higher values denoting more positive responses) the extent to which each item (i) contained new information, (ii) was more or less relevant, and (iii) more or less credible than expected prior to exposure. Thus, an item may either confirm, or positively or negatively disconfirm, the subject's pre-exposure expectation of the value of the item.]

Examining first policy-options and technology-impacts information, all judgments of these items in terms of their post-exposure relevance, credibility and novelty are positively and significantly related to the preference for regulatory control of the technology. That is, subjects whose expectations of this information are positively disconfirmed (i.e., who see this information as more relevant, credible or informative than initially expected) tent also to more strongly favor regulation as the appropriate means of control of the technology.

TABLE 3

PARTIAL CORRELATION COEFFICIENTS     POLICY PREFERENCES AS A FUNCTION OF UTILIZATION INTENSITY, CONFIRMATION/DISCONFIRMATION OF EXPECTED UTILITY.     CONTROLLING FOR PRE-UTILIZATION POLICY PREFERENCES

Judgments of descriptive information, on the other hand, are all negatively associated with preference for regulation of the technology. Thus, subjects who are positively disconfirmed by descriptive information tend to more strongly prefer investment over regulation as the appropriate means of control.

Thus, after controlling for initial policy preferences, TA utilization appears to directly affect the policy formulation process. These effects are due to differences in both the level of processing of policy-related information (much other information in the TA appears redundant), and subjects' Judgments of the actual utility of the information for the policy-formulation task.

CONCLUSION

Several issues relating to external validity and experimental procedure need to be addressed. Task environment in the current study required individual evaluations of a major new technology, based upon written information contained in a TA, with no possibility for group interaction. Some readers may feel this environment is inappropriate for the study of policy formulation since it fails to capture the dynamics associated with the policy formulation process. However, the purpose of this study is not to study policy formulation, per se, but rather the utilization of information about technology. In descriptive studies of TA utilization (Michaels and Berg 1978), instrumental task assignments of the type described in this research have been identified as established contexts for TA utilization.

The question of group processes in policy formulation is an important one. Under some situations, the policymaker may either be required to aggregate judgments of several experts or information sources, or may simply be presented with the results of a group-think, such as a Delphi procedure, etc., and incorporate these judgments into the final policy recommendation. However, this aggregation problem is preceded by the need to first understand the impact of TA utilization on individual decisions. Further, evidence is available which indicates that group participation does not produce decisions which differ from individual decisions. Vinokur and Burnstein (1974), for example, in studying the effects of group dynamics on the risky-shift phenomenon, observe that shifts are primarily dependent upon the arguments presented, and not group participation.

This research also employed computer-based procedures for data collection and presentation of information. The rationale lies principally (i) in the higher level of precision and control afforded in measurement of the complex constructs associated with information processing, and (ii) in the potential for developing prototype computer database systems for the future dissemination of technology assessment information. These objectives notwithstanding, the social judgment theory work of Hammond et al. (1978, 1980a, 1980b) indicate that computer-based procedures lead to significant improvements in the judgments of individuals in policy formulation tasks.

The importance of studying consumer response to innovation in marketing and consumer research is well documented. We have made the argument in this paper that, at least in the case of "discontinuous" innovations, the study of policy formulation is equally important to the prediction of the long-term diffusion of new technologies. The issue is captured in a recent review comment by Nicosia (1980):

Evolving new technologies have the potential of making it possible for each member of a society to interact with all other members. If this evolution cannot be stopped, then public policymakers . . . will inevitably have to choose. Either public policy will assume the authority to shape the development and use of communication technologies . . . or it will help people search for a society-wide institutional arrangement of consumer information systems that fits the variety of human needs for different kinds and amounts of information.

The diffusion of technological innovation is not, in general, a free-market process. Rather, it is a process mediated by particular policy positions taken towards the innovation. Policymakers face a variety of choices concerning the appropriate institutional arrangements and infrastructures to support the technology, as well as the means of achieving these arrangements. It follows in these cases that it is as important to study the information processing and choice behaviors of policymakers as it is to study such processes among consumers and ultimate users.

REFERENCES

Berg, M. R. (1975), "The Politics of Technology Assessment," Journal of the International Society for Technology Assessment, (December), 21-32.

Green, P. and Y. Wind (1975), "New Ways to Measure Consumers' Judgments," Harvard Business Review, 53 (July-August), 107-117.

Hammond, K. R., ed. (1978), Judgment and Decision in Public Policy Formation, Boulder: Westview Press.

Hammond, K.R., G. B. McClelland and J. Mumpower (1980a), Human Judgment and Decision Making, New York: Praeger Publishers.

Hammond, K.R., and N. E. Wascoe, eds. (1980b), "Realizations of Brunswik's Representative Design" in New Directions for Methodology of Social and Behavioral Science, No. 3, San Francisco: Jossey-Bass.

Johnson, G. P. (1981), "Review of Selected Technology Assessment Studies of Information Technologies in the U.S.A." Paper presented at the symposium on Research on Impacts of Information Technology--Hope for escaping the Negative Effects of an Information Society?", (May), Walberberg, Germany.

Martino, J. P. and R. C. Lenz, Jr. (1977), "Barriers to Use of Policy-Relevant Information by Decision Makers," Technological Forecasting and Social Change, 10, 381390.

Michaels, D. and MX R. Berg (1978), The Use of Technology Assessment Studies in Policy-Making, Center for Research on Utilization of Scientific Knowledge, Institute for Social Research, University of Michigan, Ann Arbor.

Nicosia, F. MX "Information and Public Policy: Individual versus Social Choice," in Public Policy in Marketing: Challenges and Opportunities, New York: Permagon Press (forthcoming).

Robertson, T. S. (1971), Innovative Behavior and Communication, New York: Holt, Rinehart and Winston.

Sheth, J. N. (1979), The Psychology of Innovation Resistance: The Less Developed Concept (LDC) in Diffusion Research," Working Paper No. 622, College of Commerce and Business Administration, University of Illinois (October).

Srinivasin, S. and A. Shocker (1973), "Linear Programming for Multidimensional Analysis," Psychometrika, 38, No. 3, (September), 337-369.

Srinivasin, S. (1982), Teletext and Videotex in the U.S., Institute for the Future, Menlo Park.

Tushman, M; L. and W. L. Moore, eds. (1982), Readings in the Management of Innovation, Mass.: Pitman Books.

Vinokur, A. and E. Burnstein (1974), "Effects of Partially Persuasive Arguments on Group-Induced Shifts," Journal of Personality and Social Psychology, 29, 305-315.

Wilton, P. C. (1983), "Managerial Learning of Technological Innovation." Paper presented at a conference on Strategic Planning - A Thing of the Past, or A Necessity for the Future: Can Research Contribute, European Society for Opinion and Market Research, San Francisco.

Wilton, P. C. and J. G. Myers (1984), "Response to Innovation: The Mediating Effects of Technology Assessment," Paper No. Me, Center for Research in Management, University of California, Berkeley (January).

----------------------------------------