Overview of the &Quot;Longitudinal Research Design and Analysis of Panel Data&Quot; Workshop

Citation:

Dr. Fred Cutler (1979) ,"Overview of the &Quot;Longitudinal Research Design and Analysis of Panel Data&Quot; Workshop", in NA - Advances in Consumer Research Volume 06, eds. William L. Wilkie, Ann Abor, MI : Association for Consumer Research, Pages: 31-33.

Advances in Consumer Research Volume 6, 1979      Pages 31-33

OVERVIEW OF THE "LONGITUDINAL RESEARCH DESIGN AND ANALYSIS OF PANEL DATA" WORKSHOP

Dr. Fred Cutler, Kennedy Research, Inc.

INTRODUCTION

This workshop was organized with two primary goals in mind. First, there is a need to advance consumer behavior theory through the implementation of a longitudinal research designs. At present, consumer research primarily utilizes a "snapshot" approach which examines the environment at only one particular point in time. There are alternative research methodologies available akin to a motion picture running across time; dynamic rather than static in nature. Longitudinal research designs allow for a response gathering situation in which you are re-interviewing the respondent over time, and/or acquiring response data over various time sequences.

The second purpose of this workshop was to present some methodological problems that are associated with longitudinal designs. It was felt that there is insufficient attention being paid in the literature toward the discovery of plausible solutions for some of these problems and that pointed discussion might suggest fruitful avenues to pursue. As a start, problems associated with several studies now in progress were identified, and potential solutions to these problems were advanced.

This workshop was also an example of the growing importance of the subject of "time" in consumer behavior theory. There are going to be at least two different perspectives that researchers will be using to look at the importance of time as it relates to consumer behavior theory. One is the consumer's time perspective. For example, Jon Gutman and Don Kanter at the University of Southern California have been looking at what is called planning horizons, or an individual's outlook towards the future when he is about to make a purchase decision. A second perspective relates to measuring change in attitudes and/or behavior over time. This workshop looked at the issues of longitudinal research designs which are able to measure attitudinal and behavioral change over time.

THE PARTICIPANTS

The panelists for this workshop discussion were:

* Dr. Lois Benedetti, AT&T Long Lines

* Dr. Fred Cutler, Kennedy Research

* Mr. Randall Harris, National Family Opinion, Inc.

* Dr. Ed Hart, Heublein, Inc.

* Dr. Don Lehmann, Columbia University

* Dr. Lorraine Scarpa, Heublein, Inc.

* Dr. Arch Woodside, University of South Carolina

Discussion in the workshop was wide-ranging, and difficult to easily summarize. What follows is a brief description of some of the issues which were raised, and which might prove to be provocative for some interested readers. In addition, a brief reference listing is provided for those who wish to delve more deeply into this area.

MULTI-WAVE MEASURES

There are many stated advantages to longitudinal data. One of the most basic is the possibility of measuring cause and effect. If you measure over time, you can theoretically see which comes first, the chicken or the egg. While there are recognizable difficulties in achieving this elusive goal, you are certainly better off with longitudinal data than you are with a simple snap-shot.

Longitudinal designs also allow us to start to monitor decision processes over time. This is important for researchers interested in why people make a choice decision -- not just the way they make it in a snapshot time period, but the ways decision processes change over time. One result of static research seems to be that the notion of the way consumers choose somehow gets simplified. In opposition to many of the prevailing implicit views of much consumer decision-making research, it should be understood that people do not just sit and process massive amounts of information each time they make a purchase.

There are several consumer variables that one can track via longitudinal designs. The most common procedure for tracking variables is through mean levels, such as average purchase rate or average brand share over time. A related statistic is the track of the variance or the standard deviation of some measure of how well people agree on the ratings of the brands, etc. In general, variances will decrease as a new brand gets learned about.

There are, however, problems in simplistic applications of summary statistics. If, for example, you measurement is on a 1 to 10 scale and people all start out at a 1 on the scale (meaning, "I have never heard of it," or "I don't know anything about it"), the variance has to increase simply because of the constraints of the scale.

Similarly, another interesting aspect of multi-wave data is that the non-response percentage on a question will drop considerably as something becomes better known. In a recent five wave study, the telephone panel was questioned on a new brand and an old brand. Non response tended to be around 4% for the existing brand. For the new brand, it started out around 25%-30%. However, after four or five waves of interviewing, which took about 18 months and coincided with the introductory period of the product (Vega), the item non response percent dropped to the 4% level. One can notice a brand getting learned just by the percent "don't know", "didn't response," "refused to answer the question," etc.

Another interesting construct that researchers can follow is change in consumers' attitude structures. Assuming a new brand, the first step toward uncovering structures might well involve factor analysis. Generally, only one factor will surface. Respondents either tend to like it or they don't. As time goes on, the single factor starts to break into two or three factors, reflecting that the way consumers approach brands seems to change over time. This is something you cannot easily spot in cross-sectional data, but is readily apparent through longitudinal data.

Yet another exercise that is of interest is to track model parameters. For example, if one tries to build a model of how awareness relates to comprehension and comprehension to attitude and attitude to choice, a hierarchy of effects model is often utilized. Analyses here relate to the stability of the parameters across groups and across time. Longitudinal data allows one to examine this type of issue, and many others as well.

PROBLEMS WITH LONGITUDINAL DESIGNS

There are a number of problems in this area, some well-recognized and others more hidden. For example, people can become conditioned by being asked questions, and using the same people over time can become a problem. Also, not everyone agrees to participate over time, suggesting a potential "dropout bias." What happened to the people that dropped out, are they different or not different?

Still another problem relates to what could be labeled "adjustment time." Consider adjustment time within a hierarchy of effects model, where one believes the effect of awareness on attitude is realized within a day or less. If you are measuring every six weeks, it becomes difficult to search for cause and effect looking at data the way we traditionally look at experimental data. If, on the other hand, it takes six or eight weeks after you become aware before you comprehend something, then the longitudinal designs can work a lot better. The last problem is one of collinearity. For any kind of semi-stable market, the correlation between time one and time two on most measures, is going to be high. Trying to "pull out" what's going on then requires some fairly sophisticated estimation procedures.

PROBLEMS WITH CONSUMER PANELS

The theoretical literature on longitudinal consumer panels has concentrated on several perceived problems with this methodological approach. The one problem that researchers talk about most is there presentativeness of panel data. However, some recent studies have shown that the demographic and psychographic (especially the psychographic) characteristics of people on a panel are no different from the characteristics of people who do not join a panel. Very little difference has been found in studies conducted by National Family Opinion (NFO). These results contradict much of the academic literature.

Related to this is a concern about the use of "professional'' panelists. Studies that NFO has recently conducted using matched samples of people that have been on a panel for one year versus people that have been on a panel for five or ten years, found the level of purchasing to be no different. NFO has looked at the people that drop-off a panel and found that they were not different demographically.

Do you make panel members more aware of what they are buying -- do you make them price conscious, for example -- by asking them to respond over several occasions? These potential problems are a legitimate concern, especially with such studies as a gasoline diary where people record price per gallon. However, it was also argued that use of a one-time or cross-sectional study creates more problems than using people who have been on a panel for a long period of time.

The most severe problem with longitudinal panel studies however, involves "memory pegs." Studies have shown significant differences between diary data and a one-shot multi-card or caravan study based upon a single recall question (i.e., respondents are asked what they have purchased in the last month).

NFO throws out the first two diaries that are returning by a new panel member. These initial "learning" diaries are used to enforce that respondents understand how to fill out the diary. Response overstatement is often seen in these first two monthly periods. Much to the chagrin of the client who says "I'm paying for that, why can't I have the data" - the researcher is saying that data may be lacking in quality. Respondents are not used to filling out the diary and therefore the data is questionable.

As suggested above, however, it has also been found that when one conducts one-shot research, there is often an overstatement of purchase behavior. In many instances, when people are asked what is the major brand of gasoline they buy or what is the brand they buy most often the major brands (i.e., those that advertise) come out twice as high as their market shares would suggest.

USE OF CONTROL GROUPS

All longitudinal designs have the inherent problem of test bias. Respondents are subjected to numerous interviewing situations, each of which can and often does bias the results of the succeeding interview. The use of a control group that is not subjected to questionnaire stimuli becomes almost mandatory to allow for the measurement of possible test bias.

Numerous advantages can be cited for a long period for a longitudinal design. However, if a control group is administered some form of a pre-test and then re-inter-viewed via a post-test questionnaire, this long time period produces operational problems. Such a long time period makes it very difficult to locate control respondents who have since moved or changed telephone numbers. Additionally, it would often be desirable to include a second control group that is only the advantage of measuring the effect of pre-test administration with the primary control group.

CONCLUSIONS

Longitudinal research designs offer unique advantages over cross-sectional designs for consumer research studies. Longitudinal designs appear uniquely suited for those studies that are attempting to uncover changes in consumer attitudes or behavior across different consumption situations and/or merely across time.

However, the problems associated with the proper administration and eventual data analysis of longitudinal studies are many. Some of these problems were discussed in the workshop, with positive suggestions for resolving many of the administration and analysis difficulties.

REFERENCES

Campbell, Donald T. and Julian C. Stanley. Experimental and Quasi-Experimental Designs for Research, Chicago: Rand McNally, 1963.

Enis, Ben M. and Gordon W. Paul. "Psychological and Socio-Economic Atypicality of Consumer Panel Members," presented at the Fall Educator's Conference, American Marketing Association, August 26, 1969.

McCullough, B. Claire. "Effects of Variables Using Panel Data: A Review of Techniques," Public Opinion Quarterly, February 1978, pp. 199-219.

Neyer, John and Joseph Waksberg. "Conditioning Effects from Repeated Household Interviews." Journal of Marketing, Vol. 28, April 1964, pp. 51-56.

Parfitt, John. "How Accurately Can Product Purchasing Behaviour Be Measured by Recall at Single Interview," presented at XX Esomar Conference, Vienna, Austria, August 1967.

Powers, Edward A., Willis J. Goudy and Pat M. Keith. "Congruence Between Panel and Recall Data in Longitudinal Research," Public Opinion Quarterly, June 1978, pp. 380 -389.

Pratt, Robert W., Jr. "Understanding the Decision Process for Consumer Durable Goods: An Example of the Longitudinal Approach," Marketing and Economic Development, ed. Peter D. Bennett, American Marketing Association, 1965.

----------------------------------------