The Application of Automated Concept Analysis to the Analysis of Focus Group Interviews

ABSTRACT - An Automated Concept Analysis procedure developed by Laffal is described and illustrated with two applications to the interpretation of focus group data. The first study demonstrates the basic procedure. The data were obtained in the course of developing a technique to predict television show success. The second study demonstrates the sensitivity of the procedure to differences in concepts discussed by focus groups. These data were obtained from a study designed to identify key dimensions comprising the image of a corporation or industry. Both studies demonstrated the applicability and utility of this systematic procedure in the analysis of focus group data.

Citation:

Martin R. Lautman, Larry Percy, Shel Feldman, and Abraham Wolf (1980) ,"The Application of Automated Concept Analysis to the Analysis of Focus Group Interviews", in NA - Advances in Consumer Research Volume 07, eds. Jerry C. Olson, Ann Abor, MI : Association for Consumer Research, Pages: 720-723.

Advances in Consumer Research Volume 7, 1980     Pages 720-723

THE APPLICATION OF AUTOMATED CONCEPT ANALYSIS TO THE ANALYSIS OF FOCUS GROUP INTERVIEWS

Martin R. Lautman, Associates for Research in Behavior, Inc.

Larry Percy, Creamer, Inc.

Shel Feldman, Associates for Research in Behavior, Inc.

Abraham Wolf, Associates for Research in Behavior, Inc.

[The authors would like to thank Wallace H. Wallace and Cail Kordish for their efforts in the studies reported here. Funding for this research was provided by a major television network and the Professional Development Fund of Associates for Research in Behavior, Inc., Philadelphia, PA 19104.]

ABSTRACT -

An Automated Concept Analysis procedure developed by Laffal is described and illustrated with two applications to the interpretation of focus group data. The first study demonstrates the basic procedure. The data were obtained in the course of developing a technique to predict television show success. The second study demonstrates the sensitivity of the procedure to differences in concepts discussed by focus groups. These data were obtained from a study designed to identify key dimensions comprising the image of a corporation or industry. Both studies demonstrated the applicability and utility of this systematic procedure in the analysis of focus group data.

INTRODUCTION

When analyzing a set of responses to open-ended questions, the first step is to develop a coding scheme for clustering or combining these responses into categories. This clustering is based on the assumption that these responses have something in common which allows the postulation of general categories into which specific responses can be grouped. This approach, however, would seem to be tautological. Categories are created to represent similar responses; these responses are then judged to be similar because they fall in the same category. Obviously, had some alternative categorization scheme been used they might have been judged dissimilar. On the other hand, if one can postulate a categorization scheme before coding responses and this scheme deals with concepts or dimensions with responses categorized on the basis of these a priori concepts or dimensions, the categorization procedure becomes much more defensible.

The purpose of this article is to describe an approach to categorization developed by Jules Laffal (1965; 1973) and to provide examples of its use with open-ended qualitative data obtained in two sets of focus group interviews.

At the outset it is necessary to note that utilization of this technique is recommended only in conjunction with the in-depth analysis typically provided by skilled focus group analysts. As will be demonstrated, the Automated Concept Analysis (ACA) procedure described here provides the skilled analyst with direction for looking into areas for insights which may have initially eluded him. Interpretation of these insights and recommendations for marketing strategy still rests with the focus group analyst.

The Laffal System

The Laffal system uses a concept dictionary as a "means of looking through a speaker's language to the concepts which lie behind them." The conceptual dictionary is an organization of words into higher order categories designed to reflect their synonymity, similarity, and relatedness. Laffal's assumption was that these categories reflect the "cognitive-conceptual sets which are evoked whenever a pertinent word ..." is either emitted, heard, or read.

Laffal's development of his concept dictionary derived from a technique described by him as "contextual association." Essentially, this procedure involved studying the kinds of words" ... occurring in the contexts of separate key words (which were then) compared to determine the similarity of contexts, and by implication the cognitive similarity of the key words." Thus, words observed with "experientially common meanings" were grouped and the process continued until the concept underlying this grouping emerged. Each new word was examined to determine which current category it might fit (be related to) based on commonality of meaning, with new categories formed only on "compelling grounds."

The result of Laffal's effort was over 23,000 words (about one-half of the words in Webster's New Collegiate Dictionary) grouped into 118 categories, with each category "... given independent and equal status within the system." For ease of analysis of a transcript, a computerized dictionary was created with each category designated by a maximum of four letters forming a word or mnemonic neologism. Because certain categories encompass different (but related) meanings, this abbreviated notation is not always adequate to describe a category and reference to the words in the dictionary/category itself is often necessary for interpretation.

The primary output from processing text through the Laffal system is the frequency of concept mentions by a speaker where "speaker" can be different respondents in the focus group, different focus groups, different topic areas, etc., as defined by the analyst.

Analytic Procedure

Figure 1 shows the typical analytic procedures used in an ACA analysis. Following text transcription, sampling (every nth line or page) is sometimes done in order to reduce the amount of data processed. [A typical 90 minute focus group tape will give about 8,000 to 9,000 words. Based on the number of topic areas covered, fewer words may need to be processed for the data to be reliable.] Words such as articles and prepositions are deleted. The remaining words are then examined for ambiguities. For example, the word "long" could be categorized into the categories EVER or MSMT (measurement). The analysts, knowing the context of the text, would make the appropriate categorization. Similarly, pronouns, hyphenated words and technical words not in the dictionary need to be clarified. The text is then transferred to a form for computer input and the ACA dictionary program executed.

FIGURE 1

ACA ANALYTIC SEQUENCE

Output in hand, the analyst must then make decisions about which categories are "significant." The primary criterion used here is a percent cutoff which segregates the data into frequent and infrequent concept references. Other criteria used for defining "significance" include between speaker differences in category use and within category word analysis.

At this point the analyst attempts to combine categories which, given his topic area, seem to make sense. Criteria used here include analysis of high and low "speakers" on categories and analysis of words in significant categories. This step in the analysis is critical in that the analyst here is identifying dimensions or issues (superordinate concepts) which seem to discriminate among the products, people, etc. which are represented in the output as "speakers." The final step is the profiling of the similarities and differences of the "speakers."

Examples

Two examples of an ACA will be described. The first example consists of a short transcript taken from a focus group tape which was part of a research program designed to develop a technique for predicting successful television shows. The data presented here were from the phase designed to identify the dimensions used by consumers in evaluating television shows. The second example presents the results of the analysis of one focus group (conducted in 1975) of a study designed to identify the dimensions which compose a corporation's/industry's image.

Television Study

Presented below is a part of a transcript of the group discussion of the television show "I Spy."

____________________________________________________

Moderator: The first show I want to talk about is "I Spy."

S1: I think it is one of the best shows on television. I enjoy Bill Cosby tremendously, and Robert Culp. I think they are fabulous actors, and they carry the show entirely. The episodes are well planned, and it's just an all around very good show ...

S1: Yeah, it has a comedial sense to it and also they do a lot of their versions in Hong Kong which also adds to the variety of the show and the format, which also makes it very enjoyable.

S2: Well, one episode--I can't remember the title of the show, when Bill Cosby was in a ship and the captain was a Negro of Jamaican origin, and he hinted to Cosby something about color, that color can get you someplace ...

_____________________________________________________

This text was then edited in order to delete the moderator's comments, substitute dictionary words for idioms, etc. The Laffal dictionary was then referenced to translate this text into concepts and to obtain a frequency count by concept as is shown below.

_____________________________________________________

17 PLAY

9 MALE

7 IDEA

5 ISPY, GOOD

4 LEAD, WE

3 GLAD, HAVE, PLAC, COLOR, EMPH, SIML

2 VARY, EVNT, FORM, SOME, IN

1 WHOL, SOMA, JOIN, TALK, VIEW, LARG, WHOL, AGRE, MUCH, PAST, TIME, FLOW, VEHC, BGIN, AID, GO, IN, GRUP

______________________________________________________

Obviously, the sample so characterized is not large enough for these results to be treated as stable, but one fact does seem to emerge: The members of this focus group discussed "I Spy" in terms of the actors ("MALE") involved, and they tended to be favorable to the actors and to the show. The "PLAY" concept references recreation, entertainment, parties, fiesta. Thus, the general context of the show in terms of its rather tongue-in-cheek, happy-go-lucky atmosphere seems to be a compelling reason for its popularity.

These results probably would be apparent even without the coding of the protocol, in this particular sample. But without the data reduction so accomplished, it would not be as easy to summarize the data from 4 or 6 groups each talking about the concept for 30 minutes or so; nor would it be relatively easy to compare discussions on different concepts, and to measure the degree of overlap between those concepts in terms of the discussions they elicit. The use of a standard set of categories and a relatively fixed dictionary (although it cam be added to, deleted from, etc. to accommodate specific problems) results in great savings in an analyst's time that can be shifted to concern with interpretation and explication.

Corporate Image Study

This study was designed to identify the dimensions used by male, college educated respondents in discriminating between corporations/industries. Six groups were selected--oil, pharmaceutical, chemical, restaurant, weapons and insurance. The format followed in this study differed from the typical moderator controlled focus group. Following a warm-up, the moderator gave each participant in the group a packet of cards containing leading questions about the subjects the group was to discuss. The moderator's role was limited to asking the participants to freely give their opinions and reactions to the printed questions, timing the discussion on each question to approximately l0 minutes, and probing less talkative participants.

An arbitrary cutoff of one percent mentions in a category was set (experience has shown this level to be reasonable) giving a total of 38 "significant" categories identified over the six groups. These categories accounted for about 75 percent of all of the categorizable words mentioned.

As was noted earlier, an analyst's interpretation is based on the frequency of usage of a category, the nature of the words in the category, between "speaker" differences in category usage, and, finally, by summary dimensions formed by combining related categories. The first step in the analysis is to list all words within each category meeting the one percent criterion. Table 1 shows this done for three categories later defined to form a responsibility dimension. Table 2 shows the frequency of category usage for all companies/industries. Examination of the data in these two tables for the Responsibility dimension indicates that the oil companies appear to be highest on this dimension, that this is true across the three categories and that the words "American", "country", and "patriotic" predominate. This observation was later interpreted as reflecting a patriotic responsibility ascribed to the oil companies in their actions over and above that of other industries. Using this general approach, the 12 dimensions (four formed from single categories) shown in Table 2 were selected as providing comparative insights as to differences between industries/companies.

TABLE 1

WORDS SPOKEN GROUPED BY LAFFAL CATEGORIES AND FORMING THE RESPONSIBILITY DIMENSION

TABLE 2

CATEGORY MENTIONS BY COMPANY/INDUSTRY

The company/industry profiles which follow reflect the distributions (Table 2) and the words used within each category (for example, Table l) and are oriented towards identifying between company/industry differences. It should be recalled that these profiles derived from the ACA analysis of a single focus group. Due to space limitations, interpretations are presented for only selected industries.

Oil Companies

Oil companies were viewed as being responsible in a political way, that is, being involved with the government and responsible to this country for their actions. It was desired that they conduct themselves in a patriotic way as indicated by the high frequencies of the categories LEAD, LAW and SIML which reflect the words "government", "congress", "representatives", with specific references to "America", "American." No other industry reflects this concern with responsibility, particularly on a national scale. Time, due to the then recent (1975) oil crisis, is also an important dimension for these companies as evidenced by a high frequency in the category TIME referring to "today", "month", "year."

The pecuniary dimension is relevant to oil companies, which is apparent in a high frequency for the category MONY. AID is also a high category and in this case refers to the word "profit." Although MONY will be shown to be high in other corporate groups, the notion of profit was mentioned in only one other instance, not surprisingly, for the insurance companies.

Pharmaceutical Companies

The most important dimension for these companies is research and the scientific quality of the organizations. The pharmaceuticals were seen as research oriented companies. This was evidenced by high frequencies in the categories IDEA, BLUR and POWR which refer to the words "research", "know", "wonder", "problem" and "expert." A concern with the dimension of product is evident in this group (indicated by a high category of END). It is possible that this is due to the consequence of research being an "end product" and two explanations might be offered. One is that people often expect research to lead to a better product and second that the product of pharmaceuticals--drugs--is an important one for the public to have confidence in.

The pecuniary dimension is central to this industry too, and discussed in a quantitative way. An analysis of the words in the high categories of MONY and NUMR shows that the discussion was in terms of dollars and cents. This is the only case among the company groupings where the relationship between NUMR and MONY is so clear cut. Although the above dimensions are strongly evidenced, the abstract concept of "company image" was not discussed. This may be due to a feeling of the panelists that the pharmaceuticals are a large group of which they have little knowledge and thus cannot establish a total view of the companies. This is supported by a high usage of the category LARG ("big", "large") as well as the concept of "power by knowledge" which may relate to these companies.

Chemical Companies

The profile for chemical companies is quite similar to that of the pharmaceutical industry on the dimension of research. The perception of chemical companies as a scientific, research oriented community is quite strong and the additional concept of truth is attributed to this group. This conclusion is supported by the high frequencies of IDEA, BLUR and TRUE referring to "research", "science", "think", "problem", "test", "fact" and "honest."

Like the pharmaceuticals, the profile for chemical companies also contains a strong product dimension, evidenced by a high END category. It might be hypothesized that the terminal result of research is important when considering the chemical group. Unlike the weapons group, however, there is no identification of specific products. This may be a vague area. It is interesting to note that on a social dimension these companies are very low, as noted by very few words in the TALK category. As the categories of MONY and NUMR are very low here, the pecuniary and quantitative dimensions do not seem outstanding for the chemical companies. This may result from an inability to conceptualize and deal with such dimensions for an industry which operates on such a large scale. This conclusion seems justified by the high frequency of LARG.

Finally, the image dimension should be mentioned. Although there is discussion of the chemical company image (high category VIEW with "image" as the main item), it is qualified by use of the word "seem." Thus, there appears to be a nebulous quality to the image of these "large, research oriented, non-social" corporations.

Restaurant Corporations

The social dimension is the most outstanding for this industry. High frequencies in the categories of TALK, and IN show a concentration of such words as "say", "tell", "told", "in" (specifically meaning--in a place) and "people." This reflects the notion of restaurants being seen as social gathering places. On the dimension of form and structure the restaurants are significantly low. These people were not concerned or interested in restaurants because of the "building", "design", "organization", or "system" of the industry group. The low frequency of FORM, which contains these words, supports this statement. Neither was the group concerned about money--the pecuniary dimension--when it came to restaurants. Surprisingly, they didn't even talk about the food in the restaurant--the industry product. Apparently, this industry is hardly viewed as a corporate group but instead as a people oriented, social institution.

DISCUSSION

The purpose of this article has been to introduce the Laffal ACA system into marketing and more specifically into the analysis of exploratory qualitative research. ACA is designed to explore the underlying dimensions of spoken language. It is a method of reaching beyond the specifics of conversation and thus does not catalog instances or occurrences of specific statements. Rather, it tries to distill the essential yet most general categories that are occurring in the data. Once these categories have been identified it is the analyst's task to ask: "What are the underlying elements that form these categories?" and "What marketing implications do these underlying concepts suggest?" Examination of the specific words and their contexts can then be used to develop quantitatively oriented instruments with which to measure the particulars with precision.

The essential worth of the ACA procedure is to aid the analyst providing an efficient and scientific means of extracting central issues from complex topics from qualitative data. In the absence of this tool, traditional qualitative analysis may easily suffer with different analysts tending to emphasize (or, overlook) different issues. The ACA procedure, while not insuring that different interpretations will not occur, provides a common set of data, in a quantitative format, which different analysts should consider in forming their interpretations.

As was noted earlier in this paper, it is not our belief that ACA should supplant the trained focus group analyst. Obviously, nuances in speech and body movements lend orientations and perspectives to spoken words which the ACA analysis cannot handle without coding intervention.

Focus groups, and the area of qualitative research in general, have long been thought to be areas off limit to rigorous scientific investigation. While we agree that the essence of qualitative research and its general usefulness rests on its unstructured nature, it is important to draw a distinction between the conduct of focus groups or qualitative in-depth interviews on the one hand and their analysis on the other. Much of the art of qualitative research rests in the moderator's skill in maintaining a dynamic environment conducive towards the prodding of respondents into revealing their true opinions, attitudes and beliefs. The analysis of these opinions, attitudes and beliefs, on the other hand, can be greatly aided by some means for formally organizing them. The ACA technique provides such an aid. At a more general level, the analyst must then interpret these concepts in terms of his understanding of how people conceptualize the topic area. Taking this discussion one step further requires that the analyst formalize his understanding into some theory (see, for example, Blackwell and Hilliker, 1976) which might then form the structure for integrating the conceptual data provided by the ACA technique.

One final note. The approach to qualitative data analysis through a universal system such as applied here seems to offer some scientific standardization to a technique frequently considered to be more art than science. In particular, the categorization of the data by an a priori concept analytic technique would appear to provide a means of avoiding some of the tautological reasoning typically associated with going from qualitative to quantitative analysis.

REFERENCES

Blackwell, Roger D. and Hilliker, Jo Ann S., "Clothing Decisions: A Decision Process Analysis of Focused Group Interviews," in Beverlee Anderson (ed.), Advances in Consumer Research, 3 (1976), 743-749.

Laffal, Julius, Pathological and Normal Language, New York: Atherton Press, 1965.

Laffal, Julius, A Concept Dictionary of English, New York: John Wiley and Sons, 1973.

----------------------------------------