Capturing Moment of Consumption With Smartphone: Case Study From Acapturing Meal and Snack Consumption Scenes Among Japanese Female University Students@

ABSTRACT - Focusing on data collecting procedures in qualitative research, this paper proposes a new method to capture scenes of ongoing consumption experiences by implementing a combination of ubiquitous communication technology and internet environment. A case study shows the success of a pilot system in recording informants’ activities as Adata in progress@ which leads to a better understanding of consumer behavior. Text and image data are recorded with Asmartphone@, wirelessly transmitted to a database, and used for real-time analysis. Although this system succeeds in gathering micro-ethnographical data of the informants’ diverse consuming experiences, both technological and methodological improvements are still needed.

Citation:

Satoshi Hosoe (2005) ,"Capturing Moment of Consumption With Smartphone: Case Study From Acapturing Meal and Snack Consumption Scenes Among Japanese Female University Students@", in AP - Asia Pacific Advances in Consumer Research Volume 6, eds. Yong-Uon Ha and Youjae Yi, Duluth, MN : Association for Consumer Research, Pages: 291-298.

Asia Pacific Advances in Consumer Research Volume 6, 2005      Pages 291-298

CAPTURING MOMENT OF CONSUMPTION WITH SMARTPHONE: CASE STUDY FROM "CAPTURING MEAL AND SNACK CONSUMPTION SCENES AMONG JAPANESE FEMALE UNIVERSITY STUDENTS"

Satoshi Hosoe, Keio University, Japan

ABSTRACT -

Focusing on data collecting procedures in qualitative research, this paper proposes a new method to capture scenes of ongoing consumption experiences by implementing a combination of ubiquitous communication technology and internet environment. A case study shows the success of a pilot system in recording informants’ activities as "data in progress" which leads to a better understanding of consumer behavior. Text and image data are recorded with "smartphone", wirelessly transmitted to a database, and used for real-time analysis. Although this system succeeds in gathering micro-ethnographical data of the informants’ diverse consuming experiences, both technological and methodological improvements are still needed.

1. INTRODUCTION

From a postmodern consumer research perspective, consumption is not merely a product selecting process, but an entire experience of selecting, using, and disposing of products. Research on this process should be expanded beyond ordinary commodities to services, events, ideas, people, and even to places. The fundamental concept of postmodern consumer research is to focus on the consumption phenomena (i.e. usage) rather than on the selection of products or brands. This theme has become a focus for recent research due to the recognition that traditional mass marketing approaches are ineffective in analyzing the volatile and fragmented lifestyles of contemporary consumers. Researchers realized that so many minute but important pieces of the consumption experience in our lives are neglected when scrutinizing the product selection process through quantitative research methods.

Along with increasing awareness and acceptance of the postmodern marketing concept, qualitative research methods have also come to the fore. Research methods from other disciplines such as sociology and anthropology have been introduced to consumer research methodology for exploring the individual’s consumption experience. Commonly applied qualitative methods are questionnaire inquiry, in-depth interview, and participant observation. Interpretive studies have also gained strong legitimacy for focusing on the inner experience of consumption, and several unique approaches were created. Methods such as the stereographic photo essay (Holbrook and Kuwahara, 1998) have been tested and put into practice. Some of these qualitative and interpretive research methods are further refined by the use of digital technology. Web-based questionnaires enable researchers to gather a larger number of data with ease. Mailing lists and BBS are used to form an online community, enabling researchers to utilize communication logs for qualitative analysis. The use of compact-sized video cameras and non-linear video editing software facilitate the task of the participant observation.

The impact of the application of digital media to qualitative methods is that it extends the postmodern researchers’ style of viewing the consumption phenomena. Since Morris Holbrook (1995) compared researchers’ characteristics to that of artists in his book "Consumer Research: Introspective Essays on the Study of Consumption", let us describe our assertion by showing how artifacts influenced artists’ activities. If you look back in history, you will find that artists accepted new technologies as ice breakers for acquiring new inspiration and unleashing new expression. For example, the visual sensation from a running locomotive inspired Turner to a new expression on landscape painting (landscape as a "flowing" spectacle, not a static panorama). Likewise, Beethoven’s composing style was greatly altered when the clavichord evolved into the piano. The methods of interpretation practiced by postmodern consumer researchers are similar to work done by artists mentioned above. Its constitutive aim is in obtaining new inspiration from the target consumption phenomena, and presenting the research output in an innovative style which would have an influential impact on anyone interested in that phenomena. Thus, implementation of new technology, in an appropriate manner, should lead to the development of new research "tools" for their subject of study.

Yet, in many cases, commonly used qualitative research methods are stranded in old media environments, restricting the process of capturing the consumers’ real life activities. For example, questionnaire surveys and interviews often take place in a social or cultural vacuum (ex. meeting rooms and laboratories), where ordinary human behavior is observed with difficulties. Likewise, these methods are usually designed to gather data before or after actions occur, not while the action is in progress. Thus, traditional research methods lack the ability to capture consumption experience as it is happening in the real world. One of the reasons for this deficiency has to do with the functionality of the media being used. The primary process of data collecting and analyzing is restrained because of mediocre printed questionnaire sheets.

This paper will explore the following questions: is there an alternative approach for exploring "consuming experience" by adapting new technology to existing research methods? If so, how effective is it? How will the analysis and interpretation process done by researchers change by use of these media technology? What will be the merits and demerits of future consumer research style? We search for answers to these questions by designing a ubiquitous mobile communication environment and conducting experimental research to capture the vivid real-life of consumers. Based on the method used in cognitive science referred to as Experience Sampling Method, we enhance its concept by using a multi-purpose cellular phone, or "smart phone", as a research tool. The purpose of this paper is to propose a new "viewfinder" for use in a consumer behavior study by connecting mobile communication technology to an established qualitative research method.

1.1 Key Aspects of Research

    1.1.1. Experience Sampling Method

Experience Sampling Method (ESM) is a unique research procedure originally designed by Mihaly Csikszentmihalyi (1975) in his pursuit of Optimal Experience (better known as Flow Experience). ESM is a quasi-naturalistic method that involves signaling informants at random interval throughout the day, often for a week or two, requesting them to report the nature and the quality of their experience.

In a typical procedure of ESM, informants are requested to carry a set of survey sheets and beeping devices during the whole period of research. The beeper is programmed to ring at random times, and whenever the beeper rings, informants fill out the survey sheet writing a brief description of their activities and emotions at that moment.

The most inspiring concept of ESM is that it urges informants to keep a short record of actions and emotional conditions when the actual activity is still in progress. As Russell T. Hurlburt and Christopher L. Heavey indicate in their paper (1996), the significance of ESM lies in "freezing informants’ ongoing experience and writing a brief description of the moment". This method enables exploration and recording of informants’ daily activity and fluctuation of emotion from a micro-ethnographical point of view. This method is capable of collecting clusters of vivid live recorded "situations" that informants encounter in their daily environment.

    1.1.2. Ubiquitous Communication Environment

Over the years, communication style has drastically changed among the Japanese due to broad diffusion of cellular phones. Statistical reports show that more than 60% of the Japanese use multi-purpose cellular phones (or "smartphone") on a daily basis (2003). Using the typical smartphone equipped with an internet browser application, a built-in digital camera, and colored LCD, the Japanese exchange short text messages and digital photo images with friends, families, and co-workers, while they are at home, commuting on trains, or even at work. There are many web services accessible by smartphone, such as news, weather forecast, and train schedule information. It is not too much to say that these portable devices are becoming part of the fundamental communication infrastructure among the Japanese.

This broad diffusion of the smartphone is creating an environment very similar to that which M.Weiser and others refer to as "Ubiquitous Computing" (M.Weiser, 1991). Ubiquitous computing, or "calm technology", is a concept indicating a dramatic miniaturization of communication technology so that the devices could be embedded into any surrounding environment and become virtually invisible in our lives. Nowadays, the term "ubiquitous computing" has extended its meaning to "communication technology available under any circumstances, unrestrained in time or space", which is the actual case among Japanese cell phone users.

From the viewpoint of the consumer researcher and other social science researchers, this widespread usage of ubiquitous communication media indicates a possible alternative research style that will utilize the ubiquitous media environment as an interface to capture human behavior. As a practical matter, mobile communication technology is already being used in other disciplines. For example, fieldworkers’ activities are aided with handheld computing devices and GPS (Pascoe, Morse, and Ryan, 1998). Since the devices are portable, they enable researchers to exchange data under any circumstances and free the research scheme from time and space. If the appropriate system is designed, we will be able to gather data that could be observed by the traditional approach.

2. DEVELOPMENT OF INTERACTIVE EXPERIENCE SAMPLING METHOD

2.1. Overview

(1) The outline of this project is as follows:

* Designed an integrated system of a mobile communication network and a web-based relational database to enhance the concept of ESM.

* Conducted several experimental researches using the prototype system.

* Updated the system according to the results of experimental researches.

(2) The system, Interactive Experience Sampling Method (iESM), is designed for the following purposes:

* "Barge in" with inquiry messages about the informants’ daily activities.

* Capture the informants’ situation as qualitative data by engaging them to describe what they are doing and how they are feeling at the moment with text messages and digital camera images.

* Collect these data in real-time so that the researchers can practice instant or temporal analysis.

(3) The basic architecture of iESM is composed of following media systems:

* Compact wireless communication devices (such as a smartphone or a wireless PDA) are used as data collecting interface.

* Internet servers (such as a SMTP server and a web server) are required to establish communication between observation groups and informants.

* Database application is used for storing, organizing, browsing, and sorting collected data.

The core essence of the system is in appending the use of graphic data and interactivity to ESM by replacing a mediocre questionnaire paper with a multi-purpose cell phone so that consumer researchers can remotely observe consumption experience happening in real world.

2.2. iESM Research Procedure

iESM is usually carried out for a week or two, to capture various daily activities. During the research period, the informants are engaged in recording their experience, and researchers are engaged in temporal analysis and online discussion of the collected data. Research conducted with the iESM system works in one of two ways; either inquiring at random intervals (random inquiry), or requesting informants to transmit data whenever the target consumption experience occurs (spontaneous report).

Random inquiry requests the informant to input and transmit data at unpredictable times. Informants describe their condition and reply to the questionnaire whenever they receive the cue signal from the iESM system. We can not predict when the informants will answer the inquiry since the interval between each cue is decided randomly by the system. As a result, they are more likely to behave in a natural manner. As Csikszentmihalyi had done in his researches, five to ten cues per day is preferred, depending on the attribution of the informant group.

Spontaneous report is a strategy used when the aim of research is determined to capture specific actions or events. For example, if the researchers are interested in capturing "pattern of music listening" or "usage of community facilities" or "most attractive advertisement on streets", this strategy is appropriate. The merit of this strategy is that researchers can collect pinpoint scenes of a particular consumption moment.

In either strategy, informants are instructed to do two things at the data collecting phase: describing the nature and the quality of their experience, and documenting the moment via digital photo snapshots. In order to capture the informants’ situation as accurately as possible, the informants are asked to supply the following pieces of information:

* Date and Time (usually logged automatically by the system)

* Place, describing where informant is at the moment.

* Whom the informant is with.

* Brief information on what the informant is doing

* Quality of experience (depending on research theme, particular emotion is measured with 1-5 scale)

Following up this basic information, the informants are requested to answer further questions based on the research theme. The questionnaires are deliberately changed throughout the research period, as determined by the result of online discussions held by the observation group. Informants are also instructed to take at least two snapshots using a digital camera installed on the smartphone: a symbolic picture that represents the informant’s situation, and a picture of themselves.

Each time the informant inputs responses and take snapshots using smartphone, both text data and image files are immediately stored on a web-based database server located on internet, enabling researchers to view and sort collected data in various ways. Fig.1 is an example of a single record set.

FIGURE 1

AN EXAMPLE OF A SINGLE RECORD SET STORED IN iESM DATABASE

3. CASE STUDY: "CAPTURING MEAL AND SNACK CONSUMPTION SCENES AMONG JAPANESE FEMALE UNIVERSITY STUDENTS"

Several pilot research projects were conducted in order to test the prototype iESM system between 2000 and 2002. The study, "Capturing Meal and Snack Consumption Scenes among Japanese Female University Students" which took place in 2001, is offered as an example for the successful use of the system. The purpose of this research was to study if marketers can seek a new niche market by closely observing female consumers’ daily food choices (such as meals, desserts, and between-meal snacks). Based on the result of this study, a proposal was developed for a confectionary maker in Japan.

3.1. Data collection

The research informants were recruited informally, as the main purpose of experimental research consisted in assessment of the prototype system. 16 female undergraduate students from Prof. Kuwahara’s workshop at Keio University Shonan-Fujisawa Campus were asked to serve as research informants. The average age of the informants was 21. All the informants owned their smartphone. The research took place for 14 days (10/11- 10/24, 2001). The project followed the spontaneous reports strategy mentioned earlier. The informants were given instruction to document and transmit their consumption experience every time they eat meals or snacks. The instruction was carefully given at the briefing session which took place two days before the actual research started, giving the informants sufficient time for practicing manipulation procedure.

3.2. Analysis

A trilateral research group formed from members of a confectionary company, a marketing company, and a university was organized to observe and interpret the collected data. During the research period, the role for this observation group was to constantly look over the gathered data and give temporal interpretation comments from each perspective. Since members of this group were located across the Greater Tokyo Metropolitan Area, they constantly exchanged information through an online discussion board included in the iESM system. The observation group viewed the iESM site on a daily basis adding comments on significant data. They also discussed when and how they should modify the questionnaire sets given to the informants.

4. ILLUSTRATIONS

As an example of the raw data discussed by the observation group, here are some illustrations taken from the informants’ documentation.

4.1. Eating Alone

This type of subjective information could not be collected from a traditional paper questionnaire research. Although no conclusive analysis was provided, the observation group was able to discuss this aspect of consumption experience from a variety of angles. Fig.2 and Fig.3 are contrasting for the former depicts breakfast at western style cafT and the latter shows typical Japanese style dinner with fish and chopsticks. Fig.4 surprises us with the detailed text description reported in a single experience moment. Her written data input by cell phone using 12 small buttons include rich information on her time at the library. With the combination of digital image, the observers can vividly compare the quietness of library hall and the shamefaced informant.

4.2. Food consumption and Friendship

Compared to the previously introduced data where the informants were solitary, here we can see the lively relationship between informants and their friends.

Food consumption occurs from ordinary classrooms to the baseball field, and foods (or snacks) seem to serve as bonds for deeper friendship. In most of these data, the images contained informants being surrounded by her friends. Notice how an informant expressively describes her sensation of fullness as "conquering Italy" with the delighted faces in Fig.7. For this party, Italian cuisine is illustrated as a target for ganging up and overcoming. Fig.7 may not include the actual scenes at the table, but it has been reported shortly after the meal. Therefore the written text seems very explicit. Unlike many self-report diary analysis where the informant writes at the end of the day, this method is capable of collecting self report data that is less biased by time.

FIGURES 2, 3, 4, 5, 6, 7

4.3. Spending time with boyfriend

Another interesting point that was determined through the research was that in the collected data, having traditional Japanese style dish with boyfriend was rarely observed. Many of the couples were observed having western style dishes such as fast food and pasta. They also did not seem to be spending time together at expensive restaurants, and prefer to go to reasonable priced places or eat at home. Informants’ facial expression varies, from the charming expression shown in Fig.9 to the worried gesture in Fig.10. These data gave the impression to the observers that many of the couples were spending quality time with their loved one. From this category, one demerit of exploring consumption experience through this approach arises, for giving us a hint that not all situations are recordable as data. Some very intimate situations may have been omitted from being reported. Experience sampling method is weak in capturing "special" or "private" situations, because in such occasions the informants are too preoccupied in their activity.

4.4. Categorized by Action and Events (1) Studying and Eating

Students seem to have hard time listening to lectures and doing their home work, and light meals or snacks serve as a temporary escape from reality. When observing situations related to studying, many of the images showed informants sitting in front of PCs. Based on this observation, one market researcher proposed advertisement campaign linking web banners and product package design.

FIGURES 8, 9, 10, 11, 12

4.5. Categorized by Action and Events (2) Eating before, during, and after part-time job

Part-time jobs (called "arubaito" or "baito" in Japanese) take up a great deal of off-campus student time. Although three examples above are reported from different informants, by arraying them in chronicle order, one may find contextual significance. The informants tend to purchase their food at the convenience store when they can not afford time, such as before and after their par-time jobs.

4.6. Categorized by Action and Events (3 ) Eating while in transit

Since the informants were very skillful in using their cell phones, many of the recorded data captured food consumption during walking, waiting for trains, and even during traveling on train packed with full of people. This type of consumption phenomena, when the consumers are "moving", could not be observed using the traditional qualitative research methods. Fig.20 shows a comical example of an informant’s embarrassed face while eating on crowded train.

4.7. Viewing Actual Product/Brand

In research using iESM, the questionnaire shown on the informant’s cellular phone is deliberately modified according to the propensity of collected data. The modification of questionnaire was decided through online consultation among observation group members. The research first started with no detailed instructions on what objects must be pictured in each digital image. As the research went on, one marketer complained that data would be worthless if actual product package or brand name can not be viewed. After this comment, the observation group decided to request informants to document the actual product if possible. This request led to collecting various brand figures such as Haagen-Dazs ice cream, Doritos, Diet Coke, Japanese-style cup cake, package of instant Chinese food, and others.

An interesting finding was ascertained when the observation group focused on scenes with a package of yogurt included in the picture, which is partly shown in Fig.21 to Fig.25.

In Japan, yogurt, usually packaged in a small cup or container and sold everywhere, is a symbol of "healthy food" among females. When we picked out the yogurt consumption scenes, we noticed that many of the data showed informant being alone. Perhaps one of the reasons for this is because yogurt is generally eaten as a part of breakfast, and many of the informants live alone. From the series of images and text data shown on the previous page, the observation group interpreted that yogurt relieves dullness, and might be thought of as a "healing food" or a "comfort food".

FIGURES 13, 14, 15, 16, 17, 18

5. GENERAL DISCUSSION

Due to page limitation, only partial data and categorized outputs were introduced in this paper. These examples tend to focus on the external dimension of consumption experience. During the research, several trials were made to explore the internal dimension of experience. For example, the question "how would you express your emotion with single Kanji-Character (Kanji are ideograms, i.e. every character has a meaning and corresponds to a word)" and "what kind of background music would you prefer to add for the moment" were given to informants, resulting into an interesting analysis. The output of these analyses will be presented in future papers.

One marketer who was involved in the online interpreting discussion commented that "it feels very similar to conducting an in-depth interview, except that the target group is spending their ordinary life in their real environment."

Although this system succeeds in gathering micro-ethnographical data of informants’ diverse consuming experience, both technological and methodological improvements are still needed for its development. To be more practical, the user interface of media devices must be improved. A quicker data transmitting process must be established for more effectively illustrating the natural consumption situation. Due to tremendous amount of both text and graphic data collection, data browsing application must be optimized. Obtaining cooperation from participants is a matter which needs to be seriously considered. To be honest, we were able to conduct this case research by convincing the informants that this was not only a research but also a part-time job for earning extra income (small payment was paid to the informants). A strategy to motivate informants in sharing their daily life activity as digital data must be considered.

FIGURES 19, 20, 21, 22, 23, 24, 25

In conclusion, this pilot study shows that the iESM can be used to explore the consumers’ real-life experience. Unlike traditional research methods with traditional research tools, this prototype media system shows potential of recording consumers’ activities as "data in progress" which leads to a better understanding of consumer behavior. Research aided by iESM has three important qualities: first, the usage of handheld wireless devise such as smartphone enables researchers to collect data unrestrained to time and space; next, the collection of image and text data captured in "real time" generates totally new representations of the consumption phenomena, and the researcher’s interpretative analysis is stimulated by browsing, comparing and rearranging this data; third, interactivity between observer and informants achieves an adaptive research strategy where the questionnaire is intelligently modified according to the significance of the gathered data.

REFERENCES

Brown, P.J., 1998. Triggering information by context. Personal Technologies (2).

Csikszentmihalyi, M. 1975, Beyond boredom and anxiety. Jossey-Bass Publishers, San Francisco, CA.

Kubey, R., Larson.R., and Csikszentmihalyi, M., 1996. Experience sampling method: Application to communication research questions. Journal of Communication, 46(2)Spring.

Holbrook, M.B., 1995. Consumer research: Introspective essays on the study of consumption. Sage Publications, Thousand Oaks, CA.

Holbrook, M.B., and Kuwahara, T., 1998. Collective stereographic photo essays: an integrated approach to probing consumption experience in depth. International Journal of Research in Marketing 15.

M. Weiser., 1991.The Computer for the 21st Century. Scientific American(September) .

Ministry of Public Management, Home Affaire, Post and Telecommunications, Japan, 2003. Information and communications in Japan: Building a "New, Japan-Inspired IT Society", White Paper 2003. http://www.johotsusintokei.soumu.go.jp/whitepaper/eng/WP2003/2003-index.html

Pascoe, J., Morse, D., and Ryan, N., 1998. Developing Personal Technology for the field. Personal Technologies (2).

----------------------------------------