Overcoming Technocracy: Setting the Second Stage of Internet Methodology

Citation:

J. Barlow LeVold (1999) ,"Overcoming Technocracy: Setting the Second Stage of Internet Methodology", in NA - Advances in Consumer Research Volume 26, eds. Eric J. Arnould and Linda M. Scott, Provo, UT : Association for Consumer Research, Pages: 655-656.

Advances in Consumer Research Volume 26, 1999      Pages 655-656

OVERCOMING TECHNOCRACY: SETTING THE SECOND STAGE OF INTERNET METHODOLOGY

J. Barlow LeVold, University of Illinois at Urbana-Champaign

Historically, computer technicians and software developers have purposefully conceptualized and treated the use of computers and the Internet as merely the transference of words from a sheet of paper to a video screen. Operating systems appropriate terms such as "desktop" and "file manager" in order to make unfamiliar technologies familiar and friendly to the user. One visits "pages" on the Web, "bookmarking" them along the way. It appears to the user that the new technology is nothing more than the mere shuffling of papers, albeit in an on-screen environment.

This practice has resulted in the lulling of researchers and laypersons alike into a false sense of security incumbent in the appropriation of common ideas and practices for a computing environment. Much marketing research being performed on the Web is nothing more than the direct translation of paper-based questionnaires into a electronic form. This paper argues that this practice minimizes relevant research issues about the nature of the Internet. Researchers should look beyond the fashionable metaphors and consider how the new technology fits into people’s lives and affects their participation in research efforts.

In order to learn how to use the Web as a valuable data gathering tool, researchers must first consider how the Web and Web-based surveys engage potential respondents. In short, it is necessary to do an unpopular thing; researchers must consider how the Web is different, as well as similar to existing outlets for survey research.

BACKGROUND

The Internet’s most famous child, the Web, came into being in 1989. It was not until 1992 that HTML (Hypertext Markup Language) was established as the lingua franca of the Web, making it possible for Web browsers to display formatted text, pictures, sounds, and other digitally rendered information.

HTML enabled the prodigious Web-based ability to get feedback from surfers. Check boxes, text boxes, radio buttons, and drop-down lists were enlisted in order that a Web-site owner, originally assumed to be a researcher, might obtain feedback from other researchers located anywhere around the world. Considering the almost omnipresent nature of survey research on campuses around the globe, it should not be surprising that HTML from its very inception enlisted variants of standard input schemes native to paper-based questionnaires. It is, however, surprising to see the ease with which researchers have taken the conceptual leap of applying traditional survey practices to questionnaires on the Web.

METHOD

In order to examine the all-too-intuitive act of conducting a Web survey, it is necessary to consider the perspective of an average Web surfer. Focus groups were used to gather information of experiences and opinions about Web surveys.

Forty-eight undergraduate communications students (25 male, 23 female) participated in five different focus group sessions. In order to participate, the students must have had experience filling out Internet-based questionnaires. All group members received extra-credit for university course work in exchange for their participation. Sessions ranged from 45 minutes long to an hour.

During the sessions, color printouts of actual Web surveys were presented to facilitate discussion. The sessions were recorded on audiotape and transcribed. Themes were identified on the basis of their repetition in different focus groups.

FINDINGS

Three major themes emerged. These themes include 1) reduced feelings of obligation to participate; 2) concerns over privacy and safety; and 3) considerations of ease and speed of responding. Respondents also offered suggestions on how to increase participation in Internet-based surveys.

Reduced Feelings of Obligation

In traditional intercept questionnaires, social situation or obligation keep a respondent participating. However, this feeling is not shared in Internet instruments. One focus group member stated:

You’re looking at the guy [interviewer] and he’s right there. You’re just like, god, I have to fill it out because you don’t want to look like a total bitch or anything. Whereas, when you are on the Net, you don’t have to deal with a person or anything. You can just hit a button and you’re off.

Increased Concerns of Safety and Security

There are clear lines drawn by respondents regarding the types of information they were willing to impart. Web-based questionnaires are afforded the same consideration as are strangers, where respondents provide public and non-personal information. Citing reasons of security and safety, some questions were deemed inappropriate or unwarranted.

I would answer a question [in an electronic questionnaire] that was the same when someone off the street asked it. For instance, if someone asked me about my marital status, I would answer.But if they asked me how much my parents make, I wouldn’t answer them.

Several group members expressed a willingness to give out their e-mail address. However, other data was seen as more personal. For example:

I’d even give my email address [if I was getting a prize]. But not my social security number. You don’t know who’s getting it or what they’ll do.

Considerations of Ease and Speed

When respondents choose to participate in a Web survey, they respond adversely to performing tasks that are time consuming or hard to follow. When a potential respondent encounters a survey in a non-directed exploration of the Web, he or she is only willing to contribute a small amount of time. One group member stated:

I like this one because you don’t have to type a lot of stuff. You just point and click. And maybe type in a small amount. I don’t generally do surveys if I have to write down more than a line or two.

Increasing Participation in Web-Based Surveys

The principles of good questionnaire design expressed by the participants of the focus groups include issues of persuasiveness and engagement, brevity and clarity, and informed consent. Many questionnaires found on the Web do not incorporate these basic considerations, which was duly noted by the focus group participants.

Persuasiveness and the likelihood for engagement with a questionnaire is a basic consideration for any questionnaire. The way an invitation to participate in an online questionnaire initiates contact was very important to the focus group participants. One person suggested:

You gotta have something that gets people’s attention. You have to grab their attention especially with college age kids. They don’t have a lot of time.

Another participant distilled an example questionnaire to its most salient feature,

$50.00 prize? Oh yeah, 50 bucks is 50 bucks. But I think they should have it, somewhere, like bigger. They don’t have enough attention drawn to it. That should be the first thing you should see.

The format and length of the questionnaire is also a crucial consideration. After being shown a very well designed Web-based questionnaire (according to established principles of paper-based questionnaires) for the Hermes Project at the University of Michigan, participants expressed an unwillingness to even consider responding. One participant replied:

I think my first impression is that it’s really long and I don’t think I’d even read it. Especially if it looks like I have to spend a lot of time figuring out what they want.

Another concurred:

This is like, way overload. There’s nothing on the page that even captures my attention.

In addition, it is not enough for a researcher to believe in the importance of a study placed on the Web. Without the corresponding social obligation that comes with other and more standard forms of survey research, potential respondents need additional encouagement if they are to become actual respondents. Focus group members expressed a variety of opinions about what they expected from an online questionnaire. One member said:

The Internet is almost alternative-like, so you don’t want to have a totally white background and just text....Most Web pages have little pictures and animations that keep you interested. You keep going because you want to see what’s next.

Another expressed a positive response for surveys that seemed timely and engaging:

I like [filling out surveys for] topics that are hot moral and social issues. Like what people think about Bill Clinton. You might be apt to fill out one of those surveys. But if it’s for just something like the census, to just get information, for something you don’t know with an unknown purpose, then you don’t really care.

Focus group members also expressed a desire for an opportunity to learn more about the study. They might or might not consider such information when participating in a specific study, however they considered the availability of comprehensive information necessary. Several members of the groups wanted the opportunity to get details on the security and privacy standards of the researcher conducting the survey:

I think it’s really great how they have all this stuff available to look at. I’m not saying that you have to look at it but like, it says look here for security and privacy [information] and that’s great.

It seems that the respondents want both brevity and full disclosure. Although no one suggested it, using hyperlinks could provide a method of reconciling the two executional styles of low and high information content. Hyperlinks could be used as an option for participants who want more information while retaining a concise introductory Web page.

CONCLUSIONS

There is a substantial body of literature that informs the development of paper-based and interviewer-driven questionnaires. The effects of such variables as color, font, layout, and additional suspected confounds have all been thoroughly researched. It is wrong, however, to fall into the trap of assuming that insights gained in this area directly translate into a valid, reliable Web survey instrument. Assumptions of the similarities and/or differences between paper questionnaires and electronic questionnaires on the Web must be challenged to make Web-based research a viable and productive approach for researchers.

It seems that researchers have been handed a new method of asking questions on a silver platter. However, a healthy skepticism and an energetic optimism works together to produce good scholarship. As the title of this essay suggests, we need to break free of the assumptions invested in the technology we know as the Web and plot a new course based on the principles of sound research. Having seen some of what the Internet can offer, we need to look beyond the obvious instantiations of much current research into a sophisticated outlet that considers more than the common wisdom of a campus technocrat.

----------------------------------------