Printer Friendly

Latest Bad Rap Zaps WAP.

WAP doesn't work, says Nielsen Norman

After a year of lawsuits, updates, and upgrades to the much-vaunted Wireless Access Protocol (WAP) standard, a new research report paints a starkly depressing picture of the specification's usability, reliability, and technological sophistication. Its conclusion, that WAP will not be a viable technology in the wireless market anytime soon, should sound a clear warning to computer professionals thinking about developing or implementing products based on the WAP standard.

The report, "WAP Usability Deja Vu: 1994 All Over Again" was written in December by Marc Ramsay and Jakob Nielsen of influential industry research firm Nielsen Norman Group. It is particularly startling because it studied WAP not in the United States, where WAP phones and service are just beginning to be deployed, but in London, where mobile WAP services have been in development and deployed for a year. This is also one of the first reports on WAP testing "in the field". The opening page of the research report makes plain its conclusion: WAP doesn't work.

The authors state that in terms of the most basic usability, WAP is a failure. Even the simplest tasks, such as retrieving a world news headline from a built-in WAP-based portal, take much too long (more than a minute and sometimes nearly two) to be satisfactory. (The authors surveyed a number of Internet experts--before showing them the data--which stated that a wait of 30 seconds was a reasonable amount of time to obtain such information). The time lag is especially irritating because most users of WAP service pay by the minute. It should be said that per-minute plants would probably eventually be eliminated as service providers move to include WAP airtime minutes, but such plans are limited to non-existent today.

A separate issue the authors tackle is the design of WAP-based Web sites themselves. Their conclusion: Web sites that are supposedly designed to be displayed on WAP phones are virtually useless. The usability of WAP services, the authors say, is "severely reduced because of a misguided use of design principles from previous media, especially principles of Web design."

The title of the report, "Deja Vu: 1994 All Over Again", refers to the state of the regular Internet in 1994, when the authors did a usability study of the early Web. They find that many of the same problems with WAP-enabled phones plagued the early World Wide Web: inability to connect to services, service outages and software crashes, incomprehensible error messages, poor wording and menu choices, and jargon-filled "newspeak" that confused users. Some (though by no means all) of these problems were ironed out a few years after the Web was introduced; the authors hope that this will be the case with WAP.

But one of their conclusions leads a reader to believe that WAP in 2001 may be very different from the Web in 1994, and in a disturbing way: the report states that the learning curve for WAP technology is nearly flat. That is to say, unlike virtually every successful new technology that has been introduced, users of WAP technology do not show a marked improvement in their ability to use the service over time. This lack of progress is highly unusual, and deserves examination.

WAP Facts

WAP is a protocol that is designed to display Web-based information (primarily text, at least today) on wireless devices; today it is used mainly on mobile phones. WAP supports a programming language called Wireless Markup Language (WML), which is based on HTML. As such, WAP and WML are designed to be optimized for phones in a number of ways: low bandwidth, low power consumption, support for small screen size, low latency (more on this below), and universal connectivity (multiple locations). WAP and WML are also designed to be easily personalized, since users don't have the time, battery power, or screen real estate to view information that is not pertinent to them specifically. WAP phones themselves are designed to be simple and easy to use. They use microbrowsers (in the study's case, both were WAP 1.1 compliant) to view and manipulate content.

Interestingly, the report states that the two most widely used WAP phones, which were the phones used in the study (the Ericsson R320s and the Nokia 7110e), are well designed and had little to do with WAP's deficiencies. The authors note that this is good news for phone makers but not for WAP developers; it means that WAP problems cannot be solved simply with a phone redesign. In a depressingly comic aside that says much about WAP as a mobile phone-based technology, the report says that "the main problem with the WAP phones was not their design but the very fact that they are telephones."

The report goes on to note what many observers (including this one) have been saying for some time: WAP is simply unsuited to mobile phones, with their miniscule screen size and lack of direct manipulation of data choices (such as with a stylus). WAP on a PDA is likely to offer a somewhat more rewarding experience.

WAP Service Usability

The vast majority of users in the study had a very negative impression of WAP. The most disliked feature of WAP, not surprisingly, was the slow throughput of data (Figure 1). Users also complained about the lack of depth and the uneven quality of the data itself. Even when reporting on the feature that most liked best--the ability to get news headlines--users were still unhappy with the quality of the data they got. In addition, users noticed very little differentiation of services; that is, information from very different sources looked and sounded the same. The report noted that this is probably due to the constraints on data size in WAP--headlines tend to sound the same when a limited number of works can be used. The phenomenon is completely contrary to the Web, where the same information can be packaged in any number of unique and interesting ways.

In addition to poor throughput, the report states that users consistently experienced an inability to connect to sites, and experienced numerous, repeated dropped connections. Even when users were able to complete a specific task, it often took multiple tries, which resulted in significant frustration. Users also encountered server errors, time-outs while waiting for downloads, and long delays while information loaded, only to then result in zero data. Figure 2 illustrates the various tasks users were asked to perform and their success rates and times. Because WAP is a circuit-switched technology, it is not always on. When a different site is requested, the phone must reconnect each time (the phone dials a phone number that brings up the site.) Nielsen likens the painfully slow connection process to a modem that has to dial and re-negotiate every time a new Web site is requested.

The report also makes clear that WAP has a long, long way to go to replicate many of the most basic tasks that Web suffers take for granted. For instance, using a browser's "back" button is second nature for most people, and is a critical Web browsing function, since it's easy to go down the wrong path in search of information. Ramsey and Nielsen found that the "back" command on WAP phones was a disaster. Often, previous page information was not cached, as it is on a PC, so users had to take the time to re-connect to a WAP server to retrace their steps (and connecting, as noted, was unreliable). In other cases, because of memory limitations, users could only go back a few pages before their page history was lost, forcing them to start over.

Users also had major trouble searching; many even had trouble locating a search engine at all, though six WAP-enabled search engines were on the Web when the report was written. Search results were difficult to navigate, and often even the most common of terms came back with virtually zero hits. (One user searched for "cat" and got one hit.) But again, more sites and better search engines are likely to be available as WAP catches on. The real problem was the time it took for searches to be completed: many minutes, sometimes with dropped connections. Weather forecasts presented a similar problem. Even after a week of using the WAP phones, only 30 percent of users were able to get reports of the evening weather in their areas. Travel information, theoretically a potential goldmine for both WAP users and service providers, brought yet another failure. The majority of users (65 percent) were unable to find any travel information at all. Five out of twenty users found some travel-related information, but it took' three to four minutes on average to find it.

Much has been written about WAP's ability to enable financial transactions on wireless devices. Ramsey and Nielsen asked their users to perform some very basic finance-related operations. Predictably, users failed miserably. Forget about buying and selling stocks; users were often unable to reliably get even a single stock quote, much less any in-depth financial information. One user spent more than two minutes trying to get a single quote, even though he had already bookmarked the page and had been using the WAP phone for four days. Another was repeatedly foiled by server errors. (Keep in mind also that all users were paying per-minute charges for the use of WAP services.)

Hardware Problems

While users surveyed in the study had few major complaints about the phones they were given, one question brought telling responses. When asked if "the phone's limited screen size would prevent you from using WAP in the future?" the vast majority of users either agreed or strongly agreed. These responses lead one to question the potential for success of any WAP telephone at all, since screen sizes are severely limited and not likely to get much larger--at least in a mobile phone form factor. This suggests that developers might be better off concentrating their energies on PDAs and other devices with larger footprints and screens.

Unfortunately, the survey offers little when it comes to explaining the reasons behind the low levels of satisfaction with WAP in Britain. For example, we do not know if dropped connections were due to gateway server failures or interference, or to poorly configured WML data or WML servers on the Internet. We also don't know if the lack of page caching was due to lack of phone memory, bad phone design, lack of capability of the gateway server (where data is suppose to be cached) or some combination. Ramsey and Nielsen studied only two WAP service providers in the U.K., BT Cellnet and Orange. Vodaphone, the biggest network operator, was re-organizing at the time. One would have liked to see statistics from users of Vodaphone and other services.

When asked why WAP was so difficult for his research subjects to use, Nielsen is blunt: "I don't know the technical reasons, but the problems with WAP are pervasive and fundamental," he says. "Getting information using a WAP phone is so difficult to do, everyone who does it fails." Nielsen believes that WAP is an example of a "placeholder" technology, but one that has taken so long to develop that its usefulness is sorely limited. "WAP was designed to access content on impoverished devices. But with much more advanced devices now arriving, it is too little too late," Nielsen offers. "WAP may work in two years, but by then, we will have .better systems and services in place."

WAP Or i-mode?

The report concludes: with a credible warning to those who believe that WAP is the future of wireless data access in the U.S. That warning comes in the form of i-mode, a wireless data network operated by Japan's (government-owned) telephone giant NTT DoCoMo. i-mode boasts nearly ten million users, fatter bandwidth, and a much more highly developed wireless infrastructure, including a broader range of content, better content navigation, and more reliable data delivery. But more to the point, DoCoMo is seeking to expand i-mode service, and its sights are set on Europe and the United States, markets whose potential user bases dwarf Japan's.

It should be noted that i-mode is a proprietary service, and it uses a proprietary programming language called compact HTML, or cHTML. i-mode may have difficulty succeeding in the U.S., where the market is generally unfavorable to proprietary technologies. But DoCoMo is undaunted: The company has made strategic investments and signed joint venture, agreements with several European phone companies, and it recently purchased ISP Verio Communications in the United States. DoCoMo also recently signed a deal with AT&T Wireless to take a l0-20 percent stake in that company's mobile operations. DoCoMo is also negotiating a potential investment in Cingular, a wireless joint venture between BellSouth and SBC Communications.

Nielsen believes that i-mode is simply a much better wireless technology; one good reason is that it's packet-switched. And, once i-mode is deployed by a major U.S. carrier--probably AT&T--WAP will be a sitting duck. "i-mode has very low latency, and it is simple to use," Nielson notes. "I think AT&T will use it to gain marketshare. i-mode is far superior to WAP in terms of usability."

When attempting to determine the chances of WAP's success in the U.S. market, it's crucial to remember that the standard has been deployed in the U.K. for a year, and it still doesn't work as promised. This means that, in all likelihood, it will be early 2002--if not later--before the technology is viable here. Part of the problem, and an issue that the Nielsen Norman report touches on, is that the WAP Forum and its member companies have marketed WAP as the Internet made wireless. Nielsen and Ramsey believe this has been (and continues to be) a serious strategic error. WAP, in fact, is nothing like the Internet. It is more akin to alphanumeric paging and Short Messaging Service--except significantly less reliable.

Nevertheless, it's clear that with almost 500 companies behind it, there's just no stopping WAP now. But thus far, most of WAP's praises have been sung by these companies, not by users. As the Nielsen Norman report indicates, WAP has a long way to go before service levels rise to meet the high expectations created by its marketing machine.
Fig 2
 Successful Minimum Maximum
TASK users (% of time time
 total users) (secs) (secs)
Time taken to find world headlines 20 (100%) 36.00 189.00
from home portal
Time taken to find world headlines 17 (85%) 25.00 158.00
from home portal one week later
Time taken to find world headlines 19 (95%) 30.00 126.00
from different portal
Time taken to find world headlines 18 (90%) 35.00 280.00
from different portal one week later
Time taken to find headlines from 19 (95%) 26.0 96.0
Guardian
Time taken to find headlines from 19 (95%) 24.0 177.0
Guardian one week later
Time taken to find local evening 18 (90%) 54.0 299.0
weather forecast
Time taken to find local evening 14 (70%) 49.0 270.0
weather forecast one week later
Time taken to find a TV program 20 (100%) 82.0 242.0
Time taken to find a TV program 17 (85%) 51.0 186.0
one week later
Looking for a local restaurant 16 (80%) 68.0 278.0
Looking for local travel information 7 (35%) 39.0 262.0
Looking for a sports result 17 (85%) 30.0 230.0
 Mean
TASK time
 (secs)
Time taken to find world headlines 76.3
from home portal
Time taken to find world headlines 63.5
from home portal one week later
Time taken to find world headlines 65.2
from different portal
Time taken to find world headlines 113.9
from different portal one week later
Time taken to find headlines from 52.8
Guardian
Time taken to find headlines from 46.5
Guardian one week later
Time taken to find local evening 164.3
weather forecast
Time taken to find local evening 114.9
weather forecast one week later
Time taken to find a TV program 158.6
Time taken to find a TV program 97.1
one week later
Looking for a local restaurant 137.9
Looking for local travel information 152.3
Looking for a sports result 106.5
Various tasks and their completion times
using WAP-enabled phones
Source: Nielsen Norman Group


[Graph omitted]
COPYRIGHT 2001 West World Productions, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2001, Gale Group. All rights reserved. Gale Group is a Thomson Corporation Company.

 Reader Opinion

Title:

Comment:



 

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Industry Trend or Event
Author:Piven, Joshua
Publication:Computer Technology Review
Geographic Code:1USA
Date:Feb 1, 2001
Words:2740
Previous Article:SANS And Tape Storage New Storage Paradigm For Enterprise Data.
Next Article:NAS/Tape Backup Strategies Cope With New Data Cornucopia.
Topics:


Related Articles
THE BUZZ GRAMMY TELECAST BESET BY UNCERTAINTY.
VIOLENCE BEGETS CALL FOR SUMMIT : RAP MUSIC DEATHS FUEL INDUSTRY'S WORRIES.
WHAT'S HAPPENING : TELEVISION.
Hip-hop at the movies: rappers produce reel profits on the silver screen. (The hip-hop economy: part 2 of a series).
THESE YOUNG RAPPERS HAVE A MISSION.
FROM ELECTRONICA TO SOUL, A NICE ENDING FOR THE BOWL.
Zap Mama draws inspiration from whole wide world.
DTI survey.
Zero-hour Attack Protection.
Gartner (Stamford, CT) is launching the "New Gartner Media & Technology Summit" in October in Orlando (FL).

Terms of use | Copyright © 2014 Farlex, Inc. | Feedback | For webmasters