From the punched card to the Web: data visualization is a viable strategy for almost any large information space, so why not use it to work with shared data? (Business Matters).

At the Core

This article:

* Examines the history of data visualization 

* Discusses the merits of data visualization

For as long as computers have been able to display graphics, data visualization has been a dream of users and system designers.

It's more than a dream, however. Data visualization is already a huge success, and everyone with a personal computer uses it every day. This visualization is the graphical user interface (GUI) and its "desktop metaphor." The desktop vision presented on Windows and Macintosh systems is working proof that the right visualization can be put to work today to meet the real needs of real organizations and the people they employ. But once people realize that they are using visualization to deal with their standard desktop data, another question arises: Why not use visualization to work with shared data?

Once the user moves off the desktop into a catalog, directory, or any shared information resource, he or she typically reverts back to a mode of typing queries, pressing "enter," and reading lists of results. Personal computer (PC) user interfaces used to be this way (for example, DIR/Won MS-DOS), but those interfaces are history. It doesn't take a wild-eyed futurist to predict that one way or another, the world eventually will begin using visual interfaces for shared information. No one is sure what those visual interfaces will look like, however; it is still early in the game of building the visual interfaces of tomorrow.

Still, there are some strong guideposts illustrating what the future of information interfaces and data visualization may look like. Before considering these, however, it is important to survey the history of user interfaces in order to spot the big trends, to consider the bad reasons to do visualization, and pitfalls that may stand in the way of doing a good job. Perhaps most importantly, there are a few basic rules for achieving excellence in data graphics. While data visualization may be new, graphic design isn't, and there are some good guidelines to follow.

The History of User Interfaces

It all began with Herman Hollerith and his punched card, which was the main user interface to computers for the first few decades that computers existed. Dealing with small Hollerith cards was awkward and painful. One could transfer 80 characters, more or less, back and forth between the computer and the outside world. Eventually, punched cards became obsolete in favor of screens. Primitive screens, such as the IBM mainframe application screen, remain in use today in many legacy applications. Like the punched cards, early screens were still 80 characters wide, but there were a couple of dozen rows, and the characters could now be in color. These screens, though primitive by today's standards, allowed for a lot more information than the punched card enabled.

The notion of communicating between people and computers with text-filled screens remained dominant for a long time. A more advanced method, such as a "VT100" terminal from Digital Equipment Corp., followed the mainframe screen. The difference between this and the mainframe screen is that the text came in multiple sizes and with some formatting effects, resulting in the display of more information on the screen.

Eventually, the PC swept away everything that preceded, particularly once it was equipped with data visualization muscle in the form of the GUI. Perhaps the most advanced expression of the PC-centric view of the world was the Visual Basic programming language. It offered much more information than one could build into any of the previous generations of interface technology.

Then in the early 1990s, the World Wide Web came along and literally changed the world. The Web was in one sense a step sideways, because a Web page can contain about the same amount of information as any other window in the GUI. But it offered many other advantages that enabled it to (mostly) displace desktop interfaces and become the front-end to nearly everything. But since the Web's arrival, something distinctly odd has happened: There has been remarkably little progress in user interfaces. In particular, the amount of information on the screen hasn't increased much since 1993.

Has the most obvious trend in the development of computing interfaces come to an end? It appears that the trend is just taking a reprieve and that the future for user interfaces will be to add graphics to the text, to maintain clarity while doing it, and to do all this without leaving the context of the browser, because this is where most users are happy spending the bulk of their time.

To Visualize or Not?

Most experts assume that visualization is a useful way forward. However, history shows us that there are some really bad reasons to do visualization, which does have the potential to move a project backward and waste money. The following are bad reasons for pursuing visualization:

* "A picture is worth a thousand words." This piece of conventional wisdom is simply wrong. There are very few pictures that contain as much information as a thousand words. The Gettysburg Address and the Constitution of Japan are examples of documents that are less than a thousand words in length, yet carry huge semantic weight. Don't assume that just adding graphics will result in delivering more information.

* "Dumb it down with graphics." Too many people, particularly managers, want to adopt the assumption that their audience is dumb and has to be pampered with bright colorful pictures, as would an audience of four-year-old children. In fact, most people who are going to take the trouble to work with a user interface are reasonably smart (in particular, they are apt to be well informed about the subject they're working with) and appreciate being treated that way. Graphics are not easier to understand than text, particularly if the information you are trying to deliver has any complexity or depth.

* "The future will look like a video game." Science fiction has for decades imagined the idea of "cyberspace," a world where it is always dark and our data are glowing outcrop pings of bright immersive color. The only computer interface where three dimension (3-D) has found widespread commercial acceptance is video games. Perhaps a day will come when our business user interfaces will look like "Everquest" or "Duke Nukem," but that day does not seem to be anywhere near.

* "You should see this cool VR thing I've cooked up!" This is a really insidious problem: Computer programmers love creating cool graphics programming, most likely because routine programming tasks directed at validating metadata or correlating tables of numbers tend to be boring and unglamorous. Given the choice between building a conventional user interface and a bleeding-edge graphical sensory assault, the programmer will usually choose the latter. Unfortunately, however, most programmers possess poor visual design skills. Another problem is that quantitative analytical skills are rare, as are visual design skills, and the combination is rare because active hostility often exists between the two.

Excellence in Data Graphics

If a professional finds a domain where visualization will have a business payoff and builds a visual information interface, there is no way to determine whether or not he or she has done a good job. Most disciplines have a governing set of principles that one should follow to avoid common errors. But in the data visualization area, this can be tough. To start with, it is hard to find people who have both the design talent to use graphics effectively, the engineering talent to build the systems, and the business insight to figure out what should be visualized. In fact, at the moment there is a deep gulf and a certain amount of two-way contempt between engineering professionals ("geeks with no taste") and design professionals ("people who can't do arithmetic").

Many books on visual design have been published and, fortunately, there are a few books that combine the principles of visual design and data representation. One readable, excellent, and grounded-in-clear-thinking-and-graphical-excellence example is Edward Tufte's three-book set (Visual Display of Quantitative Information, Envisioning Information, and Visual Explanations: Images and Quantities, Evidence and Narrative) that covers this subject end to end. Tufte's work does not provide everything one needs to know about excellence in data visualization, but it does supply the important things and tells readers where to go looking for the rest.

These fairly substantial books offer two important principles:

* "Maximize the Data-Ink Ratio." In a visualization graphic, the data-ink ratio is that proportion of the ink on the page (or the pixels on the screen) that is being used to convey information. Any "ink" that is not information-bearing is just decoration and can usually be discarded.

* "Maximize the Data Density." Tufte argues that the more information you can pack into your display, the better. This is a bit surprising and counterintuitive, since communication specialists are always suggesting that users simplify and reduce. But his arguments are compelling, and he offers some weighty examples and numbers to back them up. From the point of view of this principle, using an entire screen to display nine data points is inefficient.

Looking across the whole spectrum of data presentation formats, one stands out both for its data density and data-ink ratio: cartography, the discipline of making maps. Maps routinely pack the equivalent of thousands of numeric data points per square inch of display and are incredibly economical in their use of ink--every drop is information-rich.

Information Mapping Into the Future

There are a number of companies trying to address these issues. The idea is to provide a GUI to any data source. There is a wave of visual interface companies, including Antarctica Systems, WebMap, and Urbanpixel, that are using cartographic tools for their visualizations. Older players in the market such as TheBrain and Inxight use more three-dimensional "science-fiction" approaches. With a user interface based on maps, professionals can display four or five dimensions of information about their data records, whatever they may be. Many businesses are just taking first steps in a long road that, hopefully, will lead to enabling the same degree of visual richness for shared information that Windows or Mac OS already provide for desktop information.

Visualization is tricky and not straightforward; however, it can be done successfully because it has been done already. But it has to be done correctly; users will not be willing to go back to the query/list mode of navigating data once they have used any kind of a GUI. From a business perspective, one interesting question includes: Where exactly can organizations find the highest payoffs from and, thus, the earliest adopters of visualization? The research space (medical, financial, academic, marketing) is where the first big payoffs will be found. But no one should be surprised when visual interfaces take off for accessing shared information. What's surprising is that it has not happened already.

Tim Bray is founder of Antarctica Systems and co-inventor of extensible markup language (XML). He has 19 years of experience in the software industry and specializes in the problems of searching and retrieving information from large textual databases. In 1987 he managed the New Oxford English Dictionary Project at the University of Waterloo, charged with developing an indexing technology that could move the contents of the Oxford English Dictionary online and make it searchable. In 1989, Bray co-founded the Open Text Corp., where he developed text-retrieval software and later introduced what would become one of the first commercial Web search engines. He may be contacted at [e-mail].