The data game.

The year: 2173. Two scientists talk about a man who has just been roused from a 200-year sleep.

"For breakfast, he requested something called wheat germ, organic honey, and tiger's milk."

"Oh, yes, those were the charmed substances that some years ago were felt to contain life-preserving properties."

"You mean there was no deep fat, no steak or cream pie or hot fudge?"

"Those were thought to be unhealthy, precisely the opposite of what we now know to be true."

"Incredible."

--Woody Allen, Sleeper

People know how to discount some kinds of information. We usually would not take too seriously a claim by the maker of Quick 'n Crispy Crinkle Cut Fries that "In a nationwide taste test, you preferred the crispiness of Quick 'n Crispy Crinkle Cut Fries, 3 to 1." Similarly, when the National Examiner publishes a story saying, "You can slash your cholesterol level [as much as 30 percent], strengthen your heart and add years to your life with a daily can of 7-Up," many people would, rightly, not stop taking their cholesterol medication. We tend to give more weight to surveys, studies, and polls reported in the New England Journal of Medicine, Time, The New York Times, the network news shows, or The Wall Street Journal. Yet even forums like these have been slow to recognize how dubious is much of the research they publish.

People know enough to be suspicious of some numbers in some contexts, but we are at the mercy of others. We have little personal experience or knowledge of the topics of much modern research, and the methodologies are incomprehensibly arcane. Nevertheless, we respect numbers, and we cannot help believing them. Numbers bring a sense of rationality to complex decisions--the ones we used to make with common sense, experience, and intelligence.

Yet more and more of the information we use to buy, elect, advise, acquit, and heal has been created not to expand our knowledge but to sell a product or advance a cause. If the results of the research contradict the sponsor's agenda, they will routinely be suppressed. Researchers have become secretive and their sponsors greedy. The media, which can usually get the raw numbers if they want them, are stingy with data because data are boring, and many journalists are themselves innumerate.

For example: If there were two things about food that we knew for sure, it was that milk was good for children and chocolate was bad. Studies found the opposite. Surely wine, cigarettes, and pate are harmful to your health. Studies have shown the reverse. For years, whole wheat bread was thought to be better than white. No, says a study. Studies found that oat bran was good for the heart, then not good, then good. Apple a day? A study showed apples cause cancer. Hundreds of studies have exonerated coffee; hundreds have damned it.

While most of the financing for food research comes from the government, private interests with financial stake in the outcome of the studies are paying a growing share. State and federal government financing for colleges and universities to do research and development has flattened in recent years, while the amount of financing from industry has increased dramatically. In 1981, industry contributed $292 million to schools for research; by 1991, that figure had jumped to more than $1.2 billion. The theory is that if the companies may profit from the research, and they often do, the companies should pay for it. Many food researchers must choose: research funded by an interested party or no research at all.

However complex and compromised, food research commands an astonishing loyalty from consumers, resulting in alarming shifts of behavior from study to study. A study showing that the pate-consuming French have healthier hearts actually increased sales of the fatty spread in the U.S. While most people would not rush out to take an experimental drug after one small study demonstrated its effectiveness, vast numbers of people will eat or not eat a food based on a single study. That is why studies have become such a big part of the food business.

Research about food has contributed many truths to the world, resulting in longer, healthier lives for those who follow its path. Woody Allen's futuristic fantasy notwith-standing, it is unquestionably true that more fruits and vegetables and less animal fat than Americans typically eat is good. Yet beyond some broad strokes of knowledge, there is little agreement about how coffee, oat bran, margarine, wine, and nuts, just to name a few, affect human bodies.

There is no truth about food so sacred that it cannot be challenged by research. In fact, the more the study defies common wisdom, the more likely it is to enjoy wide acclaim. Consider these surprises of recent years:

* "Milk is the number one health hazard facing young children," wrote a Santa Rosa, California, doctor in support of a new report by the Physicians Committee for Responsible Medicine. The report, released at a widely covered news conference in September 1992, cited a recent study in the New England Journal of Medicine about milk contributing to juvenile diabetes. Despite its vaguely neutral name, the committee is actually a pressure group of mostly vegetarians who oppose animal research and support animal welfare groups.

* White bread will not make you gain weight and, when used in a high-fiber diet, is an okay nutritional choice, reported the Cooper Institute for Aerobic Research. Its sponsor for the study: the makers of Wonder Bread. The research: inconclusive, to say the least. The 118 subjects were divided into four groups. One ate their normal diet; one group added four slices of low-calorie bread to their daily diet; one added eight slices of low-calorie bread; and a forth added eight slices of regular bread. This ended after a mere eight weeks. Predictably, no one in the study gained or lost significant weight, but the researchers said they believed the bread eaters would have lost weight if the study continued. The study was reported by the Associated Press.

* Chocolate may actually prevent cavities, reported a newsletter from the Princeton Resource Center, citing a study about how tannins in cocoa inhibit plaque formation. The group also published reports of a study about how sticky snacks like caramel actually dissolve faster than starchy foods like potato chips. This research "should dispel myths that foods perceived as 'sticky' or 'chewy' pose the greatest threat of dental decay," said the center, which is financed by M&M/Mars.

The nutrition study, like a political poll, is a machine with a thousand knobs. Turning any one of them a notch, even well within ethical limits, will dramatically change the outcome of the study. When studies differ, there are many legitimate-sounding--and possible legitimate--reasons. But the flood of deliberately contradictory studies insures there will be no definitive proof of anything. If a study contradicts another study's position, buyers of research can simply commission more studies. They cannot be absolutely certain the new studies will confirm their position, but they know the researchers whose labs have produced agreeable results. "Usually associations that sponsor research have a fairly good idea what the outcome will be," said Joseph Hotchkiss of Cornell University. "Or they won't fund it."

Coffee Talk

There is probably no food on earth that has been as widely studied to so little effect as caffeine, the world's most popular drug. Several times a year, a new study about the effects of caffeine, usually studied in the form of coffee, is published to great media fanfare. The studies, which are often reported as though they are the first and last word on the subject, are in fact absurdly contradictory and would be funny if the media and consumers did not embrace them so fervently. "Coffee Study Finds Heavy Drinking Boosts Heart Risk," announced The Wall Street Journal in March 1990. Six months later, the Journal revisited the coffee question. "Coffee Study Finds No Link to Heart Illness," it reported, noting the earlier study and saying "controversy about coffee's effect on health probably will continue."

The hundreds, perhaps thousands, of studies on coffee have taught scientists a few things about it. For most people, coffee stimulates the nervous system, makes their muscles more resilient, and gives them a heady feeling of concentration and power. But there is much scientists do not know. Despite thousands of studies on coffee's effects on virtually every organ in the body, scientists still cannot completely rule out links to heart disease, cancer, infertility, breast cysts, and a dozen other maladies. Dueling studies suggest there may be associations--or there may not.

"You can never prove a negative," said George E. Boecklin, president of the National Coffee Association of U.S.A., the industry's trade group, which has certainly done its best, financing and publicizing studies that acquit coffee of all serious crimes.

In studying food, researchers have a choice between using animals and humans. As subjects, both are imperfect. Food research on mice or rats is flawed because animals have different physiologies and life expectancies: 30 months versus 70 years. One Food and Drug Administration study showed that when pregnant rats were fed the equivalent of 56 to 87 cups of strong coffee at one time, some of the offsprings' toes were deformed or missing. Human studies involving some 15,000 women, however, have found no association between caffeine and birth defects.

Human beings would be perfect subjects for research on human health. But unlike rats, people cannot be forced to do what they are supposed to do, and some of them lie about it. Ask a thousand men and women what they ate yesterday, and the answers will be inaccurate blends of what they should, could, and would have eaten. Even dietitcians who have been subjects of nutritional studies say they are tempted to lie about the steak, potato chips, or ice cream they ate in private.

To study long-term effects of a food, especially on diseases that can take decades to develop, like cancer, scientists need 20 years. But they find coffee addicts reluctant to give up their drug for 20 years for the sake of science. Studies on substances strongly suspected of being killers are even tougher to analyze. Scientists cannot assign half of their subjects to smoke a pack of cigarettes a day.

To further complicate the picture, every human being is different. Some people can smoke a pack a day and live to 100. Others have never picked up a cigarette but die of lung cancer at 30. Sulfite, a food additive used in wine, fruits, and vegetables, is perfectly benign to all but a few people; when those few are exposed to it, they die.

Too, coffee is not just caffeine. It is different types and ages of beans roasted at very high temperatures, releasing other chemicals. It is brewed in different ways--boiling, dripping, percolating. It can be caffeinated or not. Like people, no two pots of coffee are exactly alike. And where does moderate coffee drinking cross into beavy coffee drinking? Four cups a day? Eight? Twelve? For that matter, what is a cup? Five ounces? Six ounces? Eight ounces?

People who drink coffee also tend to have other health habits that may cause the problems blamed on coffee. The coffee drinker is also more likely to smoke, to eat pork and potato chips and to exercise less than someone who doesn't drink coffee. For whatever reason, coffee is associated with risk taking, at least where health is concerned. One study even found that heavy coffee drinkers tend to be less likely to wear seat belts. So how can scientists ever know whether the thing that is killing people is the coffee or the doughnuts?

With so many variables involved, it is no wonder there is an endless stream of coffee studies, each proving or ruling out ever smaller chunks of the mosaic: "Caffeine, moderate alcohol intake, and risk of fractures of the hip and forearm in middle-aged women" was the title of one 1991 study. As each new study arrives, newspapers and airwaves crackle with an excitement usually associated with major news events: "Caffeine Not Harmful to Health"; "Coffee's 'Perk-me-up' Effect Confirmed in Study"; "Coffee each Day Keeps Asthma Away in Italy." Such guileless enthusiasm for each coffee study is one reason scientists love to work on caffeine. So widely consumed is the addictive drug that if even a tiny association between it and disease was established, many lives could be saved and scientific careers made.

The National Coffee Association is an active player in the caffeine study game. It commissions its own research, and it trumpets other research that supports the business goals of the association members. When a critical study is published, the association quickly reacts, issuing new releases complete with critical comments from "independent experts." "The coffee industry is incredibly powerful," said Dr. Robert Superko, who did some coffee research at Stanford University. "Once you get on their bad side, they have a very heavy hammer." Superko believes the industry shapes coffee research by choosing which studies to fund partly based on its hopes, rather than scrutinizing the study design with a cool, objective eye. And industry financing, said Superko, "indeed affects the way you publish the results. The coffee industry puts pressure on you to do it their way."

George Boecklin of the coffee association said that is not true. The only reason the association gets involved in financing research, he continued, is because otherwise it would look as though it had its "head in the sand." The research is done by respected institutions that are given "no strings attached" grants, Boecklin said. But the association's credibility is doubtful, at least insofar as the way they use research to further their selfinterested goals.

During the cholesterol mania of the 1980s, for example, many researchers turned their attention to the subject of coffee and cholesterol. Some earlier studies had suggested a link between heart disease and coffee, but just what that link was remained a mystery. Could it be cholesterol?

After having put out brushfires over cancer, birth defects and heart disease, the coffee association was clearly worried about this new threat to the already depressed coffee business. The association invited several experts in the field of cholesterol and clinical trials to submit proposals for a study on the subject. Dr. Roy Fried's proposal won the financing.

Fried, then a research fellow at Johns Hopkins, received more than $200,000 from the coffee association to do his study. Fried designed a study of the effects of drinking filtered coffee (the brewing method most Americans now prefer) on cholesterol levels. Fried and his fellow researchers first asked 100 male subjects to give up coffee for eight weeks to wash out their systems. Then the men were randomly assigned to one of four groups: Drink four cups (24 ounces) of regular coffee a day; drink four cups of decaffeinated coffee a day; drink two cups of regular coffee; and drink no coffee at all. The results: The cholesterol levels of the men who drank four cups a day of regular coffee rose. But the silver lining was that both their "good" and "bad" cholesterol levels seemed to have risen, canceling out any significantly increased risk of heart disease.

"We concluded that, based on the study, drinking modest amounts of filtered coffee does raise cholesterol, but that itself wouldn't increase the risk of heart disease," Fried said. His study was published on February 12, 1992, in the Journal of the American Medical Association.

Although the only positive result of the study was that coffee increased cholesterol, both the coffee association and the media found the negative results more interesting: "Study Refutes Link Between Coffee, High Cholesterol," said the Minneapolis Star. "Coffee is off the hook again," said the Phoenix Gazette. Meanwhile, the coffee association's fact sheet said: "Most studies involving U.S.-style filter-brewed coffee, including the 1992 study published in the Journal of the American Medical Association, have not found the association between ... coffee and increased risk of cholesterol-related heart disease. Importantly, the JAMA study ... controlled for diet, exercise and smoking, all known contributing factors for heart disease."

Fried's study was far from conclusive, however. The subjects were all men, and all but seven were white; the experiment lasted only four months (whereas many people drink coffee for most of a lifetime); the study was not double-blinded, which meant the subjects knew what they were drinking; and the maximum amount of coffee anyone drank was four cups a day, while many other studies tested more than four cups a day.

Food Fights

It is almost impossible for average consumers to sort through studies like these and know what they should be eating and drinking. Unfortunately, most Americans do not have ready access to the studies and would not know how to decipher them if they did. Even relatively lucid scientific studies contain lines like "The final analysis was based on a 'pre protocol' basis since the study objective was to test a dose-response relationship of B-glucan on serum lipids." Most members of the media are ill-equipped to judge a technical study. Even if the science hasn't been explained or published in a U.S. journal, the media may jump on a study if it promises entertainment for the readers or viewers. And if the media jump, that is good enough for many Americans.

So what is to be done? A good rule is to keep in mind that a study may hint at an emerging truth, and possibly offer a diverting bit of entertainment, but in general that is all it is--a hint and a diversion. Unless and until a study has been replicated, it should be looked on with care--the experiments proving the existence of cold fusion, still unreplicated, being a good example. To establish something as widely accepted as the belief that smoking causes lung cancer took decades, and the evidence came from many different threads of research--animal studies, human studies, epidemiological studies.

Beware of research and researchers calling themselves independent. Independent often just means there are many paying clients instead of one; it does not mean that there is no financial incentive to provide agreeable results. Similarly, the word "nonprofit" means little in assessing the credibility of a study; nonprofit researchers still count on a regular paycheck.

The tacit acceptance of untruth in daily life eats away at belief in right and wrong. If nothing is true, how can one solution be better than another? Progress stalls. "We should fuss, we should be indignant," wrote Ivan Preston about disinformation in advertising. "We should call the advertisers phonies, or bullshitters, or harassers, when we think that's what they are.... We should not allow our own silence to be one of the reasons why things stay the same." It is time to reclaim our numbers, our truth.