Non-trivial pursuits: playing the research game.

Welcome to the world of science, Ms. Howard. We're not surprised that you're confused. It's not a comforting, here's-how-the-world-works place, like the textbooks portrayed it in junior high.

But you'll feel less like a ping-pong ball once you see how many--and what kind of--studies researchers have to complete before they can state with reasonable certainty that a food or supplement causes or prevents a disease.

Most of those "well-publicized studies" you hear about are just one step in a game we call "Non-Trivial Pursuits."

"New Study Finds Vitamins Are Not Cancer Preventers," ran the headline in the New York Times last July.

It was the second time in three months that the "antioxidant" vitamins had failed to prevent cancer.

In April, a study of 29,000 Finnish smokers found an increased incidence of lung cancer in men taking beta-carotene (vitamin E had no clear effect). And in July, betacarotene, E, and C failed to prevent precancerous colon polyps in 864 Americans who had already had a polyp removed.(1)

Are these results disappointing? Yes. Are they proof that antioxidants don't work? No. But to some people, the failures meant that scientists were stupid, out to discredit vitamins, or both.

PUZZLE PIECES

"True, the finding of harm from betacarotene is puzzling, and may well be refuted by future trials," wrote New York Times science editor Nicholas Wade in an op-ed piece last May. "But that cannot be presumed."

How can Wade be so unfazed when people are tearing their hair out trying to get some straight answers? Perhaps because he understands how research works.

"What medical journals publish is not received wisdom but rather working papers...," write New England Journal of Medicine editors Jerome Kassirer and Marcia Angell. "Each study becomes a piece of a puzzle....No matter how important the conclusions, they should usually be considered tentative until a body of evidence accumulates pointing in the same direction."

PROVING CAUSE AND EFFECT

"Ignoring dozens of positive studies published in the last decade, the media has given enormous coverage to a single badly flawed Finnish study which supposedly showed negative results," charged Citizens for Health, a lobbying group supported by the supplement industry, last April.

But unlike the Finnish study, virtually none of those dozens of "positive" studies were randomized, double-blind, placebocontrolled clinical intervention trials, which actually give people antioxidants (or whatever) and then wait to see what happens. Trials are less likely to suffer from pitfalls (see page 8).

For example, most of those "positive" studies simply "observed" lower cancer rates in people who consumed antioxidant vitamins, usually from a diet rich in fruits and vegetables.

But something else in fruits and vegetables could prevent cancer. Or people who eat them may do other things to protect their health.

"All that these observational studies really tell us is that there is an association between vitamins and health benefits," notes Wade. "For a biologist, that's the statement of the puzzle, not its solution."

In fact, it's never just observational studies--or just any one kind of study--that scientists need to prove cause and effect. "Cause and effect can only be based on the totality of the evidence, from all research methods--clinical, pathological, animal, experimental, epidemiological, and, when available, well-designed randomized controlled trials," says renowned epidemiologist Jeremiah Stamler of Northwestern Univerity Medical School in Chicago.

Yet the media and the vitamin industry pick and choose the types of studies that make their case.

"Four studies have been done on antioxidants and colorectal cancer, with mixed results," said a trade association of vitamin-makers last July. "In the largest study, involving 35,000 women in Iowa, supplements of vitamin E were found to be strongly protective against invasive colon cancer."

A popular health newsletter also miscast the Iowa study. "The largest and longest trial to date, lasting nearly five years, found no adverse effects from daily doses of more than 1,300 mg of vitamin E," it said.

Only one problem: The Iowa study wasn't a trial. It observed a lower rate of colon cancer in women who were taking 1,300 mg of vitamin E a day. It couldn't prove that vitamin E--and not something about people who chose to take E--was responsible.

That's not to say that trials always speak the truth. They, too, can reach erroneous results.

Confused? Our "Non-Trivial Pursuits" game on page 8 provides a snapshot of epidemiology--the type of research most often reported by the media.

Of course, researchers don't always follow the step-by-step procedure shown in our game. For example, much of the evidence that saturated fat and cholesterol in foods cause heart disease came from animal studies, small clinical studies (measuring cholesterol levels in people fed high-fat diets, for example), and studies comparing populations eating vastly different diets.

In fact, the case-control and cohort studies described in "Non-Trivial Pursuits" yielded little support for the link. And in the early 1970s, researchers decided that an intervention trial would not be feasible. Still, there is almost universal agreement that saturated fat causes heart disease.

(1)New England Journal of Medicine 331: 141, 189, 1994.