Hard-learned lessons: the benefits that models and data provide insurers have inherent limits.

Surprises--particularly unwelcome ones--can be occasions for learning, and also for correcting, if possible, what went wrong. So what can we learn from last year's surprising series of severe hurricanes that devastated homes, businesses and insurers' profits? Here are some possible conclusions, their rationales, and some implications.

Option 1: Last year was a fluke. A fluke is an instance of what is sometimes called "process risk." If we flip a fair coin a number of times, we know from experience that the proportion of heads will not be exactly half. If we flip the coin six times we may in fact obtain six heads or six tails, rather than an expected outcome of three each. In fact, the probability of obtaining five or six heads or tails is about 22%, so we should not be astonished when this occurs. Similarly, we should not be astonished by an unusually large number of severe hurricanes.

Option 2: The probability of severe hurricanes is higher than we thought. Option 1 presumes that we know the probability of flipping heads and the probability of a severe hurricane. Option 2, by contrast, recognizes that we must infer those probabilities from what we observe. The fact that our estimate may be wrong is known as "parameter risk." If we see a high percentage of heads in a long series of flips, we are certainly justified in inferring that the coin may be biased. Similarly, when we see a surprising number of severe hurricanes, we may rationally increase our estimate of their probability. In both cases, we are revising our parameters--our expectations or probabilities--to reflect additional experience.

Option 3: The probability of severe hurricanes has increased. In estimating probabilities from the events we observe, we necessarily rely on implicit or explicit mental models of what is happening. The possibility that this mental model is wrong is called "model risk." In Option 2, for example, there is an implicit presumption that the probability of flipping heads, or of observing severe hurricanes, is constant, so the crucial task is determining what that probability is. Option 3 challenges that view by asserting that the probability has changed as a consequence of global warming or some other climatic change. From this standpoint, Options 1 and 2 are both instances of model risk, and the important task is to determine how much the probability of severe hurricanes has increased and whether it will continue to do so.

Choosing among these options is more than an academic exercise, for they imply different responses. Option 1 implies that no action is necessary, since the events of 2005 were random and could not have been anticipated. Option 2 implies the need to inject greater pessimism into our catastrophe models, our pricing models, our underwriting decisions and our choices of the appropriate amount of surplus and reinsurance. Option 3 has similar implications, but also questions whether a firm can confidently write property catastrophe business given the significant uncertainty about current and future probabilities of severe losses.

But even more important than the choice among these options is the glimpse that last year gave us concerning the inherent limits to our understanding and management of risk--and not just property catastrophe risk, but risk of all kinds. Insurance as a product is necessarily concerned with events that occur very infrequently. By definition, data concerning the frequency and severity of such events is limited, so that we cannot be highly confident that the probability estimates we derive for such events are correct. Even more challenging is the related problem of estimating the correlations among different kinds of relatively rare events. This is an issue in the discipline of Enterprise Risk Management, where an essential task is to estimate a firm's distribution of overall or aggregate risk.

Our experience with events that are relatively rare is limited. So, consequently, is our ability to draw definitive conclusions from that experience. There is thus an inherent limit to the benefits that models and data can provide insurers. Models can indeed be valuable, but only as an aid to judgment, not as a substitute for it.

William H. Panning, a Best's Review columnist is executive vice president at Willis Re Inc. He can be reached at [e-mail].