Professor: cat models lack a common guide.

There is no measuring stick for catastrophe models and without one, the models can't be fairly evaluated for their effectiveness, according to a scientist at the National Center for Atmospheric Research in Colorado.

Roger Pielke Jr., also a professor at the University of Colorado, said this may be among the greatest challenges cat modelers and the insurers that use them need to overcome--more than the fine-tuning of claims data and other issues brought to light by Hurricane Katrina.

"People are using the models without a good knowledge of their skill. That's a pretty dodgy situation," Pielke said. "People should be informed. What is it that these tools are telling us?"

Allstate Corp. recently said it didn't buy reinsurance in Louisiana because models showed a "one-in-500-year" risk of a Katrina striking Louisiana. A one-in-100-year event means there's a 1% annual probability of such a strike; a one-in-500-year event means there's a 0.2% annual chance. Instead, Allstate concentrated its reinsurance program on high-risk areas where the company could recoup the cost of the reinsurance through premium increases, such as Florida.

Other insurers have questioned the use of cat models, claiming they failed to predict how disastrous a storm like Katrina could be.

Pielke said reactions like that are unwarranted. Hurricane-loss models, he said, are similar to models being used in aircraft design, weather prediction, government budgeting and financial management, all of which are accepted and unquestioned.

"The industry has benefited a great deal from having catastrophe models. But at the same rime, nobody knows how good cat models really are. No one has taken responsibility for evaluating them," he said. "Unless someone takes responsibility for providing an independent measuring stick, the measure depends on the outcomes."

But just comparing what a model predicts and what happened is insuff<cient, according to Pielke. It doesn't provide enough information by which to judge the model.

"The point is you need a baseline," Pielke said. "There's no baseline for cat models."

Pielke said past cat losses effectively would have to be adjusted to reflect the change in conditions over time to create a baseline for judging models.

Loss/Risk Management Notes is compiled by Senior Associate Editor Meg Green.