Building model trust: strong model risk management can prevent the damaging impact that model misuse or errors have on insurers.
As a result, misuse of models or errors within models can have significant adverse consequences.
Insurer models have become more intricate as computing power has increased, products have become more complex, asset/liability management more sophisticated and regulatory requirements more onerous. Furthermore, for some insurers, particularly reinsurers, the availability and quality of data might be a concern.
For innovative or small insurers, there may not be credible experience supporting the modeling assumptions. Also, new types of models are continuously being developed and used (such as predictive models in life insurance). These are just some of the factors that contribute to increased model risk within the insurance industry.
While most insurers have invested in integrated risk management for a long time, model risk management has recently become an area of emphasis for some insurers.
Leading companies have developed MRM frameworks to mitigate model risk and to become more comfortable with their models. Common elements of MRM frameworks include policies related to model governance, development and implementation, and validation.
Strong model governance is critical to effective MRM. The boards of directors and senior management at best-in-class companies have recognized this by sponsoring, supporting or leading the establishment of MRM frameworks at their companies to help foster an MRM culture.
An MRM framework is an important component of a company's overall risk management approach. As such, the steps involved in developing an MRM framework are similar to the steps used to develop an ERM framework.
For example, a critical element of any risk management approach is to identify a company's specific risks. A company can determine model risk exposures by defining what a model is, creating and maintaining an inventory of models, and categorizing the models into low, medium and high risk with due consideration for materiality. Also, a company would normally set risk limits to ensure that the degree of model risk does not exceed a specified risk tolerance.
Other important elements of governance related to MRM include policies or best practices related to model development, testing, documentation, approvals, version and change control, access rights and validation.
Roles and responsibilities are another important governance consideration. Ideally, model development and maintenance duties would be segregated from model user duties. The internal audit or risk management departments are often involved in measuring the effectiveness of MRM governance.
Some companies have established MRM officers to lead the model risk management function.
Development and Implementation
MRM begins with model development. Ideally, models would be created by experienced developers and subject matter experts using high-quality data, robust methodology and assumptions that are supported by credible experience in a controlled modeling environment.
Models in a controlled environment allow for adequate testing and approvals before implementation. Model documentation would be sufficiently detailed to describe the purpose of the model and how it will be used; data inputs to the model; and model methodology and assumptions, including their rationale. Documentation also would detail the model's compliance with regulations or other standards, the testing performed and known model limitations. Updates and changes to models would be governed by a version- and change-control process.
While some companies have developed standardi2ed templates to facilitate consistent model documentation, resource constraints, company culture or inadequate model governance can make it a struggle for many. Cases with inadequate documentation of models (in particular, no documentation of model limitations) have resulted in models being used for purposes they were not designed for.
We recommend structuring the documentation so it is easily maintained when the model is updated and revised.
Model documentation ensures the model was adequately developed; model owners are accountable; and model users understand how the model operates, how model results should be used and any limitations.
The intended purpose of model validation is regular verification that models are performing as expected. While essential, this process is time-consuming and resource-intensive. As such, priority generally is assigned to newer models, models with the greatest degree of model risk and models that have not recently been reviewed.
To avoid conflicts of interest, the validator should be objective; knowledgeable about the product being modeled, modeling standards of practice, industry practices and regulatory requirements; and not have an interest in the outcome of the validation. For example, model developers or users should not be validators. As a result, model validations are often performed by consultants. Accountability will also be encouraged if there is a sole validator per model.
Some companies have developed model validation templates for validators to use. The translation of relatively high-level validation requirements to detailed and pragmatic validation activities can be challenging.
Regardless of whether a validation template is available, model validators generally develop a validation test plan. All model components are typically validated, including inputs, calculations and output.
An MRM governance review is also typically within the scope of a model validation project. It focuses on the application of MRM governance policy; modeling oversight by senior management and the board; establishment and adherence to model risk limits; maintenance of the model inventory, roles and responsibilities; and various other model controls such as access rights, change processes and approval procedures.
The validator typically starts by reviewing existing model documentation, including support for modeling assumptions and testing that has been performed to evaluate conceptual soundness.
It is also important for model validators to understand the purpose of the model and how it is currently being used, because confirming that the model is being used as intended is an important validation element.
This is even more critical when no support for assumptions exists due to lack of company experience or industry data.
Next, the validator typically checks model output against another model that has been independently developed to confirm that the model has been properly implemented. Since full reconciliations of model output between two independent models can be time-consuming, tolerances often are used to minimize the amount of work.
Validators will often perform additional checks such as comparing model output to historical, actual audited results and testing output sensitivity to assumption changes.
We have found it useful for validators to maintain a review log that categorizes types of issues (such as approximation, error and misuse).
The review log can also contain an estimate of the impact of each issue, which can be assessed by the validator with support from model developers or users.
Thereafter, the validator prepares a model validation report that clearly details the testing performed; the findings from the validation; recommendations for improvement; a prioritized ranking of next steps; and limitations of the validation.
An executive summary is often prepared for senior management to summarize key findings.
Models that contain issues can be updated, replaced or restricted to certain purposes, depending on the company's risk limits and materiality thresholds.
Likewise, any issues related to governance of the model could lead to changes in model governance policies.
When models are changed, this marks the beginning of a new modeling cycle of development and implementation (which includes documentation, testing and approvals), use and validation.
Why MRM Matters
Models are now intrinsic to the operations of insurance companies. Their misuse or errors can have a significant, harmful impact on an insurer. And over time, models have become increasingly complex, making it even more important that insurers pay attention to model risk.
Leading insurers are developing MRM frameworks that include model governance, development and implementation, and validation to mitigate model risk in response to external forces such as regulators, or proactively as companies become more aware of the potential impact of model risk.
Actual or perceived conflicts of interest can diminish an insurer's efforts, so it is important for objectivity to guide MRM activities. When done right, MRM can provide confidence to internal and external stakeholders that a company's models are sound and that their results are reliable.
Key Model Validation Principles
1. Model design and build need to be consistent with the model's intended purpose.
2. Ensure that model validation is an independent process.
3. Establish an owner of model validation.
4. Ensure appropriateness of established model governance.
5. Make model validation efforts proportional to evidenced areas of materiality and complexity.
6. Validate the model components.
7. Address limitations of model validation.
8. Document the model validation.
Source: The North American CRO Council
Inside and Outside influences
Model risk management typically is driven by some or all of these factors:
* Increased frequency of financial restatements due to model errors.
* Ineffectiveness of some models connected with the recent financial crisis.
* Sarbanes-Oxley requirements assigning management responsibility for financial statements.
* Own Risk and Solvency Assessment requirements developed by the National Association of Insurance Commissioners for insurers to self-assess their risk management adequacy and prospective solvency under normal and severe stress scenarios.
* Board of Governors of the Federal Reserve System SR 11 -7 Supervisory Guidance on Model Risk Management.
* Solvency II and its Internal Model Application Process.
* The North American CRO Council's Model Validation Principles Applied to Risk and Capital Models in the Insurance Industry.
* Greater focus on models by internal and external auditors and regulators.
* Management's desire to understand model limitations and weaknesses.
Source: Towers Watson
The Issue: Insurance companies are using models that are more complex than ever.
* The Problem: Even slight modeling missteps can have an enormous impact on an insurance company's financials, risk management or decision-making.
* The Solution: Model risk management provides a structured, objective system for making sure they're reliable.
Contributors: Towers Watson's Dom Lebel is a director responsible for financial modeling and reporting for the Americas Life practice; and Sebastien Cimon Gagnon is a senior actuarial analyst. They can be reached at firstname.lastname@example.org and sebastien.cimon.gagnon@ towerswatson.com
|Printer friendly Cite/link Email Feedback|
|Author:||Lebel, Dom; Gagnon, Sebastien Cimon|
|Date:||Sep 1, 2014|
|Previous Article:||A trio of technologies: Insurers are tapping social media, flow analytics and cloud computing to streamline operations and deepen their bond with...|
|Next Article:||Best's credit rating actions.|