Model risk management (MRM) is the set of policies, procedures and controls that a financial institution uses to identify, measure and limit the losses that can arise from its models. Model risk is the chance of an adverse outcome, such as financial loss, a poor decision or damage to reputation, caused by a model that is flawed or by a sound model that is used in the wrong way. It is commonly treated as a part of operational risk with its own governance.
Banks, insurers and asset managers depend on models to price instruments, grant credit, set capital, detect fraud and report to regulators. A model, in the supervisory sense, is any quantitative method that turns input data into estimates by applying statistical, economic or mathematical theory. The definition covers a credit scoring model, an option pricing model and a machine learning system alike.
Sources of model risk
- Errors in design or build. A model may rest on faulty theory, wrong assumptions, poor data or coding mistakes.
- Misuse. A model built for one product, market or period may be applied to another for which it was never tested.
- Changing conditions. Relationships estimated from history can break down, so a model that once worked loses accuracy without any visible fault.
- Excessive fitting. A model tuned too closely to past data, a problem known as overfitting, performs well in testing and badly in use.
A frequently cited case is the loss of more than $6 billion on credit derivatives at JPMorgan Chase in 2012. Reviews found that a new value at risk model, run through spreadsheets containing errors, had understated the risk of the positions.
Core elements of a framework
Most frameworks follow the same life cycle. Each model is recorded in an inventory with an owner, a stated purpose and a risk rating. Development is documented so that a third party could rebuild the model. Before use, and at intervals afterwards, the model is subjected to validation by staff who are independent of its developers. Supervisors describe three parts to validation:
- Conceptual soundness. The reviewer assesses the theory, assumptions, data and design choices.
- Ongoing monitoring. The institution checks that the model is still used as intended and continues to perform.
- Outcomes analysis. Model outputs are compared with actual results, for example by backtesting.
The guiding principle is effective challenge: critical review by people with the competence, independence and standing to have their findings acted upon. Senior management and the board set the policy and receive reports on aggregate model risk. Internal audit tests whether the framework is followed. Limits on use, conservative adjustments and capital buffers are applied to models with known weaknesses.
Regulatory standards
The reference text in the United States is the Supervisory Guidance on Model Risk Management issued in 2011 by the Federal Reserve, as letter SR 11-7, and by the Office of the Comptroller of the Currency. It set out the definition of a model, the three elements of validation and the principle of effective challenge, and has been widely copied outside banking.
In the United Kingdom, the Prudential Regulation Authority published model risk management principles for banks that took effect in 2024. In the euro area, the European Central Bank reviews the internal models that banks use to calculate capital requirements under the standards of the Basel Committee on Banking Supervision. Accounting models are also in scope, including those for expected credit loss.
Artificial intelligence and machine learning
The spread of artificial intelligence has widened the task. Machine learning models may have thousands or millions of parameters and no simple account of why a given output was produced, which makes conceptual review harder. They can absorb bias from training data, a concern in lending, and their accuracy can decay as the data they meet drifts away from the data they learned from. Many are retrained often, so validation has to be closer to continuous.
Models bought from vendors, including a large language model accessed as a service, raise a further difficulty: the institution remains responsible for the model's use but cannot inspect how it was built. Supervisors have generally held that existing principles apply to such tools, and firms are adapting them with added testing for explainability, fairness and stability.
Limitations
Model risk cannot be removed, only managed. Validation is costly and slow, and large institutions hold thousands of models, so effort has to be ranked by materiality. Independent reviewers may lack the specialist knowledge of the developers. A further hazard is that a formal approval process creates false comfort, since a validated model is still a simplification that can fail in conditions outside its experience.