Scoring models and automated decisioning systems have decided who gets credit, services, attention and scrutiny for decades. The discipline for governing them is called model risk management. Its best-known reference is the US supervisory guidance known as SR 11-7, which many banks and insurers worldwide use as a model. The same ideas now apply to modern machine learning and to AI-assisted decisions in any sector. APRA has also said regulated entities are expected to manage AI risk within existing prudential standards, so these practices matter well beyond banking.
The core elements
- Model inventory. Every model, including vendor models and spreadsheets that act as models, with owner, purpose, materiality and status.
- Development standards. Documented data, assumptions, methodology and limitations, with reproducible training.
- Independent validation. A team separate from the developers assesses conceptual soundness, data quality, performance and outcomes before use and on a schedule after.
- Ongoing monitoring. Track input drift, score distributions, accuracy against outcomes and override rates, with thresholds that trigger review.
- Governance. Clear roles, change control, issue tracking and reporting to a risk committee.
Explainability and fairness
People affected by a decision need an understandable reason, and reviewers need to see what drove a score. Choose the simplest model that meets the need, use explanation methods such as feature attributions where models are complex and test whether the explanations are stable and truthful. For fairness, define what unequal treatment would mean in context, test performance and outcomes across relevant groups, examine features that act as proxies for protected attributes and record the trade-offs you accept. Discrimination and privacy laws apply to automated decisions as they do to human ones.
Champion and challenger
Run a challenger model alongside the current champion on live or shadow traffic. Compare performance, stability and fairness before promoting it. This makes improvement and retraining safe and gives validators a baseline.
Retraining controls
Retraining is a change, not a routine. Version datasets and code, validate the new model before release, document why it was retrained, keep the ability to roll back and avoid feedback loops where the model’s own decisions shape its future training data.
Human review and the right to challenge
Design the route for human review and appeals into the process. For decisions that significantly affect people, give a clear explanation, a contact route and a timely review by someone with authority to change the outcome. From 10 December 2026, Privacy Act transparency rules require covered organisations to disclose in their privacy policy the kinds of decisions that are made or substantially assisted by computer programs using personal information. That includes rules engines as well as machine learning.
Applying it to generative AI
The same structure applies to LLM-based decisions: inventory, validation using evaluation sets, monitoring and human review. The techniques differ, but the discipline carries over.
See the architectures for ML model risk management and the MLOps lifecycle and automated decisioning with explainability, review and appeals.
How EITSAA helps
We help you design and evidence AI governance that matches your risk, from the register and policy through to technical guardrails and audit-ready evidence. Explore AI Governance & Assurance, talk to our team or call 0402 401 584.
This is general guidance, not legal advice. Check current requirements with the relevant authority and take advice on your own position.