Model complexity
Prefer the simplest model that meets the need.
Why. Easier to validate, explain and defend.
Trade-off. May give up some accuracy.
AI governance and assurance · ML · model risk · MLOps
ML model risk management and MLOps lifecycle: feature store, validation, registry, drift monitoring and challenger models
A governed lifecycle for scoring and prediction models: reproducible training from a feature store, independent validation including fairness and explainability, a registry with approval status, controlled deployment, drift monitoring and champion and challenger comparison.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
- Model risk management practice (SR 11-7 style)
- NIST AI RMF
- ISO/IEC 42001
- Voluntary AI Safety Standard
- Privacy Act and APPs
- APRA CPS 230 and CPS 234
Architecture
- Monitoring, audit and analytics
- AI and ML
- Security and identity
- Compute and containers
- Request or data flow
- Asynchronous or optional
- Boundary: region, account, network or trust zone
Request and data flow
Source data is governed and lineage is recorded. Features are computed once and published to a shared, versioned feature store so training and serving use identical logic.
The training pipeline pulls versioned features and code, trains reproducibly and logs data, parameters and metrics.
A candidate model goes to independent validation, which is a team separate from the developers.
Validation tests conceptual soundness, data quality, performance, stability, explainability and fairness across relevant groups, and documents limitations.
Validated models are recorded in the registry with a model card, lineage and approval status.
The model inventory records owner, purpose, materiality, validation date and next review.
A model risk committee reviews the validation report and approves, approves with conditions or rejects.
Approved models are promoted as the champion, with canary release and one-step rollback.
A challenger model runs on shadow traffic so performance, stability and fairness can be compared before any switch.
Monitoring receives champion and challenger outcomes and tracks input drift, score distribution, overrides and fairness. Thresholds trigger revalidation or retraining under change control.
Components and why they were chosen
| Component | What it does | Why this choice |
|---|---|---|
| Feature store (SageMaker Feature Store, Feast, Databricks, Azure ML) | Consistent, versioned features. | Prevents training and serving skew. |
| Experiment and training tracking (MLflow) | Reproducible runs. | Evidence and rollback. |
| Model registry | Versions, cards and status. | Single source of truth for approved models. |
| Validation tooling (SHAP, fairness toolkits, backtesting) | Explainability and bias tests. | Independent, repeatable evidence. |
| Serving platform with shadow and canary | Safe release. | Limits blast radius. |
| Monitoring (Evidently, CloudWatch, Azure Monitor) | Drift and performance. | Catches quiet decay. |
Design decisions and trade-offs
Validation independence
A separate validation team or external reviewer for high materiality.
Why. Effective challenge.
Trade-off. Capacity and cost.
Fairness metric
Select metrics with stakeholders and document trade-offs.
Why. No single metric fits every context.
Trade-off. Metrics can conflict.
Retraining trigger
Scheduled review plus threshold-triggered retraining under change control.
Why. Avoids silent decay and uncontrolled change.
Trade-off. Process overhead.
Feedback loops
Hold out data and monitor for loops where decisions shape future data.
Why. Prevents self-reinforcing bias.
Trade-off. Needs experiment design.
Well-architected pillars
Security
Access control on features and models, signed artifacts, protected training data and model extraction defences for exposed endpoints.
Reliability
Versioned rollback, multi-zone serving and a documented fallback such as a rules baseline.
Performance
Low-latency feature retrieval and autoscaled serving.
Cost optimisation
Right-size training, reuse features and retire unused models.
Operational excellence
Scheduled revalidation by materiality and a monitored inventory.
Sustainability
Avoid needless retraining and use efficient model classes.
Network, identity and data
Networking
Training and serving run in private networks. Data access is through private endpoints. Validation environments are separate from development. Exposed endpoints sit behind an authenticated gateway.
Identity and least privilege
Developers cannot approve their own models. Validators have read access to artifacts and write access only to validation records. Service identities are scoped by environment.
Data and encryption
Document the training data, its provenance, representativeness and known gaps. Minimise personal information and protect sensitive attributes used for fairness testing. Retain evidence for the model's life.
Security and compliance mapping
| Control | How this design addresses it |
|---|---|
| Model inventory and governance | Materiality-based review aligns with model risk management practice. |
| Independent validation | Effective challenge supports the Guidance for AI Adoption testing practice and ISO/IEC 42001. |
| Privacy and fairness | Data minimisation and fairness testing support Privacy Act obligations and AI Ethics Principles. |
| Change management | Registry approval and canary release control change. |
| Operational resilience | Rollback and fallback support CPS 230 expectations. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
Failure modes, targets and scaling
- Target RTOFollows the decision process. Fall back to the previous champion or a rules baseline immediately.
- Target RPORegistry, features and monitoring data are backed up. Training can be reproduced from versions.
- Scaling and limitsSized by scoring volume and retraining cadence.
| Failure mode | Mitigation |
|---|---|
| Training and serving features differ | Shared feature store and parity tests. |
| Model decays unnoticed | Drift and outcome monitoring with thresholds and owners. |
| Fairness regression after retraining | Fairness checks as a release gate. |
| Validator and developer share blind spots | Independent validation and periodic external review. |
| Vendor model changes silently | Contract notice terms, version pinning and benchmark tests. |
Observability and runbooks
- Input and score drift by feature.
- Outcome accuracy and calibration.
- Fairness metrics by group.
- Override rates and latency.
Cost drivers and levers
Training compute, feature computation, serving and validation effort.
- Reuse features across models.
- Schedule retraining by evidence.
- Retire unused models.
- Use batch scoring where real time is not needed.
Implementation and fit
Phased approach
- Create the model inventory and materiality tiers.
- Stand up tracking, registry and a reproducible training pipeline.
- Introduce independent validation and approval for material models.
- Add drift and fairness monitoring, challengers and retraining controls.
Anti-patterns to avoid
- Models living only in notebooks.
- Validation by the developer.
- Deploying without rollback.
- Monitoring uptime but not accuracy.
- Ignoring models bought from vendors.
When to use it
- Scoring, prediction, fraud and risk models.
- Models that inform decisions about people or money.
When not to
- One-off analyses that do not drive decisions.
- Where a transparent rule is sufficient and safer.
More in ai governance and assurance
Automated decisioning · human review
Automated decisioning platform: rules and models with explainability, human review and appeals
A decisioning pattern for decisions that affect people: rules and model scores combine in a versioned decision service, every outcome carries reasons, uncertain or adverse cases go to a trained reviewer and affected people have a clear route to challenge.
Evaluation · red-teaming · CI
AI evaluation and red-team pipeline: golden sets, adversarial tests, release gates and CI
A pipeline that turns testing into a release gate: golden sets and adversarial suites run in CI against every change, scores are compared with thresholds by risk tier, failures block promotion and results become evidence.
Observability · incident response
AI observability and incident response: tracing, quality metrics, drift detection, alerts and runbooks
How to see what AI systems are actually doing and respond when they misbehave: end-to-end tracing, quality and safety scoring on sampled traffic, drift and cost detection, alerting tied to runbooks and post-incident learning that feeds testing.
Related insights
Discuss this architecture
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.