Where to enforce
Central gateway plus library checks in apps.
Why. Consistency with defence in depth.
Trade-off. Gateway latency and a shared dependency.
AI governance and assurance · LLM guardrails · policy engine
LLM guardrails architecture: input and output guard layers, policy engine, PII redaction, grounding checks and audit
A layered guardrail pattern around an LLM application: authenticate and screen the request, redact personal data, retrieve only what the user may see, call the model, then validate grounding and policy on the way out, with every step logged for audit.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
- OWASP Top 10 for LLM Applications
- NIST AI RMF Generative AI Profile
- Voluntary AI Safety Standard
- ISO/IEC 42001
- ACSC Essential Eight
- Privacy Act and APPs
Architecture
- People and clients
- Network and edge
- Security and identity
- AI and ML
- Monitoring, audit and analytics
- Request or data flow
- Asynchronous or optional
- Boundary: region, account, network or trust zone
Request and data flow
A user or application calls the gateway with a workload or user identity. Requests without valid identity or quota are rejected.
The policy engine decides whether this caller may use this use case with this data class and model, and sets limits for tokens, steps and spend.
The input filter screens for injection patterns, harmful content and out-of-scope topics, and treats retrieved content as data.
The request passes through PII detection so that personal and confidential data is masked or tokenised unless the policy allows it to go to the model.
Retrieval queries the index with the caller's entitlements, so only documents the user may see can enter the prompt.
The prompt, built from a versioned system prompt and minimal context, is sent to the approved, pinned model in the permitted region.
The answer returns to the guardrail layer rather than directly to the user. A grounding check compares claims with retrieved sources, requires citations and abstains when support is weak.
The output filter applies content and format policy, removes anything that should not leave, and restores tokenised values where permitted.
Prompts, retrieved context references, filter decisions and costs are written to a masked, immutable audit trail with trace identifiers.
Components and why they were chosen
| Component | What it does | Why this choice |
|---|---|---|
| AI gateway (API gateway with an AI proxy such as LiteLLM, Azure API Management or Amazon API Gateway) | Single entry with identity and quotas. | Every app inherits the same controls. |
| Policy engine (OPA, Cedar) | Policy as code on use case, data class and model. | Versioned and testable. |
| Guardrail services (Amazon Bedrock Guardrails, Azure AI Content Safety, NeMo Guardrails, Llama Guard) | Input and output classification. | Defence in depth around the model. |
| PII detection (Amazon Comprehend, Microsoft Presidio, Purview) | Detect and mask personal data. | Reduces leakage risk and privacy exposure. |
| Retrieval index with access control | Authorised retrieval only. | Prevents oversharing. |
| Audit store (object storage with retention lock) | Immutable traces. | Evidence and investigation. |
Design decisions and trade-offs
Filter strictness
Tier-based thresholds measured against false positives.
Why. Balances safety and usefulness.
Trade-off. Needs tuning and appeals.
Model-based or rule-based checks
Combine deterministic rules with classifier models.
Why. Rules are predictable, models catch variants.
Trade-off. Classifiers add cost and can be evaded.
Redaction method
Tokenise reversible values and mask the rest.
Why. Preserves utility without exposing data.
Trade-off. Token vault becomes sensitive.
Fail open or closed
Fail closed for high tiers, degrade gracefully for low.
Why. Matches the consequence of failure.
Trade-off. Closed failure reduces availability.
Well-architected pillars
Security
Defence in depth across input, retrieval, model and output, with least privilege and treatment of content as untrusted.
Reliability
Gateway deployed across zones with timeouts, fallbacks and tested fail-closed behaviour.
Performance
Run independent checks in parallel and cache policy decisions. Budget latency per layer.
Cost optimisation
Classifier calls and extra tokens have a cost. Tier checks by risk and sample low-risk traffic.
Operational excellence
Version and test policies, review blocked-request samples and tune with evidence.
Sustainability
Use small classifier models and cache repeated checks.
Network, identity and data
Networking
The guardrail layer runs inside a private network. Model, retrieval and logging endpoints are reached by private connectivity in the permitted Australian region. Applications cannot call model providers directly.
Identity and least privilege
Callers are authenticated with SSO or workload identity. The user's entitlements flow to retrieval. Guardrail services use scoped roles, and secrets are held in a managed vault.
Data and encryption
Classify data before it reaches the layer. Mask personal information in logs and keep tokens in a restricted vault. Set retention for prompts and outputs by class and records obligations.
Security and compliance mapping
| Control | How this design addresses it |
|---|---|
| Prompt injection and insecure output handling | Input screening, untrusted-content handling and output validation address OWASP LLM01 and LLM05. |
| Sensitive information disclosure | PII redaction and access-controlled retrieval address OWASP LLM02 and privacy obligations. |
| Unbounded consumption | Rate, token and spend limits address OWASP LLM10. |
| Event logging | Masked, immutable traces support investigation and audit. Essential Eight. |
| Testing and monitoring | Guardrail effectiveness is tested and monitored, supporting the Guidance for AI Adoption. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
Failure modes, targets and scaling
- Target RTOFollows the application. Typically under one hour for the gateway with multi-zone deployment.
- Target RPONear zero for policy and configuration in version control. Audit records are replicated.
- Scaling and limitsSized by requests per second and the cost of the most expensive check.
| Failure mode | Mitigation |
|---|---|
| Filter blocks legitimate work | Measure false positives, offer an appeal path and tune by tier. |
| Novel injection bypasses the filter | Layered controls, tool least privilege and regular red-teaming. |
| Redaction misses an identifier | Test with seeded data, use multiple detectors and mask logs. |
| Grounding check rejects correct answers | Calibrate on labelled samples and allow abstention with escalation. |
| Guardrail service is unavailable | Fail closed for high tiers, degrade to a safe fallback and alert. |
Observability and runbooks
- Block, redact and abstain rates by rule.
- Injection attempts and blocked exfiltration.
- Groundedness scores on sampled traffic.
- Latency by layer and cost per request.
Cost drivers and levers
Classifier calls, extra tokens, gateway compute and log storage.
- Tier checks by risk.
- Cache repeated decisions.
- Use small models for classification.
- Sample low-risk traffic for scoring.
Implementation and fit
Phased approach
- Introduce the gateway with identity, quotas and logging.
- Add PII redaction and access-controlled retrieval.
- Add input and output filters and grounding checks, tested against an evaluation set.
- Add policy as code, red-team regression and monitoring dashboards.
Anti-patterns to avoid
- Relying on the system prompt as the only control.
- Filtering input but not output.
- Passing model output directly to a database or shell.
- Logging raw prompts with personal data.
- Never measuring false positives.
When to use it
- Customer, staff or citizen-facing LLM applications.
- Any application that sends organisational data to a model.
When not to
- A personal experiment with no organisational data.
- Systems where a deterministic workflow would do the job.
More in ai governance and assurance
AI agents · safety controls
AI agent safety architecture: tool permissions, sandboxing, approval gates, spend limits and a kill switch
How to let an AI agent act without letting it do harm: a permission broker issues narrow, short-lived tool access, risky code runs in a sandbox, high-impact actions wait for human approval, budgets cap spend and a kill switch stops everything.
ML · model risk · MLOps
ML model risk management and MLOps lifecycle: feature store, validation, registry, drift monitoring and challenger models
A governed lifecycle for scoring and prediction models: reproducible training from a feature store, independent validation including fairness and explainability, a registry with approval status, controlled deployment, drift monitoring and champion and challenger comparison.
Automated decisioning · human review
Automated decisioning platform: rules and models with explainability, human review and appeals
A decisioning pattern for decisions that affect people: rules and model scores combine in a versioned decision service, every outcome carries reasons, uncertain or adverse cases go to a trained reviewer and affected people have a clear route to challenge.
Related insights
Discuss this architecture
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.