AI, LLM and real-time · RAG · Government records
An assistant that answers staff questions from agency records. Retrieval enforces each user's access and classification before text reaches a model, answers carry citations to the authoritative record, and every interaction is logged so it can be managed under the State Records Act and produced under FOI.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
Staff sign in with SSO and MFA. Group membership and any clearance or caveat attributes arrive as claims, refreshed frequently so a role change takes effect quickly.
A question reaches the assistant API with the user's token. The API holds the session, but not the authority to decide access.
The retrieval service receives the question with the user's identity. It rewrites the query, then builds a filter from the user's entitlements and the classification labels they are cleared to see.
The search index applies the access filter inside the query (document-level security), so documents the user cannot open are never retrieved, scored or sent to a model. Hybrid keyword and vector search with re-ranking improves precision.
The top passages go to the model through the gateway with an instruction to answer only from the sources and to cite them. An Australian-region model and guardrails run through the gateway. Responses without supporting passages are declined.
The API writes a log entry: user, time, question, retrieved record identifiers and versions, model, answer and feedback. The log is retained under the relevant records authority and is searchable for FOI and investigations.
An ingestion pipeline crawls source repositories, extracts text (OCR for scans), splits it into sections that keep headings and context, and attaches metadata: record ID, version, owner, classification, security markings and retention class.
Chunks, embeddings and ACL metadata are written to the index. Permission changes in the source are synchronised on a short cycle and by event, and deletions remove the chunks.
Citations link to the authoritative record in the records system, not to a copy. Users open the source to confirm. Where the assistant output is itself a record of a decision, it can be saved to the records system with its sources.
| Component | What it does | Why this choice |
|---|---|---|
| Microsoft Entra ID (or agency IdP) | SSO, MFA, groups and clearance attributes. | One identity source for access decisions. |
| Assistant API | Chat sessions, citation rendering, feedback and rate limits. | Thin layer, no access decisions. |
| Retrieval service | Query rewriting, entitlement filter, hybrid search, re-ranking and context assembly. | Central place to enforce access before generation. |
| Search index (Azure AI Search, Amazon OpenSearch Service, Vertex AI Search or pgvector) | Vectors, keywords and ACL metadata with document-level security. | Filtering inside the index prevents leakage. |
| Model gateway | Australian-region model, guardrails and cost control. | Shared with other AI workloads. |
| Ingestion pipeline (Azure Functions or Lambda, Textract or Document Intelligence) | Parse, OCR, chunk, label and embed. | Quality of retrieval depends on ingestion quality. |
| Records system (an EDRMS such as Content Manager, or SharePoint with retention labels) | Authoritative records, versions and disposal. | Citations point to the record of truth. |
| Audit log (immutable storage plus SIEM) | Queries, sources, answers and feedback. | Supports FOI, privacy and quality reviews. |
Where access control is enforced
Why. A model cannot reliably withhold content it has been given.
Trade-off. Needs reliable ACL sync and a clean permission model in the sources.
Classification handling
Why. Mixing classifications in one index multiplies the cost of a mistake.
Trade-off. More deployments, but simpler assurance.
Chunking and context
Why. Better answers and better citations.
Trade-off. Higher ingestion complexity and re-indexing cost when a record changes.
Answer policy
Why. Public sector staff need verifiable answers.
Trade-off. Some questions get no answer. Measure and tune rather than relax the rule.
Logging content
Why. These can be subject to FOI and may inform decisions.
Trade-off. The logs hold sensitive material and need the same protection as the sources.
Document-level security, claims-based access, private endpoints, customer-managed key encryption, DLP on outputs, guardrails and injection testing on retrieved text.
Multi-AZ search and API tiers, idempotent ingestion with a dead-letter queue and a documented re-index procedure.
Hybrid search with filters, caching of embeddings and streaming answers. Pre-filtering must be tested at agency scale.
Embed once per version, tier search capacity, use a small model for query rewriting and a larger one only for answers.
Evaluation sets per knowledge domain, source-freshness dashboards, and a review loop with records managers.
Incremental indexing, no duplicate copies of source files and right-sized search replicas.
API, retrieval, search and ingestion run in private subnets or a private environment. Search, storage, key vault and model endpoints use private endpoints. Source connectors use agency private connectivity. Public access is disabled on all data services, and egress is limited to the model gateway.
Users authenticate with SSO and MFA. The retrieval service receives and validates user claims and builds the filter. Connectors and the ingestion pipeline use distinct workload identities with read-only access. The audit log is writable only by the API and readable by a small governance group.
Records keep their classification and retention class in index metadata. Deleting or disposing of a source record must remove its chunks and embeddings. Treat embeddings as derived from the source. Do not use the content to train models. Retain logs under the applicable general disposal schedule and confirm with the agency's records manager.
| Control | How this design addresses it |
|---|---|
| Restrict administrative privileges | Separate duties for platform, index and audit administration. Essential Eight. |
| Multi-factor authentication | Phishing-resistant MFA for staff and administrators. Essential Eight. |
| Records management | Records identified, retained and disposed of under State Records Act obligations, with citations to authoritative copies. |
| Information access | Interaction logs searchable to support FOI processing. |
| Privacy and security of information | Minimisation, access filtering and encryption support APP 6 and APP 11 and SA ISMF controls. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
| Failure mode | Mitigation |
|---|---|
| Permission change not yet reflected in the index | Short sync interval, event-driven updates and a final entitlement check against the source before display. |
| Hallucinated or unsupported answer | Citation requirement, groundedness checks and a decline path. Sample reviews with subject matter experts. |
| Stale or superseded policy retrieved | Version and effective-date metadata, prefer current versions and show the date in the citation. |
| Ingestion backlog or poison document | Queue with dead-letter handling and alerts, and per-source health on a dashboard. |
| Prompt injection hidden in a document | Treat retrieved text as data, strip active content, give the assistant no write tools and monitor outputs. |
Search index capacity, embedding generation, model tokens, OCR and engineering time to clean sources.
AWS · chatbot · guardrails
A customer-facing chatbot with flow control and guardrails kept apart. A deterministic engine owns intents, slots, validation and retries. A catalog of controls across six layers (input, context, model, tool and action, output, human oversight) screens every message in and every reply out, and a small Bedrock model is called only between questions, all on pay-per-use serverless services.
AWS and Azure AI
A grounded question-answering system for internal knowledge: governed ingestion, hybrid retrieval with permissions, a guarded model call, citations, an audit log and a repeatable evaluation harness.
AWS and Azure AI
An agent that can do things, not just answer: a bounded planner, a registry of narrowly scoped tools, policy checks outside the model, human approval for consequential actions and a full action ledger.
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.