AI, LLM and real-time · Agentic AI · MCP
A platform for agents that act in enterprise systems: a supervisor and specialist agents, an MCP gateway that holds every tool credential, a policy engine deciding each call, human approval for consequential steps, continuous evaluation and red-teaming, and a complete audit trail.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
A user submits a request through a channel. The gateway authenticates with SSO, resolves entitlements and session context, and applies rate limits.
A supervisor agent receives the task with the user's identity context and a budget for steps, time and tokens. It decides whether to use a deterministic workflow, a single specialist or several.
The supervisor delegates sub-tasks to specialist agents (case, finance, HR, IT). Each specialist has a narrow prompt, a short tool allow-list and its own evaluation set, which keeps behaviour testable.
Reasoning calls go through the model gateway, which applies guardrails, routes to an approved model, caches where safe and attributes cost to the team and use case.
When an agent proposes an action, the call goes to the MCP gateway. Tools are small, typed, versioned servers. Agents never hold credentials, and the gateway validates arguments against the tool schema.
The policy engine decides allow, deny or escalate using the user, the agent, the tool, the arguments and the risk tier. Prompt content is never trusted as authority.
Consequential actions (payments, entitlements, external communications, deletions) pause in an approval inbox showing the exact arguments and evidence. The approver can approve, edit or reject, cannot approve their own request, and the decision resumes the workflow.
Approved calls reach the enterprise system with a per-tool identity (a workload identity or a narrowly scoped service account), idempotency keys and timeouts. Results return through the gateway, which filters them before the agent sees them.
Every call, decision, approval and result is written to an append-only ledger with a hash chain, linked to the conversation trace.
Evaluation runs on every change: regression scenarios, tool-use accuracy, groundedness and a red-team suite covering prompt injection, tool abuse and data exfiltration. Failing runs block release.
| Component | What it does | Why this choice |
|---|---|---|
| Agent gateway | Authentication, session handling, throttling and channel adapters. | One control point and one identity story. |
| Orchestration (LangGraph, Microsoft Agent Framework, Amazon Bedrock AgentCore, Google ADK or Temporal for durable flows) | Supervisor and specialist graphs with step limits and checkpoints. | Use a framework for control flow, but keep policy and state outside it. |
| Model gateway | Routing, guardrails, quotas and cost attribution. | Model choice becomes configuration. See the sovereign gateway architecture. |
| MCP tool gateway | Hosts or proxies MCP servers, enforces schemas, brokers credentials and logs every call. | Agents get capability without secrets, and every tool is a governed product. |
| Policy engine (Cedar, OPA) | Authorisation by user, agent, tool, arguments and risk tier. | Deterministic enforcement outside the model. |
| Approval service (Teams, ServiceNow or a console) | Human decisions with evidence and separation of duties. | Accountability for consequential actions. |
| Workflow engine (Temporal, Step Functions, Camunda) | Durable execution for long processes, timers and compensation. | Processes outlive a chat session. |
| Evaluation and red-team harness (promptfoo, Inspect, Azure AI Evaluation, Bedrock Evaluations) | Scenario replay, scoring and adversarial testing. | Quality and safety as release gates. |
| Audit ledger (S3 Object Lock, immutable SIEM index or a hash-chained table) | Tamper-evident action history. | Supports investigation, assurance and FOI. |
Single agent or multi-agent
Why. Narrow agents are easier to test and to restrict.
Trade-off. More moving parts and extra latency from delegation. Do not split without a clear reason.
Where credentials live
Why. A hijacked agent cannot exfiltrate a key it never sees.
Trade-off. The gateway is a critical and high-value component and needs strong hardening.
How authority is decided
Why. Autonomy scaled to consequence.
Trade-off. Policy authoring is real engineering work and needs product owners.
End-user delegation
Why. Avoids a super-powered service account.
Trade-off. Not every legacy system supports delegation, so those need a policy-checked service identity.
Evaluation approach
Why. Agents regress silently when prompts, tools or models change.
Trade-off. Ongoing cost to curate scenarios and review samples.
Credentials only in the tool gateway, deny-by-default policy, result filtering, injection defences at every trust boundary, sandboxing for code-execution tools and egress allow-lists.
Checkpointed agent state, durable workflows, idempotent tools, timeouts and circuit breakers, model fallbacks and safe hand-over to people.
Parallel read-only tool calls, streaming, small models for routing and caching of tool results.
Step and token budgets per task, per-team quotas, model tiering and retirement of low-value agents based on evidence.
Versioned prompts, tools and policies, canary releases, per-tool kill switches and trace-based debugging.
Prefer code over model calls where the task is deterministic, and cap loops.
Platform services run in private subnets or a private container environment. Tool servers sit behind the MCP gateway and are reachable only from it. Egress is limited to approved enterprise endpoints and model providers through private endpoints or a controlled proxy. Browsing or code-execution tools run in isolated sandboxes with no access to internal networks.
Users authenticate with SSO and their identity flows to policy and, where supported, to tools through token exchange. Each tool has its own workload identity or scoped service account with short-lived credentials from a secrets manager. Agents are first-class principals in the policy engine with an owner, a purpose and an expiry. Approvers cannot approve their own requests.
Treat all retrieved content and tool output as untrusted input. Classify data at ingestion and filter results by user entitlement before they enter a prompt. Define retention for traces, conversation memory and approvals. Keep the action ledger immutable, and keep prompt content separate from ledger metadata so privacy requests can be handled.
| Control | How this design addresses it |
|---|---|
| Restrict administrative privileges | Per-tool least privilege and no standing admin credentials. Essential Eight. |
| Multi-factor authentication | Phishing-resistant MFA for users and approvers. Essential Eight. |
| Human oversight | Risk-tiered approvals and a kill switch support the Voluntary AI Safety Standard on human control. |
| Event logging | Ledger and traces support ISM event logging and incident investigation. |
| LLM-specific threats | Mapped to the OWASP LLM Top 10: prompt injection, insecure output handling, excessive agency and sensitive information disclosure. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
| Failure mode | Mitigation |
|---|---|
| Prompt injection in a retrieved document or email | Content treated as untrusted, policy by user and tool not by prompt, approval for impactful actions, and result filtering. |
| Agent loops or overspends | Step, time and token budgets with hard stops and alarms. |
| Tool succeeds but the agent crashes | Idempotency keys and durable workflow state so the process resumes without repeating the action. |
| A tool server is compromised | Per-tool identity limits blast radius, mTLS from the gateway, egress allow-lists and rapid credential revocation. |
| Quality regression after a prompt or model change | Evaluation gate, canary release and rollback. |
Model tokens (agent loops multiply them), orchestration and workflow infrastructure, evaluation runs and reviewer time.
RAG · Government records
An assistant that answers staff questions from agency records. Retrieval enforces each user's access and classification before text reaches a model, answers carry citations to the authoritative record, and every interaction is logged so it can be managed under the State Records Act and produced under FOI.
AWS · chatbot · guardrails
A customer-facing chatbot with flow control and guardrails kept apart. A deterministic engine owns intents, slots, validation and retries. A catalog of controls across six layers (input, context, model, tool and action, output, human oversight) screens every message in and every reply out, and a small Bedrock model is called only between questions, all on pay-per-use serverless services.
AWS and Azure AI
A grounded question-answering system for internal knowledge: governed ingestion, hybrid retrieval with permissions, a guarded model call, citations, an audit log and a repeatable evaluation harness.
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.