Most security problems in new software are designed in, not typed in. A form that trusts a hidden field, an API that never checks who owns a record, or a chatbot that can call a tool nobody meant it to reach are decisions made at the whiteboard, long before testing. Threat modelling is the habit of asking “what could go wrong here?” while those decisions are still cheap to change. This guide explains two frameworks that South Australian councils, agencies, not-for-profits and SMEs can use: STRIDE for ordinary applications and APIs, and MAESTRO for AI agents and chatbots that retrieve data and use tools. It is general guidance, not a substitute for an assessment of your own system.
What threat modelling is
A threat model is a short, structured conversation about a system, written down. The OWASP and Threat Modeling Manifesto community frame it as four questions:
- What are we working on? A simple diagram of users, components, data stores and the trust boundaries between them.
- What can go wrong? Threats found by walking the diagram with a framework such as STRIDE or MAESTRO.
- What are we going to do about it? A control, a design change, an accepted risk with a named owner, or a test.
- Did we do a good enough job? A review that checks the controls exist and the model still matches the system.
The output is a short list of threats, each with an owner and a decision. It does not need special tools. A whiteboard photo, a table and a ticket for each action are enough to start.
When to threat model
- At design, before a new application, portal, integration or AI feature is built or bought.
- When something important changes, such as a new data source, a new user group, a move to the cloud, a new third-party API or giving an AI agent a new tool.
- Before go-live for internet-facing or higher-risk systems, so security testing can target the threats that matter. Our software testing and UAT guide shows where this fits.
- After an incident or near miss, to find the design gap behind it. See our cyber incident response plan and AI incident response playbook.
- Each year for systems that hold personal information or support essential services.
Keep it proportionate. A brochure website needs a short conversation. A resident portal with payments, or a chatbot connected to your records system, deserves a proper session with the people who build, run and own it.
STRIDE: six questions for any application
STRIDE came out of Microsoft. Loren Kohnfelder and Praerit Garg described it in an internal 1999 paper, The threats to our products, and it became the core of Microsoft’s threat modelling practice. Each letter names a kind of threat, and each threat breaks a security property you want the system to have. Walk every component and every data flow on your diagram and ask the six questions.
| Threat | What it breaks and typical examples | Typical controls and how to check them |
|---|---|---|
| Spoofing | Authentication. Someone pretends to be another user or system: stolen session cookies, reused passwords tried at scale, forged or replayed tokens. | MFA, short-lived sessions, signed tokens validated on every call. Check that MFA cannot be skipped and that sessions expire and cannot be replayed. |
| Tampering | Integrity. Data or code is changed without permission: edited prices or IDs in a request, cross-site request forgery, altered files or build steps. | Server-side validation, CSRF protection, signed builds and protected pipelines. Check by modifying requests in an intercepting proxy and by fuzzing inputs. |
| Repudiation | Accountability. Someone can deny an action because there is no trustworthy record: missing, editable or deleted audit logs. | Audit logs for sensitive actions, stored where the actor cannot change them, with synchronised time. Check that a test action can be traced end to end. |
| Information disclosure | Confidentiality. Data reaches people who should not see it: changing a record number in a URL to see someone else’s file, SQL injection, error pages that reveal internals. | Object-level access checks, parameterised queries, generic error messages, encryption. Check with a second test account that one user cannot read another’s records. |
| Denial of service | Availability. The service is slowed or stopped: floods of requests, very large uploads, expensive queries, abuse of AI usage quotas. | Rate limits, size limits, timeouts, autoscaling with budgets. Check with load and stress tests and by confirming the limits actually trigger. |
| Elevation of privilege | Authorisation. Someone gains more access than intended: a standard user reaching admin functions or another customer’s area, broken role checks. | Least privilege, role checks on the server for every function. Check by repeating admin actions with lower-privilege accounts. |
STRIDE works well for web apps, mobile apps, APIs, integrations and cloud platforms. It maps neatly to the Essential Eight and to OWASP’s Application Security Verification Standard when you turn threats into requirements.
Why AI agents need more than STRIDE
STRIDE still applies to an AI system, but it was designed for software whose behaviour is written by people. An agent built on a large language model behaves differently. It reads untrusted text that can carry instructions, it chooses which tools to call, its answers vary from run to run, and several agents may pass work to one another. A poisoned document in a knowledge base or an instruction hidden in an email can change what the agent does without anyone touching the code. You need a way to look for those threats layer by layer.
MAESTRO: threat modelling for agentic AI
MAESTRO stands for Multi-Agent Environment, Security, Threat, Risk and Outcome. Ken Huang published it through the Cloud Security Alliance (CSA) on 6 February 2025. It builds on STRIDE and similar methods and adds AI-specific threats, organised around a seven-layer reference architecture for agentic systems. You model threats at each layer, then look for cross-layer chains, where a weakness in one layer opens a path in another. The OWASP GenAI Security Project points practitioners to MAESTRO in its Agentic AI: Threats and Mitigations guide and applies it in its Multi-Agentic System Threat Modelling Guide (version 1.0, April 2025).
| Layer | What it covers and example threats | Questions to ask |
|---|---|---|
| 1. Foundation models | The underlying model. Jailbreaks, poisoned training or fine-tuning data, backdoors, model theft, unannounced model changes by a vendor. | Which model and version are we using, where does it run, and how do we notice if its behaviour shifts? |
| 2. Data operations | Knowledge bases, vector databases, ingestion pipelines and retrieval (RAG). Poisoned documents, indirect prompt injection in retrieved text, data leaking across users. | Who can add content to what the agent reads, and does retrieval respect each user’s access rights? |
| 3. Agent frameworks | Orchestration, prompts, tools and plugins. Tool misuse, unvalidated tool arguments, an agent being talked into actions outside its purpose. | What can each tool do, what arguments does it accept, and what needs human approval? |
| 4. Deployment and infrastructure | Hosting, containers, serverless functions, networks and identities. Over-privileged service roles, exposed endpoints, misconfiguration, resource exhaustion. | Which cloud roles does the agent run as, and could they be narrower? |
| 5. Evaluation and observability | Logs, traces, evaluations and guardrail monitoring. Blind spots, manipulated metrics, guardrails bypassed over several turns or with disguised wording. | Could we reconstruct what the agent saw, decided and did for any conversation? |
| 6. Security and compliance | A vertical layer across all the others: identity, policy enforcement, privacy and regulatory obligations. Controls that can be evaded, compromised oversight or security agents. | Which controls apply at every layer, who owns them, and what evidence would an auditor see? |
| 7. Agent ecosystem | Where agents meet users, other agents, marketplaces and external services. Impersonated agents, manipulated goals, agents colluding or passing false instructions through inter-agent protocols such as A2A or MCP. | How does each agent prove who it is, and what happens if a partner agent misbehaves? |
The OWASP guides add useful threat names to attach to each layer, such as memory poisoning, tool misuse, privilege compromise and intent manipulation. Combine them with our guardrails for LLM applications and AI agents to pick controls.
Worked example: a council or business chatbot with RAG and tools
Picture a customer chatbot for a council or a service business. It answers questions from a knowledge base of web pages and policies (RAG), it can look up the status of a request through an API, and it can create a service request or booking. Staff manage the knowledge base, and conversations are logged in the cloud.
Step 1: draw it. Visitor, chat widget, chat API, model provider, vector database, ingestion job, request-lookup tool, request-creation tool, logs, staff admin console. Mark the trust boundaries: the public internet, your cloud account, the model provider and your line-of-business system.
Step 2: STRIDE the plumbing.
- Spoofing: can a visitor look up someone else’s request by guessing a reference number? Require a second factor such as a verified email link before showing status.
- Tampering: can the chat API be called directly with altered fields? Validate everything on the server, not in the widget.
- Repudiation: can you show which conversation created a request? Log the conversation ID with every tool call.
- Information disclosure: do error messages or logs expose personal information? Redact before logging and set retention.
- Denial of service: can a script run up your model bill? Rate-limit per session and per address, and cap daily spend.
- Elevation of privilege: does the admin console sit behind the same login as everything else? Put it behind MFA and restrict it to staff networks or devices.
Step 3: MAESTRO the AI.
- Layer 1: pin the model version, record which vendor region processes data, and re-run your evaluation set when the vendor announces a change.
- Layer 2: only approved staff can publish to the knowledge base, with review for new sources. Treat retrieved text as data, never as instructions. Test with a planted page that says “ignore your rules”.
- Layer 3: the lookup tool is read-only and scoped to the verified visitor. The create tool accepts a fixed schema, validates every field and asks the visitor to confirm before submitting.
- Layer 4: each tool runs under its own narrow cloud role. The chatbot cannot reach the records database directly.
- Layer 5: trace each turn: retrieved documents, guardrail decisions, tool calls and outcomes. Alert on spikes in refusals or tool errors, and test guardrails with multi-turn and disguised attempts as described in our red-teaming and evaluation guide.
- Layer 6: tell people they are talking to AI (see AI transparency and chatbot disclosure), record the chatbot in your AI inventory, and check privacy and records obligations.
- Layer 7: if the chatbot hands work to another agent or a vendor’s assistant, authenticate that link and limit what each side may ask of the other.
Step 4: look across layers. The riskiest chain here is a poisoned knowledge base page (layer 2) that persuades the agent to call the create tool with a visitor’s details (layer 3) under a role that can write to the records system (layer 4). Breaking any one link stops it, so fix the cheapest first: confirmation before the create tool runs.
A practical checklist
- A current diagram for each important system, with trust boundaries marked.
- STRIDE walked for every component and data flow, and MAESTRO for any AI feature.
- Each threat has an owner, a decision (fix, reduce, transfer or accept) and a ticket.
- Server-side access checks for every record and every function.
- Tamper-resistant audit logs for sensitive actions and every AI tool call.
- Rate limits, size limits and spend caps on public endpoints and AI usage.
- Retrieved content and tool output treated as untrusted data.
- AI tools with narrow permissions, fixed schemas and human confirmation for consequential actions.
- Security tests planned from the threat list, including an independent test for higher-risk, internet-facing systems.
- A review trigger for design changes, new tools, new data sources and incidents.
A 30-day plan
- Week 1: list your internet-facing systems and AI features, pick the two that matter most, and name an owner for each.
- Week 2: run a 90-minute session for each with the people who build, run and own it. Draw the diagram, walk STRIDE, and walk MAESTRO for the AI parts.
- Week 3: turn the top threats into tickets, fix the quick wins such as rate limits, logging and tool permissions, and record any risk you accept and who accepted it.
- Week 4: test the fixes, add the threat model to your project documentation and AI governance records, and set the triggers for the next review.
How EITSAA can help
We build threat modelling and secure design reviews into our application, cloud and AI work, and we can run a facilitated session on a system you already have. Talk to us about a threat modelling session or call 0402 401 584, and we will quote it to your scope. Where independent penetration testing is needed, we can help you scope it from the threat model and fix what it finds. Learn more about our AI governance, application development, DevOps and cloud and AI agents and chatbots services, and see our generative AI acceptable use policy guide for the staff side of AI risk.
Sources
- Microsoft Learn: Integrating threat modelling with DevOps (origin of STRIDE)
- Shostack + Associates: early writing on threat modelling, including Kohnfelder and Garg, The threats to our products (1999)
- Threat Modeling Manifesto
- OWASP Threat Modeling Cheat Sheet
- Cloud Security Alliance: Agentic AI threat modeling framework, MAESTRO (6 February 2025)
- OWASP GenAI Security Project: Agentic AI, Threats and Mitigations
- OWASP GenAI Security Project: Multi-Agentic System Threat Modelling Guide v1.0