AI Agent Threat Modeling Guide: Model the Path to Action

An AI agent threat model is useful only when it shows how an attacker can cross a real trust boundary and affect a named asset. This guide produces a diagram and test set that connect identities, tool arguments, controls, and downstream state.

Threat modeling is useful only when each arrow ends in a named system action. For an agent path, record the identity that crosses the boundary, the tool arguments it carries, and the final asset that could change.

When a modeled action uses an MCP tool, MCP protocol architecture makes the client, server, tool, and downstream handoff explicit; draw each as a separate node instead of labeling the whole path as the agent.

Draw the agent path to effect

Map user and external inputs, model context, memory, planner, tools, identities, downstream systems, and irreversible actions.

The diagram shows where an attacker can influence the system and where a control can stop an effect. Do not accept a model diagram that omits the downstream system or the identity used there.

Name assets and trust boundaries

List secrets, customer data, money movement, code, access rights, and operational systems. Mark transitions between users, tenants, processes, and vendors.

Each asset has an owner and each boundary has an enforcement point. Escalate when a shared service identity crosses a tenant or privilege boundary without a policy decision.

Shared service accounts can erase the distinction between a user request and an agent action. Use agent credential leak controls when the model context, connector setup, or trace exposes delegated credentials.

Write abuse cases around capability gain

For each path, ask how hostile content, a compromised tool, a stolen identity, or a confused workflow could gain a capability it should not have.

The abuse case ends with a verifiable effect such as an exported record, changed setting, or unauthorized tool call. Discard vague cases that have no asset, path, attacker action, or observable result.

Place controls and failure signals

Choose controls for provenance, identity, data filtering, argument validation, approval, execution, and audit. Define what a failed control looks like.

Each high-impact path has a prevention or detection control and a testable failure signal. Fail closed before the effect when the enforcement point cannot verify the needed context.

Validate the model against a running system

Run representative abuse cases, including indirect injection and tool compromise, and compare the observed trace to the model.

The model is updated when a real path or side effect differs from the diagram. Roll back the changed integration or policy if testing uncovers an unmodeled route to a protected asset.

Add product evidence to the threat model

Draw AgentGuard as an evidence source, not as the authorization or downstream-enforcement node. Use AgentGuard documentation to establish what the modeled host can expose. Place a Deep Scan finding on the exact component-admission node and, only for a supported checkpoint, place the selected Runtime Guard decision on a separate pre-execution observation node.

Connect both to the caller identity, application policy, bypass route, downstream enforcement, and final asset state. The separate nodes make missing coverage visible instead of implying that one product result closes the attack path.

Update the model when the real path changes

Keep the diagram revision with one exercised abuse path and its trace. When the observed request takes an unmodeled route, update the model and the regression suite together.

MITRE ATLAS supplies adversary techniques that make abuse cases more concrete, and the NIST AI RMF supplies a risk-management frame. Use the pair to challenge the arrows in your own diagram, not to certify an architecture by citation.