Model Security Scanning: What to Test Before a Model Ships

A model is not a single file in a security review. The release includes its weights or API, serialization format, tokenizer, loaders, dependencies, serving configuration, adapters, prompts, and the application that exposes it. Model security scanning should identify defects in those components and test the behaviors that static inspection cannot observe.

Define the release object

Record the model identifier and hash, source, license, format, tokenizer, adapters, quantization, loader version, runtime dependencies, serving image, and intended interface. For hosted models, record the provider version or alias, region, retention setting, and contract assumptions. A scanner cannot produce a meaningful result if the artifact under review is not the artifact that ships.

Use MITRE ATLAS to select adversarial techniques relevant to the model and application. Treat technique names as test ideas, not vulnerability findings.

Inspect provenance and unsafe artifact behavior

Verify the download source, checksums, signatures where available, and custody between acquisition and deployment. Avoid loading untrusted pickle-like formats in a privileged environment. Scan archives before extraction, cap file sizes, reject path traversal, and execute unavoidable conversion in an isolated worker without production credentials.

The OWASP Machine Learning Security Top 10 includes model and supply-chain risks that help frame artifact checks. Version the taxonomy because categories evolve.

Scan code and dependencies around the model

A clean weight file can still ship with a vulnerable web server, unsafe custom loader, exposed debug endpoint, or overprivileged container. Generate a software bill of materials, scan packages and images, review custom model code, and test the serving interface for authentication, authorization, input limits, and error disclosure.

Map these components in AI agent threat modeling when the model can select tools. Model behavior and application authority are separate review surfaces.

Run behavioral security tests

Test prompt injection, sensitive-data disclosure, cross-tenant retrieval, excessive agency, resource exhaustion, and unsafe output handling where they apply. Include allowed baselines and inspect downstream effects. Behavioral testing must state its coverage: a finite corpus can reveal failures but cannot prove the absence of every adversarial behavior.

The OWASP Agentic AI Top 10 guide becomes relevant when the application adds planning, memory, identities, or side effects. Do not report an agent-control result as a property of the underlying model.

Turn findings into release conditions

Deduplicate findings by root component and reachable impact. A vulnerable package that is absent from the runtime path is different from one exposed to user input. For each finding, record evidence, severity rationale, owner, fix, exception, and retest. Preserve the scanned hashes so later teams can tell whether the approved object changed.

AgentGuard Deep Scan is designed to inspect agent-related components and support review evidence. It should be one input to a broader release decision, not a claim that model quality or application authorization has been proven. Review the release surface when model, tool, and runtime responsibilities are unclear.

Apply the NIST Secure Software Development Framework to the code, build, dependency, and release portions of the model package. It does not replace model-specific tests, but it gives the surrounding software a traceable development and provenance baseline.

Run the scan again after changing the artifact, loader, serving image, adapter, or exposed interface. Compare findings by stable component identity rather than raw count; a lower count can simply mean the scanner lost visibility. The release record should name the scan policy and tool version, list suppressed findings with owners and expiry dates, and distinguish an accepted risk from an untested path.

Finish with a clean-room rebuild from the recorded source, lockfile, model digest, and build instructions. Compare the rebuilt hashes and deployment configuration with the object that was scanned. This catches a quiet but common release gap: reviewers approve one artifact while the pipeline later packages another. If exact reproduction is not possible, document which inputs are nondeterministic, how their provenance is checked, and what compensating verification runs before deployment. Keep that result beside the scan evidence so the approval remains tied to a reproducible release object.