Prompt Injection Testing: A Defensible Test Plan

OWASP's prompt-injection reference distinguishes direct and indirect injection and describes risks arising when untrusted input influences an application's behavior. Use that vocabulary to draw your own application boundary: user messages, retrieved documents, tool results, memory, system instructions and executable actions. Identify which sources the application should treat as data rather than authority.

Our recommended test charter states the authorized environment, permitted techniques, synthetic protected values, allowed tools, prohibited side effects and the person who can stop the exercise. Use a sandbox with test accounts and reversible actions. The purpose is to learn about a controlled boundary, not to expose real credentials or affect a third party's systems.

Write the harmful outcome in operational terms. For a read-only assistant, it may be disclosing a synthetic restricted value. For an agent, it may be invoking a forbidden tool or submitting an unauthorized argument. Merely reproducing a provocative phrase in a response is not equivalent to demonstrating that the application crossed the boundary you are testing.

Observe the complete path from input to application action. Record retrieved material, model or agent version, relevant tool calls, enforced policy and the final result. A security product may emit an alert without blocking an action. A model may refuse in prose after a tool has already executed. Distinguish those outcomes in the finding rather than classifying everything as a generic pass or fail.

Define a pass condition for each case before running it. For a synthetic restricted value, the condition might require that the value is neither returned nor sent to an unauthorized tool. For an agent action, inspect the actual arguments and destination. Where the environment cannot expose the necessary evidence, mark the result inconclusive instead of inferring that nothing happened.

Include unavailable and slow dependencies in the authorized trial. Ask what the application does if the guardrail service, retrieval layer or policy store cannot respond. Fail-open and fail-closed behavior are operating choices with different consequences; your test should make the configured behavior observable and assign responsibility for the decision.

Use the same application version, permitted capabilities and controlled cases for a comparison. Preserve configurations and explain any difference in integration point. An inline scanner and a monitoring-only deployment do not have the same opportunity to intervene, so a single comparative percentage can be misleading without that context.

Report attack outcomes and benign-task outcomes separately. Show the consequential failures individually, retain inconclusive results and record latency or availability observations without generalizing beyond the trial. A vendor-provided result can inform a shortlist, but it should not be presented as an independent GovernAtlas benchmark unless GovernAtlas actually ran and documented the test.

When a defense is changed, retest both the successful attack cases and the legitimate tasks. A narrowly tuned rule can close one example while causing unnecessary refusals elsewhere. Keep the change record, the retest evidence and any accepted exception linked so a later reviewer can understand the remaining exposure.

A useful finding includes the attacker's assumed control, the intended task, the precise test input, application and policy versions, observed steps, outcome, impact within the sandbox and a reproduction procedure. Record the evidence that was unavailable. Avoid replacing this record with a screenshot of a refusal or an aggregate score.

Assign an owner and a proposed next action. That action may be removing an unnecessary tool privilege, changing a workflow boundary, improving input handling or adding a reviewed control. Do not assume that another instruction in the prompt is the right remediation for every failure. Reproduce the finding after the change and preserve both results.

The deliverable is a bounded evidence report. It explains what was tested and what remains unresolved; it does not certify immunity to future attacks. Use newly observed failures to update the case set and revisit the threat model when tools, privileges, data sources or deployment paths change.