When a document tries to change the task
A procurement team asks an AI assistant to summarise a supplier proposal. In this hypothetical example, the file also contains instructions to retrieve internal pricing notes and email them outside the company. The employee requested a summary; the document attempts to redirect the work.
An assistant can mistake content it is reading for instructions it should follow. OWASP describes this as indirect prompt injection. It can affect an answer or lead to a request to use a connected tool. Reading the document does not itself grant new permissions: the consequences depend on the surrounding application and its controls.
Map what the assistant can reach
I would begin by listing the information and actions available during that task. Can the assistant search unrelated folders, read another team's records, send messages or change business data? Identify the account each connection uses and why that access is necessary.
A summarisation task may need access to a selected document without needing a messaging or deletion tool. Even read-only access needs boundaries: information available to the assistant could appear in its answer. Restrict retrieval to the requester's permitted scope and review where the result can be delivered.
Enforce permissions outside the model
Keep application instructions separate from retrieved content, and tell the assistant how to handle untrusted material. These measures can help, but they cannot establish that every malicious instruction will be rejected.
The code handling a tool request must check the user's authority, the permitted operation and its target before execution. A document should have no authority to expand that access. Give each connection only the capabilities it needs, and keep credentials out of the text passed to the model.
Make approval cover the actual action
For a sensitive action, such as external sharing or changing payment details, I would require a separate approval enforced by the application. The reviewer should see the actual operation, recipient and affected data, derived from the proposed tool request.
The application should check that execution matches what was approved. A changed destination or file needs a fresh decision. The model's description of an action as safe is no substitute for that control. Record the decision and outcome without placing confidential document contents or credentials in routine logs.
Test beyond the assistant's final answer
Use sample documents containing invented data in an agreed test environment, with test versions of tools that cannot send real messages or change production data. Put the misleading instructions inside the document being read. Include ordinary documents to confirm that the intended summary still works.
Inspect the answer, attempted tool calls, approval decisions and any data passed to those test tools. A reassuring refusal at the end does not reveal whether an earlier action already occurred. Missing evidence leaves the result unresolved. Repeat relevant tests when models, retrieval sources, tools or permissions change.
For a technical review, I would ask the team to demonstrate which controls still hold when document content influences the assistant's plan. Passing a selected test set provides evidence for that workflow; it does not guarantee protection against every future attempt.