Release and operation
Keep malicious document instructions away from business actions
An assistant that reads outside content and uses business tools can be steered by hidden malicious instructions.
Who this helps
CISO, CIO, Security engineering leader, Application owner
Discuss this workThe situation
An assistant reads emails, documents or websites and can use business tools. A malicious instruction hidden in that material could redirect an answer or attempt an unauthorized action. The model reads useful facts and malicious directions through the same language interface. A stronger instruction or a retrieved citation cannot guarantee the attack is ignored.
What we build
We build controls outside the model, so a manipulated answer cannot become an unauthorized action.
- Retrieved content is marked as untrusted input. Credentials and policy are kept out of model context, and tools are limited to typed operations.
- Actions and destinations are validated in application code under the actual caller's authority, with outbound policy and role and tenant limits enforced outside the model.
- We run labeled attack fixtures, inspect attempted calls and show whether the independent boundary blocked the consequence. Tests use decoy data and simulated tools, never real credentials or live systems.
- A suspected attack pauses the affected action and keeps a minimal trace for review. A blocked write is distinguished from a manipulated answer. Recovery may revoke access, isolate a source or restore a previous configuration.
- We inspect local files, retrieval, connectors, model access and telemetry. Private hosting changes exposure paths but does not remove prompt injection.
What you get
- A threat map. Trust boundaries, permitted tools and allowed data destinations for the bounded workflow.
- An attack and control trace. Each attack attempt, the call it tried to make and whether the boundary blocked it.
- A residual-risk register. Remaining exposure and the control changes that address it, for security to review.
Where people stay in control
- Security approves the attack scope and reviews residual exposure.
- A person approves consequential permitted actions with the object and values visible.
- The approval interface never treats document text as authorization, and a filter pass never declares the system immune.
When another approach fits
A narrow, read-only assistant can reduce action exposure when tools are unnecessary. Ordinary secure software controls remain necessary even if a model is replaced or privately hosted.
Related work
Scope it with us
Bring the workflow, the data you can share and the decision it supports. We agree how the work will be evaluated before anything is built.
Discuss the workSee all use cases