Safe synthetic demo

See Firefish stop a poisoned-document-to-tool-call attack.

This public demo uses a fake internal support agent, fake RAG content, fake customer data, and a fake email tool. It shows the story without raw dangerous payloads, real secrets, or external services.

View benchmark Open Operator App Run quickstart

Scenario setup

Everything here is fake and safe by design

Fake internal support agent

A deterministic local agent fixture that reads support context and proposes tool calls.

Fake RAG document

A synthetic document with a de-weaponized poisoned-instruction marker.

Fake customer secret

A labeled synthetic canary shown only as [REDACTED_FAKE_SECRET].

Fake email/send tool

A simulated outbound-send action that never contacts a real service.

Local-only authorized lab

The scenario is designed for localhost review and safe CI regression checks.

Attack path

Poisoned content tries to become trusted action

Poisoned RAG document

Retrieved support context

Agent proposes send

Firefish evaluates

Security Ops evidence

Without Firefish

Unsafe behavior can cross the boundary

Poisoned document enters RAGThe unsafe instruction is mixed into retrieved context.
Agent retrieves itThe model sees untrusted content beside the user task.
Agent proposes unsafe external sendThe fake send tool would move data outside the trusted workflow.
Secret could leakThe synthetic customer secret could leave the local lab if no control intervened.

With Firefish

The unsafe chain is interrupted

RAG firewall quarantines suspicious chunkPoisoned context is labeled and kept out of trusted retrieval.
DLP redacts or tokenizes sensitive dataEvidence keeps [REDACTED_FAKE_SECRET], not the raw value.
Agent Firewall blocks or requires approvalThe fake external-send tool call cannot run silently.
Security Ops records the incidentOperators get decision metadata, reason codes, and redacted evidence.
CI runner catches regression laterThe same style of synthetic case can guard future releases.

Evidence preview

Audit-ready without raw payloads

Redacted event IDevt_demo_poisoned_doc_tool_call

DecisionBLOCK or REQUIRE_APPROVAL depending policy

Reason codesrag_indirect_instruction_override, dlp:synthetic_secret, unsafe_external_send

Risk levelHIGH

Evidence[REDACTED_FAKE_SECRET], source hash, tool-call hash, policy trace

Raw payloadsNot displayed by default

Try the real controls

Move from story to proof

Use the benchmark for reproducible local evidence, the Operator App for investigation, and the quickstart to wire Firefish into your own AI workflow.

View benchmark Open Operator App Run quickstart