AI Pentest Modules

Firefish modules are small, discoverable security checks for authorized AI testing. Each module declares the target types it supports, the run modes it allows, the safety level it requires, and the evidence or findings it produces.

Available Module Families

Firefish currently includes modules for:

• Reconnaissance.
• Prompt injection.
• Jailbreak and policy-boundary smoke checks.
• RAG and vector weakness checks.
• Tool hijacking and excessive agency.
• Sensitive data disclosure.
• Output handling.
• Model behavior smoke checks.
• Resource-consumption probes.
• Benchmark and regression campaigns.

Example Modules

Useful starting points:

• recon.echo_target
• recon.target_fingerprint
• recon.capability_probe
• recon.tool_inventory
• benchmark.smoke_test
• prompt_injection.direct_instruction_override
• prompt_injection.data_exfiltration_canary
• jailbreak.policy_bypass_smoke
• rag.indirect_prompt_injection
• rag.citation_integrity_check
• agent.tool_hijack_simulation
• agent.unapproved_external_send
• data_exposure.system_prompt_leakage_canary
• output_handling.unsafe_markdown_html
• resource_consumption.long_context_cost_probe

What A Module Declares

Each module includes:

• Module ID and version.
• Category and description.
• Compatible target types.
• Required capabilities.
• Run parameters.
• Safety level.
• Supported run modes.
• OWASP LLM Top 10 2025 mappings.
• MITRE ATLAS mappings.
• Evidence types.
• References and author/source.

Safe Payload Design

Modules use synthetic placeholders and canaries. They do not include real secrets, real third-party targets, destructive instructions, or unsafe payload text in public docs or normal UI surfaces.

Local Lab Compatibility

Module tests and demos can use the built-in local lab fixtures:

• Chat modules compare vulnerable_chatbot with protected_firefish_gateway.
• RAG modules compare vulnerable_rag_corpus with protected_rag_corpus.
• Tool-governance modules compare vulnerable_tool_agent with protected_tool_agent.

The fixtures are deterministic and networkless. They return safe vulnerable markers, Firefish blocked markers, or synthetic canary observations so modules can prove expected pass/fail outcomes without touching real infrastructure.

How To Use Modules In Security Workbench

1. Open Security Workbench.

2. Choose a target.

3. Search or filter the module library.

4. Select one or more modules.

5. Review safety warnings and required approvals.

6. Run a dry-run campaign.

7. Review findings, evidence, and coverage.

Extension Point

Custom modules can be added by subclassing the Firefish pentest module base class, declaring a typed module spec, validating scope before running, emitting attack-trace events, and returning redacted findings and evidence.