feat(attacks): AgentPoison-style memory/RAG backdoor triggers · Issue #601 · AISecurityLab/hackagent · GitHub
Skip to content

feat(attacks): AgentPoison-style memory/RAG backdoor triggers #601

Open

Description

Problem

HackAgent’s RAG attack poisons documents for indirect instruction following at retrieval time. It does not yet implement AgentPoison-style memory/RAG backdoors: optimized triggers mapped into a unique embedding region so that, when the user instruction contains the trigger, malicious demonstrations are retrieved with high probability while benign queries stay clean.

That is a different threat model from one-shot document instruction injection: stealthy backdoor triggers, low poison rate, and transfer across agent memory stores.

Taxonomy

Field Value
Primary category adaptive
Tags indirect, rag

Register as AttackCategory.ADAPTIVE + AttackTag.INDIRECT + AttackTag.RAG.

Adaptive: trigger/payload optimization over retrieval success. Distinct from current rag technique, which is classified static (poison once, then query) — this issue is a new attack_type (e.g. agentpoison / memory_backdoor), not a rename of rag.

Scope for v1

  1. Optimize or search for a backdoor trigger that co-locates poisoned memory/RAG entries in embedding space.
  2. Inject a small poison set into agent long-term memory or RAG KB.
  3. Evaluate: triggered instructions retrieve malicious demos and induce attacker-chosen behavior; untriggered instructions remain near baseline utility.
  4. Target at least one HackAgent-supported agent path (RAG-backed or memory-backed).
  5. Docs + taxonomy + CLI/TUI; clear comparison section vs existing rag attack.

Out of scope

References

  • AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases (arXiv:2407.12784).
  • Related: ASB memory poisoning; PoisonedRAG.

Acceptance criteria

  • New attack_type (not a silent behavior change to rag) registered as adaptive + indirect + rag
  • Docs contrast with existing static rag attack
  • Metrics: triggered ASR, benign utility delta, poison rate
  • CLI/TUI discoverable; Result-per-goal + traces
  • Tests for trigger retrieval and benign control

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions