IssueTrojanBench Shows Two-Thirds of Malicious GitHub Issues Bypass Coding-Agent Guardrails

This paper introduces IssueTrojanBench, a benchmark built to answer an uncomfortable question: if you point an autonomous coding agent like Cursor, Claude Code, or Codex Desktop at a real GitHub issue, and that issue was written by an attacker rather than a real user, how often does the agent actually get tricked into doing something harmful? The researchers built malicious issues around four attack categories and six different delivery vectors, plain issue text, issue comments, embedded images with malicious alt-text, PDFs attached to the issue, and more, then ran them against agents powered by GPT-5.3 Codex, GPT-5.4, and Claude Sonnet 4.6, layering in additional prompt perturbations to see how robust the results were. The headline number is stark: 66.5% of the malicious issues in the benchmark got past every guardrail the agents and their underlying models had, whether that was model-level refusal or agent-framework-level filtering. Digging into why, the paper finds that almost all of the successful blocking came from the underlying LLM refusing to comply, not from any protective logic in the agent scaffolding itself, meaning the agent framework layer that developers often assume is doing safety work is mostly just passing instructions straight through. It also found meaningful differences between models: GPT-family models were broadly more vulnerable, while Sonnet 4.6 was more selective about blocking high-impact actions specifically, suggesting model choice measurably changes exposure. For anyone building or deploying coding agents that read and act on issues, PRs, or other untrusted repository content autonomously, this is a direct warning that today's defenses are largely cosmetic, and that treating issue text as trusted input is a real, demonstrated attack surface rather than a theoretical one.

Source

View on ShipDigest