Most coding-agent benchmarks, SWE-bench included, hand the agent a fully specified task upfront: here's the bug, here's the exact fix expected. ICAE-Bench, posted to arXiv this week, argues that's a poor proxy for what coding agents actually get asked to do in the 'vibe coding' era, where a person describes a rough product idea and expects the agent to build it out, clarifying and changing scope as it goes. The benchmark pairs the coding agent under test with an automated 'User Agent' that starts from only a fuzzy initial requirement and doles out clarifications, feedback, and scope changes over the course of the build, rather than dumping a complete spec at the start. That forces the coding agent under test to demonstrate a bundle of skills together: eliciting requirements, planning under real uncertainty, using tools, debugging its own output, and doing repository-level construction, which is closer to what a contractor actually has to do than what a static benchmark measures. Why this matters for people building or buying coding agents: current leaderboard scores on saturated, fully-specified benchmarks say less than they used to about how an agent will behave once it's embedded in an actual product workflow with a non-technical stakeholder who doesn't know exactly what they want yet. As agent vendors increasingly market autonomous or semi-autonomous software engineering capability, benchmarks like ICAE-Bench are a more honest test of whether that claim holds up outside curated demo tasks, and teams evaluating agents for real internal use should weight interactive, under-specified evaluations over static ones when deciding what to trust.