UK's AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do
In a study covering seven benchmarks, the UK's AI Security Institute shows that standard AI evaluations systematically underestimate agent capabilities by capping the compute budget. On software engineering tasks, success rates jumped about 25 percent when the token budget was increased tenfold. Newer models benefit the most. Depending on the token budget, actual progress at the frontier is about 60 percent steeper than previous measurements suggested, according to AISI.<br /> The article UK's AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do appeared first on The Decoder. [...]
Rating
Innovation
Pricing
Technology
Usability
We have discovered similar tools to what you are looking for. Check out our suggestions for similar AI tools.
Microsoft says ungoverned AI agents could become corporate 'double agents.' Its fix costs $99 a month.
Microsoft today announced the general availability of Agent 365 and Microsoft 365 Enterprise 7, two products designed to bring security and governance to the rapidly growing population of AI agents op [...]
OpenAI unveils Workspace Agents, a successor to custom GPTs for enterprises that can plug directly into Slack, Salesforce and more
OpenAI introduced a new paradigm and product today that is likely to have huge implications for enterprises seeking to adopt and control fleets of AI agent workers.Called "Workspace Agents," [...]
Most enterprises can't stop stage-three AI agent threats, VentureBeat survey finds
A rogue AI agent at Meta passed every identity check and still exposed sensitive data to unauthorized employees in March. Two weeks later, Mercor, a $10 billion AI startup, confirmed a supply-chain br [...]
AI benchmarks systematically ignore how humans disagree, Google study finds
A Google study finds that the standard three to five human raters per test example often aren't enough for reliable AI benchmarks, and that splitting your annotation budget the right way matters [...]
Upwork study shows AI agents excel with human partners but fail independently
Artificial intelligence agents powered by the world's most advanced language models routinely fail to complete even straightforward professional tasks on their own, according to groundbreaking re [...]
The agent security gap: 54% of enterprises have already had an AI agent incident, and most still let agents share credentials
Across 107 enterprises, AI agents are being given real access to systems and data while the controls meant to contain them lag behind. More than half have already had a confirmed agent security incide [...]
Resolve AI says the AI coding boom is breaking production systems. It wants to fix that.
Resolve AI, the production-operations startup backed by Greylock and Lightspeed Venture Partners, today announced a sweeping expansion of its platform that introduces always-on background agents, a re [...]
Agentic security: Enterprises enforce agent permissions two-thirds of the time — and isolate high-risk agents less than one in five
Across 116 enterprises, agents are in production and so are the incidents: A majority have already had a confirmed agent security event or a near-miss. Two-thirds of enterprises enforce scoped permiss [...]
RSAC 2026 shipped five agent identity frameworks and left three critical gaps open
“You can deceive, manipulate, and lie. That’s an inherent property of language. It’s a feature, not a flaw,” CrowdStrike CTO Elia Zaitsev told VentureBeat in an exclusive interview at RSA Conf [...]