Elon Musk called Grok 4.5 an Opus-class model that runs faster and costs less. An independent, agentic benchmark now provides real backing for that claim.
On Artificial Analysis’s AutomationBench-AA, Grok 4.5 ranked first with a 51.4% score while costing $0.34 per task. It beat both Claude Fable 5 (48.6%) and Claude Opus 4.8 (48.5%).
An Independent Check on Elon Musk’s Claim
SpaceXAI took Grok 4.5 public this week, built on its 1.5 trillion-parameter V9 foundation. Musk’s pitch rested on early internal evaluations.
AutomationBench-AA changes that. Artificial Analysis runs the benchmark independently and keeps its task set private to prevent contamination.
The test spans 657 tasks across 40 simulated apps, including Gmail, Slack, Salesforce, and HubSpot. It scores the share of objectives an agent completes without breaking guardrails.
Follow us on X to get the latest news as it happens
Grok 4.5: Cheaper, Faster, and Mostly Compliant
Grok 4.5’s $0.34 per task sat far below Fable 5’s $1.35 and Opus 4.8’s $1.46. Gemini 3.5 Flash came closest at $0.49.
The model used about 8,000 output tokens per task, roughly a quarter of Opus 4.8’s total. That efficiency drives most of the cost gap, matching the framing Musk used at launch.
“Its total token usage of 0.44M per task is among the lowest on the leaderboard. Low cost is driven by this efficiency as well as low token pricing,” Artificial Analysis said.
In addition, Grok 4.5 completed 79.9% of task objectives and fully passed 21.9% of tasks. In Finance, the hardest domain, it led with 71%, ahead of Fable 5’s 64% and Opus 4.8’s 62%.
However, the model broke more rules than its closest rivals. It logged 0.63 guardrail violations per task, above Opus 4.8’s 0.55 and Gemini 3.5 Flash’s 0.46.
That gap matters for firms that deploy agents to live financial systems, where a single violation can incur real costs.
Subscribe to our YouTube channel to watch leaders and journalists provide expert insights
Bitrue