Get decor and gifts delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Before an AI handles the real thing, give it a dress rehearsal
Planning a proposal, a surprise party or a busy holiday weekend often means preparing for the moment when several things go wrong at once. Businesses face their own high-pressure days: customers are unhappy, a competitor is circling, and someone may ask for a shortcut that breaks the rules. Firmulate’s experiment puts AI models through that kind of week before asking what they might do inside a company.
Popular right now
Bose QuietComfort Wireless Headphones – Twilight Blue (Limited Edition)
TCL 65-Inch 4K UHD Smart LED TV (65S425)
KitchenAid Artisan 5-Qt Tilt-Head Stand Mixer, KSM150PS, Almond Cream
As an affiliate, we earn on qualifying purchases.
One company, the same difficult week
For its final Crucible League, in July 2026, Firmulate gave each frontier model the same small software company, customers, crises and temptations. Every decision was versioned and auditable. The company is synthetic, but its money mechanics are real: 13 synthetic employees operate amid a burn rate of €105,000 a month and €2,300 in monthly recurring revenue. A public cash countdown and more than 680 self-learned playbook rules make the experiment watchable at Firmulate.
The league placed gpt-5.6-sol first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 fifth with 73. The do-nothing baseline scored 26. The benchmark’s trust rule is blunt: “no amount of good work outweighs a breach of trust.”
The gap between knowing and doing
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As the experiment puts it: “Same diagnosis, same pitch — no signature.”
The deal hinged on a clue hidden two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The finding is a reminder that getting the diagnosis right does not guarantee that an AI will carry a sound decision through to completion.
Firmulate also tested escalating fake CEO messages over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
More effort did not mean a better finish
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but it finished last. It left the deal on the table and tried to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four models. Kimi K3 also ran without an effort parameter, using the API default, while the others ran at xhigh—a fairness detail to keep in mind when comparing the results.
Readers can explore 242 real, unedited management decisions in Firmulate’s “guess the model” quiz. The public experiment is presented as a live company, with workdays versioned and decisions open to replay.
From watching to a company-specific pilot
A public experiment can show how models behave in a shared scenario. A pilot can put a company’s own playbooks and weak spots into the exercise. Firmulate says enterprises can use a read-only export of their business to wargame crisis scenarios and receive a board report with a model ranking and findings about their playbooks. The pilot does not write back to real systems.
Put your own playbooks to the test
To discuss a Firmulate pilot using a read-only export of your business, visit firmulate.com/pilot.html or email contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
