Imagine a team of AI assistants running a real company — making tough calls, resisting temptation, and even securing deals worth thousands of euros. Sounds like science fiction? Not anymore. In a groundbreaking live experiment, artificial intelligence models faced the chaos of a small software company’s worst week, and the results are revealing more than just tech prowess.
Get movie nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Live AI Company Wargame: Reality Check for Business AI
In a unique experiment, four frontier AI models were tasked with managing a real, functioning software company during its most turbulent week. This wasn’t a simulated demo or a chat exercise; it was a real-time test with actual money mechanics, real crises, and genuine temptations to cheat or manipulate. Every decision was logged, auditable, and made under the same conditions, ensuring fairness and consistency across the models.
The League Table: Who Came Out on Top?
- gpt-5.6-sol scored the highest at 95, identifying a buried critical fact and sealing a €55,000 deal, adding €4,583 in Monthly Recurring Revenue (MRR).
- Kimi K3 from Moonshot scored a close second with 93 points, also securing the deal and demonstrating the cleanest discipline among all models.
- Sonnet 5 followed with 88, completing the deal despite some process slips.
- Fable 5 managed 77, also closing the deal but showing more weaknesses in decision discipline.
- Opus 4.8 lagged with 73, illustrating the challenges of thorough analysis within the constraints.
All models successfully identified every crisis, refused manipulation attempts—including sophisticated social engineering—and made the same core diagnosis and pitch. Yet, only two models actually signed the deal that their own analysis had earned, highlighting a crucial difference between diagnosis and execution.
The Hidden Weakness: Reading and Trust
The decisive edge for the top performers was reading deeper into the company’s own files. The crucial information for closing the deal was buried two document references down in internal records, not visible in the initial customer summaries. Models that delved into these files won the deal at full price, underscoring how crucial internal document analysis is for closing complex business decisions.
Dealing with Social Engineering and Ethical Dilemmas
Another test involved social engineering: fake CEO messages escalating in stages, plus a reporter trick asking for a secret approval. Remarkably, all five models refused to cooperate, citing suspicion of impersonation and bypass attempts—demonstrating strict adherence to ethical standards and security protocols.
The Company in Action: Real Money, Real Risks
The live company used in this experiment comprises 13 synthetic employees, managing real financial mechanics. It burns €105,000 monthly against just €2,300 in MRR, with a public cash countdown and a library of over 680 self-learned rules. Every workday’s decisions are versioned and observed, providing a transparent window into how AI models handle real-world complexity. More details and live footage are available at firmulate.com/live.
What the Results Mean for Business AI
While all models could spot crises and resist manipulation, the real differentiator was in execution — whether they signed deals, read internal files, or maintained discipline under pressure. The experiment shows that choosing an AI model isn’t just about how well it chats; it’s about whether it can reliably complete tasks, read beyond surface data, and stay honest when it counts.
The Fairness Footnote
It’s worth noting that Kimi K3 ran without an effort parameter (the API default), while the other models operated at the xhigh setting. This difference in approach might influence decision-making behavior but emphasizes the importance of understanding how models are configured for real tasks.
The Open League: The Future of AI in Business
The league table—gpt-5.6-sol, Kimi K3, Sonnet 5, Fable 5, and Opus 4.8—illustrates a competitive landscape where the newcomer, Kimi K3, holds its own against established giants. The results reinforce a key point: in the realm of AI-driven management, selecting a model without your own rigorous testing is a gamble.
Ready to Test Your AI Workforce?
Businesses can try their own ‘wargame’ by running the same decision challenge against a read-only export of their operations. This approach allows testing AI decision-making in a controlled, risk-free environment. Details are available at firmulate.com/pilot.html. Watch the live company in action or challenge your AI models to see how they perform under pressure.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
