Forezai · Polybot: When the AI Disagrees With the Odds

📊 Full opportunity report: Forezai · Polybot: When the AI Disagrees With the Odds on ThorstenMeyerAI.com — validation score, market gap, and execution plan.

TL;DR

Polybot is an experimental open-source AI designed to assess when its probability estimates diverge meaningfully from prediction market prices. It aims to evaluate whether AI can identify genuine mispricings without overtrading. The project emphasizes risk management and transparency but remains a research tool, not a profit generator.

Polybot, an open-source AI trading system, is testing whether an artificial intelligence can independently identify market mispricings by comparing its probability estimates with prediction market prices. This experiment raises questions about the potential for AI to challenge market consensus and the risks involved.

The Polybot project is designed to research when and how an AI can reliably detect discrepancies between its own probability assessments and the implied probabilities of prediction markets like Polymarket. It operates by researching publicly available information, forming a probability estimate, and then comparing it to the market’s implied price. The system only trades when the disagreement exceeds a predefined threshold, accounting for costs such as fees and slippage.

Built with transparency in mind, each estimate includes recorded reasoning, allowing post-trade analysis of why the AI believed a mispricing existed. The approach emphasizes calibration over time, requiring the system to demonstrate that its predictions are statistically aligned with actual outcomes across many estimates. It deliberately adopts a risk-averse stance, trading rarely and only on the strongest signals, to avoid common pitfalls such as overtrading and excessive fees.

At a glance
reportWhen: ongoing; the project is currently activ…
The developmentPolybot, an open-source AI trading bot for Polymarket, tests whether AI can reliably identify market mispricings by comparing independent estimates to market prices.
Forezai · Polybot — When the AI Disagrees With the Odds · Built in Public Day 13/19
Built in Public · Day 13 / 19 ThorstenMeyerAI.com · the operator portfolio
The Markets Layer · Day 13 · Forezai

Polybot — when the AI disagrees with the odds

A prediction market puts a price on the future. Polybot asks: can an AI’s own estimate diverge from that price for real — and should it ever act on the gap?

Not financial advice — and not a recommendation to trade, invest, or use this software. Automated trading carries a substantial risk of loss, up to all of your capital. Prediction-market access is legally restricted or prohibited in some jurisdictions (including for US persons) — know your local law. Experimental open-source software; no guarantee of accuracy or profit. Figures below are illustrative of the logic, not a track record.
01 Estimate vs price → the gap → a decision
AI estimate compared to market price · trade only on a real, cost-clearing edgeillustrative
Market questionMarketAI est.EdgeDecision
Will event A resolve YES by Q3? 62%71%+9 clears threshold → small, risk-capped
Will metric B exceed target? 48%50%+2 too small → SKIP
Will outcome C happen by year-end? 30%34%+4 · low conf. too uncertain → SKIP
default = NO TRADE most markets → skip. Trade rarely, small, only on the strongest disagreements — and even those can be wrong. Each estimate’s reasoning is recorded.
02 A research tool, not a money machine
open & auditable
MIT — and every estimate records why it disagreed, so a decision can be inspected, not just executed.
edge = hypothesis
the gap is a guess, not a property. Backtests flatter; costs are merciless; markets adapt and fight back.
mostly skip
the sane system finds action almost nowhere — and is honest that it can still be wrong.
03 The thesis the whole series inherits
01
Local-first
Runs on owned compute — the experiment costs compute, not a subscription.
02
Provider-agnostic
The forecasting model is swappable — no single model is trusted as an oracle, least of all about the future.
03
Non-developer build
An open, inspectable way to study AI forecasting against a live, adversarial market.
04
Edit by subtraction
The default action is nothing. Trade rarely, small, only on the strongest, cost-clearing disagreements.
04 The operator constellation
18 products · one foundation
Today: Polybot lit — the first Markets node. The portfolio’s instincts meet the most unforgiving test: a live market that keeps score in cash.
Content
DojoClaw
RoundupForge
Stenvrik
ChannelHelm
IdeaNavigator
Decision
IdeaClyst
Threlmark
Outcome-First
Platform
Grimfaste
Delvasta
Open / Reg
Glasspane
QAtrial
Markets
Polybot
TradingAgents
Defense / Intel
Argus
VigilSAR
VigilSAR-Bench
Diagnostic
World Model Readiness
Local-first · Provider-agnostic foundation

Not financial, investment, legal or tax advice; not a recommendation or solicitation to trade, invest or use any software. Forezai · Polybot is experimental open-source software (MIT), provided “as is” without warranty of accuracy or profitability. Trading and automated trading carry a substantial risk of loss including total loss of capital; past or backtested performance does not indicate future results. Prediction-market participation is restricted or prohibited in some jurisdictions (including for US persons) — you are solely responsible for compliance with applicable law. Consult a licensed professional before any financial decision. Produced with AI assistance under human editorial oversight; independent commentary, the author’s own views. Product and company names are trademarks of their respective owners; mention does not imply endorsement.

ThorstenMeyerAI.com · Built in Public · Day 13 of 19 · © 2026 Thorsten Meyer

Potential Impact of AI Market Disagreement Detection

Polybot’s experiment explores the capabilities and limitations of AI in identifying market mispricings within prediction markets. Its focus on transparency and calibration could inform future developments in AI applications for financial analysis, particularly regarding risk management and interpretability. While not designed as a commercial trading tool, the project contributes to understanding the challenges and potential of AI-driven market assessments.

Amazon

AI trading bot for prediction markets

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Background on Prediction Markets and AI Testing

Prediction markets like Polymarket allow participants to buy and sell contracts based on future events, effectively putting a market-implied probability on outcomes. These markets are known for their informational density, often making their prices difficult to beat. Polybot’s approach is to see if an AI, using public data, can identify when the market’s consensus is mispriced, and whether it should act on these signals.

Previous attempts to outperform markets often fail due to costs, market adaptation, and the inherent difficulty of consistently beating aggregated information. Polybot’s design aims to address these issues by trading only when the AI’s estimate significantly diverges from the market price, and by recording reasoning for transparency and calibration purposes.

“Polybot is an open-source experiment that tests whether an AI can reliably identify mispricings in prediction markets by comparing its own probability estimates with market prices.”

— Thorsten Meyer, source author

Investing for Programmers: Understanding markets through data and code

Investing for Programmers: Understanding markets through data and code

Maximize your portfolio, analyze markets, and make data-driven investment decisions using Python and generative AI.

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Limitations and Unanswered Questions in Polybot’s Approach

It remains uncertain how well Polybot’s estimates will calibrate over time and whether the system can consistently outperform or match market accuracy. The experiment is still in early stages, and real-world factors such as slippage, liquidity, and adversarial market behavior may influence its effectiveness. The long-term viability of AI trading based on disagreement detection has yet to be established.

Amazon

automated trading system for Polymarket

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Next Steps for Polybot and Its Research Goals

Polybot’s developers plan to continue testing and refining the system, focusing on long-term calibration and robustness. Future work includes analyzing historical data, expanding to other prediction markets, and documenting lessons learned. The project aims to contribute insights into AI’s capacity for independent market assessment and risk-aware trading strategies.

Amazon

risk management trading algorithms

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Key Questions

Can Polybot reliably beat prediction markets?

Currently, Polybot is an experimental tool designed to assess whether AI can identify mispricings. Its effectiveness in consistently beating markets remains unproven and is part of ongoing research.

Is Polybot meant for live trading or just research?

Polybot is intended as a research artifact, not a commercial trading system. It emphasizes transparency, calibration, and risk management rather than profit generation.

What are the risks of using AI like Polybot in prediction markets?

Using AI in prediction markets involves substantial risks, including financial loss, market manipulation, and the costs associated with slippage and fees. The system is experimental and should be used with caution and only with risk capital.

How does Polybot determine when to trade?

Polybot trades only when its independent probability estimate significantly diverges from the market price, after accounting for costs and uncertainties, and only if the disagreement exceeds a predefined threshold.

Source: ThorstenMeyerAI.com

You May Also Like

The Co-Founder’s Black Hole — A Structural Read on Jack Clark’s Automated AI R&D Essay

Jack Clark predicts over 60% chance that autonomous AI research systems could emerge by 2028, raising concerns about institutional readiness and future risks.

Every Benchmark Launched 2023-2024 Has Fallen — The METR / SWE-Bench / CORE-Bench / MLE-Bench / PostTrainBench Sequence

Every major AI research benchmark launched in 2023-2024 has now saturated or is nearing saturation, signaling advancements in AI capabilities.

Jack Clark Says It Out Loud — Reading the Co-Founder’s 60%/2028 Estimate on Automated AI R&D

Anthropic’s co-founder Jack Clark publicly estimates over 60% probability that autonomous AI systems capable of building their own successors will emerge by 2028.

The Compounding Error Problem — Why 99.9% Alignment Decays to 60% in 500 Generations

Research shows that even 99.9% per-generation alignment accuracy drops to 60% after 500 generations, raising concerns over recursive self-improvement safety.