"Prediction markets are usually right" is the kind of claim people repeat without ever checking. It is also testable — a market that says 70% should win about seven times out of ten, and if it does not, the market is not as accurate as its fans think. Accuracy is not a vibe; it is an arithmetic property you can measure against resolved outcomes, one venue at a time.
If you are still forming a picture of what prediction markets are and how their prices become probabilities, start there. This guide is about the harder question: once a market prints a number, how good is that number, and how would you ever know? The short answer to how accurate are prediction markets is that the best ones are strikingly well-calibrated, the worst are not, and the only honest way to tell them apart is to score them.
TL;DR
- Accuracy has a precise meaning: a well-calibrated market's 70% events happen ~70% of the time. That is measurable against resolved markets.
- Two numbers do the work: calibration error (ECE) — how far predicted probabilities drift from realised rates — and the Brier score, which also rewards confidence.
- Good calibration needs a fair test: enough resolved markets, provider-confirmed outcomes, and real pre-resolution history. Short-lived markets and thin books distort it.
- Real-money venues with deep liquidity (Polymarket, Kalshi) tend to be well-calibrated; play-money and forecast platforms can be too, but the incentives differ.
- CoinRithm scores calibration per venue from resolved outcomes and publishes it on the sources page — so "accurate" is a number you can inspect, not a marketing line.
- No market is a crystal ball. Calibration measures honesty of the odds, not certainty about the future.
What "accurate" actually means
There are two different things people mean by an accurate prediction market, and conflating them causes most of the confusion.
The first is resolution accuracy: did the outcome the market said was most likely actually happen? This is the intuitive one — "the market called the election" — but on its own it is almost useless. A market that says 55% and the favourite wins looks "right", yet a 55% call that lands is barely more informative than a coin flip. Judging a probabilistic forecast by whether its favourite won throws away the number that made it a forecast in the first place.
The second, and the one that matters, is calibration: across all the times a venue said "70%", did the thing happen about 70% of the time? A perfectly calibrated venue is one whose stated probabilities match reality in aggregate. It can be wrong on any single market — that is what a probability means — while being extraordinarily reliable across hundreds of them. Calibration is the property that lets you trust a price you have never seen before, which is exactly what you need when you read a market cold.
Everything below is about measuring the second thing.
Calibration, in one picture
Take every resolved market on a venue and bucket its forecasts by the probability it assigned: all the 0–10% forecasts in one bin, 10–20% in the next, and so on up to 90–100%. Now, inside each bin, compute the realised rate — what fraction of those events actually happened.
For a well-calibrated venue the two line up: the 20–30% bin resolves Yes about a quarter of the time, the 70–80% bin about three-quarters of the time. Plot predicted probability against realised rate and the points hug the diagonal. Where the points sag below the line, the venue was overconfident (it said 80% but those events happened 65% of the time); where they ride above, it was underconfident.
CoinRithm builds exactly these reliability bins for each venue with enough history, using the probability the market assigned roughly 24 hours before resolution — late enough that most of the information is in the price, early enough that it is a genuine forecast and not a settlement formality. It is the same discipline a weather service uses when it checks whether its "30% chance of rain" days actually rained 30% of the time.
The two scores that summarise it
A reliability chart is honest but hard to compare at a glance, so two single numbers condense it.
Calibration error (ECE)
Expected Calibration Error is the sample-weighted average gap between each bin's predicted probability and its realised rate. Zero is perfect; the larger it gets, the more a venue's stated odds drift from reality. ECE is the cleaner headline for "is this venue's number trustworthy?" because it ignores nothing and flatters nothing — it just measures distance from the truth. On CoinRithm's sample, the best-calibrated real-money venues sit close to the diagonal, with calibration error small enough that their probabilities are worth taking at face value; you can read each venue's current figure on the sources page.
The Brier score
The Brier score is the mean squared error of a probabilistic forecast: for each market, take the gap between the probability assigned to what actually happened and 1 (or 0 for what did not), square it, and average. It rewards being both calibrated and confident — a venue that hedges everything toward 50% can look calibrated while being nearly useless, and Brier punishes that timidity. Lower is better; 0 is a perfect confident forecaster, 0.25 is the score you get by saying 50% to everything.
The two scores answer different questions. Calibration error asks "are the odds honest?" Brier asks "are the odds honest and sharp?" A serious accuracy claim cites both.
Why a fair test is harder than it sounds
Most "prediction markets are 90% accurate" headlines quietly cheat, usually in one of these ways — and avoiding them is why our numbers are more conservative than the marketing figures you will see elsewhere.
- Cherry-picked resolution accuracy. Counting "did the favourite win" instead of scoring the probability inflates any venue with a lot of lopsided, near-certain markets. Score the probabilities, not the winners.
- Grading on settled-anyway markets. Including a market's probability the instant before it resolves — when the answer is already obvious — makes every venue look clairvoyant. We score the price ~24h out so it is a real forecast.
- Tiny or thin samples. Ten resolved markets prove nothing. Calibration needs a minimum sample (we require a floor before we publish a venue's figure) and enough liquidity that the prices reflect real conviction rather than one stale order.
- Excluding the hard stuff. Short-duration markets with no real pre-resolution history get dropped from a clean calibration test, which honestly reduces some venues' sample sizes — we would rather show a smaller, fair number than a bigger, flattering one.
This is why accuracy is a per-venue, per-methodology quantity, not a property of "prediction markets" as a category. Two venues can both call themselves accurate and mean wildly different things.
So — are they accurate?
For the venues that clear a fair test, yes, and often impressively so. Deep real-money markets like Polymarket and CFTC-regulated Kalshi tend to be well-calibrated across large samples: their 70% really is about 70%. That is not magic — it is money. When being wrong costs real capital, overconfident prices get arbitraged toward the truth, which is the same mechanism our probability divergence guide describes across venues.
Play-money and forecast platforms such as Manifold and Metaculus can also be well-calibrated — sometimes strikingly so — because reputation and scoring rules give forecasters their own incentive to be honest. But because nothing is at financial stake, we never blend their odds into a money-backed number; the reference probability is real-money only for exactly that reason.
The essential caveat: calibration is a statement about the odds, not the future. A perfectly calibrated market that says 30% is telling you the truth even when the 30% outcome happens. Accuracy means the price is an honest probability — it does not mean the market knows what will occur. Treating a well-calibrated 80% as a certainty is one of the most common prediction market mistakes.
How CoinRithm measures and shows it
We compute calibration from resolved markets with provider-confirmed outcomes — meaning the venue itself reported how the market settled, not a guess — and provider-confirmed resolution times, then score each (outcome, probability) pair against reality. The per-venue results, including how much of each venue's history is even usable for a fair calibration test, appear on the sources page alongside its fee model, settlement currency, and resolution health.
Accuracy also drives our trust layer. Markets that are too thin, too ambiguous, or too close to resolution to be safely relied on are flagged by our data-quality engine rather than quietly trusted — a low-quality price is exactly the kind that erodes a venue's calibration. And when AI agents trade on our platform, we score their forecasts with the same Brier discipline, described in how we grade agent forecasts, so an agent cannot claim skill it has not demonstrated.
If you want to sanity-check any of this yourself, every resolved market and its provider-confirmed outcome is reachable through the free prediction market data API, and the full scoring approach is written up on our methodology page.
FAQ
How accurate are prediction markets, in one sentence?
The well-run, deep-liquidity ones are well-calibrated — their 70% events happen about 70% of the time — while thin or short-lived markets are far less reliable, which is why accuracy has to be measured per venue rather than assumed for the whole category.
What is the difference between calibration and a Brier score?
Calibration error (ECE) measures whether a venue's stated probabilities match realised rates — "are the odds honest?" The Brier score measures the same thing but also rewards confidence, penalising a forecaster who hedges everything toward 50% — "are the odds honest and sharp?" A credible accuracy claim reports both.
Are prediction markets more accurate than polls?
Often, in the aggregate and closer to resolution, because markets continuously price in new information and put money behind conviction — but they are not infallible and can move with sentiment. We compare the two directly in prediction markets vs polls.
Why don't you just count how often the favourite won?
Because that throws away the probability. A market that says 55% and wins is barely more informative than a coin flip, while a well-calibrated 90% that loses one time in ten was still a good forecast. Scoring the probability against the realised outcome — not the winner — is the only fair test.
Can a prediction market be well-calibrated and still wrong?
Yes, and that is the point. A calibrated market that says 30% is telling you the truth about the odds even when the 30% outcome happens. Calibration measures the honesty of the price, not certainty about the future — no market is a crystal ball.
Where can I see CoinRithm's accuracy numbers?
On the sources page, which shows per-venue calibration and resolution health, and on Arena agent scorecards for AI agents. The underlying resolved-market data is free to read through the public API.
