A prediction market price looks authoritative — a clean number, a percentage, a probability. But some of those numbers are backed by millions of dollars and thousands of traders, and some are one stale order on a market nobody is watching. Read them the same way and you will get burned by the second kind. The hard part of aggregating prediction markets is not collecting prices; it is telling the reliable ones from the noise.
That is what a data-quality layer does, and it is the piece most aggregators skip. If you have read our guide to reading prediction market probabilities, you know a price is only a probability if the market behind it is real. CoinRithm scores every market for exactly that — and, crucially, we never hide the ones that fail. prediction market data quality is a visible, inspectable property on our platform, not a silent filter.
TL;DR
- Every market on CoinRithm carries a quality verdict: is it reliable enough to base a decision on, and if not, why.
- Two signals drive it — a machine-readable decision-eligibility flag with explicit warning and block reasons, and a graded quality score with liquidity, volume, spread, and ambiguity tiers.
- Common failure modes: thin markets (almost no money), inactive markets (nothing trading), highly ambiguous resolution criteria, and markets near resolution where the price is a formality.
- Low-quality markets stay visible and labelled — we never 404 a real market or quietly delete it. Hiding data is its own dishonesty.
- The verdict gates automated trust: which markets feed the reference probability, what AI agents may act on, and what carries a warning badge for you.
- Quality is versioned (
pm-quality-2today), so the standard is auditable and improves over time.
Why a price is not automatically trustworthy
Prediction markets fail quietly. A market can show 63% while having ten dollars of liquidity, no trades in a week, and resolution criteria vague enough that two reasonable people would settle it differently. Nothing about the number 63% warns you of any of that. The price renders exactly as crisply as one backed by a deep, active, unambiguous market — and that visual equivalence is the trap.
Four failure modes account for most untrustworthy prices:
- Thin liquidity. With almost no money in the book, a single small order sets the "probability". It is a quote, not a consensus. The number can lurch ten points because one person felt like it.
- Inactivity. A market with real historical liquidity but no recent trading is showing you a stale belief. The world moved; the price did not.
- Ambiguity. If the resolution criteria leave room for interpretation — "will the economy improve", "will the deal happen" — the price is partly noise about what the question even means, and settlement itself may become a dispute.
- Near-resolution formality. Minutes before a market settles, its price is no longer a forecast; it is the answer arriving. Treating that as predictive skill flatters everyone, which is one reason it distorts accuracy measurement.
An aggregator that treats all four the same as a healthy market is not aggregating quality — it is laundering noise into something that looks like signal.
What CoinRithm attaches to every market
We assess each market and attach two complementary objects: a strict yes/no gate, and a graded scorecard.
The decision-eligibility verdict
The gate is a machine-readable verdict: is this market reliable enough that an automated system — an agent, a ranking, the reference number — should act on it? It carries three things:
decisionEligible— a boolean. Eligible markets are safe to build automated decisions on; ineligible ones are not.warningReasons— issues that should make you cautious but do not disqualify the market (elevated ambiguity, softening liquidity).blockReasons— issues serious enough to keep the market out of automated trust entirely (effectively no liquidity, resolution imminent, criteria unresolvable).
Because the reasons are explicit rather than a mystery score, you always know why a market was flagged. And because the whole verdict is versioned — the current policy is pm-quality-2 — the standard is auditable: when we tighten it, the version bumps and old assessments are not silently rewritten.
The quality scorecard
Alongside the gate sits a graded scorecard for human judgement. It rolls up into a single quality score and tier (high / medium / low), backed by component tiers you can reason about:
- liquidity tier — how much capital stands behind the price;
- volume tier — how much has actually traded recently;
- spread tier — how tight the bid/ask is (a proxy for how contested and tradable the price is, covered in depth in our liquidity and spreads guide);
- and boolean flags for the four failure modes: thin, inactive, highly ambiguous, near resolution.
A market can be decision-eligible and still carry a warning flag — say, high quality on liquidity and volume but elevated ambiguity. The scorecard is designed to preserve that nuance instead of collapsing it into a single green light.
The rule that matters: we never hide a flagged market
Here is the decision that separates an honest trust layer from a convenient one. When a market fails a quality check, we keep it visible and label it. We do not 404 it, delist it, or drop it from search.
This is deliberate. A curious user who arrives from a post looking for a specific market should find it — and find the truth about it — not hit an empty page that pretends the market does not exist. Hiding low-quality data is just a different way of lying about it: it robs you of the context that the price is thin, stale, or ambiguous. The right move is transparency, not concealment. So a flagged market shows its price and its warning, on the event page, on cards across the site, and in the cross-venue compare matrix.
What the verdict does gate is automated trust — the places where a bad price would silently propagate:
- Only eligible, real-money markets feed the cross-venue reference probability, so a thin outlier cannot drag the consensus number.
- AI agents trading on the platform receive the eligibility flag, so an autonomous strategy is not tricked into "arbitraging" a market that is thin for a reason.
- Rankings and highlights lean toward markets that clear the bar, so the strongest data surfaces first without the weak data disappearing.
The principle is simple: gate what machines trust automatically; never gate what a human is allowed to see.
How the quality layer connects to everything else
Data quality is not a standalone feature; it is the foundation the rest of the platform stands on.
It protects the reference probability by keeping junk out of the median. It sharpens divergence detection — a 20-point gap between two venues means something very different when both markets are high quality versus when one is a thin outlier. It underpins accuracy scoring, because a venue's calibration is only meaningful over markets that were real forecasts in the first place. And it is what lets us describe CoinRithm as an evaluation layer for prediction markets, not merely a mirror of them.
You can see the raw inputs — liquidity, volume, spread, resolution health — per venue on the sources page, read the full standard on the methodology page, and pull the same signals programmatically through the free prediction market data API.
FAQ
What does "decision eligible" mean on a market?
It means the market is reliable enough — sufficient liquidity, active trading, clear resolution, not settling imminently — that an automated system can safely act on its price. Ineligible markets are still shown to you with their price and a reason; they are simply kept out of automated trust like the reference probability and agent decisions.
Do you delete or hide low-quality prediction markets?
No. We keep every real market visible and label its quality. Hiding a flagged market would strip away the very context you need — that its price is thin, stale, or ambiguous. We gate what machines trust automatically, never what a person can see.
What makes a prediction market "thin", and why does it matter?
A thin market has almost no money behind its price, so a single small order can move it several points. The number looks like a consensus probability but is really one trader's quote. Our quality scorecard flags thin markets explicitly so you do not mistake a quote for a crowd.
Why is the quality policy versioned?
Because a trust standard should be auditable. The current policy is pm-quality-2; when we tighten or refine the rules, the version changes rather than silently rewriting past verdicts. That way you can tell which standard a given assessment was made under.
How is data quality different from accuracy?
Quality is a forward-looking judgement about whether a market's current price is trustworthy — enough liquidity, activity, and clarity to be a real forecast. Accuracy is a backward-looking measurement of whether a venue's past probabilities matched reality. Quality is a precondition for accuracy: a venue can only be well-calibrated over markets that were real in the first place.
Can I get the quality signals through the API?
Yes. The liquidity, volume, spread, and resolution-health inputs are available per venue on the sources page and through the free public API, so downstream tools and agents can apply the same standard.
