Home / Forecasting evidence

Evidence base · November 2026

Forecasting evidence for policymakers

Foresight explores many futures; forecasting puts numbers on a few of them. Policy needs both. This page collects the track record: which forecasters are well calibrated, which institutions err in a predictable direction, and what that means for anyone who plans with official projections.

At a glance

Six numbers that change how you should read a forecast

5–10 yrs

how early AI reached IMO gold-medal level

In 2022, domain experts expected it by 2030 and superforecasters by 2035. It happened in July 2025.

Forecasting Research Institute (FRI), 2026 [6]
32–60×

overestimate of direct air capture

2022 forecasts for 2024: 0.32 to 0.60 MtCO₂ a year. Projected actual: 0.01 Mt. Hardware-bound tech moved far slower than expected.

FRI, XPT near-term accuracy [47]
602

IMF crisis programmes audited

Across 123 countries (1992–2020): export and growth forecasts systematically optimistic, reserve forecasts systematically pessimistic.

Eicher and Kawai [69]
≈12%

lower error from weighted, extremised team polls

They beat prediction markets (Brier 0.210 vs 0.227) in the IARPA Good Judgment Project, across 261 geopolitical events.

Atanasov et al., Management Science, 2017 [7]
99%

of naive baselines beaten by disease ensembles

The CDC and Reich Lab multi-model ensembles also beat 84–91% of trend and complex baselines, across more than 2.5 million audited US forecasts.

Hubverse evaluation [22]
+1.25

points per question: humans still ahead

Metaculus Pro Forecasters beat the best AI bots on 99 matched questions in Spring 2026, although a bot won the Summer 2026 Metaculus Cup outright.

Metaculus, AI forecasting in 2026 [13]
Official baselines

Institutional forecasters err in predictable directions

Public-sector projections steer trillions in budgets and investment. Their errors are not just random shocks: audits find biases that run in a consistent direction, often driven by incentives or by assumptions baked into the model.

Institution and productScopeHow it is auditedDocumented bias
Philadelphia Fed Survey of Professional Forecasters (since 1968; Philly Fed since 1990)Oldest US quarterly panel: GDP, CPI and PCE inflation, unemployment, density rangesIndividual, ID-tracked errors published onlineDensity forecasts historically overestimated the probability of large inflation shocks; inflation surprises are serially correlated [66][67]
ECB Survey of Professional Forecasters (since 1999)About 75 euro-area forecasters, around 59 respondents a roundMean error, RMSE, probability distributionsLimited inflation bias after adjusting for shocks, but consistent GDP overprediction since the 2001 slowdown, sharpest at the two-year horizon [68]
IMF World Economic Outlook and crisis programmesGlobal surveillance; 602 lending programmes in 123 countriesHolden-Peel and Mincer-Zarnowitz tests; IEO (2014); Eicher and Kawai (2021)WEO unbiased for advanced economies, but crisis forecasts optimistic on exports and growth; reserve coverage averaged 55% below actuals in the worst balance-of-payments quartile because known conditionality effects were left out [69][70][71]
IEA World Energy Outlook and US EIA Annual Energy OutlookEnergy demand, fuel mix, transition scenarios (WEO 2026 published 19 November 2026)Projected versus realised capacity additions (Hoekstra audits, 2002–2026)Multi-decade under-forecasting of solar PV. WEO 2004 saw renewables' share of electricity rising only from 18% to 19% by 2030; global solar share grew ninefold from 0.15% (2010) to 1.3% (2016) [73][74][77][78]
US CDC and Reich Lab: COVID-19 Forecast Hub, FluSight, HubverseAbout two dozen disease hubs; more than 120 COVID teams over 160 weeksWeighted interval score, relative WIS, 95% coverageThe counter-example: open, standardised multi-model ensembles beat almost all baselines; FluSight four-week skill now exceeds its first season's one-week skill. Ensembles still struggle at sudden turning points [22][79][81]

The pattern matters for foresight practice. Scenario baselines built on current or stated policies systematically flatten exponential-to-logistic adoption curves, and growth forecasts made inside an institution that must defend a programme lean optimistic. Treat an official projection as one perspective, with a known direction of error, not as the probable future of the futures cone.

Challenge baseline assumptions in Module 3: assumption premortem
Aggregation

Crowds, superforecasters and prediction markets

The strongest finding in forecasting research is that how you combine judgements matters as much as whose judgements you collect.

The Good Judgment Project (IARPA ACE, 2011–2015)

Four seasons, 100–150 geopolitical questions a year, more than 2,800 volunteers. The top 2% each season became “superforecasters”. The widely quoted 30% edge over intelligence analysts came mostly from the aggregation algorithm (weighting and extremising), not from the play-money market, which only matched the analysts' own market [7][17][107].

Foxes, hedgehogs and pundits

Tetlock's Expert Political Judgment (2005) found that eclectic “foxes” beat big-idea “hedgehogs” on calibration and discrimination, simple statistical extrapolation beat the average expert, and media prominence was inversely related to accuracy [86].

Superforecasters versus experts

On ForecastBench, superforecasters scored a Brier of 0.081 in 2024–25 and a Brier Index of 70.2 in May 2026. Their edge rests on probabilistic literacy, reference classes, small frequent updates and team extremising; roughly 70% stay in the top tier year on year [19][49].

Prediction markets at scale

Dollar volume on the four largest venues reached $137.5 billion in January–October 2026, more than six times the $21.9 billion traded in all of 2025, mostly on sports and crypto rather than politics. A Kalshi study of 2.24 million markets found Brier scores falling from about 0.08–0.09 three months out to about 0.02 at close [1][8].

Markets are calibrated, not infallible. Political markets on Kalshi and Polymarket are consistently underconfident (prices squeezed towards 50%) across 353 million trades [25]; thin markets under $10,000 score a close Brier of 0.0635 against 0.0099 for markets over $200,000 [8]; and 58% of Polymarket's national presidential markets in 2024 showed day-after price reversals, a sign of noise from internal order flow [8]. Kalshi's own audit of 427 races in the 2026 US midterms reports an Election Day Brier of 0.0139 with a slight Republican lean, matching or beating polling averages at every horizon [23].
Resolved forecasts

The ledger: what forecasters said, and what happened

Concrete, timestamped predictions with known outcomes are the only honest test. The Forecasting Research Institute's audits show a striking split by domain.

QuestionElicitedForecastOutcomeVerdict
Year AI reaches IMO gold-medal level2022 (XPT)2030 experts / 2035 superforecastersJuly 2025Underestimate
AI matches the baseline on a biorisk troubleshooting test (VCT)Nov 2024–Feb 20252030 experts / 2034 superforecastersApril 2025Underestimate
Cost of the largest AI training run by 20302022 (XPT)$180M experts / $100M superforecasters$388M already by mid-2025Underestimate
Highest annual recurring revenue of an AI company, end 2026FRI economic study$16–25BAbout $100B (Sept 2026)4–6× under
Share of US work hours assisted by generative AI, 2025Sept–Oct 2025 (LEAP)4.0% experts / 3.6% superforecasters5.7%Moderate under
Top Epoch Capabilities Index score, end 20262025–26165 experts / 169 superforecasters167Accurate
Autonomous share of US ride-hailing trips, 2027LEAP panel7.3% experts / 2.0% superforecastersAbout 2.5% (tracking)Experts over
Direct air capture and storage, 20242022 (XPT)0.60 Mt experts / 0.32 Mt superforecasters0.01 Mt (projected)32–60× over

Sources: FRI, “How accurate have AI progress forecasts been so far?” [6] and the XPT near-term accuracy report [47]. Some 2026 outcomes are tracking estimates, not final resolutions.

Two lessons. First, near-term accuracy is not a reliable proxy for long-term credibility: on the 38 near-term XPT questions resolved by mid-2025, superforecasters and domain experts were equally accurate, and both shared the same blind spots [47]. Second, the errors are not uniformly pessimistic. Software and revenue curves compounded faster than anyone expected, while physical-world deployment, from carbon removal to driverless taxis, met friction that rewarded outside-view scepticism. That boundary is a useful rule of thumb when you build scenario axes.

2025–2026

AI forecasters: parity on paper, a narrow human edge in practice

In two years AI forecasting went from trailing the public to winning open tournaments. How you read that depends on what is being measured.

Tournament stamina

A bot won the Summer 2026 Metaculus Cup (5 September 2026); bots took 1st, 2nd and 5th. Cups reward 24/7 coverage, instant news and frequent updating as well as accuracy [3].

Matched-question skill

On identical questions in controlled windows, Metaculus Pros beat the best bot teams in five straight comparisons, most recently by 1.25 points per question. Their edge is mainly discrimination, not calibration [13].

Benchmark parity

FRI reported in July 2026 that AI had likely reached parity with superforecasters on ForecastBench. Parity holds for scaffolded, tool-using ensembles; on the tool-free baseline board the public median (62.1) still edges o3 (61.2) and GPT-5.5 (60.2) [5][45].

Metaculus's survey of 39 bot builders shows what separates winning pipelines: a frontier model backbone; capping probabilities away from 0 and 100% (r = +0.48); two or more independent research feeds (r = +0.42); explicit base rates and lookups of similar resolved questions; around 28 model calls per question rather than 7; and Platt-scaling recalibration [13]. These are, almost one for one, the habits of good human forecasters.

Read leaderboards with care. Some November 2026 ForecastBench rankings reported in financial news (for example scores above the superforecaster benchmark) have not been independently audited. Retrospective back-tests suffer measurable search leakage (about 1.65% of results in one institutional study). Prefer prospective, contamination-free benchmarks such as ForecastBench's nightly rounds, Metaculus FutureEval and MiniBench [13][19].
See how we use AI in synthetic panels, and where we do not
Unresolved, late 2026

Long-horizon beliefs: read the spread, not the median

Where resolution lies decades away, disagreement between groups is itself the signal.

TopicGroupVintageMedian estimateTrend
General AI arrivalMetaculus communityOct 2026January 2031Compressed sharply since 2022
Human-level machine intelligenceAI Impacts survey (n = 1,580)Dec 202420422061 → 2059 → 2047 → 2042 across four waves
AGI (strict definition)FRI superforecasters / expertsMid-20262047 / 2050Pulled in from about 2060 in 2022
AI catastrophe killing >10% of humanity by 2100FRI experts / superforecastersMid-20265% / 2.4%Superforecasters at about half the expert view
Societal impact of AI by 2040FRI LEAP panel2025 → 2026“Technology of the century” (electricity-scale)Rising weight on “technology of the millennium”
Nuclear conflict with more deaths than WWII within 25 yearsFRI nuclear risk panel2022–20245%Non-AI tail-risk anchor

Respondents are sometimes internally inconsistent on long tails, for example rating transformative growth by 2070 as likelier than strong growth by 2100 [64]. Use these numbers as comparative belief distributions and weigh the direction of updates across methods (here: timelines compressing in surveys, platforms and compute models at once) more than any single median [63][65][96][112].

Put these signals in context on Horizon 2026–2050
Measurement

Scoring forecasts properly

If a unit cannot score its own forecasts, it cannot learn from them. Five rules cover almost every case.

RuleUse it forWatch out for
Brier scoreBinary questions. Mean squared error of the probability; 0 is perfect, a constant 50% scores 0.25Legacy datasets (Good Judgment Project, Good Judgment Open) use a 0–2 scale: divide by 2 before comparing [7]
Murphy decompositionSplitting a Brier score into reliability (calibration), resolution (discrimination) and uncertainty (question difficulty)Uncertainty is outside the forecaster's control, so raw scores on easy questions flatter [8]
Brier skill score / Brier IndexComparing across question sets: BSS = 1 − BS / BSreference; ForecastBench's index adjusts for difficulty (higher is better)The reference forecast must be stated [12]
Log and peer scoresPenalising overconfident misses harshly; Metaculus's time-averaged scores reward early, frequent updatingNot comparable with Brier values [11][21]
CRPS and weighted interval scoreContinuous and quantile forecasts, as in epidemic hubsNeeds full distributions, not point estimates [114]

Two practical controls: record when the outcome was actually decided, not only when the question formally closed (14% of markets in one 2026 study were settled more than a day before close [2]), and score at fixed offsets such as 30 days, 7 days and 1 day before the decision. A free tool such as Fatebook is enough to start a team track record [43].

For policy units

Six lessons for anticipatory governance

  1. Know the direction of your baseline's errorOfficial energy outlooks have under-forecast solar for two decades; IMF crisis programmes lean optimistic on growth. Adjust or bracket, rather than adopt.
  2. Split software from steelExpect digital and revenue curves to surprise upwards and physical deployment (infrastructure, carbon removal, autonomous vehicles) to surprise downwards. Build that asymmetry into scenario axes.
  3. Aggregate before you deliberateWeighted, extremised pooling of independent forecasts beat both markets and unweighted polls. Collect numbers privately first, then discuss.
  4. Keep score, publicly where possibleLog forecasts with dates and resolution criteria, and score them with Brier or interval scores. The Philadelphia Fed has published individual forecasters' errors for decades.
  5. Use markets and AI as inputs, not oraclesLiquid markets are well calibrated near resolution but noisy far out and in thin books; AI ensembles are fast and cheap but strongest when tool-augmented and checked against humans on matched questions.
  6. Treat long-range probabilities as disagreement mapsFor 2040–2100 questions, the spread between experts, superforecasters and the public tells you where to place signposts and safe-to-fail experiments.

Sources

Compiled from the research dossier Global Forecasting Ecosystem Dataset (November 2026). Numbers keep the source's original reference numbers. Exchange-authored and vendor figures are labelled as such in the text; raw Brier scores are not directly comparable across venues.

  1. Totalis, Which prediction market has the most volume? Kalshi vs Polymarket (industry reconciliation)
  2. PredictionScout, How accurate are prediction markets? Kalshi and Polymarket data
  3. explainx.ai, AI wins Metaculus Cup: humans still lead head-to-head
  4. ForecastBench, About
  5. Forecasting Research Institute, How accurate have AI progress forecasts been so far?
  6. Are markets more accurate than polls? The surprising informational value of “just asking”, Judgment and Decision Making
  7. Horus Analytics, How accurate are prediction markets? (review of Dickerson et al. 2026, Le 2026, Kagan and Baiocchi 2026 and others)
  8. Metaculus vs Good Judgment Open vs Manifold: which score you
  9. Glypse, Brier score guide
  10. Metaculus, AI forecasting in 2026: what 11 analyses say
  11. Crowd prediction systems: markets, polls and elite forecasters (Wharton)
  12. BigGo Finance, NeuroFusion tops AI future-forecasting benchmark (media report, not independently audited)
  13. Metaculus, A primer on the Metaculus scoring rule
  14. Quantifying advances in infectious disease forecast skill across seasons and pathogens
  15. Kalshi Research, Kalshi and the 2026 midterms: calibration and accuracy (exchange-authored)
  16. Le (2026), Domain-specific calibration dynamics in prediction markets, arXiv:2602.19520
  17. Fatebook API
  18. ForecastBench leaderboards
  19. FRI, Assessing near-term accuracy in the Existential Risk Persuasion Tournament
  20. Slate Star Codex, Book review: Superforecasting
  21. agitracker.io, AGI timeline: 74 dated predictions (2005–2026)
  22. Artificial general intelligence forecasting and scenario analysis, arXiv:2604.22766
  23. Virev AI, Odds and dates compilation
  24. Federal Reserve Bank of Philadelphia, Survey of Professional Forecasters
  25. Survey of Professional Forecasters (overview)
  26. ECB Occasional Paper 59, The ECB Survey of Professional Forecasters
  27. Eicher and Kawai, IMF trade forecasts for crisis countries
  28. The accuracy of IMF crises nowcasts
  29. Systemic bias of IMF reserve and debt forecasts for programme countries
  30. Hoekstra chart: the IEA once again predicts solar installations
  31. IEA, Renewable energy: RD&D priorities
  32. IEA, World Energy Outlook 2026: executive summary
  33. Breakthrough Institute, Solar power: the case for tempered optimism
  34. RenewEconomy, Reality keeps surprising the IEA on global solar growth
  35. The US COVID-19 and Influenza Scenario Modeling Hubs
  36. Coordinating collaborative infectious disease modelling projects with the hubverse
  37. Tetlock, Expert Political Judgment: how good is it? (chapter 1)
  38. What do XPT forecasts tell us about AI timelines?
  39. Comparing top forecasters and domain experts
  40. FRI, Can humanity achieve a century of nuclear peace?
  41. Evaluating forecasts with scoringutils in R

Put a number on your policy's future

Run your policy through four structurally different futures, then decide which signposts to score.