The Knowable Future
The Knowable Future:
Forecastability and the
Limits of Prediction
The limit on prediction is set by the information in the data, not the sophistication of the model.
A research programme by Dr Peter Catt
Auckland, Aotearoa New Zealand.
The core research question: How much of the future is knowable from the information available now?
Forecastability is the predictive information available about a target at a given forecast horizon, given a declared information set: the data a forecast is allowed to use, stated in advance. That may be the target’s own history alone, or it may add other observed variables and external signals available when the forecast is made. The Knowable Future studies how forecastability varies across horizons, how it can be estimated, and what it implies for forecasting and decisions.
​
There is a limit to how well anything can be forecast, and it is set by the information available, not by the forecasting model. Measured at each forecast horizon h, that limit is the forecastability profile, written F(h). No method, however sophisticated, can extract more predictive information than its data contains. In practice the profile is estimated from data, so any measured value carries some uncertainty. The exact statement is on the Definitions page.
​
The consequence is forecast triage. Where information is rich, sophistication earns its keep; where it is thin, simple baselines are near-optimal; where it is absent, the real work is decision design. Model choice is the second question; how much of the future is knowable is the first.
How Much of the Future Is Knowable from the Past?
Why prediction has limits
In 1814 Pierre-Simon Laplace imagined an intellect that knew the position and momentum of every particle in the universe, a figure now known as Laplace's demon. For such an intellect, he argued, nothing would be uncertain; the future would lie open like the past. The demon did not survive the twentieth century.
The first failure is quantum mechanical. Bell's theorem, confirmed by loophole-free experiments in 2015, rules out any account in which measurement outcomes are locally predetermined. Deterministic interpretations of quantum mechanics survive, but only by abandoning locality, and none restores a world in which the future is simply read off the present.
The second failure is classical. Even a perfectly deterministic system defeats prediction when its dynamics are chaotic. The smallest error in the starting state grows exponentially, at a rate set by the Lyapunov exponent, and no measurement is ever fine enough to pin the state down. That is why atmospheric predictability ends near two weeks: not for want of better models, but because the atmosphere forgets its own past.
Information theory turns that forgetting into a number. The past holds a measurable quantity of information about the future, and the data processing inequality guarantees that no method, and no amount of computation, can extract more. What survives Laplace is not the demon but the question of degree: how much of the future the past still contains. Prediction has a ceiling, and the ceiling belongs to the data. This programme measures it.
​
​