The Knowable Future:
Forecastability and the
Limits of Prediction
The limit on prediction is set by the information in the data, not the sophistication of the model.
A research programme by Dr Peter Catt
Auckland, Aotearoa New Zealand.
The core research question:
How far into the future does a time series contain usable information about its own evolution?
Forecastability is the extent to which the past contains exploitable information about the future. It is a property of the series and the horizon, not of any model.
The Knowable Future measures forecastability, maps how predictive information changes across forecast horizons, and identifies the limits of prediction in time series. It begins one step before model selection, with a prior question: how much of the future is actually knowable from the past?
The answer varies across series and horizons. Some systems retain enough structure to support meaningful prediction; others do not. The programme develops pre-modelling diagnostics that estimate forecastability from training data alone, clarifying when sophisticated forecasting is justified, when simpler approaches are sufficient, and when the structure of the problem places hard limits on what can reasonably be predicted, with direct consequences for forecasting practice, model selection, and economic value.
Why the world is not fully deterministic
In 1814, Pierre-Simon Laplace imagined an intellect that knew the position and momentum of every particle in the universe. For such an entity, nothing would be uncertain: past and future would be equally visible. This became known as Laplace’s demon. Two findings from physics defeat it, independently.
Quantum mechanics rules out local hidden-variable determinism. Bell’s theorem (1964), confirmed by loophole-free experiments in 2015, shows that no account of underlying local facts can reproduce the observed correlations. Non-local deterministic interpretations, such as de Broglie-Bohm pilot-wave theory, remain logically possible, but only by allowing instantaneous action across arbitrary distances. What is gone is the comfortable Laplacian picture of a locally deterministic world awaiting a diligent calculator.
Deterministic chaos defeats the demon independently, and more deeply. Even in a perfectly deterministic universe, prediction remains bounded. In chaotic systems, tiny initial errors grow exponentially at a rate set by the Lyapunov exponent, and no starting state can be measured precisely enough to constrain a trajectory indefinitely. Heisenberg’s uncertainty principle guarantees the imprecision even in principle; chaos then amplifies it. Atmospheric predictability, limited to roughly two weeks, is the canonical case.
That is where physics hands the problem to information theory. Shannon’s data processing inequality states that no transformation can recover information that was never present. Once mutual information between past and future has decayed, no model can reconstruct it. In many systems, the ceiling on predictive performance is set before modelling begins. Model capability is not the binding constraint. Information content is.
Physics ended Laplace’s dream of perfect prediction. Information theory explains why. Forecasting therefore begins not with models, but with a measurement of how much of the future is encoded in the past.
What the past can tell us
Forecasting is often treated as a modelling problem. This research begins one step earlier, asking a more fundamental question: how much of the future is actually knowable from the past? The answer varies across series and horizons. Some systems retain enough structure to support meaningful prediction. Others do not. The Knowable Future is concerned with measuring that difference.
Drawing on information theory, specifically the mutual information between past observations and future values, it examines how predictive signal persists, weakens, or collapses as the forecast horizon extends. The aim is to estimate forecastability before committing substantial modelling effort, a question with practical weight across business, economics, engineering, public policy, and the physical sciences.
The deeper question, then, is not simply which model performs best, but whether the underlying process contains enough recoverable information to make forecasting worthwhile at all. By shifting attention from model choice to the limits of inference itself, this work aims to clarify when sophisticated forecasting is justified, when simpler approaches are sufficient, and when the structure of the problem places hard limits on what can reasonably be predicted.