It is commonplace to deploy increasingly complex tools in order to evaluate the current, and expected benefits of a given policy initiative, or program. Academic disciplines compete amongst each other for the primacy of their explanatory framework, and try as best as they can to convince media and decision-makers that their outlook is best.

In this, I will be no different! Defenders of common-sense have a long tradition behind them, not only among most regular people with no particular intellectual ideological axe to grind, but among some pretty sophisticated heavyweight thinkers. The Scottish Enlightenment gave birth to many wise and prudent thinkers who were careful not to get carried away in fits of abstraction, and to check their prejudices against the world of concrete facts. What was early ’empiricism’ in early modern philosophy of course has roots in Aristotle and quite a few other thinkers after him, but in the hands of the Scots, it was more common-sensical, if not then, at least to many practically minded people today.

These days, academics, and policy wonks of the empiricist bent may be slightly better than their rationalist counterparts. Yet, they are also fond of engaging in complex modelling done on the basis of a limited set of idealized assumptions that try to capture increasingly complex social realities and predict on the basis of findings from statistical samples, years into the future.

I am no postmodern skeptic of the empirical sciences, statistical methods, and of the accumulated store of general knowledge about patterns in macroeconomics and sociology. However, there is definitely a limit to this data-driven predictive thinking and its connection to policy. Here’s why.

There is a problem with empirically-based predictions, in general. Many thinkers have come to similar conclusions about the limitations of empirical prediction from very different starting points.

René Descartes was engaged in a search for the most certain foundations, upon which a theory of knowledge could be built. He was motivated by an inability to separate the wheat from the chaff in the claims being thrown around by some of his contemporaries. We are of course in the same predicament today, and always will be. He tried to doubt the truth of everything he could, until he stumbled upon something that could not be doubted. From here, we would deduce further truths, and build a philosophy that could be applied to all areas of inquiry. The Cogito ergo sum was that foundation – I think therefore I am. He could not doubt that he was thinking, because it was always true that he was thinking, even when he was doubting! He had arrived at a species of logical truth, guaranteed by the principle of noncontradiction. Unfortunately, through this method, he couldn’t prove that something like an ‘I’ (René) was thinking, only that thinking was taking place. To establish that further claim – that he himself exists as a 17th century Frenchman – he would have to have recourse to other claims, which of course, could not be proven absolutely without further recourse to others, and so on and so forth.

The Problem of Induction, as it has become known, has a long history in philosophy. One standout figure who helped crystallize the contemporary version of it was David Hume. The common practice of predicting outcomes on the basis of past experience is something enabled by the process of repeated exposure of like experiences. We infer that the sun will rise again tomorrow because it has done so everyday of our lives thus far. Though each one of these repeated instances of experience are similar, they are never identical, and are in fact rather different in that the state of affairs that accompanies every particular like instance is different – the state of the matter in the world, the position of the sun vis-à-vis other planets, and so on. This is not to say that we cannot aid our predictions with better theories, but points only to the fact that there is no set of additional criteria that we can appeal to that can get us beyond the problem of having to rely upon the comparison of like experiences. We can have no further proof that like experiences will always be connected with what they have been in the past. The problem here is with justification. How are we to be justified in the validity of our prediction, when what we have to go on are just repeated exposures of like experiences?

Writing about the predictive power of scientific theories in general, and about their use in social contexts in particular, W.V.O. Quine famously argued that by necessity, theories underdetermine their conclusions. This is simply because there can never be enough evidence to completely rule out alternative explanations. The evidence in support of your preferred explanation is always, and by necessity incomplete. It is practically – though not logically – impossible to rule out all of the possible subtle ways in which the pieces in your explanatory puzzle may be incorrect, or at least lacking in some way. Quine thought that this implied that we best think of our knowledge holistically – in order to evaluate an isolated claim, we make reference to a set of other considerations that we think we’re also warranted in believing, and upon which the truth of the claim in question depends. The set of these claims and beliefs fit together in a web, that cannot be evaluated without reference to one another.

F.A. Hayek applied similar insights to the social world, and the domain of policy. he noted that knowledge was not localized, but distributed among many different people in a society. This precludes the ability of planners to make decisions about the allocation of resources whose effects could be predicted with great certainty. It is not the case that planners could not make the ‘right’ decisions in some cases, but provides a strong argument to leave many decisions, and the specifics of them, up to local decision-makers.

All these arguments go to show that we cannot have predictive certainty in principle, and that the more complex the phenomenon to be explained, the less successful our models will be in making accurate predictions. Furthermore, the bigger the scope of the entity that we are trying to understand, and affect with a decision, and the further removed the decision-maker is from the situation ‘on the ground’, the less likely it is to be successful, and to achieve the results desired.

These arguments are not undermined by the rise of data collection. What data-driven decisions must necessarily leave out is a good deal of intangible knowledge that individuals have about their relationships with other people, their knowledge of how to do things, and their constrained, but nonetheless free capacity to choose and act in new, ultimately unpredictable ways. There are many ways to increase predictive power, but the principle remains the same, and indeed, insofar as the system is increasingly complex, it is more difficult to predict.

From a certain perspective, our lives are correctly characterized as the predicament of choosing what to do in the face of limited knowledge, time, and resources – it is the very description of the state that we constantly find ourselves in. We like to feel confident in our decisions, and also think highly of them since they are always made in light of alternatives. We rarely get a payoff from presenting our opinions in a guarded manner, by using terms like ‘maybe’ and ‘probably’ to qualify our justifications and suggestions. Thus, the aforementioned theoretical insights of philosophers quickly lose whatever appeal they might have in the abstract, when you find yourself in the heat of a tough decision.

It might be helpful if some people thought a bit more like some economists, who understand the meaning of the definition of their subject – the study of the alternative use of scarce resources. This definition, in spite of what some economists and people may think, is in fact rooted in common sense. It touches on many decisions that we make, even though they may not be strictly speaking ‘economic’. This is simply for the reason that this definition provides the conceptual tools to think about decision-making in practical terms, in light of scarcity, in light of our own abilities and resources, and the impact of our choices, and those of others on the micro and the macro level that we are always simultaneously operating in.

Each one of the words in this statement refers to something important. First, take resources. These are not just natural, or physical things that we consume or use in producing other things. They include a person’s abilities, time, effort, productivity, and network of relationships. In terms of physical resources, it is not just their availability, but the processes, and institutions around it that give it value. It does you no good to have an excess of bronze, when you need copper, or an air conditioning unit in the winter, and a fleet of vehicles without access to the right amount of fuel, and at the right price.

These resources are always scarce, because there is a finite amount of them, even if many are nowhere near the point of exhaustion. When people put more time and effort into some things, this means there is necessarily less to go around for the alternative uses of these things. Thus, even though there may be a whole lot of some natural resource lying around, an ‘excess’ of money in someone’s bank account, and food on the shelves in grocery stores, these things are still ‘scarce’. Why? – because the mix of resources in the society is not being deployed in such a way so as to activate and make them available to the critic who reasons that, ‘everyone should have food because there is enough food in the world to ensure that no one goes hungry’. This is to miss the point of how scarce resources are allocated in complicated societies where information is distributed among billions of people, and is outside the scope of any one person to understand, and act upon.

When we evaluate some given policy, we should be careful to note the facts pertaining to the availability of resources, what the alternatives are, and how the choice of one path over another will impact others, and the way they change their behaviour in accordance with the new relationships of availability and scarcity in the mix of resources in the domain of concern – a city, a province, a country, or a network of countries.

Furthermore, when we combine our principle of humility about our (in)ability to know the future, and predict in situations of increasing complexity, with the economists basic toolkit, we should steer clear of big government programs and overly ambitious social engineering.