Sweepstake Knowledge: the tyranny of orphaned deterministic numbers
November 2017
Conmen down the ages have found their way in to our pride by challenging us to hazard a guess – go on! “Have a guess: how many grains of rice (almonds, chick peas or hemp) are there in these bottles? Place a bet - winner takes all!” It has to be easy money surely because you can use your judgment and your judgment is better than a lottery, right? We can’t resist the draw, either as a form of idle entertainment, where being proven wrong is part of the fun, or as a profession, where being proven wrong is an every-day fact of life.
“SOCIETY'S TO BLAME...”
As geoscientists, we know we don’t have the answer but we can estimate that the truth lies in a range described by an uncertainty distribution function (expressed as a probability), keeping the aperture wide on uncertainty in the face of our own ignorance. Unfortunately, not everyone is as honest as us because neither conmen nor honest brokers can put money on a range. They know that there is only one truth and that truth is a single number, which requires sampling the distribution function somewhere along the curve so that those who seek a single number can at least know how representative that number is. This is where the tyranny begins: in order of descending honesty…
Economists: Economic models are deterministic, honestly represented by building scenarios, honouring where the numbers have been sampled from an uncertainty distribution function and recording exactly what the expectation results are. There is nothing intrinsically wrong with deterministic numbers if we know where they come from. The problem is that these numbers have a habit of being taken out of context, acquiring a life of their own, becoming misappropriated along the way through no fault of the economist. That is business.
Conmen: Conmen ask you for a deterministic number (your best guess) but they are honest enough to tell you that your guess exists within a range of possible outcomes. They sell the dream not your part in it (you choose). That is gambling.
Politicians: Our representatives know that you need a single number to focus the power of your vote. They know there is a range of potential outcomes but they tell you what they want you to hear and sample anywhere in the uncertainty distribution function without telling you where, with intent to mislead. One thing you can be quite certain of, however, is that the donkey party will hold up the P10 as the most likely and the elephant party will give you a P90, depending whether you are facing north or south. That is democracy.
Whether you adopt a business, gambling or democratic model, popular outcomes will always tend to seek protection in the middle (P50) but there will also be biases that will skew the distribution, depending on which particular interest group (extrinsic to the distribution itself) is winning the argument. We can remove the bias if we choose to…
CROWD-SOURCED SOLUTIONS
More recently, we have been invited as explorers by a number of pundits to participate in crowd-sourced deterministic sweepstakes to give our best (educated) estimate about “how close to prognosis is the top reservoir?” or “how much oil is in this prospect?”. We might have little or no information to help guide our response but we do it anyway because after all, it’s only harmless fun (what do we have to lose?) and so we can give free reign to our bias. We do this by dipping into our experience and come out with a deterministic “best guess”, knowing there is an uncertainty distribution function behind that guess and allowing our gut feel to guide us to the truth. We aren’t completely stupid; we know our answer will be wrong but we can’t resist seeing how close we can get to the truth just to test our judgement (preferably in private) or justify our confirmation bias. The organisations running these crowd-sourcing websites (who have also contributed to this experiment) are doing pioneering work motivated by a genuine desire to harvest the “wisdom of the crowd” by establishing if crowd-sourcing knowledge can improve decision making in the future. What they currently lack is sufficient granularity to distinguished between cognitive bias and native observations. The former expresses who we are, which is interesting; the latter can be tied back to the original data, which is essential for verification.
What if we were to do this differently? what if I told you that instead of one big guess per bottle, that we break down a sequence of smaller guesses that combine to help you get closer to the truth? would that help to make us more aware of gaps in our knowledge and thereby our confidence? would it improve our accuracy? In this case, let’s say you can visually estimate two component quantities:
- The internal dimensions of the bottle
- The average grain size
· Grain sorting
· Grain shape (based on an ellipsoid)
· Settling or compaction (estimate intergranular void space)
Number of grains = total volume available / (the volume occupied by a single average grain + average associated porosity)
There remain uncertainties but the metrics combine to provide you with a calculated estimate of the number of grains in the bottle, which you can then compare with your original big guess. The big guess taps in to your judgement; the composite estimate is more evidence based. The number you get is still a guess but the composite is more accountable, more credible because it is under-pinned by premises that we recognise. In effect, we establish the credibility of the whole by summing its parts and then comparing the two as separate perspectives to test for accuracy. Pulling apart value statements in to measurable quantities and reassembling them is at the heart of the experiment reported in the article on stacking knowledge in September (click HERE). It is starting to reveal that the whole may indeed be greater than the sum of its parts but it is not necessarily closer to the truth.
THE REAL WORLD
If the conman approached you and said “Have a guess: how many grains of rice (almonds, chick peas or hemp) are there in these bottles?” would you be interested now? You can’t see the grains, you don’t know what the internal dimensions of the bottles are, you have no idea if they are full, partially filled or even empty; there is just too much here for you to make a judgment on and you would be inclined to “pass”. Not even a politician would attempt this one because there is an overwhelming over-print of risk (perception of loss) associated with the uncertainty. And yet this is what explorers do every day in the oil and gas sector. Not by guessing but by introducing appropriate technology. You can’t see through the bottles any more but you can weigh them and you know from analogue studies what the density of the bottle is, and how the weight changes from an empty bottle to a partially filled one. Not a precise measurement but we have successfully polarised the risk and are back to having an estimate supported by evidence, which may be worth taking a punt on. Is this science or is it just luck… or a little bit of both? As we all know, fortune only smiles on those who are prepared for it.
WHY KNOWLEDGE?
“There is nothing intrinsically wrong with deterministic numbers”. If anything, they are more honest than hiding behind a smeared distribution function, which is itself underpinned by deterministic scenarios. But deterministic numbers are only dangerous if they become separated from their origins – from knowledge.
Einstein famously declared that imagination is more important than knowledge and who can deny it? But what is imagination without knowledge? Just dreaming without reference or accountability?