Do you trust your data?

Over the last week, my attention was taken by three snippets of ‘news’.

The first was an interview with a GQ journalist who has written a biography of David Bowie.  Instead of the usual form of celebrity life story, the book is a series of interviews with people who interacted with Bowie over his life, some like Brian Eno who worked closely with the star and other, more casual acquaintances.

During the interview the journalist mentioned a conversation with his father about a critical Top of the Pops appearance by Bowie.  The tape of this appearance was lost by the BBC for many years but it has recently been rediscovered.  It is often considered as the point where glam rock exploded on the UK scene. In the clip, Bowie is seen to drape his arm around guitarist Mick Ronson during the closing chorus of Starman; for the early 1970s, a rather suggestive action.

The journalist described to his father what had impressed him about Bowie in the performance.  It wasn’t just the act of putting his arm around Ronson.  it was the colour of Bowie’s jumpsuit and his shock of carrot red hair; it was Ronson’s gold page-boy outfit; it was the colourful stage set and lighting.

Then the journalist’s father spoke, “you do realise son, at that time we had a black and white television?”

The second thing which caught my attention was the article by the foreign secretary, Boris Johnson in the Daily Telegraph on his vision of Britain post-Brexit.  In the article, Boris reiterated the claim that Britain will have £315 million a week to spend on services like the NHS.

Now, I do not believe that Boris Johnson believes that figure for one moment.  Boris is clearly on manoeuvres and preparing the ground for a leadership bid.  he knows that figure is inaccurate but he also knows that a large part of the constituency he is talking to have been taken in by it.

The £350 million a week claim was widely debunked during the EU referendum.  So much so, the Electoral Commission told the Leave camp to stop using it (a warning which was ignored).

The truth is that the £350m a week figure ignores the rebate on EU membership fees achieved by Margaret Thatcher.  it also ignores money sent back to the UK in the form of grants and subsidies.

The UK treasury publishes data on the UK’s EU membership fees.  We paid last year £230 million a week to the EU; £120 million less that the figure quoted by Johnson.  Net, once grants and subsidies are subtracted, we paid £153 million a week.

Now £153 million a week is still a lot of money.  To Brexit supporters it is an anathema.  However, if you consider that the single market is worth £450 billion a year in trade to the UK, our EU membership offers a Return on Investment of 5100%.  If you, as a business were offered such a return, you would grab it with both hands and not let go.

Boris also ignores the costs of leaving the EU.  Only today, the head of HMRC has said that to toughen border controls to the level wanted by many Brexiteers, the cost will be £800 million.  The Office for National Statistics has calculated that the cost of lost tax receipts from EU migrants post-Brexit and in replicating functions currently undertaken by EU institutions will be £315 million a week.

The implications of these figures are clear, on leaving the EU, all of the money reclaimed from our membership fees will be swallowed up replacing lost government income.  in fact, the government may have to find an additional £200 million a week through cuts to services or increased taxes.

The third thing which caught my attention were two internet surveys.  One was listing the best places in the UK to live.  the other was listing the most depressing places in the UK to live.  Edinburgh topped both polls.

Now how can Edinburgh be simultaneously the best and worst place to live?

The answer to nearly all of this is confirmation bias.

Confirmation bias is the tendency of human beings to search for; to favour; or to recall information in away which confirms pre-existing beliefs or hypotheses.  There are three basic forms of confirmation bias:

  1. Biased search for information
  2. Biased interpretation of information
  3. Biased memory

The GQ journalist is suffering from biased memory.  In his mind Bowie’s appearance is full of colour; no doubt based on recent showings of the Top of the Pops performance; when the truth is that he first saw it in black and white.

Many Brexit supporters are guilty of a biased search for information as they only read sources of information which match their political viewpoint e.g. The Daily Mail.  It has been rumoured that David Davis, the cabinet minister for exiting the EU, will only read civil service papers which match his existing point of view.

Biased interpretation of information is where both sides of a discussion have the same information but they interpret it differently based on their beliefs.

Confirmation bias used to be common in miscarriages of justice.  Take as an example the Birmingham Six.  In that case, the investigation of an IRA bombing in Birmingham, the West Midlands police were found to have bent the evidence to suit their case.  The crucial evidence was forensics tests on the hands of the accused.  To the police these were prime suspects, Ulster Catholics with some links, although peripheral to the IRA.  The forensic tests appeared to show that the accused had recently handled explosives.  The police thought they had an open and shut case.  They applied significant physical and psychological pressure to obtain confessions.  However, the police officers involved were so tied to their preconceptions they ignored evidence that put the forensic tests in doubt; the fact that the accused had been playing cards on the night of the bombing and that the coating of the playing cards gave the same results in the forensic test as if the accused had been handling explosives.  In that case, the investigators were clearly guilty of a biased search for information.

Following a number of such miscarriages of justice, new procedures were placed on investigators not just to investigate one potential course of events but all potential courses.  If an accused person gives an alibi, the investigation of that alibi must be treated in the same manner as all other potential evidence.

Then we come to the two internet surveys.  These are likely a case of biased interpretation.  Where two similar sets of information are interpreted in a different manner.

I is also highly probable that these two polls have been carried out in an unscientific manner where little or no care was taken to appropriately select respondents.

This is often an accusation placed at political psephologists, that they way they weight their opinion polls has such an effect as to skew the polls findings.

Professional market researchers take great care to minimise the effects of confirmation bias.  They use information from previous surveys; they carefully select the respondents to ensure that a wide range of demographics are included; they weight polls to take account of potential bias; they ensure the numbers of respondents are statistically significant; etc.

All too often, in small businesses, decisions are made on the views of individuals; often the views of the business owner.  Little or no regard is taken as to the potential for confirmation bias.  As a result there is a real danger that incorrect, costly or improper strategies are pursued.