Research and decision making – A Bayesian Approach

Pre-existing beliefs can’t help but cloud our judgment of new information, this is true when presenting unexpected findings to decision-makers that may be contrary to their former intuitions.

This is a problem because interesting insights might be overlooked just because they do not fit with a previous picture of the market, causing long-term and old ideas to persist potentially past their period of maximum utility.

One solution to this problem is an approach known as ‘Bayesian decision making’, originating from a recent trend in statistics that is replacing the conventional ‘statistical significance’ approach to statistics.

Whilst statistical significance testing creates an absolute cut-off between significance (likely to be true) vs. non-significance (unlikely to be true), Bayesian statistics considers findings more holistically with no absolute distinction between either PASSES or FAILS for any given finding.

Instead, Bayesian statistics advise that we continually update our confidence in our beliefs as new information arrives.

Using this logic, we can help decision-makers incorporate new findings, and make improved decisions, even if they challenge their existing beliefs.

To take an example, let’s say a decision-maker is 90% confident their brand should partner with brand A as part of a new brand partnership initiative. Research, speaking to their customers, conversely suggests brand B is the preferred partner. Should the decision-maker disregard all their prior experience and knowledge (which might be vast) in favor of the research findings?

Absolutely not, this would be the old ‘pass/fail’ approach promoted by significance testing. Instead, the research should be considered as a warning light, that maybe the decision-maker is missing something important, or maybe there is another opportunity that requires exploration.

At first, they were 90% confident in brand A, if the research had supported this they may have increased that confidence to 99% and felt good contributing even more resources to this decision.

Given that the research didn’t support this, instead, they might reduce their confidence to 85% and take a more cautious approach to the brand partnership, perhaps spending fewer resources on this initially, or taking the time to explore Brand B further.

Bayesian decision-making helps us ‘hold the hand’ of decision-makers at inevitable times when the research findings give us mixed signals (research is messy!). It is not likely or advised, to ever reach 100% confidence in any real-life decision, but thinking about research and informed decision making in terms of levels of confidence is a useful tool in ensuring research findings, however unexpected, are incorporated into the decision-making process.

A version of this talk was recently presented at the Big forum Online hosted by Business Intelligence Group. 

This talk was presented at the Big Forum Online (hosted by Business intelligence Group) by Steven Nicholson