Bayesian Updating

A payment incident has more than one plausible cause

At 09:00, payment success falls from 96% to 70%. A release went out ten minutes earlier, so the first explanation feels obvious: the release broke payments.

The team rolls it back. The failures continue.

One engineer had warned that the change was unrelated, but the coincidence looked persuasive. He now spends another hour defending his work while customers wait and support deals with the consequences.

The team has at least three live explanations:

• The release introduced a defect.
• An upstream payment route is failing.
• The monitoring dashboard is wrong.

Then new evidence arrives. The decline began seven minutes before the release. Failures are concentrated in transactions using one bank route. Support can see customers stuck in pending states, and direct database counts agree with the dashboard.

Every fact can fit more than one story, though the stories don't predict every fact equally well.

Bayesian Updating means changing how much weight you give competing explanations as evidence arrives. Ask:

How much more likely would I be to see this evidence if one explanation were true than if its rivals were true?

Give more weight to evidence that the competing explanations predict differently.

Start with what was plausible, then update from there

A Bayesian update has three parts.

Before: what was already plausible

Start with what was plausible before the new evidence. A defect near a release deserves attention because releases sometimes cause failures. A payment provider also deserves attention if it has failed before. Keep a dashboard fault on the list until an independent count makes it less likely.

These starting weights are called priors. They come from past frequency, context, and what you already knew. They should change with the context.

Evidence: does it separate the stories?

Now ask how expected the observation would be under each explanation.

If both a release defect and a provider failure commonly produce a payment drop, the drop itself distinguishes little. The fact that the decline began before the release is different: it would be surprising if this particular release caused the problem, but unsurprising under the provider explanation.

That comparison is called a likelihood ratio. If both explanations predict the evidence equally well, it should barely change your view. If one would predict it and the others would struggle to, it should change your view much more.

After: where the evidence leaves you

The revised comparison is called posterior odds. Odds compare how much weight one explanation has against another:

Posterior odds = prior odds × likelihood ratio

In plain language, start with what was plausible, compare what each explanation predicted, then change your confidence by how well the evidence separates them.

A plausible explanation is easy to find, so look for evidence the other explanations would have difficulty producing.

What the team should do with the update

The evidence now favours the upstream route, though “favours” is short of proof. The updated view should change the next check: route a small set of payments through another provider, compare its success rate and be ready to undo the change.

The incident response should stop treating the release as the established cause, correct the unsupported blame placed on the engineer, and investigate the failing route while keeping other explanations open.

How this moves across fields

Medicine. A positive screening result doesn't mean the condition is probably present without considering how common it was beforehand and how often the test is positive in healthy people. A result shared by many harmless conditions moves belief less than a follow-up test that's unusual unless the disease is present. The same calculation applies: start with prevalence, compare how likely the result is with and without the illness, then update. Treatment still requires a separate decision about side effects, delay and the patient's preferences.

Law and historical inquiry. Evidence is useful when competing accounts predict it differently. A fact consistent with both prosecution and defence adds little even if it makes one side's story vivid. Historians use the same discipline when comparing explanations for an event: a source matters because rival accounts may have difficulty accommodating it. Courts also impose rights and standards of proof that probability alone can't supply.

Software, product and strategy. During an incident, a metric change may fit a release bug, a seasonal pattern, broken tracking, or a supplier failure. In product work, one retention increase may fit a successful feature, a changed customer mix, or a temporary promotion. Write down what each explanation predicts before checking the dashboard, so checking the data can separate them and the team can't rewrite the story afterwards. Three dashboards fed by one event stream are one stream of evidence seen three times.

In each field, compare the live explanations against the same evidence instead of checking only whether the first story sounds plausible.

A practical loop for using it

Use Bayesian Updating when diagnosing incidents, interpreting experiments, evaluating forecasts, screening candidates, conducting due diligence or making any decision where several causes remain plausible.

1. Name live alternatives. Include at least one explanation you don't currently prefer.
2. Set rough starting weights. Use how often each explanation has been true before and the current context; ranges are often enough.
3. Predict before looking. What would each explanation make you expect to observe?
4. Seek discriminating evidence. Prefer observations that rivals predict differently.
5. Update by degree. Don't convert one clue into certainty.
6. Choose the next reversible test. Let the new belief change what you do.
7. Record what would change your mind, so the team can't quietly change the standard later.

Exact numbers help when good data exist. When they don't, call an explanation unlikely, plausible, or leading rather than inventing precision.

What this discipline can't solve

Bayesian Updating can produce disciplined confidence in the wrong set of explanations. If every live hypothesis is mistaken, updating among them never discovers the missing cause.

People can still pick convenient starting weights, judge how strong the evidence is only after seeing the outcome, or count several measurements from one source as independent confirmation. Strategic actors may change their behaviour once they know what evidence you watch.

The model updates belief and doesn't choose action for you. A 10% risk may justify immediate containment when harm is irreversible, or no action when the intervention is worse than the threat. Costs, rights, reversibility, and error asymmetry still matter.

Nearby models

Signal Detection Theory asks when uncertain evidence should trigger action and which errors the threshold will create. Bayesian Updating asks how the evidence should change the relative credibility of explanations. The updated confidence from Bayesian reasoning can inform when you act, but changing a belief and deciding when to act are different decisions.

Confirmation bias is the tendency to seek or interpret evidence in ways that support an existing belief. Bayesian Updating is a corrective discipline because it forces you to ask what rival explanations predict. It's no automatic cure: conveniently chosen priors and likelihoods can make bias look mathematical.

When the first story feels obvious, ask what else could explain the facts and which observation would separate the explanations.