Reference Class Forecasting
Before you trust the plan, check how comparable work really went.
Start an uncertain forecast with how similar work actually went, then adjust only for differences backed by evidence.
A sensible task list can still give you the wrong date
A small hardware team wants to build a stair-climbing accessory for a home robot.
They break the first prototype into sensible pieces: two weeks for the mechanical design, two for fabrication, one for control software and one for testing. The work adds up to six weeks. Nobody has made an obviously careless estimate.
Six weeks later, the parts have arrived in three batches. The first wheel assembly climbs one step but tips with the intended battery. Moving the battery changes the enclosure, the new enclosure changes the weight, and the control tuning has to begin again.
A collaborator has arranged a demonstration around the original promise. She postpones it and keeps a backup plan because the delivery date is no longer dependable. The team works late because the six-week commitment didn't include the rework and supplier delays seen in comparable builds.
The task list showed one possible sequence. The estimate left out the delays and rework that showed up in similar completed prototypes.
Start with completed cases
Reference Class Forecasting means starting with what similar completed work actually took, then adjusting only for differences that have a believable effect on the result.
Suppose the team finds twelve recent prototypes with similar mechanical uncertainty, supplier dependence and safety testing. Half reached a first safe unit people could test in eighteen weeks, and eighty per cent did so within twenty-eight. The usual delays were ordinary: late parts, components affecting one another, enclosure rework, battery and weight trade-offs, test-rig changes and supplier iteration.
That range becomes the starting point. The team already has a compatible chassis, a known fabricator and a working test rig, so it has real reasons to test a fourteen-week internal target. It keeps capacity through twenty-eight weeks and uses earlier tests to decide when it has enough evidence to promise an external demonstration.
Instead of adding an arbitrary buffer to six weeks, start from the completed-project range and adjust only for differences with evidence, such as the existing chassis, fabricator and test rig.
Build the forecast from completed work first
- Name the event. Decide exactly what you are forecasting: first safe test, production readiness, customer adoption, total cost or something else.
- Choose comparable completed cases. Make the group broad enough to contain useful evidence and narrow enough that the main reasons for delay or failure are similar.
- Look at the full range. Include the middle result, the slow outcomes and the failures. Use enough completed cases to see the pattern, because one comparison can be unusual.
- Place the current case before adjusting it. Start inside the historical range rather than treating the new case as exempt.
- Adjust for demonstrated differences. Existing components, a proven supplier or a changed test method may matter. Confidence, talent and strategic importance aren't adjustments by themselves.
- Choose how cautious the estimate needs to be. The median is the middle completed case: half were faster and half were slower. A customer promise, safety-critical launch or fixed budget may need a later date that most comparable projects managed to beat.
- Update with actual evidence. Once work starts, its real pace, test results and supplier performance can justify moving away from the initial range.
Give a range and explain how the cost of missing the commitment shaped the date you chose.
Where the outside view helps
Psychology and statistics. The inside view focuses attention on the steps and intentions of the current plan. Daniel Kahneman and Dan Lovallo described how this can anchor people on optimistic scenarios while they neglect the statistics of comparable outcomes in their paper “Timid Choices and Bold Forecasts”. The completed cases give you a starting range. Move away from it only when the current project produces evidence that it behaves differently.
Strategy, economics and public investment. Large projects are often approved using specific narratives about local competence, urgency and exceptional opportunity. The same people asking for resources may also benefit from a low initial estimate. The completed-project range makes the estimate less dependent on the sponsor's narrative. Publish the projects you compared too, so reviewers can see whether the examples were chosen to make the estimate look better.
Software, product and hardware. A task list covers the visible work, while completed projects also include integration, waiting, rework and coordination. Software teams can compare work of a similar size that relied on a similar number of people and systems. Hardware also includes physical changes and supplier delays that a clean plan often misses. Automation, reusable foundations or a different design may shorten the work, but adjust the range only after a small trial shows that they have.
Start with completed work that already includes integration, waiting and rework, then use specific evidence to explain why this case should differ.
When to use the outside view
Use Reference Class Forecasting when:
- A detailed internal plan feels convincing before integration or user testing begins.
- Time, cost, demand, adoption or failure risk matters to a commitment.
- The project is called unique even though important parts have been attempted before.
- Optimistic estimates help a proposal win resources or approval.
- Several small uncertainties can interact and the tail matters more than the average.
- Another person will schedule work or spend money based on the estimate.
Ask three questions: What exact outcome are we estimating? Which completed cases share the main causes of delay? What evidence would justify moving this case away from their range?
The reference class can be wrong
The hardest part is choosing the class.
A broad group may hide the cause that matters. A group chosen too narrowly may include only favourable examples or too few examples to help. When the work is genuinely new, compare it at several levels, such as all consumer hardware, similar physical-and-electronic prototypes and stair-climbing mechanisms, then keep the range wide enough to show where those comparisons disagree.
Historical data can also preserve obsolete constraints. A new manufacturing method may really shorten the work. Past outcomes may contain exclusion, weak infrastructure or incentives that shouldn't be treated as natural law. Use historical outcomes as a starting range, and change that range when new evidence shows an old constraint no longer applies.
When no close reference class exists, compare several broader classes, show the disagreement between them and keep the forecast range wide.
How this works with Bayesian Updating and a premortem
Bayesian Updating is the broader habit of changing what you believe as evidence arrives. Reference Class Forecasting gives it a starting belief based on similar completed work, and evidence from the current project can then change the forecast.
A Premortem asks people to imagine that the current plan failed and work backwards to possible reasons. It can reveal risks that past records do not include. Reference Class Forecasting uses real completed cases to show how often plans miss, while the premortem helps you see how this one might miss.
Before making an external commitment, put the task estimate beside the outcome range from comparable completed work.