6: Risk – Account for the as yet unknown unknowns
Why it matters
Often significant variance happens because of something you weren’t even aware of as being a possibility.
There are known unknowns and there are unknown unknowns. Unknown unknowns are hard to predict by definition – we are not even aware of their existence and we have no previous personal experience of them. The other reason why unknown unknowns make forecasting difficult is that there is only a small chance that they might happen.
The essence of the practice is finding ways to uncover the unknown unknowns.
How it works
“Whenever I run retro on a forecast that went wrong, someone in the team had anticipated the eventual reasons for failure, but it was either ignored or deemed too unlikely to matter”
– Troy Magennis
The first step towards discovering unknown unknowns is to create a conducive environment to voice the possibility of failure. If it is known to someone, we need to remove the reasons why we wouldn’t listen to that person.
The second step is to look at reasons for variance in the past and challenge why they wouldn’t occur this time around.
Using these sources of data, you now have a bunch of factors.
The third step is to estimate the likelihood of the unknown unknowns happening. When doing so, you should articulate clear reasons why this likelihood would diminish. We do this to overcome optimism bias by applying the “err on the side of caution” or the precautionary principle. We are basically saying, “There is value in resolving things that may happen.”
A naive example of the precautionary principle would be “Climate change”. Even if the likelihood may be low, the mere fact that the impact may result in disastrous irreversible damage, means it would be better to take active measures to prevent this from happening (e.g. burn less fuel and at a slower rate).
The fourth step is to estimate the effort required to resolve it, should each factor occur.
The fifth and final step is to put an economic value on reducing the likelihood early.
We do this by combining likelihood with resolution effort producing “Risks”. We apply these inside a Monte Carlo simulation in order to find the economic value of reducing them. The basic rule of thumb in the Monte Carlo simulation is to introduce the risks into the simulation and correlate them to their likelihood percentages.
Takeaways
- Encourage people to voice the reasons why things could go wrong.
- Elicit the current likelihood for each factor identified.
- Understand the work that could entail, should it happen.
- Find the reasons why the likelihood could be diminishing.
- Put economic value on reducing the likelihood early.
Unknown unknowns are rarely unknown to everyone, they are only unknown to you!