Which scenarios?

Over the years, I've written many risk scenarios¹. They are narratives about possible, but undesirable situations. Scenarios help understand the complexities and are useful as a basis for further modelling and monitoring. However, a question often comes up: "Which scenarios should we develop?" Imagine that you developed various scenarios to understand the risks you're facing. How do you know you've selected the right scenarios, and that you've chosen enough of them?

There is an approach that at least ensures your set of scenarios is coherent and comprehensive. By following this approach, you will at least be able to say that your scenarios cover most possibilities, with only a small likelihood that a risk event has not been anticipated.

Firstly, we will group scenarios into categories. Scenarios are often written as specific events, where the details are just an example of what might happen. But such a particular scenario is just an example of a category of scenarios – variants of what might happen within a particular theme. For example, you might have written a scenario about a hacking event that impacts a bank, but that scenario is just an example to understand aspects of a broader category of "cyber attacks".

A useful feature of categories is to make them MECE (mutually exclusive and collectively exhaustive). In our context, a risk event will only belong to one category, and all risk events will have a category to belong to. The MECE idea goes back a long way and was already clearly formulated in the 13th century by John Duns Scotus². The feature that all risks are covered by a category is especially useful – it means that our categories are complete: we haven't missed any risks.

But how do we ensure our categories are MECE? A simple procedure is to use combinations of binary properties. A binary property is something that is either true or false of an entity. For example, "is white" is a binary property of a billiard ball (it is true of the white ball, but false for the others). For risks, we might have a bunch of such properties, like "resulting in human injury", "caused by hacking", or "resulting in a lawsuit". Now, to make a set of MECE categories, we list all the combinations of the properties. So, for properties A, B, and C we would have the categories "only A", "only B", "only C", "A and B", "A and C", "B and C", "A, B and C" and "other" (the last one is "other" in the sense of "something other than A, B, or C").

Notice that these categories are mutually exclusive by design. For example, by definition, an event cannot belong to "only A", as well as "A and B". The categories are also collectively exhaustive because we have the "other" category.

To try this out, let's categorise risk scenarios by their possible impact. I'll use the following properties for national-scale risk impacts. I'll assume here "best effort". That is, not deliberately selecting irrelevant or otherwise inappropriate properties. So, here's my selection:

1. "has a financial impact"

2. "results in human injury"

3. "causes ecological loss" (e.g. species extinction)

4. "causes institutional loss" (e.g. disruption in the ability to vote)

5. "causes loss of quality of life" (e.g. reduced freedom of movement)

For the mathematically inclined, this is the formula to calculate the number of categories resulting from N properties:

$$\sum_{0\le k\le n} nCk \quad \small \text{where }nCK \text{ is combinations of n items k at the time}$$

So, for five properties, there would be 32 categories (including "other"). This is useful because it lets us play with some basic probability. If we don't have any idea what the relative probabilities are, then we can assume that all categories are equally likely. In our example, it would mean that if we don't assume any prior knowledge about the risks, then "causes ecological loss" is equally likely as "has a financial impact". So, each impact category would have a 1 in 32, or roughly 3.1% chance of occurring.

In practice, we do have prior knowledge about risk impacts. For example, for any national impact type risk event, there will likely be financial impact. So, we can sort all the categories that include "financial impact" to the top of the list. And we can increase the probability to something a bit more likely, say, 80%. These revised estimates can be subjective (based on collective expertise and experience) or based on incident data. Note that the estimates are not risk estimates, but rather estimates that the impact of a risk event has certain properties.

The basic probabilities applied to 32 independent categories also mean that the "other" category tends to become very unlikely fairly quickly. For example, if there is a dominant set of categories (like our 80%), then the probability of the "other" category can easily reduce to the order of magnitude of a thousandth of a percent. In many cases, that is enough to defend the claim that the categories are sufficiently comprehensive.

If the number of categories still seems too many to write individual scenarios for, consider that scenarios are most useful where we don't understand the risks very well. In other words, the less likely a risk, the more chance that a scenario will increase our understanding. So, a reasonable strategy is to focus on scenarios for the less likely risk categories. To make that worthwhile, we would also focus on high-impact categories.

Notes

1. This article is a revision of a post from 2015.

2. Scotus, John Duns (1300-1305, para 1.4) De primo principio, http://www.ewtn.com/library/theology/godasfir.htm