Data Informed Decision Making[1]
It feels somewhat ghoulish to write about the recent Amtrak train derailment near DuPont, Washington, but at this stage of the investigation, the information is quite disturbing.
There's a news clip of the mayor of the adjacent city of Lakewood, WA questioning some of the safety decisions and the risk to the community. While as a lay observer the mayor did not attempt to predict the exact nature of the possible accident his concerns proved to be more accurate than the “professionals.”
While hindsight is always 20-20, as decision makers it’s instructive to look at a decision-making process to find the failure point. By failure point I mean the place in the process that some other factor than the data changed the decision-making process.
Where did the data give a clear indication that the community concerns about an accident were either groundless or has such a small chance of happening that the potential loss of life was an acceptable risk?
It appears, from the available information, that at some point in the data-informed-decision-making process the data was modified by cost considerations. As we have seen in so many high-profile accidents as far back as the Challenger space shuttle disaster “other considerations” than the data drove the decision.
Every organization has cost constraints and has to make decisions limited by what they can afford. We could build cars that would allow the passengers to survive any crash BUT they would be wickedly inefficient and no one could afford to buy them. Tradeoffs will always be necessary.
The big problem with this particular decision-making process is that the outcome was predetermined. The premise was “We will start this route and within this budget.” Then came the big failure – the people designing the system were told: “If the numbers don’t support the outcome, recalculate until they do.” The only way to adjust the numbers on an engineering project is to take out some parts. Removing a nob on the dashboard of a car saves some cost of parts and some cost of assembly reducing the cost to manufacture. Keep taking out parts or features until you reach the cost goal.
So far that is a normal design process. The issue, in this case, was that the only place to cut costs was safety and with the hard decision that “We are going to do this” there was no place for the introduction of the possibility that the project was not economically viable. There was no place for the concept that if we can’t afford the safety protections it shouldn’t be done at all.
There is a well-understood concept in gambling that it's not the odds that matter, it's the consequences. Many of us are willing to risk two bucks on a vanishingly low chance of winning the $250 million dollar Power Ball but would be completely unwilling to risk $10,000 on the same odds. We feel comfortable with the extremely high risk of losing simply because two bucks is a small consequence while a ten thousand is too big to risk.
The reality is that life has risks and there will always be the possibility of a catastrophic failure. The problem, in this case, was cost considerations were allowed to color the decision-making process to artificially minimize the risks and to artificially inflate the chance of success.
All because the decision makers began with the idea that “we have to create this service at all cost!” The cost, in this case, being at least 3 people dead and 72 passengers and crew injured.
Perhaps if the decision makers had truly understood that the possibility of an accident was far less important than the results of an accident they would have stopped the project until the (obviously in hindsight) necessary safety precautions could be included.
[1] “Data-Informed-Decision-Making" (DIDM) gives reference to the collection and analysis of data to guide decisions that improve success. DIDM is used in education communities (where data is used with the goal of helping students and improving curricula) but is also applicable to (and thus also used in) other fields in which data is used to inform decisions. While data based decision-making is a more common term, data-informed decision-making is a preferable term since decisions should not be based solely on quantitative data. (Wikipedia)