Author: Gerd Doeben-Henisch
Email: gerd@doeben-henisch.de

Time: April 2, 2020 — Oct 20, 2023

CONTEXT

This post is part of the REVIEW section of the uffmm blog.

Review of Nancy Leveson
Are you sure your software will not kill anyone?

A Review from the Point of View of the DAAI Paradigm

POSTSCRIPT

Three years after the above reviewed paper Nancy Leveson published a new paper to the subject of Safety Critical Systems together with John P. Thomas with the title Inside Risks. Certification of Safety-Critical Systems. Seeking new approaches toward ensuring the safety of software-intensive systems. in the COMMUNICATIONS OF THE ACM, OCT 2023, VOL . 66, NO. 1 0, 22-26

In a first reading one can get the impression that the task of securing safety critical systems seems to be more and more unsolvable. The complexity of the task seems to be is beyond all paradigms we know today. Is this the end of modern technology? Nancy and John exclude explicitly that a ‘solution’ could be based on ‘more AI’ only; without human persons it will not work. What does this mean?

My personal judgment: We have to go back to ‘start’. We have to consider this challenge from scratch in a new way: What do we really need? What methods are known? What is still missing?

A personal guess: We have to think about the human factor more radically: the human factor is not only one more ‘factor’ besides others in the scenario; the human factor is indeed an ‘object’ but at the same time also the ‘author’ inducing implicitly all the conditions of thinking, the medium of communications, as well as the decisions. To be silent about this causes hiding many important factors which are effective.