THE BIG PICTURE: HCI – HMI – AAI in History – Engineering – Society – Philosophy

ISSN 2567-6458, 20.April 2019
Author: Gerd Doeben-Henisch

A first draft version …


The context for this text is the whole block dedicated to the AAI (Actor-Actor Interaction)  paradigm. The aim of this text is to give the big picture of all dimensions and components of this subject as it shows up during April 2019.

The first dimension introduced is the historical dimension, because this allows a first orientation in the course of events which lead  to the actual situation. It starts with the early days of real computers in the thirties and forties of the 20 century.

The second dimension is the engineering dimension which describes the special view within which we are looking onto the overall topic of interactions between human persons and computers (or machines or technology or society). We are interested how to transform a given problem into a valuable solution in a methodological sound way called engineering.

The third dimension is the whole of society because engineering happens always as some process within a society.  Society provides the resources which can be used and spends the preferences (values) what is understood as ‘valuable’, as ‘good’.

The fourth dimension is Philosophy as that kind of thinking which takes everything into account which can be thought and within thinking Philosophy clarifies conditions of thinking, possible tools of thinking and has to clarify when some symbolic expression becomes true.


In history we are looking back in the course of events. And this looking back is in a first step guided by the  concepts of HCI (Human-Computer Interface) and  HMI (Human-Machine Interaction).

It is an interesting phenomenon how the original focus of the interface between human persons and the early computers shifted to  the more general picture of interaction because the computer as machine developed rapidly on account of the rapid development of the enabling hardware (HW)  the enabling software (SW).

Within the general framework of hardware and software the so-called artificial intelligence (AI) developed first as a sub-topic on its own. Since the last 10 – 20 years it became in a way productive that it now  seems to become a normal part of every kind of software. Software and smart software seem to be   interchangeable. Thus the  new wording of augmented or collective intelligence is emerging intending to bridge the possible gap between humans with their human intelligence and machine intelligence. There is some motivation from the side of society not to allow the impression that the smart (intelligent) machines will replace some day the humans. Instead one is propagating the vision of a new collective shape of intelligence where human and machine intelligence allows a symbiosis where each side gives hist best and receives a maximum in a win-win situation.

What is revealing about the actual situation is the fact that the mainstream is always talking about intelligence but not seriously about learning! Intelligence is by its roots a static concept representing some capabilities at a certain point of time, while learning is the more general dynamic concept that a system can change its behavior depending from actual external stimuli as well as internal states. And such a change includes real changes of some of its internal states. Intelligence does not communicate this dynamics! The most demanding aspect of learning is the need for preferences. Without preferences learning is impossible. Today machine learning is a very weak example of learning because the question of preferences is not a real topic there. One assumes that some reward is available, but one does not really investigate this topic. The rare research trying to do this job is stating that there is not the faintest idea around how a general continuous learning could happen. Human society is of no help for this problem while human societies have a clash of many, often opposite, values, and they have no commonly accepted view how to improve this situation.


Engineering is the art and the science to transform a given problem into a valuable and working solution. What is valuable decides the surrounding enabling society and this judgment can change during the course of time.  Whether some solution is judged to be working can change during the course of time too but the criteria used for this judgment are more stable because of their adherence to concrete capabilities of technical solutions.

While engineering was and is  always  a kind of an art and needs such aspects like creativity, innovation, intuition etc. it is also and as far as possible a procedure driven by defined methods how to do things, and these methods are as far as possible backed up by scientific theories. The real engineer therefore synthesizes art, technology and science in a unique way which can not completely be learned in the schools.

In the past as well as in the present engineering has to happen in teams of many, often many thousands or even more, people which coordinate their brains by communication which enables in the individual brains some kind of understanding, of emerging world pictures,  which in turn guide the perception, the decisions, and the concrete behavior of everybody. And these cognitive processes are embedded — in every individual team member — in mixtures of desires, emotions, as well as motivations, which can support the cognitive processes or obstruct them. Therefore an optimal result can only be reached if the communication serves all necessary cognitive processes and the interactions between the team members enable the necessary constructive desires, emotions, and motivations.

If an engineering process is done by a small group of dedicated experts  — usually triggered by the given problem of an individual stakeholder — this can work well for many situations. It has the flavor of a so-called top-down approach. If the engineering deals with states of affairs where different kinds of people, citizens of some town etc. are affected by the results of such a process, the restriction to  a small group of experts  can become highly counterproductive. In those cases of a widespread interest it seems promising to include representatives of all the involved persons into the executing team to recognize their experiences and their kinds of preferences. This has to be done in a way which is understandable and appreciative, showing esteem for the others. This manner of extending the team of usual experts by situative experts can be termed bottom-up approach. In this usage of the term bottom-up this is not the opposite to top-down but  is reflecting the extend in which members of a society are included insofar they are affected by the results of a process.


Societies in the past and the present occur in a great variety of value systems, organizational structures, systems of power etc.  Engineering processes within a society  are depending completely on the available resources of a society and of its value systems.

The population dynamics, the needs and wishes of the people, the real territories, the climate, housing, traffic, and many different things are constantly producing demands to be solved if life shall be able and continue during the course of time.

The self-understanding and the self-management of societies is crucial for their ability to used engineering to improve life. This deserves communication and education to a sufficient extend, appropriate public rules of management, otherwise the necessary understanding and the freedom to act is lacking to use engineering  in the right way.


Without communication no common constructive process can happen. Communication happens according to many  implicit rules compressed in the formula who when can speak how about what with whom etc. Communication enables cognitive processes of for instance  understanding, explanations, lines of arguments.  Especially important for survival is the ability to make true descriptions and the ability to decide whether a statement is true or not. Without this basic ability communication will break down, coordination will break down, life will break down.

The basic discipline to clarify the rules and conditions of true communication, of cognition in general, is called Philosophy. All the more modern empirical disciplines are specializations of the general scope of Philosophy and it is Philosophy which integrates all the special disciplines in one, coherent framework (this is the ideal; actually we are far from this ideal).

Thus to describe the process of engineering driven by different kinds of actors which are coordinating themselves by communication is primarily the task of philosophy with all their sub-disciplines.

Thus some of the topics of Philosophy are language, text, theory, verification of a  theory, functions within theories as algorithms, computation in general, inferences of true statements from given theories, and the like.

In this text I apply Philosophy as far as necessary. Especially I am introducing a new process model extending the classical systems engineering approach by including the driving actors explicitly in the formal representation of the process. Learning machines are included as standard tools to improve human thinking and communication. You can name this Augmented Social Learning Systems (ASLS). Compared to the wording Augmented Intelligence (AI) (as used for instance by the IBM marketing) the ASLS concept stresses that the primary point of reference are the biological systems which created and create machine intelligence as a new tool to enhance biological intelligence as part of biological learning systems. Compared to the wording Collective Intelligence (CI) (as propagated by the MIT, especially by Thomas W.Malone and colleagues) the spirit of the CI concept seems to be   similar, but perhaps only a weak similarity.

AASE – Actor-Actor Systems Engineering. Theory & Applications. Micro-Edition (Vers.9)

eJournal:, ISSN 2567-6458
13.June  2018
Authors: Gerd Doeben-Henisch, Zeynep Tuncer,  Louwrence Erasmus



1 History: From HCI to AAI …
2 Different Views …
3 Philosophy of the AAI-Expert …
4 Problem (Document) …
5 Check for Analysis …
6 AAI-Analysis …
6.1 Actor Story (AS) . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.1 Textual Actor Story (TAS) . . . . . . . . . . . . . . .
6.1.2 Pictorial Actor Story (PAT) . . . . . . . . . . . . . .
6.1.3 Mathematical Actor Story (MAS) . . . . . . . . . . .
6.1.4 Simulated Actor Story (SAS) . . . . . . . . . . . . .
6.1.5 Task Induced Actor Requirements (TAR) . . . . . . .
6.1.6 Actor Induced Actor Requirements (UAR) . . . . . .
6.1.7 Interface-Requirements and Interface-Design . . . .
6.2 Actor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2.1 Actor and Actor Story . . . . . . . . . . . . . . . . .
6.2.2 Actor Model . . . . . . . . . . . . . . . . . . . . . .
6.2.3 Actor as Input-Output System . . . . . . . . . . . .
6.2.4 Learning Input-Output Systems . . . . . . . . . . . .
6.2.5 General AM . . . . . . . . . . . . . . . . . . . . . .
6.2.6 Sound Functions . . . . . . . . . . . . . . . . . . .
6.2.7 Special AM . . . . . . . . . . . . . . . . . . . . . .
6.2.8 Hypothetical Model of a User – The GOMS Paradigm
6.2.9 Example: An Electronically Locked Door . . . . . . .
6.2.10 A GOMS Model Example . . . . . . . . . . . . . . .
6.2.11 Further Extensions . . . . . . . . . . . . . . . . . .
6.2.12 Design Principles; Interface Design . . . . . . . . .
6.3 Simulation of Actor Models (AMs) within an Actor Story (AS) .
6.4 Assistive Actor-Demonstrator . . . . . . . . . . . . . . . . . .
6.5 Approaching an Optimum Result . . . . .
7 What Comes Next: The Real System
7.1 Logical Design, Implementation, Validation . . . .
7.2 Conceptual Gap In Systems Engineering? . . .
8 The AASE-Paradigm …


This text is based on the the paper “AAI – Actor-Actor Interaction. A Philosophy of Science View” from 3.Oct.2017 and version 11 of the paper “AAI – Actor-Actor Interaction. An Example Template” and it   transforms these views in the new paradigm ‘Actor- Actor Systems Engineering’ understood as a theory as well as a paradigm for and infinite set of applications. In analogy to the slogan ’Object-Oriented Software Engineering (OO SWE)’ one can understand the new acronym AASE as a systems engineering approach where the actor-actor interactions are the base concepts for the whole engineering process. Furthermore it is a clear intention to view the topic AASE explicitly from the point of view of a theory (as understood in Philosophy of Science) as well as from the point of view of possible applications (as understood in systems engineering). Thus the classical term of Human-Machine Interaction (HMI) or even the older Human-Computer Interaction (HCI) is now embedded within the new AASE approach. The same holds for the fuzzy discipline of Artificial Intelligence (AI) or the subset of AI called Machine Learning (ML). Although the AASE-approach is completely in its beginning one can already see how powerful this new conceptual framework  is.




For the Integrated Engineering of the Future (SW4IEF)
Campaining the Actor-Actor Systems Engineering (AASE) paradigm

eJournal:, ISSN 2567-6458

Last Update June-22, 2018, 15:32 CET.  See below: Case Studies —  Templates – AASE Micro Edition – and Scheduling 2018 —


This is a complete new restart of the old uffmm-site. It is intended as a working place for those people who are interested in an integrated engineering of the future.


A widely known and useful concept for a general approach to the engineering of problems is systems engineering (SE).

Open for nearly every kind of a possible problem does a systems engineering process (SEP) organize the process how to analyze the problem, and turn this analysis into a possible design for a solution. This proposed solution will be examined by important criteria and, if it reaches an optimal version, it will be implemented as a real working system. After final evaluations this solution will start its carrier in the real world.


In a meta-scientific point of view the systems engineering process can become itself the object of an analysis. This is usually done by a discipline called philosophy of science (PoS). Philosophy of science is asking, e.g., what the ‘ingredients’ of an systems-engineering process are, or how these ingredients do interact? How can such a process ‘fail’? ‘How can such a process be optimized’? Therefore a philosophy of science perspective can help to make a systems engineering process more transparent and thereby supports an optimization of these processes.


A core idea of the philosophy of science perspective followed in this text is the assumption, that a systems engineering process is primarily based on different kinds of actors (AC) whose interactions enable and direct the whole process. These assumptions are also valid in that case, where the actors are not any more only biological systems like human persons and non-biological systems called machines, but also in that case where the traditional machines (M) are increasingly replaced by ‘intelligent machines (IM)‘. Therefore the well know paradigm of human-machine interaction (HMI) — or earlier ‘human-computer interaction (HCI)’  will be replaced in this text by the new paradigm of Actor-Actor Interaction (AAI). In this new version the main perspective is not the difference of man on one side and machines on the other but the kind of interactions between actors of all kind which are necessary and possible.


The  concept of intelligent machines (IM) is understood here as a special case of the general Actor (A) concept which includes as other sub-cases biological systems, predominantly humans as instantiations of the species Homo Sapiens. While until today the question of biological intelligence and machine intelligence is usually treated separately and differently it is intended in this text to use one general concept of intelligence for all actors. This allows then more direct comparisons and evaluations. Whether biological actors are in some sense better than the non-biological actors or vice versa can seriously only be discussed when the used concept of intelligence is the same.


And, as it will be explained in the following sections, the used paradigm of actor-actor interactions uses the two main concepts of actor story (AS) as well as actor model (AM). Actor models are embedded in the actor stories. Whether an actor model describes biological or non-biological actors does not matter. Independent of the inner structures of an actor model (which can be completely different) the actor story is always  completely described in terms of observable behavior which are the same for all kinds of actors (Comment: The major scientific disciplines for the analysis of behavior are biology, psychology, and sociology).


In analogy to the so-called ‘Object-Oriented (OO) approach in Software-Engineering (SWE)’ we campaign here the ‘Actor-Actor (AA) Systems Engineering (SE)’ approach. This takes the systems Engineering approach as a base concepts and re-works the whole framework from the point of view of the actor-actor paradigm.  AASE is seen here as a theory as well as an   domain of applications.

Ontologies of the AASE paradigm
Figure: Ontologies of the AASE paradigm

To understand the different perspectives of the used theory it can help to the figure ‘AASE-Paradigm Ontologies’. Within the systems engineering process (SEP) we have AAI-experts as acting actors. To describe these we need a ‘meta-level’ realized by a ‘philosophy of the actor’. The AAI-experts themselves are elaborating within an AAI-analysis an actor story (AS) as framework for different kinds of intended actors. To describe the inner structures of these intended actors one needs different kinds of ‘actor models’. The domain of actor-model structures overlaps with the domain of ‘machine learning (ML)’ and with ‘artificial intelligence (AI)’.


What will be described and developed separated from these theoretical considerations is an appropriate software environment which allows the construction of solutions within the AASE approach including e.g. the construction of intelligent machines too. This software environment is called in this text emerging-mind lab (EML) and it will be another public blog as well.



How we proceed

Because the overall framework of the intended integrated theory is too large to write it down in one condensed text with  all the necessary illustrating examples we decided in Dec 2017 to follow a bottom-up approach by writing primarily case studies from different fields. While doing this we can introduce stepwise the general theory by developing a Micro Edition of the Theory in parallel to the case studies. Because the Theory Micro Edition has gained a sufficient minimal completeness already in April 2018 we do not need anymore a separate   template for case studies. We will use the Theory Micro Edition  as  ‘template’ instead.

To keep the case studies readable as far as possible all needed mathematical concepts and formulas will be explained in a separate appendix section which is central for all case studies. This allows an evolutionary increase in the formal apparatus used for the integrated theory.


(Still not final)

Here you can find the actual version of the   theory which will continuously be updated and extended by related topics.

At the end of the text you find a list of ToDos where everybody is invited to collaborate. The main editor is Gerd Doeben-Henisch deciding whether the proposal fits into the final text or not.

Last Update 22.June 2018

Philosophy of the Actor

This sections describes basic assumptions about the cognitive structure of the human AAI expert.

From HCI to AAI. Some Bits of History

This sections describes main developments in the history from HCI to AAI.


The Milestone for a first outline in a book format has been reached June-22, 2018. The   milestone for a first final version   is  scheduled   for October-4, 2018.