Predicting human error in action sequences is of immense value in numerous private and professional activities. In the case of dangerous or otherwise critical activities, for example if wrong actions would be irreversible, it is of crucial importance to anticipate and prevent mistakes. However, even in the case of less critical activities, corresponding predictions can serve to prevent errors and thus to carry out actions more smoothly and efficiently than if erroneously executed actions had to be corrected afterwards.
The assessment of individual, task-related prior knowledge is also highly important in the area of technical assistance systems. Especially when using augmented reality smart~glasses, which enrich the natural perception of environments by projecting virtual elements directly into the user's field of view, it is essential to spare users' attentional and other limited cognitive resources. Such a system should therefore have sufficient knowledge of the actual need for assistance and offer meaningful cognitive assistance through appropriately targeted support, instead of distracting users with unnecessary overlays. Otherwise, cognitive overload would reduce the system's usability and lead to a lack of user acceptance.
The structural-dimensional analysis of mental representations (SDA-M) is a method originating from cognitive psychology, which has been established in movement and sports science, as well as in cognitive robotics. Using a special, semi-automatic survey, the so-called “split procedure”, SDA-M retrieves data about the individual, task-related memory structures of a particular person. So far, this data has commonly been analyzed using hierarchical clustering and visualized in the form of dendrograms. These dendrograms and the associated statistical quantities can be evaluated by appropriately trained experts in order to assess problems related to the execution of actions and to develop suggestions for improvement, for example in the area of training, manual actions, and clinical rehabilitation. However, this procedure requires special expertise and time.
The present work therefore examined how the analysis of task-related mental representation structures can be further automated. Various algorithmic approaches based on different cognitive architecture models were developed for this purpose.
A total of four empirical studies have evaluated these algorithmic approaches for predicting the overall task-related competence and the individual probabilities of errors for each action in different action sequences in comparison with the traditional expert-based approach, as well as in different practical areas of application. This includes a controlled laboratory study with a standard task for assembly processes, a movement sequence from traditional martial arts training, as well as a manual assembly task in an applied industrial context.
The empirical evidence shows that the new computer-aided analysis methods are at least equal to the previous expert-based approach and that they were able to correctly predict the majority of the actual human errors in the various areas of application. The accuracy of the algorithmic predictions was also significantly above chance level in all studies.
Complementary to this theoretical and empirical work in the field of cognitive science, the present work describes a new agile methodology and a corresponding iterative process model, which systematically analyzes and considers the intended worth and outcomes of system usage, ethical issues, and relevant stakeholder characteristics during the design and development of technical systems. This methodology was specially developed to optimize the chances that advanced user-adaptive cognitive assistance systems turn out valuable and successful.
