Better person-machine communication designed to help prevent accidents

Better person-machine communication designed to help prevent accidents
HoliDes. Credit: Elhuyar Fundazioa

The basic components of communication are transmitter, receiver, code, message, channel and context. This setup is the same for communication between humans and machines. In fact, this concept of human-machine communication is becoming increasingly widespread in the increasingly connected digital world, with elements such as the IoT (internet of things), and interactivity with smartphones and wearables.

Human-machine communication is constant across the automotive sector, tourism, industry and others. A number of European aeronautics companies have joined forces to advance this work. Let us assume that incorrect encryption or decryption of messages between humans and machines could have disastrous consequences in some contexts, and the fact is that some air accidents could have been avoided if conflicts of this type between the crew and control had been corrected.

Holistic Human factors and System Design (HoliDes) of Adaptive Cooperative Human-Machine Systems (AdCoS), has emerged on the assumption that pilots and automated cabin systems need to communicate with each other and act together in a cooperative way and in a highly adaptable manner. Achieving correct communication, co-operation and mutual adaptability would heighten safety.

Work in the field is being done using a range of sensors and algorithms that measure the contextual state of the human operators so that once the data of the parameters has been processed and analysed, they can form part of a system that assists in decision making and thus has the "co-operative adaptation" capacity between the human and the machine and vice versa.

To achieve all this, work is being done in various areas of technological expertise in which the following four are the most salient ones: specifying of patterns of human behaviour and design of the inference tool, unification and storage of information on experiments, the modelling ontology of human factors, and the drawing up of recommendations for correct human-machine communication.

The first of these blocks, the tool known as the Pilot Pattern Classifier, consists mainly of sensorizing and measuring the relevant parameters in the conduct of a pilot on duty that can establish and personalise predictive models in real time for human behaviour (fatigue, distraction, etc.).  This calls for a set of devices or sensors fitted to the pilot that monitor brain waves, eye tracking, heart rate, sweating and other biometric measurements; this work is being undertaken by the Universita Degli Studi Suor Orsola Benincasa.

This information must be stored in an accessible and structured way if it is to be truly operative. An information repository is being developed by Tecnalia and is called the Experiment Data Archive (EDA).

At the same time, the vocabulary of this technology needs to be standardised so that all the systems that use this information can understand each other. In other words, an ontology that models human factors has to be generated so that the systems can be interoperable. Tecnalia is participating in this area as the leading company.

Finally, researchers are drawing up recommendations for communication. Guides or user manuals provide recommendations so that the messages arrive optimally between the transmitter and receiver—in other words, between the human and the machine.  So the researchers consider how the information is presented—any variable that could affect the way a message is understood; the visual aspect, the interfaces (information architecture, the colours used, final look of the user interface, etc.), the acoustic aspect (type of sounds used, volume, etc.) or the tactile aspect (vibrations).

In short, HoliDes technology consists of tools to build a system of emotional intelligence based on human-machine co-operative communication; this is done by using systems of sensors and algorithms that measure the state of the human operator (human factors) in a range of contexts and domains, and that way the first steps are being taken toward Adaptive Cooperative Human-Machine Systems.

Explore further: Removing the communication barrier between humans and computers