3D sensor technology: two high-speed cameras and a gobo projector capture the 3D data of the moving person. Credit: Fraunhofer IOF Humans and machines will increasingly support each other in the workplace. For processes to be efficient, the machine must respond to the human worker without any time delay. Thanks to sophisticated high-speed 3-D measurement and sensor technology, researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF are making this real-time interaction possible. They will be demonstrating how this works in practice at the Hannover Messe trade show from April 1 to 5, 2019 using the example of an interactive wall of spheres that reacts immediately, contact-free and in three dimensions to every movement of a person standing in front of it (Hall 2, Booth C22).
As gesture control represents a seamless interface between humans and machines, more and more machines, robots and devices are able to respond to gestural cues. Researchers at Fraunhofer IOF in Jena are raising human-machine interaction to a new level: the high-speed 3-D measurement and sensor technology developed in the 3-D-LivingLab research project (see "3-D-LivingLab" box) enables them to capture and interprete even complex movements – and does so in real time. At Hannover Messe 2019, the research team is demonstrating its gesture-based human-machine interaction technology using the example of a wall made up of 150 spheres, which copies in 3-D every head, arm and hand movement of a person standing in front of it. The wall of spheres effectively imitates the body movements with contact-free 3-D reactions in real time, free from irritating time lags. The wall of spheres was created as part of the "3-D-LivingLab" project.
Workflows are greatly simplified
The system is made up of several modules: 3-D sensor, 3-D data processing and image fusion as well as the actuator system itself comprising 150 individual actuators. "The wall of spheres is not only a great toy, it also represents cutting-edge technology. Real-time 3-D capture and interpretation of multiple gestures without tracking sensors can radically simplify workflows – from production scenarios to health and safety," says Dr. Peter Kühmstedt, scientist and group leader at Fraunhofer IOF.
The demonstrator system responds to the behavior of people, captures complex movements such as gestures and physical actions, and gives real-time feedback through a technical actuator system that converts electrical signals into movement on the wall of spheres. It is the person's posture that controls the actuators. Specially developed algorithms enable human 3-D movements to trigger control of the actuators, thus causing the spheres to move. "We are demonstrating very rapid measurement technology – the data is captured by a new generation of 3-D sensors –, very rapid low-latency processing – the data is interpreted and converted immediately – and very rapid reactions in real time. According to the calculation results, the wall of spheres immediately mirrors the movement of the person in front of it," says the researcher. In production environments, for example, the technology could be used to monitor a worker who is interacting with a robot and handing it parts. It could also be transferred to other application fields, such as health and safety, where it can make processes safer and more efficient. Other conceivable applications for the 3-D sensor technology and the interaction components are in assembly assistance and quality control systems. Moreover, they also qualify for the monitoring of biometric access points.
The spheres respond instantaneously to the movements of the person standing in front of the wall. Credit: Fraunhofer IOF
Explore further: Interactive control to guide industrial robots