New 3D tech allows real-time hand gesture interaction between humans and computers

German researchers reveal a new technology that makes machines much more responsive to the input of human workers. It lets users interact with devices through simple hand gestures, thus saving time and increasing efficiency.

People have invented machines to make jobs simpler and safer. In the situations that they support humans in the workplace, the device must instantly respond to the input of the worker.

The Fraunhofer Institute (Fraunhofer) research team developed sensors and systems that rapidly measured 3D movements and spaces. They made it possible for human workers to interact with the machines they operate in real time.

Members of the university’s Institute for Applied Optics and Precision Engineering (IOF) unveiled their 3D control system at the Hannover Messe trade show in Germany. Their demonstrator unit featured a wall of spheres that immediately responded to any input made by the user.

The interactive wall didn’t require the user to touch it. Instead, its sensors scanned the person and responded to every 3D movement. (Related: Nanotechnology breakthrough helps diabetics track their blood sugar levels.)

Humans may one day control machines by making gestures

Gesture control systems offer a smoother way for people to interact with technology. Many devices, machines, and robots recognize human gestures and respond to those cues.

The Fraunofer researchers refined the interface between humans and machines. They developed high-speed 3D measurement devices and sensors that not only captured and interpreted gestures but did so instantaneously.

They unveiled the abilities of their gesture-based human-machine interaction system at the Hannover Messe 2019 event. Their smart device was an interactive wall made up of 150 spheres and fitted with sensors.

A user stood in front of the wall so that it would scan him. Whenever he moved his head, arms, and hands, the device moved the equivalent spheres to imitate his gestures.

The wall did not require the human user to touch it. It also responded in real time and did not suffer any lags.

“The wall of spheres is not only a great toy, it also represents cutting-edge technology,” explained Fraunhofer researcher Dr. Peter Kühmstedt, the leader of the team. “Real-time 3D capture and interpretation of multiple gestures without tracking sensors can radically simplify workflows – from production scenarios to health and safety.”

New 3D sensor technology eliminates time lag

The Fraunhofer-developed system succeeded in spotting a human’s gestures, physical actions, and other fancy movements. Its sensors gathered data and sent them through a technical actuator system, which turned the electrical signals into movements.

The posture of the user controls the actuators of the wall. When a person made a 3D movement, he triggered the actuators, which moved the spheres matching his moving body parts.

“We are demonstrating very rapid measurement technology – the data is captured by a new generation of 3D sensors – very rapid low-latency processing – the data is interpreted and converted immediately – and very rapid reactions in real time,” explained Kühmstedt.

His demonstration showed that the wall of spheres instantly copied the movement of the person before it.

Once perfected, the gesture control technology may see use in many applications. For example, in environments where humans and robots work together, a properly programmed machine will respond to the actions of the human worker.

If the human hands it an object, the robot will identify the gesture and respond by taking the item. Likewise, if the user is controlling the robot directly to perform a task, the machine may match the careful movements of the person.

The 3D sensor technology may also improve the efficiency and safety of other processes. Assembly assistance, quality control systems, and biometric access points will benefit from the ability to recognize human gestures.

Sources include:

Fraunhofer.de

HannoverMesse.de