A Cooperative Hand Gesture Recognition System for Human Machine Interface to Control Robotic Arms Kettering University

Format:

Book

Conference/Event

Author/Creator:

DeHaven, Aaron Lee, author.

Park, Jungme, author.

Conference Name:

WCX SAE World Congress Experience (2026-04-14 : Detroit, Michigan, United States)

Language:

English

Subjects (All):

Artificial intelligence (AI).

Wearable technology.

Machine learning.

Human machine interface (HMI).

Consumer electronics.

Hand.

Local Subjects:

Artificial intelligence (AI).

Wearable technology.

Machine learning.

Human machine interface (HMI).

Consumer electronics.

Hand.

Physical Description:

1 online resource

Place of Publication:

Warrendale, PA SAE International 2026

Summary:

In the context of Industry 5.0, effective humanmachine collaboration requires seamless and natural interaction. Hand-Gesture Recognition (HGR) has emerged as a promising technology for developing humanmachine interfaces (HMI) that enable users to control robotic systems without physical controllers or wearable devices. This research presents a real-time HGR system designed to control a 6-Degree-of-Freedom (DoF) robotic arm using YOLOv10, a state-of-the-art deep learning model for hand gesture detection and classification. While YOLOv10 delivers high recognition accuracy, its computational demands surpass the capabilities of edge devices typically mounted on robotic platforms, creating a hardware bottleneck. To address this challenge, a cooperative clientserver architecture is proposed, distributing computational workload between the edge device and a more powerful remote server. An RGB camera attached to the robotic arm captures hand gesture images and transmits them to the server via the User Datagram Protocol (UDP). The server performs real-time inference using YOLOv10 and returns the detection results to the edge device, which translates the recognized gestures into corresponding robotic arm movements. Experimental evaluation demonstrates an interfacing speed of approximately 15.7 frames per second and an 11.54 times improvement in performance compared with standalone edge-based processing. The proposed cooperative HGR system successfully integrates advanced computer vision techniques with robotic control to deliver a responsive, touch-free interface, enabling smooth, natural HMI. By overcoming edge-computing limitations, this research contributes to the advancement of Industry 5.0, supporting applications in healthcare, assistive robotics, industrial automation, and collaborative robotics, and promoting effective and safe humanmachine collaboration

Notes:

Vendor supplied data

Access Restriction:

Restricted for use by site license