AI-Based Hand Gesture Recognition Through Camera on Robot

Csonka, Gergo; Khalid, Muhammad; Rafiq, Husnain; Ali, Yasir

Abstract

This paper presents an innovative approach to real-time hand gesture recognition for robot control using Artificial Intelligence (AI). The core of this project is a machine learning model trained on a custom data set of hand gestures, which was meticulously hand-annotated to ensure accuracy. To enhance the model's performance and generalization, data augmentation techniques were employed. Furthermore, the model leverages the power of transfer learning, with a ResNet backbone serving as the foundation, to efficiently learn from the data set. In addition to the development of the AI model, a custom robot was designed and built using Arduino and Raspberry Pi. This robot is equipped with a camera to capture images of hand gestures, which are then transmitted to the machine learning model for real-time analysis. The hardware of the robot was meticulously optimized to ensure smooth operation and accurate data capture. The resulting system enables real-time hand gesture recognition on the robot, opening up a plethora of applications, from industrial automation to smart home technology. By synergistically combining AI, computer vision, and robotics, this project not only demonstrates the potential for innovative solutions to real-world problems but also significantly enhances the functionality and usability of robots. It paves the way for improved human-computer interaction through the practical implementation of advanced AI and computer vision techniques.

Citation

Csonka, G., Khalid, M., Rafiq, H., & Ali, Y. (2023, December). AI-Based Hand Gesture Recognition Through Camera on Robot. Presented at 2023 International Conference on Frontiers of Information Technology (FIT), Islamabad

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2023 International Conference on Frontiers of Information Technology (FIT)
• Start Date: Dec 11, 2023
• End Date: Dec 12, 2023
• Acceptance Date: Oct 13, 2023
• Online Publication Date: Feb 5, 2024
• Publication Date: Feb 5, 2024
• Deposit Date: Nov 2, 2023
• Publicly Available Date: Feb 6, 2026
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 256-261
• Book Title: 2023 International Conference on Frontiers of Information Technology (FIT)
• DOI: https://doi.org/10.1109/FIT60620.2023.00054
• Keywords: Artificial Intelligence; Human-Computer Inter- action; Hand Gesture Recognition; Robotics; Computer Vision; Arduino; Raspberry Pi
• Public URL: https://hull-repository.worktribe.com/output/4430425
• Publisher URL: https://ieeexplore.ieee.org/document/10410366