Skip to main content

Research Repository

Advanced Search

AI-Based Hand Gesture Recognition Through Camera on Robot

Csonka, Gergo; Khalid, Muhammad; Rafiq, Husnain; Ali, Yasir


Gergo Csonka

Husnain Rafiq

Yasir Ali


This paper presents an innovative approach to real-time hand gesture recognition for robot control using Artificial Intelligence (AI). The core of this project is a machine learning model trained on a custom data set of hand gestures, which was meticulously hand-annotated to ensure accuracy. To enhance the model's performance and generalization, data augmentation techniques were employed. Furthermore, the model leverages the power of transfer learning, with a ResNet backbone serving as the foundation, to efficiently learn from the data set. In addition to the development of the AI model, a custom robot was designed and built using Arduino and Raspberry Pi. This robot is equipped with a camera to capture images of hand gestures, which are then transmitted to the machine learning model for real-time analysis. The hardware of the robot was meticulously optimized to ensure smooth operation and accurate data capture. The resulting system enables real-time hand gesture recognition on the robot, opening up a plethora of applications, from industrial automation to smart home technology. By synergistically combining AI, computer vision, and robotics, this project not only demonstrates the potential for innovative solutions to real-world problems but also significantly enhances the functionality and usability of robots. It paves the way for improved human-computer interaction through the practical implementation of advanced AI and computer vision techniques.


Csonka, G., Khalid, M., Rafiq, H., & Ali, Y. (2023, December). AI-Based Hand Gesture Recognition Through Camera on Robot. Presented at 2023 International Conference on Frontiers of Information Technology (FIT), Islamabad

Presentation Conference Type Conference Paper (Published)
Conference Name 2023 International Conference on Frontiers of Information Technology (FIT)
Start Date Dec 11, 2023
End Date Dec 12, 2023
Acceptance Date Oct 13, 2023
Online Publication Date Feb 5, 2024
Publication Date Feb 5, 2024
Deposit Date Nov 2, 2023
Publicly Available Date Feb 6, 2026
Publisher Institute of Electrical and Electronics Engineers
Pages 256-261
Book Title 2023 International Conference on Frontiers of Information Technology (FIT)
Keywords Artificial Intelligence; Human-Computer Inter- action; Hand Gesture Recognition; Robotics; Computer Vision; Arduino; Raspberry Pi
Public URL
Publisher URL