Deep Learning Based Hand Gesture Recognition for Assistive Communication

  • Unique Paper ID: 209159
  • Volume: 13
  • Issue: 5
  • PageNo: 928-931
  • Abstract:
  • People who are deaf or have a speech impairment rely on sign language, yet most members of the public cannot read it, and the gap becomes serious in emergencies where a message must be understood at once. This paper presents ASL2VOICE, a real-time system that watches a person’s hand signs through an ordinary camera and turns them into on-screen text and spoken words. The system uses an SSD MobileNet V2 detector trained with the TensorFlow Object Detection API on ten emergency signs (Ambulance, Call, Danger, Emergency, Fire, Help, Medicine, Police, Stop and Water) plus a No Sign class for frames that hold no gesture. A FastAPI server receives frames from a Next.js web interface, runs detection and returns the sign together with a confidence score. Predictions below an adjustable threshold are rejected; accepted ones are displayed and read aloud with text-to-speech. The interface also lets the user choose a camera, switch voice output on or off and review recently detected signs. The paper describes the design, dataset, training procedure and evaluation plan of the system.

Copyright & License

Copyright © 2026 Authors retain the copyright of this article. This article is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

BibTeX

@article{209159,
        author = {Arlangki Sungoh and Abdul Samad K A and Ahmed Nazim Nisham and Abdul Rahman Arshad and Asst. Prof. Shruthi},
        title = {Deep Learning Based Hand Gesture Recognition for Assistive Communication},
        journal = {International Journal of Innovative Research in Technology},
        year = {2026},
        volume = {13},
        number = {5},
        pages = {928-931},
        issn = {2349-6002},
        url = {https://ijirt.org/article?manuscript=209159},
        abstract = {People who are deaf or have a speech impairment rely on sign language, yet most members of the public cannot read it, and the gap becomes serious in emergencies where a message must be understood at once. This paper presents ASL2VOICE, a real-time system that watches a person’s hand signs through an ordinary camera and turns them into on-screen text and spoken words. The system uses an SSD MobileNet V2 detector trained with the TensorFlow Object Detection API on ten emergency signs (Ambulance, Call, Danger, Emergency, Fire, Help, Medicine, Police, Stop and Water) plus a No Sign class for frames that hold no gesture. A FastAPI server receives frames from a Next.js web interface, runs detection and returns the sign together with a confidence score. Predictions below an adjustable threshold are rejected; accepted ones are displayed and read aloud with text-to-speech. The interface also lets the user choose a camera, switch voice output on or off and review recently detected signs. The paper describes the design, dataset, training procedure and evaluation plan of the system.},
        keywords = {Assistive Communication, Deep Learning, Emergency Signs, Hand Gesture Recognition, Object Detection, SSD MobileNet V2, Text-to-Speech.},
        month = {October},
        }

Cite This Article

Sungoh, A., & A, A. S. K., & Nisham, A. N., & Arshad, A. R., & Shruthi, A. P. (2026). Deep Learning Based Hand Gesture Recognition for Assistive Communication. International Journal of Innovative Research in Technology (IJIRT), 13(5), 928–931.

Related Articles