A Hybrid CNN-Vision Transformer Framework with Behavioural Feature Fusion for Driver Drowsiness Detection

  • Unique Paper ID: 211313
  • Volume: 13
  • Issue: 6
  • PageNo: 4207-4213
  • Abstract:
  • Driver drowsiness is a major road safety concern that can contribute to traffic accidents. This paper presents HCAF-ViT-M,a hybrid deep learning framework designed to detect driver drowsiness using facial images and behavioural features. The proposed framework integrates a ResNet18-based Convolutional Neural Network (CNN), a Vision Transformer (ViT), and a behavioural feature extraction branch. The CNN captures local visual patterns, while the Vision Transformer learns global facial representations. Behavioural features provide additional information about eye and mouth movements and head position. These complementary representations are fused to classify the driver's state into two categories: Active and Fatigue. The framework is developed using the UTA-RLDD dataset and integrated into a real-time monitoring application. The system aims to support continuous driver monitoring and provide fatigue alerts.

Copyright & License

Copyright © 2026 Authors retain the copyright of this article. This article is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

BibTeX

@article{211313,
        author = {Pallavi T and Lavanya G and Monika S and Gumma Mythri and Dr krishna Kumar P R and Radhika R},
        title = {A Hybrid CNN-Vision Transformer Framework with Behavioural Feature Fusion for Driver Drowsiness Detection},
        journal = {International Journal of Innovative Research in Technology},
        year = {2026},
        volume = {13},
        number = {6},
        pages = {4207-4213},
        issn = {2349-6002},
        url = {https://ijirt.org/article?manuscript=211313},
        abstract = {Driver drowsiness is a major road safety concern that can contribute to traffic accidents. This paper presents HCAF-ViT-M,a hybrid deep learning framework designed to detect driver drowsiness using facial images and behavioural features. The proposed framework integrates a ResNet18-based Convolutional Neural Network (CNN), a Vision Transformer (ViT), and a behavioural feature extraction branch. The CNN captures local visual patterns, while the Vision Transformer learns global facial representations. Behavioural features provide additional information about eye and mouth movements and head position. These complementary representations are fused to classify the driver's state into two categories: Active and Fatigue. The framework is developed using the UTA-RLDD dataset and integrated into a real-time monitoring application. The system aims to support continuous driver monitoring and provide fatigue alerts.},
        keywords = {Driver drowsiness detection, Convolutional Neural Network, Vision Transformer, Feature Fusion, Facial Behaviour Analysis, Deep Learning, Road Safety.},
        month = {November},
        }

Cite This Article

T, P., & G, L., & S, M., & Mythri, G., & R, D. K. K. P., & R, R. (2026). A Hybrid CNN-Vision Transformer Framework with Behavioural Feature Fusion for Driver Drowsiness Detection. International Journal of Innovative Research in Technology (IJIRT), 13(6), 4207–4213.

Related Articles