Author(s)

Chuwei Wang ¹, Wenhui Zeng ¹, Zicheng Wang ¹, Bing Wang ¹

Affiliation(s)

¹ School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan, China

Corresponding Author

Bing Wang

ABSTRACT

Continuous sign language recognition (CSLR) is crucial for bridging communication gaps for hearing-impaired people. Traditional methods relying on manual feature extraction suffer from poor adaptability and low accuracy. To address this, this paper designs a high-performance CSLR system based on CNN-BiLSTM-Attention, integrated with comprehensive regularization strategies to suppress overfitting. Using the RWTH-PHOENIX-Weather 2014 dataset, the system conducts standard data preprocessing and constructs a hybrid model: CNN extracts spatial features, BiLSTM captures bidirectional temporal dependencies, and attention enhances key frames. Regularization measures include L2 regularization (λ=0.001), Dropout(rate=0.3), model pruning, and early stopping. With Adam optimizer and learning rate decay, the model achieves 93.2%test accuracy, 15.8/12.0 percentage points higher than single CNN/BiLSTM, and 4.1 percentage points higher than CNN-BiLSTM without regularization/attention. It balances accuracy, robustness, and inference efficiency, providing a feasible solution for practical CSLR applications.

KEYWORDS

Continuous Sign Language Recognition; Deep Learning; CNN; BiLSTM; Attention Mechanism; RWTH-PHOENIX-Weather 2014 Dataset; L2 Regularization; Dropout; Model Pruning

CITE THIS PAPER

Chuwei Wang, Wenhui Zeng, Zicheng Wang, Bing Wang. Design and Implementation of a Continuous Sign Language Recognition System Based on Deep Learning. Automation and Machine Learning (2026). Vol. 7, No. 1, 48-54. DOI: http://dx.doi.org/10.23977/autml.2026.070106.

REFERENCES

[1]  Ben Zaid, F., Benaddy, M., Boukdir, A., & El Meslouhi, O. "A dataset for Moroccan sign language recognition and translation." Data in Brief, 64, 112395, 2026.
[2]  Alotaibi, N., Al Dayil, R., Aljehane, N. O., & Rizwanullah, M. "Enhanced feature fusion with hand gesture recognition system for sign language accessibility to aid hearing and speech impaired individuals." Scientific Reports, 2026.
[3]  Nemri, N., Alzahrani, M. Y., Bouchelligua, W., & Alneil, A. A. "Improving sign language recognition system for assisting deaf and dumb people using pathfinder algorithm with representation learning model." Scientific Reports, 2025.
[4]  Tao, T., Che, X., Zhao, Y., & Yang, Z. "Local attention and contrastive clustering network for sign language recognition." Pattern Recognition, 173, 112941, 2026.
[5]  Talaat, F. M., & Hassan, B. M. "A multistream attention based neural network for visual speech recognition and sign language understanding." Scientific Reports, 15(1), 44675, 2025.
[6]  Jiang, Y., Yang, D., & Chen, C. "Enhancing Continuous Sign Language Recognition via Spatio-Temporal Multi-Scale Deformable Correlation." Applied Sciences, 16(1), 124, 2025.
[7]  Hao, J., & Pan, H. "A novel deep transformer based CvT model for sign language recognition in visual communication." Scientific Reports, 2025.
[8]  Kishore, P. V. V., Bindu, G. H., Prasad, B., Kumar, D. A., Kumar, P. P., Suneetha, M., & Kumar, E. K. "Ppent: a pose embedding refinement framework aligning estimated and motion-captured skeletons for real-time word-level sign language recognition." International Journal of Information Technology, (prepublish), 1-19, 2025.
[9]  Harrouch, H., Trabelsi, L., Jebali, M., & Gammoudi, O. "A deep learning-based method combines manual and non-manual features for sign language recognition." Scientific Reports, 2025.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Computing, Performance and Communication Systems
  
  
• Journal of Artificial Intelligence Practice
  
  
• Advances in Computer, Signals and Systems
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks