Human Action Recognition by Learning Spatio-Temporal Features with Deep Neural Networks

¹ Dept. of CSE- Data Science, KG Reddy College of Engineering & Technology, Hyderabad, Telangana, India.
² Dept. of CSE-AI&ML, CMR Technical Campus, Kandlakoya, Hyderabad, Telangana, India
^3,4 Dept. of CSE- Data Science, CMR Technical Campus, Kandlakoya, Hyderabad, Telangana, India
⁵ Professor, Department of Computer Science and Engineering, GRIET, Bachupally, Hyderabad, Telangana
⁶ Uttaranchal Instiiute of Technology, Uttaranchal University, Dehradun, 248007

^* Corresponding author: veerender57@gmail.com

Abstract

Human action recognition plays a crucial role in various applications, including video surveillance, human-computer interaction, and activity analysis. This paper presents a study on human action recognition by leveraging CNN-LSTM architecture with an attention model. The proposed approach aims to capture both spatial and temporal information from videos in order to recognize human actions. We utilize the UCF-101 and UCF-50 datasets, which are widely used benchmark datasets for action recognition. The UCF-101 dataset consists of 101 action classes, while the UCF-50 dataset comprises 50 action classes, both encompassing diverse human activities. Our CNN-LSTM model integrates a CNN as the feature extractor to capture spatial information from video frames. Subsequently, the extracted features are fed into an LSTM network to capture temporal dependencies and sequence information. To enhance the discriminative power of the model, an attention model is incorporated to improve the activation patterns and highlight relevant features. Furthermore, the study provides insights into the importance of leveraging both spatial and temporal information for accurate action recognition. The findings highlight the efficacy of the CNN-LSTM architecture with an attention model in capturing meaningful patterns in video sequences and improving action recognition accuracy. You should leave 8 mm of space above the abstract and 10 mm after the abstract. The heading Abstract should be typed in bold 9-point Arial. The body of the abstract should be typed in normal 9-point Times in a single paragraph, immediately following the heading. The text should be set to 1 line spacing. The abstract should be centred across the page, indented 17 mm from the left and right page margins and justified. It should not normally exceed 200 words.