Frame Differencing Based Temporal Feature Extraction in Human Action Recognition

Abstract

Human Action Recognition (HAR) is applicable in many research domains such as video retrieval, autism care etc. Action can be recognized from the video whose content are temporal in nature. The two major challenges that HAR system faces are the temporal feature extraction and its computation cost. In this paper, these two challenges are rectified to some extent by introducing temporal feature extraction in residual frames. The Frame Differencing (FD) method is used to extract spatial and temporal features in order to recognize action. Keyframes are utilized to extract spatial features, whereas residual frames are utilized to recover temporal features. Both the features are fused to form spatio-temporal features and classified using Multiclass Support Vector Machine (MSVM) classifier. The proposed method is tested on HMDB51, UCF101 and UCF Sports datasets and the performance is measured using precision, recall, specificity and accuracy. It is also compared with most recent methods and found that the proposed method outperforms all compared methods by achieving an accuracy of 85.8%, 98.83% and 96.6% on HMDB51, UCF101 and UCF Sports action datasets respectively.