Human Activity Recognition in Video Sequences Based on the Integration of Optical Flow and Appearance of Human Objects

Abstract

Recognition of human activities has emerged as a critical research area due to its potential applications in many automated monitoring applications. However, it is still a challenging problem due to inter- and intra-class variations in human activities, varying illumination conditions, viewpoint changes, etc. This work presents human activity recognition framework for motion activities recorded in realistic and multi-view environments. To represent complex motion activities, we designed a novel feature descriptor based on the integration of motion information and the appearance of moving human objects. For this purpose, first, we employed an object segmentation technique capable of dealing with camera motion, varying lighting conditions, and scale of the human object. Then we used the optical flow technique to compute each moving pixel’s velocity vector and orientation by avoiding background noise. Then histogram of oriented gradients of velocity and orientation information is computed to get the relative distribution of motion information for samples of each activity category. Finally, a feature fusion strategy integrates a local-oriented histogram of velocity and orientation information to construct the final feature vector. Support vector machine is used to compute the class score of activity categories. In order to give an empirical justification, we conducted several extensive experiments on three publically available datasets, namely IXMAS, UT Interaction, and CASIA. To demonstrate the effectiveness of the proposed methods, we compared their results with several state-of-the-art methods. The recognition results demonstrate the supremacy of the proposed method over the other state-of-the-art methods considered for comparison.