Fast Real-Time Video Analytics for Human Action Recognition in Compressed Domain

Herein, a novel methodology is proposed for real-time human activity detection and recognition in compressed domain of videos by using motion vectors and long-term recurrent convolutional networks (MVLRCN). The videos in MPEG-4 and H.264 compression formats are considered for the present study. Any video source without any prior setup could be considered by adapting the proposed method to various video codecs and camera settings. Existing algorithms for human action recognition in a compressed domain video have some limitations in this regard such as (i) requirement of keyframes at a fixed interval, (ii) usage of only P frames, and (iii) normally support only a single codec. These limitations are overcome in the proposed method by using arbitrary keyframe intervals, using both P and B frames, and supporting both MPEG-4 and H.264 codecs. The experimentation is carried out using the benchmark datasets, namely, UCF101, HMDB51 and THUMOS14, and the recognition accuracy in compressed domain is found to be comparable to that observed in raw video data by using other recent methods. The proposed MVLRCN method has outperformed other methods in the literature significantly by improving the video action recognition inference in the compressed videos.