Deep Learning-based Video Classification using CNN and LSTM

Video classifying is a difficult field of computer vision, as successfully utilizing spatial and temporal features is essential. In this study, we describe a method that leverages dl methods for identifying human actions in videos. First, we implement a ConvLSTM model that jointly utilizes convolutional operations within the LSTM cells to capture both spatiotemporal features. Additionally, we implement a Longterm Recurrent Convolutional Network (LRCN) that applies a CNN to drawn out spatial features and then uses an LSTM for temporal modeling. We conduct experiments on a subset of the UCF50 dataset to illustrate both models performance in the classification task and the results show both models achieved better performance than expected in classification and the LRCN model slightly outperformed the ConvLSTM in accuracy. The paper includes a meticulous description of the preprocessing pipeline and the architectural design of the models for training and evaluation. Lastly, we provide some ideas for improvement and future work.