Stochastic Gradient Descent–Whale Optimization Algorithm-Based Deep Convolutional Neural Network To Crowd Emotion Understanding

Abstract

Crowd emotion understanding is an interesting research area that assists the security personnel to read the emotion/activity of the crowd in the locality. Most of the traditional methods utilize the low-level visual features to understand the crowd emotions that extend the gap between the low- and the high-level features. With the aim to develop an automatic method for emotion recognition, this paper utilizes the deep convolutional neural network (deep CNN). For the effective emotion recognition, it is essential to select the key frames of the video using the wavelet-based Bhattacharya distance. The key frames are fed to the space-time interest points descriptor that extracts the features and forms the input vector to the classifier. Deep CNN is trained using the proposed Stochastic Gradient Descent–Whale Optimization Algorithm, which is the unification of the standard stochastic gradient descent algorithm with whale optimization algorithm. The proposed classifier recognizes the emotions of the crowd, such as angry, escape, fight, happy, normal, running/walking and violence. The analysis of the method proved that the proposed approaches acquired a maximal accuracy, specificity and sensitivity of 0.9693, 0.9936 and 0.9675, respectively.