Ensemble Neural Networks for Multimodal Acute Pain Intensity Evaluation using Video and Physiological Signals

Abstract

Automated pain recognition is essential in healthcare. Research done previously suggests that for conventional algorithms, automated pain recognition relies on features gathered from video and physiological inputs electrodermal activity (EDA). The article presents investigations into designing a collaborative neural network structure that combines a fine-tuned, 3-stream hybrid deep neural network (HEDLM) with Convolutional Neural Network (CNN) to extract face image and EDA signal characteristics and recognize and precisely identify pain level. The experiments we conducted demonstrate that multimodal data on context works much better than uni-level data on context. Our research results for pain task recognition in Part A of the BioVid Heat Pain database contain pain levels 0 and 1, pain levels 0 and 2, pain levels 0 and 3, and pain levels 0 and 4. During LOSO (leave one subject out cross validation techniques) research, the classification task among levels of pain 0 and 4 reached an average accuracy of 84.8% for 87 individuals. This suggested approach takes advantage of deep learning capacity utilization to perform better than standard techniques by integrating facial images and physiological signals.