圖像整體與局部區域嵌入的視覺情感分析

蔡國永 賀歆灝 儲陽陽

摘 要：目前多數圖像視覺情感分析方法主要從圖像整體構建視覺情感特征表示，然而圖像中包含對象的局部區域往往更能突顯情感色彩。針對視覺圖像情感分析中忽略局部區域情感表示的問題，提出一種嵌入圖像整體特征與局部對象特征的視覺情感分析方法。該方法結合整體圖像和局部區域以挖掘圖像中的情感表示，首先利用對象探測模型定位圖像中包含對象的局部區域，然后通過深度神經網絡抽取局部區域的情感特征，最后用圖像整體抽取的深層特征和局部區域特征來共同訓練圖像情感分類器并預測圖像的情感極性。實驗結果表明，所提方法在真實數據集TwitterⅠ和TwitterⅡ上的情感分類準確率分別達到了75.81%和78.90%，高于僅從圖像整體特征和僅從局部區域特征分析情感的方法。

關鍵詞：社交媒體;情感分析;圖像局部對象檢測;深度學習;神經網絡

中圖分類號：?TP181

文獻標志碼：A

Visual sentiment analysis by combining global and local regions of image

CAI Guoyong， HE Xinhao*， CHU Yangyang

Guangxi Key Laboratory of Trusted Software （Guilin University of Electronic Technology）， Guilin Guangxi 541004， China

Abstract：?Most existing visual sentiment analysis methods mainly construct visual sentiment feature representation based on the whole image. However， the local regions with objects in the image are able to highlight the sentiment better. Concerning the problem of ignorance of local regions sentiment representation in visual sentiment analysis， a visual sentiment analysis method by combining global and local regions of image was proposed. Image sentiment representation was mined by combining a whole image with local regions of the image. Firstly， an object detection model was used to locate the local regions with objects in the image. Secondly， the sentiment features of the local regions with objects were extracted by deep neural network. Finally， the deep features extracted from the whole image and the local region features were utilized to jointly train the image sentiment classifier and predict the sentiment polarity of the image. Experimental results show that the classification accuracy of the proposed method reaches 75.81% and 78.90% respectively on the real datasets TwitterⅠand TwitterⅡ， which is higher than the accuracy of sentiment analysis methods based on features extracted from the whole image or features extracted from the local regions of image.

Key words：?social media; sentiment analysis; image local object detection; deep learning; neural network

0 引言

當前，越來越多社交媒體用戶喜歡用視覺圖像來表達情感或觀點，相較于文本，視覺圖像更易于直觀表達個人情感，由此，對圖像的視覺情感分析引起了人們的廣泛關注和研究[1-2]。視覺情感分析是一項研究人類對視覺刺激（如圖像和視頻）做出的情感反應的任務[3]，其關鍵挑戰問題是情感空間與視覺特征空間之間存在的巨大鴻溝問題。

早期的視覺情感分類主要采用特征工程的方法來構造圖像情感特征，如采用顏色、紋理和形狀等特征[4-6]?！?br>