跨媒體分析的理論和方法

盧漢清 劉靜 黃萱菁

摘 要：該年度的研究主要圍繞多媒體對象的多粒度語義分析與關聯挖掘等方面展開，考察媒體對象與語義標簽關聯矩陣的縱橫不同視角，充分發掘底層特征之間的關聯性，語義特征之間的關聯性，以及媒體對象在底層特征空間相似度和語義標簽空間相似度的一致性，同時注重與應用背景的緊密結合，力爭將研究成果做實做細。依照“課題計劃任務書及其后3年調整方案”要求，該課題在多媒體異構特征拓撲結構分析、媒體對象的多粒度語義解析等方面取得了突破性進展，課題整體進展順利，已完成本年度計劃的各項預期目標。在媒體數據的層次化語義分析方面，重點關注社會標簽在媒體信息理解任務中的重要作用，引入標簽行為的參與者（即用戶）以及地理位置等多屬性信息，以提高社會媒體網站中多媒體對象的語義理解性能。同時，我們還在多媒體內容的細粒度語義解析方面展開研究工作。在基于語義的媒體內容檢索與應用方面，重點考慮媒體數據的多模態與多關聯特性，在已取得層次化語義分析成果的基礎上，進一步關注用戶對媒體檢索的高、精、準的實際需求，力圖實現網絡媒體數據檢索的快速性與準確性，并結合實際應用開發了相關的檢索服務原型系統。

關鍵詞：跨媒體 多粒度語義分析 關聯挖掘

Abstract：Our work in this year focuses on the multi-granularity semantic analysis and correlation mining for the multimedia information. We attempt to utilize correlations within low-level and high-level features， and their similarity consistence to better understand multimedia objects. Our project goes well， and has reached the goals of this year. There are totally 33 publications in this year， in which 18 papers are published on international journals or transactions （SCI indexed）， and 15 papers are published on international conferences （e.g.， ACM Multimedia， ICCV， CIKM， and CVPR， EI indexed ）. Besides， we have one authorized patent and two pending patents. In the following， we will introduce our finished work in this year in details. （1）Multimedia feature representation and correlation construction：We have proposed a set of effective methods to solve the problem of the multi-modal feature fusion and selection when given a large-scale， noisy， and high dimensional multimedia dataset. One is the multi-view learning approach considering the consistency and complementarity of different features， one is the sub-space learning based robust feature selection， and the other is the topological feature structure analysis. The related works have published on important journals of TNNLS and CVIU， and top conferences of ACM MM and WWW， etc. （2）Hierarchical semantic analysis of multimedia data：We attempt to semantically understand multimedia data （video and image） from different semantic levels including low-level visual appearance， object part， object， and scene. To this goal， we utilized the important role of social tags to enhance the performance of multimedia semantic understanding. Other relevant attributes to social tags， i.e.， tagging users and geographic positions， are also considered for the task. The related works have published on important journals of TMM， Pattern Recognition， and TALSP， and top conferences of CVPR， ICCV， and ICME， etc. （3）Semantic retrieval and other applications：To integrate and verify our proposed approaches in the project， we attempt to develop and design some prototype systems for multimedia retrieval. The systems can meet user real requirements in retrieval process. The related works have published on important journals of TKDE， TOMCCAP， and TMM， and top conferences of ACM MM， WWW， and ICIP， etc.

Key Words：Cross-Media；Multi-granularity Semantic Analysis；Correlation Mining

閱讀全文鏈接（需實名注冊）：http：//www.nstrs.cn/xiangxiBG.aspx？id=51008&flag=1