一種基于色彩補(bǔ)償?shù)乃聢D像綜合增強(qiáng)算法

楊 淼，王海文，胡 珂，殷 歌，胡金通

一種基于色彩補(bǔ)償?shù)乃聢D像綜合增強(qiáng)算法

楊 淼1,2，王海文1，胡 珂1，殷 歌1，胡金通1

(1. 江蘇海洋大學(xué)電子工程學(xué)院，江蘇 連云港 222005； 2. 青島海洋科學(xué)與技術(shù)試點(diǎn)國(guó)家實(shí)驗(yàn)室，山東 青島 266237)

針對(duì)退化的水下圖像在高級(jí)視覺(jué)分析任務(wù)中無(wú)法進(jìn)行有效的目標(biāo)檢測(cè)及識(shí)別的問(wèn)題，提出了一種通過(guò)色彩補(bǔ)償和對(duì)比度拉伸，HSV空間校正和亮度通道去模糊系列方法實(shí)現(xiàn)了對(duì)水下圖像的色彩校正、色彩對(duì)比度、飽和度和細(xì)節(jié)清晰度的綜合提高。其中，提出了基于高斯濾波的亮度通道去散射方法，并對(duì)典型水體水下圖像綜合增強(qiáng)參數(shù)進(jìn)行了分析。實(shí)驗(yàn)對(duì)比了綜合增強(qiáng)方法和其他增強(qiáng)方法對(duì)偏藍(lán)、偏綠、偏黃、白色近岸淺灘水下圖像的處理結(jié)果并通過(guò)目標(biāo)檢測(cè)網(wǎng)絡(luò)對(duì)7種算法增強(qiáng)后的水下圖像數(shù)據(jù)集進(jìn)行訓(xùn)練與測(cè)試，對(duì)比了平均水下目標(biāo)識(shí)別準(zhǔn)確率和檢測(cè)到的目標(biāo)數(shù)量與實(shí)際目標(biāo)數(shù)量的比值來(lái)評(píng)估各個(gè)增強(qiáng)算法對(duì)于水下目標(biāo)識(shí)別和檢測(cè)任務(wù)中的作用。實(shí)驗(yàn)表明，與現(xiàn)有方法相比，該算法不僅可以有效地實(shí)現(xiàn)各類水下圖像清晰度和色彩增強(qiáng)，適用范圍廣，而且可以有效地提高水下圖像目標(biāo)識(shí)別任務(wù)的準(zhǔn)確率和檢測(cè)數(shù)量。

水下圖像增強(qiáng)；高斯濾波；亮度通道去散射；目標(biāo)檢測(cè)；水下圖像質(zhì)量評(píng)價(jià)

水下視覺(jué)是海洋探測(cè)、海洋生物調(diào)查、水下工程監(jiān)測(cè)中重要的科學(xué)研究依據(jù)[1-5]。近年來(lái)國(guó)內(nèi)外學(xué)者在基于Retinex模型[6]、暗通道先驗(yàn)[7-12]、暗通道與色彩校正相結(jié)合[13]、色彩平衡和補(bǔ)償衰減[14]、圖像融合[15]等方面提出了很多水下圖像復(fù)原、增強(qiáng)方法。這些傳統(tǒng)方法無(wú)法綜合地解決由多種退化混合疊加造成水下圖像質(zhì)量退化，且適應(yīng)水下環(huán)境變化的能力較差。隨著人工智能技術(shù)的發(fā)展，基于深度學(xué)習(xí)的水下圖像增強(qiáng)算法逐漸成為了研究熱點(diǎn)，如YU等[16]基于生成式對(duì)抗網(wǎng)絡(luò)的水下圖像復(fù)原網(wǎng)絡(luò)，模擬水下成像模型獲得一個(gè)水下圖像數(shù)據(jù)集；PENG和COSMAN[17]提出了基于深度學(xué)習(xí)的水下圖像增強(qiáng)網(wǎng)絡(luò)，從訓(xùn)練集中學(xué)習(xí)退化及恢復(fù)的水下圖像之間的特征映射關(guān)系，從而實(shí)現(xiàn)對(duì)測(cè)試集中退化的水下圖像進(jìn)行增強(qiáng)；LI等[18]提出了基于水下場(chǎng)景先驗(yàn)知識(shí)和卷積神經(jīng)網(wǎng)絡(luò)的水下圖像增強(qiáng)網(wǎng)絡(luò)，無(wú)需估計(jì)水下成像模型的參數(shù)，可直接生成清晰的水下圖像；YANG等[19]基于條件生成式對(duì)抗網(wǎng)絡(luò)的水下圖像增強(qiáng)方法，用多尺度生成器實(shí)現(xiàn)水下圖像增強(qiáng)。

本文提出了一種可用于水下目標(biāo)檢測(cè)預(yù)處理的水下圖像綜合增強(qiáng)方法(underwater image competive enhancement method，UICEM)。處理流程如圖1所示。通過(guò)對(duì)近岸水下圖像衰減的分析，在補(bǔ)償紅通道衰減的基礎(chǔ)上，進(jìn)行色偏校正、HSV ((色調(diào))Hue，(飽和度) Saturation，(亮度)Value)顏色空間校正和亮度通道去模糊處理來(lái)分別達(dá)到衰減補(bǔ)償、提高對(duì)比度和清晰度的目的。另外，基于水下圖像光亮度分布的局部尺度特征，提出了一種用于亮度空間的單通道去模糊方法。通過(guò)對(duì)多種類型水下圖像的主觀和目標(biāo)檢測(cè)實(shí)驗(yàn)證明，本文算法在視覺(jué)效果和增強(qiáng)后的評(píng)估指標(biāo)上均明顯優(yōu)于現(xiàn)有的水下圖像復(fù)原、增強(qiáng)算法。

圖1 水下圖像綜合增強(qiáng)方法流程圖

1 水下圖像綜合增強(qiáng)算法

根據(jù)Jaffe-McGlamery水下成像模型，退化的水下圖像可描述為

1.1 水下圖像紅通道色彩補(bǔ)償

1.2 色彩校正

1.3 HSV均衡和去模糊

1.3.1 HSV均衡

γ校正曲線如圖2所示，對(duì)歸一化后灰度圖像，當(dāng)α3k<1時(shí)，低灰度值動(dòng)態(tài)范圍變大，高灰度值動(dòng)態(tài)范圍變小，圖像整體的灰度值變大。相反，當(dāng)α3k>1時(shí)，低灰度值動(dòng)態(tài)范圍變小，高灰度值動(dòng)態(tài)范圍變大。因此對(duì)于存在低對(duì)比度的水下圖像，選取大于1的α3k值，可提高目標(biāo)對(duì)比度。

1.3.2 亮度去模糊

1.4 參數(shù)估計(jì)

2 實(shí) 驗(yàn)

本節(jié)對(duì)比了UICEM和其他增強(qiáng)方法對(duì)偏藍(lán)、偏綠、偏黃、白色近岸淺灘水下圖像的處理結(jié)果。選取的比較方法主要包括：文獻(xiàn)[7]、[10]、[13]、[6]和文獻(xiàn)[15]提出的方法。還將本文方法與基于深度學(xué)習(xí)的UWCNN (underwater image enhancement convolutional neural network)方法[18]進(jìn)行了單獨(dú)比較。在實(shí)驗(yàn)中，暗通道和高斯濾波器的濾波窗口大小為15×15，1=1.0，2=2.3，3H=2.9，3S=2.3，3V=2.3，其他用于對(duì)比的各種方法中的參數(shù)均參照原文設(shè)置。

2.1 4類水下圖像增強(qiáng)結(jié)果對(duì)比

實(shí)驗(yàn)結(jié)果如圖3和圖4?？梢钥闯?，文獻(xiàn)[13]、文獻(xiàn)[6]和文獻(xiàn)[15]提出的方法對(duì)4類水下圖像都取得了一定程度的色彩復(fù)原，但無(wú)法同時(shí)取得比較好的增強(qiáng)效果。文獻(xiàn)[7]的紅通道法對(duì)偏藍(lán)水下圖像色彩復(fù)原效果不佳，對(duì)存在黃色目標(biāo)的水下圖像，由于補(bǔ)償紅色而使得復(fù)原后圖像中的黃色目標(biāo)偏綠，如圖3(b)中第1幅圖片。文獻(xiàn)[13]方法存在暗區(qū)域模糊和色彩偽影問(wèn)題；文獻(xiàn)[10]和文獻(xiàn)[13]增強(qiáng)后的圖像暗區(qū)域的對(duì)比度均較低；文獻(xiàn)[6]方法輸出圖像存在邊緣模糊的問(wèn)題。而本文方法對(duì)4類水下圖像均取得了色彩和對(duì)比度的提高，且圖像細(xì)節(jié)的清晰度也得到了增強(qiáng)。本文從UIEBD (an underwater image enhancement benchmark dataset and beyond)數(shù)據(jù)集[21]中選取了偏色和低對(duì)比度的水下圖像用UWCNN方法[18]和本文方法分別進(jìn)行增強(qiáng)。實(shí)驗(yàn)結(jié)果如圖4所示，可以看出，對(duì)偏色水下圖像進(jìn)行增強(qiáng)時(shí)，本文方法可以更有效地校正偏色，并保留圖片細(xì)節(jié)；對(duì)低對(duì)比度水下圖像進(jìn)行增強(qiáng)時(shí)，本文方法能夠更顯著地提高對(duì)比度，且未導(dǎo)致偏色。

圖3 4類水下圖像處理結(jié)果對(duì)比((a)原始圖像；(b)文獻(xiàn)[7]；(c)文獻(xiàn)[10]；(d)文獻(xiàn)[13]；(e)文獻(xiàn)[6]；(f)文獻(xiàn)[15]；(g)本文UICEM)

圖4 本文UICEM與UWCNN[18]處理結(jié)果對(duì)比((a)偏色水下圖像；(b)UWCNN[18]處理結(jié)果；(c)UICEM處理結(jié)果；(d)低對(duì)比度水下圖像；(e)UWCNN[18]處理結(jié)果；(f)UICEM處理結(jié)果)

2.2 目標(biāo)檢測(cè)對(duì)比

水下圖像增強(qiáng)的目的不僅是為了主觀視覺(jué)質(zhì)量的改進(jìn)，更主要的是為了能夠完成更高級(jí)別的視覺(jué)分析任務(wù)(例如目標(biāo)識(shí)別和檢測(cè))。本文通過(guò)目標(biāo)檢測(cè)網(wǎng)絡(luò)模型對(duì)經(jīng)過(guò)7種算法增強(qiáng)后的水下圖像數(shù)據(jù)集進(jìn)行訓(xùn)練與測(cè)試，對(duì)比了平均水下目標(biāo)識(shí)別準(zhǔn)確率(mean average precision, mAP)和檢測(cè)到目標(biāo)數(shù)量與實(shí)際的目標(biāo)數(shù)量的比值(Num)來(lái)評(píng)估各個(gè)增強(qiáng)算法對(duì)于水下目標(biāo)識(shí)別和檢測(cè)任務(wù)中的作用。實(shí)驗(yàn)的圖像數(shù)據(jù)來(lái)自于RUIE (real-world underwater image enhancement)[22]近海水下圖像數(shù)據(jù)集，該數(shù)據(jù)集提供3類海洋生物，即扇貝、海膽和海參的邊界框和標(biāo)簽。由于這3種海洋生物的外觀和周?chē)h(huán)境十分相似，因此對(duì)該數(shù)據(jù)集進(jìn)行目標(biāo)檢測(cè)及識(shí)別均具有一定的挑戰(zhàn)性。本文從該數(shù)據(jù)集中選取2 400張標(biāo)記過(guò)的水下圖像作為實(shí)驗(yàn)圖像，圖像歸一化大小為300×400。其中2 000張圖像作為訓(xùn)練集，400張圖像為測(cè)試集。用各種方法增強(qiáng)后的RUIE數(shù)據(jù)集重新訓(xùn)練目標(biāo)檢測(cè)網(wǎng)絡(luò)YOLO-V3[23]并對(duì)3種目標(biāo)進(jìn)行檢測(cè)，結(jié)果如圖5和表1所示。3類目標(biāo)的識(shí)別準(zhǔn)確率和檢測(cè)率如圖6所示。在圖像未經(jīng)過(guò)增強(qiáng)處理前，mAP和Num分別為0.323和0.347。

圖5 部分測(cè)試集中水下圖像增強(qiáng)后的目標(biāo)檢測(cè)結(jié)果((a)水下圖像原圖；(b)文獻(xiàn)[11]；(c)文獻(xiàn)[8]；(d)文獻(xiàn)[7]；(e)文獻(xiàn)[6]；(f)文獻(xiàn)[13]；(g)文獻(xiàn)[10]；(h)本文UICEM算法)

表1 目標(biāo)識(shí)別準(zhǔn)確率(mAP)及檢測(cè)率對(duì)比(Num)(%)

圖6 目標(biāo)識(shí)別準(zhǔn)確率及檢測(cè)數(shù)量比((a)目標(biāo)識(shí)別準(zhǔn)確率；(b)檢測(cè)數(shù)量比)

從表1可以看出，大多數(shù)增強(qiáng)方法由于改善了水下圖像存在的退化，提高了清晰度和對(duì)比度，mAP和Num相較于未增強(qiáng)的圖像均有相應(yīng)的提高，因此說(shuō)明了水下圖像增強(qiáng)在高級(jí)視覺(jué)任務(wù)中作為預(yù)處理的必要性。圖像對(duì)比度的提高對(duì)于目標(biāo)檢測(cè)任務(wù)具有重要的影響，因此，文獻(xiàn)[6]的方法和本文方法的mAP和Num均高于其他增強(qiáng)方法。而本文的UICEM改善了由于后向散射等造成的模糊，提高了目標(biāo)清晰度，獲得了最高的93.3%的目標(biāo)檢測(cè)率和85.24%的目標(biāo)識(shí)別準(zhǔn)確率。由圖6可以看出，對(duì)其他增強(qiáng)方法輸出的圖像進(jìn)行檢測(cè)時(shí)，對(duì)海參這種與海底環(huán)境較為相似的目標(biāo)的識(shí)別率相對(duì)于其他2種目標(biāo)較低。而本文方法增強(qiáng)后的水下圖像進(jìn)行識(shí)別時(shí)，對(duì)3類目標(biāo)均有相近的識(shí)別正確率。

3 結(jié) 論

本文提出的UICEM方法旨在通過(guò)提高水下圖像質(zhì)量有效地實(shí)現(xiàn)對(duì)水下彩色圖像進(jìn)行目標(biāo)檢測(cè)和識(shí)別。本文基于色彩補(bǔ)償、色偏校正和HSV空間的對(duì)比度提高和去模糊，結(jié)合水下圖像分類參數(shù)分析，實(shí)現(xiàn)了對(duì)4類水下圖像的有效增強(qiáng)。與其他方法相比，本文UICEM所產(chǎn)生的結(jié)果具有更好的色調(diào)復(fù)原及清晰度提高效果，適用于各種水下環(huán)境拍攝的圖像，采用本文方法進(jìn)行水下圖像預(yù)處理，可有效提高高層次水下視覺(jué)任務(wù)性能。

[1] LI Y J, LU H M, ZHANG L F, et al. Real-time visualization system for deep-sea surveying[J]. Mathematical Problems in Engineering, 2014(1): 1-10.

[2] SPAMPINATO C, PALAZZO S, BOOM B, et al. Understanding fish behavior during typhoon events in real-life underwater environments[J]. Multimedia Tools and Applications, 2014, 70(1): 199-236.

[3] HUANG P X, BOOM B J, FISHER R B. Fish4 knowledge deliverable D1.3 fish recognition and clustering[EB/OL]. (2012-11-01) [2020-07-10]. http://homepages.inf.ed.ac.uk/rbf/Fish4Knowledge/DELIVERABLES/Del13.pdf.

[4] YANG M, HU J T, LI C Y, et al. An in-depth survey of underwater image enhancement and restoration[J]. IEEE Access, 2019, 7: 123638-123657.

[5] YANG M, SOWMYA A, WEI Z Q, et al. Offshore underwater image restoration using reflection- decomposition-based transmission map estimation[J]. IEEE Journal of Oceanic Engineering, 2020, 45(2): 521-533.

[6] FU X Y, ZHUANG P X, HUANG Y, et al. A retinex- based enhancing approach for single underwater image[C]//2014 IEEE International Conference on Image Processing (ICIP). New York: IEEE Press, 2014: 4572-4576.

[7] GALDRAN A, PARDO D, PICóN A, et al. Automatic red-channel underwater image restoration[J]. Journal of Visual Communication and Image Representation, 2015, 26: 132-145.

[8] HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.

[9] YANG M, YIN B, WEI Z Q, et al. Effective estimation of background light in underwater image dehazing[J]. OSA Continuum, 2019, 2(3): 767-785.

[10] PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594.

[11] CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing[C]//IEEEOceans 2010. New York: IEEE Press, 2010: 1-8.

[12] EMBERTON S, CHITTKA L, CAVALLARO A. Hierarchical rank-based veiling light estimation for underwater dehazing[C]//2015 British Machine Vision Conference. Durham: British Machine Vision Association Press, 2015: 125.1-125.12.

[13] LI C Y, GUO J C, GUO C L, et al. A hybrid method for underwater image correction[J]. Pattern Recognition Letters, 2017, 94: 62-67.

[14] 楊淼, 紀(jì)志成. 基于模糊形態(tài)篩和四元數(shù)的水下彩色圖像增強(qiáng)[J]. 儀器儀表學(xué)報(bào), 2012, 33(7): 1601-1605. YANG M, JI Z C. Underwater color image enhancement based on quaternion and fuzzy morphological sieves[J]. Chinese Journal of Scientific Instrument, 2012, 33(7): 1601-1605 (in Chinese).

[15] ANCUTI C O, ANCUTI C. Single image dehazing by multi-scale fusion[J]. IEEE Transactions on Image Processing, 2013, 22(8): 3271-3282.

[16] YU X L, QU Y Y, HONG M. Underwater-GAN: underwater image restoration via conditional generative adversarial network[C]//2018 International Conference on Pattern Recognition. Cham: Springer International Publishing, 2018: 66-75.

[17] PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594.

[18] LI C Y, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038.

[19] YANG M, HU K, DU Y X, et al. Underwater image enhancement based on conditional generative adversarial network[J]. Signal Processing: Image Communication, 2020, 81: 115723.

[20] HOU M J, LIU R S, FAN X, et al. Joint residual learning for underwater image enhancement[C]//The 25th IEEE International Conference on Image Processing (ICIP). New York: IEEE Press, 2018: 4043-4047.

[21] LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376-4389.

[22] LIU R, FAN X, ZHU M, et al. Real-world underwater enhancement: challenges, benchmarks, and solutions[EB/OL]. [2020-07-10]. https://arxiv.org/abs/1901.05320.

[23] REDMON J, FARHADI A. Yolov3: an incremental improvement[EB/OL]. [2020-07-10]. https://arxiv.org/abs/1804.02767.

An underwater image comprehensive enhancement algorithm based on color compensation

YANG Miao1,2, WANG Hai-wen1, HU Ke1, YIN Ge1, HU Jin-tong1

(1. School of Electronic Engineering, Jiangsu Ocean University, Lianyungang Jiangsu 222005, China; 2. Pilot Qingdao National Laboratory for Marine Science and Technology, Qingdao Shandong 266237, China)

A novel underwater image compositive enhancement method was proposed to improve the quality of underwater images, thereby synthetically boosting the performance of high-level visual analysis. A series of operations, including color compensation and correction, gamma correction in the HSV space, and final brightness de-blurring, were combined to realize color restoration, contrast and clarity improvements for underwater images. A method of brightness channel de-scattering based on Gauss filtering was proposed, and the comprehensive enhancement parameters of typical underwater images were analyzed. The experiments in this paper compared the processing results of the compositive enhancement method and other enhancement methods for the bluish, greenish, yellowish, and whitish nearshore shoal underwater images, and trained and tested the underwater image data sets enhanced by seven algorithms through the target detection network. Comparisons were also made between the average underwater target recognition accuracy rate and the ratio of the number of detected targets to the actual target number, so as to evaluate the effect of each enhancement algorithm on underwater target recognition and detection tasks. The experiment results demonstrate that the proposed method can achieve substantial image clarity improvement and color restoration, and is widely applicable, compared with the existing methods. At the same time, it can effectively improve the accuracy of underwater target recognition and the number of the detected objects.

underwater image enhancement; Gauss filtering; brightness channel de-scattering; object detection; underwater image quality evaluation

TP 391

10.11996/JG.j.2095-302X.2021010059

A

2095-302X(2021)01-0059-06

2020-07-17；

17 July，2020；

2020-08-13

13 August，2020

江蘇省基礎(chǔ)研究計(jì)劃(自然科學(xué)基金) (BK20191469)；江蘇科技大學(xué)海洋裝備研究院高技術(shù)協(xié)同創(chuàng)新項(xiàng)目(HZ20190005)；江蘇省研究生科研創(chuàng)新項(xiàng)目(KYCX19_2314，KYCX20_2768，KYCX20_2769)；國(guó)家自然科學(xué)基金青年項(xiàng)目(61601194)

：Jiangsu Basic Research Program (Natural Science Fund) (BK20191469); High-Tech Collaborative Innovation Project of Marine Equipment Research Institute of Jiangsu University of Science and Technology (HZ20190005); Jiangsu Province Graduate Research and Innovation Project (KYCX19_2314, KYCX20_2768, KYCX20_2769); National Natural Science Foundation Youth Project (61601194)

楊 淼(1978-)，女，黑龍江五常人，教授，博士，碩士生導(dǎo)師。主要研究方向?yàn)樗乱曈X(jué)、圖像處理、計(jì)算機(jī)視覺(jué)。 E-mail：lemonmiao@gmail.com

YANG Miao (1978-), female, professor, Ph.D. Her main research interests cover underwater vision, image processing and computer vision. E-mail：lemonmiao@gmail.com