基于雙模態(tài)融合的鋼軌表面缺陷分割研究

摘要：因長(zhǎng)期受反復(fù)荷載作用，高速鐵路鋼軌會(huì)產(chǎn)生表面缺陷，為了提升復(fù)雜場(chǎng)景下多類(lèi)多尺度鋼軌表面缺陷檢測(cè)的精度與速度，設(shè)計(jì)了一種基于雙模態(tài)融合的鋼軌表面缺陷分割網(wǎng)絡(luò)（DAFNet）。首先構(gòu)建了一個(gè)包含可見(jiàn)光和紅外通道的鋼軌表面缺陷數(shù)據(jù)集，并采用改進(jìn)的雙分支網(wǎng)絡(luò)架構(gòu)，提高了分割速度；同時(shí)，設(shè)計(jì)了雙模態(tài)自適應(yīng)融合模塊（BAFM），實(shí)現(xiàn)了特征的自適應(yīng)融合，提高了復(fù)雜場(chǎng)景下鋼軌表面缺陷的分割精度；此外，設(shè)計(jì)了空間細(xì)節(jié)提取模塊（SDEM）和關(guān)鍵信息增強(qiáng)模塊（KIEM），進(jìn)一步提高了對(duì)缺陷邊緣的感知度，解決了復(fù)雜場(chǎng)景下缺陷與背景對(duì)比度不高的問(wèn)題。實(shí)驗(yàn)表明，所設(shè)計(jì)網(wǎng)絡(luò)分割的精確度和mIoU分別達(dá)到了68.13%， 59.96%，明顯優(yōu)于其他主流網(wǎng)絡(luò)；且FLOPs、參數(shù)量和模型大小分別為17.41 GFLOPs， 1.38 M和5.67 MB，優(yōu)于大多數(shù)主流網(wǎng)絡(luò)。所設(shè)計(jì)的網(wǎng)絡(luò)顯著提高了鋼軌表面缺陷的分割精度，并且具有較高分割速度，對(duì)保障高鐵的安全運(yùn)營(yíng)具有重要意義。

關(guān)鍵詞：語(yǔ)義分割；鋼軌表面缺陷；深度學(xué)習(xí)；紅外圖像；可見(jiàn)光圖像；雙模態(tài)融合

中圖分類(lèi)號(hào)：U213；TP39 文獻(xiàn)標(biāo)志碼：A

本文引用格式：羅暉，韓岳霖，馬治偉，等. 基于雙模態(tài)融合的鋼軌表面缺陷分割研究[J]. 華東交通大學(xué)學(xué)報(bào)，2025，42（1）：52-60.

Research on Rail Surface Defect Segmentation

Based on Bimodal Fusion

Luo Hui， Han Yuelin， Ma Zhiwei， Si Chenghao

（School of Information and Software Engineering， East China Jiaotong University， Nanchang 330013， China）

Abstract： Due to the long-term repeated loading， surface defects occur in high-speed railway steel rails. In order to improve the accuracy and speed of surface defect detection for multiple classes and scales of steel rails in complex scenarios， a steel rail surface defect segmentation network based on multimodal fusion （DAFNet） is designed. Firstly， a steel rail surface defect dataset containing visible light and infrared channels is constructed， and an improved dual-branch network architecture is adopted to increase segmentation speed. Simultaneously， a bimodal adaptive fusion module （BAFM） is designed to achieve adaptive feature fusion， improving the segmentation accuracy of steel rail surface defects in complex scenarios. Additionally， a spatial detail extraction module （SDEM） and a key information enhancement module （KIEM） are designed to further enhance the perception of defect edges and address the low contrast between defects and backgrounds in complex scenarios. Experiments show that the accuracy and mIoU" of the designed network segmentation reach 68.13% and 59.96% respectively， which are significantly better than other mainstream networks. Moreover， FLOPs， parameter quantity， and model size are 17.41 GFLOPs， 1.38 M， and 5.67 MB respectively， which are better than most mainstream networks. The designed network significantly improves the segmentation accuracy of steel rail surface defects and has a high segmentation speed， which is of great significance for ensuring the safe operation of high-speed railways.

Key words： semantic segmentation; rail surface defects; deep learning; infrared image; visible light image；bimodal fusion

Citation format： LUO H， HAN Y L， MA Z W， et al. Research on rail surface defect segmentation based on bimodal fusion[J]. Journal of East China Jiaotong University， 2025， 42（1）： 52-60.

在長(zhǎng)期的重復(fù)荷載影響下，鋼軌的健康狀況和性能會(huì)逐漸下降，同時(shí)在其表面形成多種類(lèi)型和尺度的缺陷。當(dāng)前鋼軌表面缺陷檢測(cè)主要依賴(lài)非破壞性檢測(cè)技術(shù)[1]和傳統(tǒng)圖像處理方法[2-3]，這些方法效率低，且存在誤檢漏檢的情況，難以確保準(zhǔn)確性和實(shí)時(shí)性。……