| Citation: |
XU Huilin, ZHAO Xin, YU Bo, WEI Xiaoya, HU Peng. Multi-resolution Feature Extraction Algorithm for Semantic Segmentation of Infrared Images[J]. Infrared Technology , 2024, 46(5): 556-564.
|
A multi-resolution feature extraction convolution neural network is proposed for the problem of inaccurate edge segmentation when existing image semantic segmentation algorithms process low-resolution infrared images. DeepLabv3+ is used as the baseline network and adds a multi-resolution block, which contains both high and low resolution branches, to further aggregate the features in infrared images. In the low-resolution branch, a GPU friendly attention module is used to capture high-level global context information, and a multi-axis-gated multilayer perceptron module is added in this branch to extract the local and global information of infrared images in parallel. In the high resolution branch, the cross-attention module is used to propagate the global features learned on the low resolution branch to the high resolution branch, hence the high resolution branch can obtain stronger semantic information. The experimental results indicate that the segmentation accuracy of the algorithm on the dataset DNDS is better than that of the existing semantic segmentation algorithm, demonstrating the superiority of the proposed method.
| [1] |
刘致驿, 孙韶媛, 任正云, 等. 基于改进DeepLabv3+的无人车夜间红外图像语义分割[J]. 应用光学, 2020, 41(1): 180-185. https://www.cnki.com.cn/Article/CJFDTOTAL-YYGX202001031.htm
LIU Zhiyi, SUN Shaoyuan, REN Zhengyun, et al. Semantic segmentation of nocturnal infrared images of unmanned vehicles based on improved DeepLabv3+[J]. Journal of Applied Optics, 2020, 41(1): 180-185. https://www.cnki.com.cn/Article/CJFDTOTAL-YYGX202001031.htm
|
| [2] |
夏威. 基于卷积神经网络的热红外图像语义分割研究[D]. 合肥: 安徽大学, 2020.
XIA Wei. Thermal Image Semantic Segmentation Based on Convolutional Neural Networks[D]. Hefei: Anhui University, 2020.
|
| [3] |
景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述[J]. 计算机工程, 2020, 46(10): 1-17. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202208003.htm
JING Zhuangwei, GUAN Haiyan, PENG Daifeng, et al. Survey of research in image semantic segmentation based on deep neural network[J] Computer Engineering, 2020, 46(10): 1-17. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202208003.htm
|
| [4] |
ZHAO L, WANG M, YUE Y. Sem-aug: improving camera-lidar feature fusion with semantic augmentation for 3d vehicle detection[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 9358-9365. DOI: 10.1109/LRA.2022.3191208
|
| [5] |
WANG J, LIU L, LU M, et al. The estimation of broiler respiration rate based on the semantic segmentation and video amplification[J]. Frontiers in Physics, 2022, 10: 1-13.
|
| [6] |
XUE Z, MAO W, ZHENG L. Learning to simulate complex scenes for street scene segmentation[J]. IEEE Transactions on Multimedia, 2021, 24: 1253-1265.
|
| [7] |
WANG Y, TIAN S, YU L, et al. FSOU-Net: Feature supplement and optimization U-Net for 2D medical image segmentation[J]. Technology and Health Care, 2023, 31(1): 181-195. DOI: 10.3233/THC-220174
|
| [8] |
郭尹. 基于深度学习的电力设备热红外图像语义分割方法研究[D]. 合肥: 安徽大学, 2022.
GUO Yin. Research on Electrical Thermal Image Semantic Segmentation Method Based on Deep Learning[D]. Hefei: Anhui University, 2022.
|
| [9] |
LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 3431-3440.
|
| [10] |
Adrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615
|
| [11] |
OLAF R, PHILIPP F, THOMAS B. U-Net: Convolutional networks for biomedical image segmentation[J]. CoRR, 2015, abs/1505.04597.
|
| [12] |
ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017: 2881-2890.
|
| [13] |
LIN G, MILAN A, SHEN C, et al. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1925-1934.
|
| [14] |
CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848.
|
| [15] |
CHEN L C, ZHU Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation [C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 801-818.
|
| [16] |
Chollet F. Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.
|
| [17] |
练琤, 张宝辉, 江云峰, 等. 基于语义分割的红外图像增强方法[J]. 红外技术, 2023, 45(4): 394-401. http://hwjs.nvir.cn/cn/article/id/012a14e0-e0f5-4854-94fa-7b0392f63498?viewType=HTML
LIAN Zheng, ZHANG Baohui, JIANG Yunfeng, et al. An infrared image enhancement method based on semantic segmentation[J]. Infrared Technology, 2023, 45(4): 394-401. http://hwjs.nvir.cn/cn/article/id/012a14e0-e0f5-4854-94fa-7b0392f63498?viewType=HTML
|
| [18] |
WANG J, GOU C, WU Q, et al. RTFormer: efficient design for real-time semantic segmentation with transformer[J]. arXiv e-prints, 2022: arXiv: 2210.07124.
|
| [19] |
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. arXiv, 2017. DOI: 10.48550/arXiv.1706.03762.
|
| [20] |
TU Z, TALEBI H, ZHANG H, et al. Maxim: Multi-axis MLP for image processing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5769-5780.
|
| [21] |
JADON S. A survey of loss functions for semantic segmentation[C]//IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB). IEEE, 2020: 1-7.
|
| [22] |
Sandler M, Howard A, ZHU M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
|
| [23] |
于营, 王春平, 付强, 等. 语义分割评价指标和评价方法综述[J]. 计算机工程与应用, 2023, 59(6): 13. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202306005.htm
YU Ying, WANG Chunping, FU Qiang, et al. Survey of evaluation metrics and methods for semantic segmentation[J]. Journal of Computer Engineering & Applications, 2023, 59(6): 13. https://www.cnki.com.cn/Article/CJFDTOTAL-JSGG202306005.htm
|
| [24] |
WANG P, CHEN P, YUAN Y, et al. Understanding convolution for semantic segmentation[C]//IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018: 1451-1460.
|
| [1] | ZHAO Yating, HAN Long, HE Huihuang, CHEN Chu. DSEL-CNN: Image Fusion Algorithm Combining Attention Mechanism and Balanced Loss[J]. Infrared Technology , 2025, 47(3): 358-366. |
| [2] | LI Ruihong, FU Zhitao, ZHANG Shaochen, ZHANG Jian, WANG Leiguang. Nighttime Object Detection in Infrared and Visible Images Based on Multi-Attention Mechanism[J]. Infrared Technology , 2024, 46(12): 1371-1379. |
| [3] | WANG Yan, ZHANG Jinfeng, WANG Likang, FAN Xianghui. Underwater Image Enhancement Based on Attention Mechanism and Feature Reconstruction[J]. Infrared Technology , 2024, 46(9): 1006-1014. |
| [4] | LIU Xiaopeng, ZHANG Tao. Global-Local Attention-Guided Reconstruction Network for Infrared Image[J]. Infrared Technology , 2024, 46(7): 791-801. |
| [5] | DI Jing, LIANG Chan, REN Li, GUO Wenqing, LIAN Jing. Infrared and Visible Image Fusion Based on Multi-Scale Contrast Enhancement and Cross-Dimensional Interactive Attention Mechanism[J]. Infrared Technology , 2024, 46(7): 754-764. |
| [6] | ZHAO Songpu, YANG Liping, ZHAO Xin, PENG Zhiyuan, LIANG Dongxing, LIANG Hongjun. Object Detection in Visible Light and Infrared Images Based on Adaptive Attention Mechanism[J]. Infrared Technology , 2024, 46(4): 443-451. |
| [7] | LI Xiangrong, SUN Lihui. Multiscale Infrared Target Detection Based on Attention Mechanism[J]. Infrared Technology , 2023, 45(7): 746-754. |
| [8] | CHEN Xin. Infrared and Visible Image Fusion Using Double Attention Generative Adversarial Networks[J]. Infrared Technology , 2023, 45(6): 639-648. |
| [9] | LUO Di, WANG Congqing, ZHOU Yongjun. A Visible and Infrared Image Fusion Method based on Generative Adversarial Networks and Attention Mechanism[J]. Infrared Technology , 2021, 43(6): 566-574. |
| [10] | WANG Hao, ZHANG Jingjing, LI Yuanyuan, WANG Feng, XUN Lina. Hyperspectral Image Classification Based on 3D Convolution Joint Attention Mechanism[J]. Infrared Technology , 2020, 42(3): 264-271. |
| 1. |
赵洪山，王惠东，刘婧萱，杨伟新，李忠航，林诗雨，余洋，吕廷彦. 考虑局部纹理特征和全局温度分布的电力设备红外图像超分辨率重建方法. 电力系统保护与控制. 2025(02): 89-99 .
| |
| 2. |
徐浙君. 基于优化深度学习的低照度图像超分辨率重建方法的研究. 科技通报. 2024(04): 39-43+53 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: