基于语义分割的红外图像增强方法

练琤; 张宝辉; 江云峰; 蒋志芳; 张倩; 袁茜琳

基于语义分割的红外图像增强方法

练琤^1,,
张宝辉^{1, 2, ,},
江云峰¹,
蒋志芳²,
张倩²,
袁茜琳¹

1.
昆明物理研究所, 云南昆明 650223
2.
北方夜视科技(南京)研究院有限公司, 江苏南京 211106

详细信息

作者简介:
练琤（1998-），女，硕士研究生，研究方向为红外图像处理，语义分割。E-mail：1449069814@qq.com

通讯作者:
张宝辉（1984-），男，博士，研高工，主要研究方向为红外探测与图像处理。E-mail：zbhmatt@163.com

中图分类号: TP391
计量
- 文章访问数: 344
- HTML全文浏览量: 100
- PDF下载量: 79
出版历程
- 收稿日期: 2022-11-16
- 修回日期: 2022-12-10
- 刊出日期: 2023-04-19

An Infrared Image Enhancement Method Based on Semantic Segmentation

1.
Kunming Institute of Physics, Kunming 650223, China
2.
North Night-Vision Science & Technology (Nanjing) Research Institute Co. LTD., Nanjing 211106, China

摘要

摘要: 针对对比度受限的自适应直方图均衡化（contrast limited adaptive histogram equalization, CLAHE）强行分块造成的视觉不自然现象，本文提出了一种基于语义分割的红外图像增强方法。语义分割网络将整个红外图像分割成种类块而不是传统的矩形图像块。然后，每个种类块各自进行对比度受限的直方图均衡化，以减少过度增强。最后，采用了一种新的边缘过渡方法来避免种类块之间的突变。实验结果表明，本文所提出的红外图像增强方法在对比度和熵上优于其他对比算法，而且避免了传统CLAHE的视觉不自然现象，具有更好的视觉效果。
- 图像增强 /
- 红外图像 /
- 语义分割 /
- 直方图均衡 /
- CLAHE
Abstract: To solve the problem of visual unnaturalness caused by contrast-limited adaptive histogram equalization (CLAHE) forced blocking, this study proposes an infrared image enhancement method based on semantic segmentation. The semantic segmentation network segments the entire infrared image into category blocks instead of traditional rectangular image blocks. Each category block is individually subjected to contrast-limited histogram equalization to reduce over-enhancement. Finally, a new edge transition method is introduced to avoid abruptness between category blocks. The experimental results show that the proposed image enhancement method outperforms other contrast algorithms in terms of contrast and entropy and avoids the visual unnaturalness of traditional CLAHE with better visual effects.
- image enhancement /
- infrared image /
- semantic segmentation /
- histogram equalization /
- CLAHE

HTML全文

图 1 CLAHE整体框架图

Figure 1. Framework of CLAHE

下载: 全尺寸图片幻灯片

图 2 CLAHE处理后图像

Figure 2. The image after CLAHE processing

下载: 全尺寸图片幻灯片

图 3 本文方法整体框架图

Figure 3. The overall framework diagram of the proposed method

下载: 全尺寸图片幻灯片

图 4 语义分割示意图

Figure 4. The diagram of semantic segmentation

下载: 全尺寸图片幻灯片

图 5 直方图的裁剪和重映射

Figure 5. Clipping and redistribution of histogram

下载: 全尺寸图片幻灯片

图 6 种类块边缘过渡图

Figure 6. The diagram of category blocks edge transition

下载: 全尺寸图片幻灯片

图 7 场景一不同算法的增强效果对比图

Figure 7. Enhanced comparison of scene one by different algorithms

下载: 全尺寸图片幻灯片

图 8 场景二不同算法的增强效果对比

Figure 8. Enhanced comparison of scene two by different algorithms

下载: 全尺寸图片幻灯片

图 9 场景三不同算法的增强效果对比

Figure 9. Enhanced comparison of scene three by different algorithms

下载: 全尺寸图片幻灯片

图 10 场景四不同算法的增强效果对比

Figure 10. Enhanced comparison of scene four by different algorithms

下载: 全尺寸图片幻灯片

表 1 各种语义分割模型在Cityscapes数据集上的参数、精度、运行时间分析

Table 1 Parameters, accuracy and time analysis of various semantic segmentation models Cityscapes dataset

Method Parameter(M) MIoU/(%) Time/ms

SegNet 29.5 56.1 89.2

ENet 0.4 58.3 19.3

PSPNet 65.6 73.6 > 1000

RefineNet 118.4 78.4 > 1000

CGNet 0.5 64.8 56.8

下载: 导出CSV

表 2 不同算法的对比度

Table 2 Contrast of different algorithms

Original BBHE 2DHE CALHE Proposed

Fig.7 23.45 35.10 34.07 38.29 44.94

Fig.8 10.11 15.31 14.82 15.16 17.07

Fig.9 8.11 12.79 16.98 14.45 17.47

Fig.10 13.06 22.93 27.52 32.52 37.13

下载: 导出CSV

表 3 不同算法的熵

Table 3 Entropy of different algorithms

Original BBHE 2DHE CALHE Proposed

Fig.7 5.54 5.86 6.67 6.54 7.04

Fig.8 7.23 7.72 7.59 7.31 7.94

Fig.9 7.07 7.71 7.73 7.49 7.87

Fig.10 7.30 7.62 7.80 7.76 7.97

下载: 导出CSV

参考文献(24)

[1]	石川凌. 海陆场景的红外实时仿真研究[D]. 杭州: 浙江大学, 2016. SHI Chuanling. Study on Infrared Real-time Simulation of Sea-land Scene[D]. Hangzhou: Zhejiang University, 2016.
[2]	曾庆杰. 红外成像中图像质量提升算法研究[D]. 西安: 西安电子科技大学, 2021. ZENG Qingjie. Research on Image Quality Improvement Algorithm in Infrared Imaging[D]. Xi 'an: Xidian University, 2021.
[3]	William K. Pratt. Introduction to Digital Image Processing[M]. Taylor and Francis: CRC Press, 2013.
[4]	Jain A. Fundamentals of digital image processing[J]. Computer Vision, Graphics, and Image Processing, 1989, 46(3): 400-400.
[5]	Yeong-Taeg Kim. Contrast enhancement using brightness preserving bi-histogram equalization[J]. IEEE Transactions on Consumer Electronics, 1997, 43(1): 1-8. DOI: 10.1109/30.580378
[6]	Stark J A. Adaptive image contrast enhancement using generalizations of histogram equalization[J]. IEEE Transactions on Image Processing, 2000, 9(5): 889-96. DOI: 10.1109/83.841534
[7]	YANG Maoxiang, TANG Guijin, LIU Xiaohua, et al. Low-light image enhancement based on Retinex theory and dual-tree complex wavelet transform[J]. Optoelectronics Letters, 2018, 14(6): 470-475. DOI: 10.1007/s11801-018-8046-5
[8]	Zuiderveld K. Contrast limited adaptive histogram equalization[J]. Graphics Gems, 1994: 474-485.
[9]	Suharyanto, Hasibuan Z A, Andono P N, et al. Contrast limited adaptive histogram equalization for underwater image matching optimization use SURF[J]. Journal of Physics, 2021, 1803(1): 012008.
[10]	Ali M Reza. Realization of the contrast limited adaptive histogram equalization(CLAHE) for real-time image enhancement[J]. The Journal of VLSI Signal Processing, 2004, 38(1): 35-44. DOI: 10.1023/B:VLSI.0000028532.53893.82
[11]	Arici Tarik, Dikbas Salih, Altunbasak Yucel. A histogram modification framework and its application for image contrast enhancement[J]. IEEE Transactions on Image Processing, 2009, 18(9): 1921-1935. DOI: 10.1109/TIP.2009.2021548
[12]	Kim Wonkyun, You Jongmin, Jeong Jechang. Contrast enhancement using histogram equalization based on logarithmic mapping[J]. Optical Engineering, 2012, 51(6): 067002-1-067002-10. DOI: 10.1117/1.OE.51.6.067002
[13]	WU T, TANG S, ZHANG R, et al. CGNet: a light-weight context guided network for semantic segmentation[J]. IEEE Transactions on Image Processing, 2021, 30: 1169-1179. DOI: 10.1109/TIP.2020.3042065
[14]	Badrinarayanan Vijay, Kendall Alex, Cipolla Roberto. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615
[15]	CHEN Soong-Der, Ramli Abd Rahman. Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation[J]. IEEE Trans. Consumer Electronics, 2003, 49(4): 1301-1309. DOI: 10.1109/TCE.2003.1261233
[16]	刘程威. 红外成像系统架构及图像处理关键技术研究[D]. 南京: 南京理工大学, 2020. LIU Chengwei. Research on Infrared Imaging System Architecture and Image Processing Key Technologies[D]. Nanjing: Nanjing University of Science and Technology, 2020.
[17]	Pizer S M, Amburn E P, Austin J D, et al. Adaptive histogram equalization and its variations[J]. Computer Vision, Graphics, and Image Processing, 1987, 39(3): 355-368. DOI: 10.1016/S0734-189X(87)80186-X
[18]	LI C, XIA W, YAN Y, et al. Segmenting objects in day and night: edge-conditioned CNN for thermal image semantic segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(7): 3069-3082.
[19]	Silva Eric A, Panetta Karen, Agaian Sos S. Quantifying image similarity using measure of enhancement by entropy[J]. Mobile Multimedia/Image Processing for Military and Security Applications, 2007, 6579: 65790U-65790U-12. DOI: 10.1117/12.720087
[20]	ZHANG Yujin. Handbook of Image Engineering[M]. Singapore: Springer, 2021.
[21]	GAO Ce, YUN Lijun, WANG Kun, et al. Infrared image enhancement method based on discrete stationary wavelet transform and CLAHE[C]//IEEE International Conference on Computer Science and Educational Informatization(IEEE CSEI 2019), 2019: 52: DOI: 10.1109/CSEI47661.2019.8938871.
[22]	Gonzalez R C, Woods R E. Digital Image Processing[M]. Singapore: Pearson Prentice Hall, 2002: 75-215.
[23]	KIM Yeong-Taeg. Contrast enhancement using brightness preserving bi-histogram equalization[J]. IEEE Transactions on Consumer Electronics, 1997, 43(1): 1-8. DOI: 10.1109/30.580378
[24]	Turgay Celik. Two-dimensional histogram equalization and contrast enhancement[J]. Pattern Recognition, 2012, 45(10): 3810-3824. DOI: 10.1016/j.patcog.2012.03.019