室外环境下红外热图像内眼角定位

孙磊; 陈树越; 戚亚明

室外环境下红外热图像内眼角定位

常州大学微电子与控制工程学院, 江苏常州 213164

基金项目:

江苏省科技厅社会发展项目 BE2018638

详细信息

作者简介:
孙磊（1995-），男，硕士，机器视觉、模式识别，E-mail：19081203594@smail.cczu.edu.cn

通讯作者:
陈树越（1963-），男，教授，计算机视觉、图像处理，E-mail：csyue2000@163.com

中图分类号: TP391.41
计量
- 文章访问数: 100
- HTML全文浏览量: 36
- PDF下载量: 19
- 被引次数: 0
出版历程
- 收稿日期: 2021-07-06
- 修回日期: 2021-08-20
- 刊出日期: 2022-10-20

Inner-Canthus Localization in Infrared Thermal Images in Outdoor Environments

School of Microelectronics and Control Engineering, Changzhou University, Changzhou 213164, China

摘要

摘要: 针对室外环境下红外热图像中目标区域受背景过热与周围环境影响，导致目标边界模糊、噪声大等问题，提出了一种室外环境下红外热图像内眼角定位算法。该算法首先对采集的图像进行面部倾斜校正，接着采用Gentle-AdaBoost与HAAR特征相结合进行人脸、人眼粗定位，并引入几何校正对眼睛区域精确定位，最终依据内眼角区域特性提出区域精化与区域生长分割相结合的内眼角定位。在3种不同的红外热图像数据集以及自主采集不同季节的温度区间室外的数据集上进行实验。结果表明：在室外环境下，所提出的方法可有效地定位内眼角，人眼定位准确率达到98.1%，内眼角定位准确率达到97.7%。
- 室外环境 /
- 红外热图像 /
- 内眼角定位 /
- 特征提取 /
- 区域精化
Abstract: Target areas in infrared thermal images in outdoor environments are affected by background overheating and the surrounding environment, causing fuzzy target boundaries and large noise. An inner-canthus location algorithm for infrared thermal images in outdoor environments is proposed to solve this problem. First, the algorithm corrects the facial tilt of collected images. Then, Gentle-Adaboost and Haar features are combined to perform approximate localization of human faces and eyes, and geometric correction is applied to accurately locate the eye region. Finally, based on the characteristics of the inner-canthus region, an inner-canthus location is proposed by combining region refinement and region growth segmentation. Experiments are conducted on three different infrared thermal image datasets and outdoor datasets independently collected at different temperature ranges in different seasons. The results show that the proposed method can effectively locate the inner canthus in the outdoor environment, and the accuracy for human eyes and inner-canthus can reach 98.1% and 97.7%, respectively.
- outdoor environment /
- infrared thermal image /
- inner-canthus location /
- feature extraction /
- regional refinement

HTML全文

图 1 内眼角定位流程

Figure 1. Flow chart of positioning of inner-canthus

下载: 全尺寸图片幻灯片

图 2 双前沿演化过程，从左到右：(a) 初始轮廓；(b) 形态学膨胀；(c) 新分割区域

Figure 2. Double frontier evolution process from left to right: (a) initial contour; (b) morphological expansion; (c) newly segmented region

下载: 全尺寸图片幻灯片

图 3 HAAR级联分类器筛查过程

Figure 3. HAAR cascading classifier screening process

下载: 全尺寸图片幻灯片

图 4 矩阵特征其中(a)、(b)边缘特征；(d)、(f)、(g)线性特征；(c)对角线特征；(e)中心环绕特征

Figure 4. Matrix features Where (a), (b) Edge features; (d), (f), (g) Linear features; (c) Diagonal features; (e) Center surround feature

下载: 全尺寸图片幻灯片

图 5 样本示例：(a) 0℃~10℃区间拍摄；(b) 10℃~20℃区间拍摄；(c) 20℃~30℃区间拍摄

Figure 5. Sample diagram (a) Shooting at 0℃-10℃; (b) Shooting at 10℃-20℃; (c) Shooting at 20℃-30℃

下载: 全尺寸图片幻灯片

图 6 实验效果图(a)、(b)为E40采集的原图像，(c)、(d)为其校正后的图像

Figure 6. Experimental renderings (a) and (b) are the original images collected by E40, and (c) and (d) are the corrected images

下载: 全尺寸图片幻灯片

图 7 实验效果图（从左往右，从上往下）(a) TTF数据集图；(b) 人脸、人眼定位；(c) 几何校正；(d) 内眼角定位；(e) FLIR E40采集图；(f) 人脸、人眼定位；(g) 几何校正；(h) 内眼角定位

Figure 7. Experimental renderings from left to right, top to bottom (a) TTF data set; (b) Face and human eye positioning; (c) Geometric correction; (d) Inner corner of the eye positioning; (e) FLIR E40 acquisition map; (f) Face and human eye positioning; (g) Geometric correction; (h) Inner corner of the eye positioning

下载: 全尺寸图片幻灯片

表 1 HAAR级联分类器训练参数

Table 1. HAAR cascading classifier training parameters

Region	Size of sample	Feature	Detection rate	Fall-out ratio	Number
Eye	19×19	4	0.999	0.5	63960
Eye	12×16	7	0.999	0.5	23408

下载: 导出CSV

表 2 测试与训练数据的基本信息

Table 2. Basic information of test and train data

Dataset	Number/sheet	Resolution	Filming spot
Train-OTC	22419	320×240	-
Train-Sci	3000	1024×768	-
Test-TTF	1400	336×256	-
Test-E40	1000	320×240	School gate

下载: 导出CSV

表 3 感兴趣区域定位平均准确率

Table 3. Average accuracy of region of interest location

Region	Test	Number/sheet	ACC/%
Face	OTC	400	96.8
Face	TTF	400	100
Eye	TTF	400	LE 97.5
	TTF	400	RE 97.3
	E40 collected	400	LE 94.2
	E40 collected	400	RE 93.4
Inner-Canthus	TTF	400	LIC 100
	TTF	400	RIC 100
	E40 collected	400	LIC 98.7
	E40 collected	-	RIC 98.2

下载: 导出CSV

表 4 面部倾斜校正的结果

Table 4. Results of facial tilt correction

Data set	Number/sheet	Correct revise	Mistake revise
TTF	400	396	4
E40	400	397	3

下载: 导出CSV

表 5 几何校正前后检测结果对比

Table 5. Comparison of test results before and after

Model	Eyes	Number of tests	False detection
Origin	800	1178	401
Geometric correction	800	786	9

下载: 导出CSV

表 6 本文方法与基本算法定位效果对比

Table 6. Comparison of positioning results with the basic algorithm

Algorithm	Eyes	Inner-Canthus ACC/%	t/ms
HAAR(4 species)+RG	59.1	62.0	72.1
HAAR(7 species)+RG	69.0	66.5	91.1
HAAR(7 species)+ geometric correction +RG	98.1	67.4	228.3
HAAR(7 species)+ regional refinement +RG	68.7	97.7	321.5
Proposed	98.1	97.7	238.7

下载: 导出CSV

表 7 5种算法定位效果对比

Table 7. Comparison of positioning effect of 5 algorithms

Algorithm	Data set	Eyes AACC/%	Inner-Canthus AACC/%	t/ms
HAAR+RG	TTF	85.0	74.8	166.9
HAAR+RG	E40	69.0	66.5	91.1
TS+FP	45 thermal images of complex places	80.0	100	-
RHT+RG	125 thermal images of faces with the up-right position	L 97.1	L 97.5	-
RHT+RG	125 thermal images of faces with the up-right position	R 97.1	R 97.5	-
Neural Network+RG	Thermal images included 198 thermograms which invloving 128 subjects	88.0	77.4	70.4
Proposed	TTF	L 97.5	L 100	230.1
	TTF	R 97.3	R 100	230.1
	E40	L 94.2	L 98.7	238.7
	E40	R 93.4	R.98.2	238.7

下载: 导出CSV

参考文献(23)

[1]	陈红强. 非接触式红外热成像测温技术的探析[J]. 中国安防, 2020(5): 35-40. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGAF202005007.htm CHEN Hongqiang. Analysis of non-contact infrared thermal imaging temperature measurement technology [J]. China Security, 2020(5): 35-40. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGAF202005007.htm
[2]	吕事桂, 刘学业. 红外热像检测技术的发展和研究现状[J]. 红外技术, 2018, 40(3): 214-219. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201803003.htm LV Shigui, LIU Xuexue. Development and research status of infrared thermography detection technology[J]. Infrared Technology, 2018, 40(3): 214-219. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201803003.htm
[3]	Nguyen A V, Cohen N J, Lipman H, et al. Comparison of 3 infrared thermal detection systems and self-report for mass fever screening[J]. Emerging Infectious Diseases, 2010, 16(11): 1710. doi: 10.3201/eid1611.100703
[4]	STELLA A B, MANGANOTTI P, FURLANIS G, et al. Return to school in the COVID-19 era: considerations for temperature measurement[J]. Journal of Medical Engineering & Technology, 2020, 44(8): 468-441.
[5]	MERCER J B, RING E F J. Fever screening and infrared thermal imaging: concerns and guidelines[J]. Thermology International, 2009, 19(3): 67-69.
[6]	IX-IXO. Medical Electrical Equipment - Part 2-59: Particular Requirements for Basic Safety and Essential Performance of Screening Thermographs for Human Febrile Temperature Screening: IEC 80601-2-59 [S]. [2008-12-02].
[7]	US-ASTM. Standard Practice for Minimum Resolvable Temperature Differ Ence for Thermal Imaging Systems: ASTM1213-14 [S]. [2018-11-01].
[8]	Friedrich G, Yeshurun Y. Seeing people in the dark: Face recognition in infrared images[C]//International Workshop on Biologically Motivated Computer Vision, 2002: 348-359.
[9]	BUDZAN S, WYŻGOLIK R. Face and eyes localization algorithm in thermal images for temperature measurement of the inner canthus of the eyes[J]. Infrared Physics and Technology, 2013, 60: 225-234. doi: 10.1016/j.infrared.2013.05.007
[10]	BERG A. Detection and Tracking in Thermal Infrared Imagery[M]. Linköping University Electronic Press, 2016.
[11]	CHENNA Y N D, GHASSEMI P, PFEFER T J, et al. Free-form deformation approach for registration of visible and infrared facial images in fever screening[J]. Sensors, 2018, 18(1): 125.
[12]	VIOLA P, JONES M J. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2): 137-54. doi: 10.1023/B:VISI.0000013087.49260.fb
[13]	Martinez B, Binefa X, Pantic M. Facial component detection in thermal imagery[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-workshops, 2010: 48-54.
[14]	WONG W K, LIM H S, BINTI ISHAK N I N, et al. Fuzzy logic based viola Jones fever detection method in thermal imaging system[C]//Fuzzy Logic Based Viola Jones Fever Detection Method in Thermal Imaging System, ICETCSIT-2013, 2013: 23-26.
[15]	Michael Kass, Andrew Witkin, Demetri Terzopoulos. Snakes: active contour models[J]. International Journal of Computer Vision, 1988, 1(4): 321-331. doi: 10.1007/BF00133570
[16]	Cohen L, Kimmel R. Global minimum for active contour models: a minimal path approach[J]. International Journal of Computer Vision, 1997, 24(1): 57-78. doi: 10.1023/A:1007922224810
[17]	KOPACZKA M, NESTLER J, MERHOF D. Face detection in thermal infrared images: a comparison of algorithm-and machine -learning-based approaches[C]//Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, 2017: 518-529.
[18]	FRIEDMAN J, HASTIE T, TIBSHIRANI R. Additive logistic regression: a statistical view of boosting: rejoinder [J]. The Annals of Statistics, 2000, 28(2): 337-407.
[19]	LEVINE M D, NAZIF A M. Dynamic measurement of computer generated image segmentations [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1985(2): 155-64.
[20]	BAILEY D G, JOHNSTON C T. Single pass connected components analysis[C]// Proceedings of Image and Vision Computing, 2007: 282-287.
[21]	Fitriyah H, Rachmadi A, Setyawan G E. Automatic measurement of human body temperature on thermal image using knowledge -based criteria[J]. Journal of Information Technology and Computer Science, 2017, 2(2): DOI: 10.25126/JITECS.20172235.
[22]	FITRIYAH H, WIDASARI E R, PUTRI R R M. Inner-Canthus localization of thermal images in face-view invariant[J]. International Journal on Advanced Science, Engineering and Information Technology, 2018, 8(6): 2570-2576.
[23]	Marzec M, Lamża A, Wróbel Z, et al. Fast eye localization from thermal images using neural networks[J]. Multimedia Tools and Applications, 2016: 1-14.