Inner-Canthus Localization in Infrared Thermal Images in Outdoor Environments
-
摘要: 针对室外环境下红外热图像中目标区域受背景过热与周围环境影响,导致目标边界模糊、噪声大等问题,提出了一种室外环境下红外热图像内眼角定位算法。该算法首先对采集的图像进行面部倾斜校正,接着采用Gentle-AdaBoost与HAAR特征相结合进行人脸、人眼粗定位,并引入几何校正对眼睛区域精确定位,最终依据内眼角区域特性提出区域精化与区域生长分割相结合的内眼角定位。在3种不同的红外热图像数据集以及自主采集不同季节的温度区间室外的数据集上进行实验。结果表明:在室外环境下,所提出的方法可有效地定位内眼角,人眼定位准确率达到98.1%,内眼角定位准确率达到97.7%。Abstract: Target areas in infrared thermal images in outdoor environments are affected by background overheating and the surrounding environment, causing fuzzy target boundaries and large noise. An inner-canthus location algorithm for infrared thermal images in outdoor environments is proposed to solve this problem. First, the algorithm corrects the facial tilt of collected images. Then, Gentle-Adaboost and Haar features are combined to perform approximate localization of human faces and eyes, and geometric correction is applied to accurately locate the eye region. Finally, based on the characteristics of the inner-canthus region, an inner-canthus location is proposed by combining region refinement and region growth segmentation. Experiments are conducted on three different infrared thermal image datasets and outdoor datasets independently collected at different temperature ranges in different seasons. The results show that the proposed method can effectively locate the inner canthus in the outdoor environment, and the accuracy for human eyes and inner-canthus can reach 98.1% and 97.7%, respectively.
-
图 7 实验效果图(从左往右,从上往下)(a) TTF数据集图;(b) 人脸、人眼定位;(c) 几何校正;(d) 内眼角定位;(e) FLIR E40采集图;(f) 人脸、人眼定位;(g) 几何校正;(h) 内眼角定位
Figure 7. Experimental renderings from left to right, top to bottom (a) TTF data set; (b) Face and human eye positioning; (c) Geometric correction; (d) Inner corner of the eye positioning; (e) FLIR E40 acquisition map; (f) Face and human eye positioning; (g) Geometric correction; (h) Inner corner of the eye positioning
表 1 HAAR级联分类器训练参数
Table 1. HAAR cascading classifier training parameters
Region Size of sample Feature Detection rate Fall-out ratio Number Eye 19×19 4 0.999 0.5 63960 Eye 12×16 7 0.999 0.5 23408 表 2 测试与训练数据的基本信息
Table 2. Basic information of test and train data
Dataset Number/sheet Resolution Filming spot Train-OTC 22419 320×240 - Train-Sci 3000 1024×768 - Test-TTF 1400 336×256 - Test-E40 1000 320×240 School gate 表 3 感兴趣区域定位平均准确率
Table 3. Average accuracy of region of interest location
Region Test Number/sheet ACC/% Face OTC 400 96.8 TTF 400 100 Eye TTF 400 LE 97.5 RE 97.3 E40 collected 400 LE 94.2 RE 93.4 Inner-Canthus TTF 400 LIC 100 RIC 100 E40 collected 400 LIC 98.7 - RIC 98.2 表 4 面部倾斜校正的结果
Table 4. Results of facial tilt correction
Data set Number/sheet Correct revise Mistake revise TTF 400 396 4 E40 400 397 3 表 5 几何校正前后检测结果对比
Table 5. Comparison of test results before and after
Model Eyes Number of tests False detection Origin 800 1178 401 Geometric correction 800 786 9 表 6 本文方法与基本算法定位效果对比
Table 6. Comparison of positioning results with the basic algorithm
Algorithm Eyes Inner-Canthus ACC/% t/ms HAAR(4 species)+RG 59.1 62.0 72.1 HAAR(7 species)+RG 69.0 66.5 91.1 HAAR(7 species)+ geometric correction +RG 98.1 67.4 228.3 HAAR(7 species)+ regional refinement +RG 68.7 97.7 321.5 Proposed 98.1 97.7 238.7 表 7 5种算法定位效果对比
Table 7. Comparison of positioning effect of 5 algorithms
Algorithm Data set Eyes AACC/% Inner-Canthus AACC/% t/ms HAAR+RG TTF 85.0 74.8 166.9 E40 69.0 66.5 91.1 TS+FP 45 thermal images of complex places 80.0 100 - RHT+RG 125 thermal images of faces with the up-right position L 97.1 L 97.5 - R 97.1 R 97.5 Neural Network+RG Thermal images included 198 thermograms which invloving 128 subjects 88.0 77.4 70.4 Proposed TTF L 97.5 L 100 230.1 R 97.3 R 100 E40 L 94.2 L 98.7 238.7 R 93.4 R.98.2 -
[1] 陈红强. 非接触式红外热成像测温技术的探析[J]. 中国安防, 2020(5): 35-40. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGAF202005007.htmCHEN Hongqiang. Analysis of non-contact infrared thermal imaging temperature measurement technology [J]. China Security, 2020(5): 35-40. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGAF202005007.htm [2] 吕事桂, 刘学业. 红外热像检测技术的发展和研究现状[J]. 红外技术, 2018, 40(3): 214-219. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201803003.htmLV Shigui, LIU Xuexue. Development and research status of infrared thermography detection technology[J]. Infrared Technology, 2018, 40(3): 214-219. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201803003.htm [3] Nguyen A V, Cohen N J, Lipman H, et al. Comparison of 3 infrared thermal detection systems and self-report for mass fever screening[J]. Emerging Infectious Diseases, 2010, 16(11): 1710. doi: 10.3201/eid1611.100703 [4] STELLA A B, MANGANOTTI P, FURLANIS G, et al. Return to school in the COVID-19 era: considerations for temperature measurement[J]. Journal of Medical Engineering & Technology, 2020, 44(8): 468-441. [5] MERCER J B, RING E F J. Fever screening and infrared thermal imaging: concerns and guidelines[J]. Thermology International, 2009, 19(3): 67-69. [6] IX-IXO. Medical Electrical Equipment - Part 2-59: Particular Requirements for Basic Safety and Essential Performance of Screening Thermographs for Human Febrile Temperature Screening: IEC 80601-2-59 [S]. [2008-12-02]. [7] US-ASTM. Standard Practice for Minimum Resolvable Temperature Differ Ence for Thermal Imaging Systems: ASTM1213-14 [S]. [2018-11-01]. [8] Friedrich G, Yeshurun Y. Seeing people in the dark: Face recognition in infrared images[C]//International Workshop on Biologically Motivated Computer Vision, 2002: 348-359. [9] BUDZAN S, WYŻGOLIK R. Face and eyes localization algorithm in thermal images for temperature measurement of the inner canthus of the eyes[J]. Infrared Physics and Technology, 2013, 60: 225-234. doi: 10.1016/j.infrared.2013.05.007 [10] BERG A. Detection and Tracking in Thermal Infrared Imagery[M]. Linköping University Electronic Press, 2016. [11] CHENNA Y N D, GHASSEMI P, PFEFER T J, et al. Free-form deformation approach for registration of visible and infrared facial images in fever screening[J]. Sensors, 2018, 18(1): 125. [12] VIOLA P, JONES M J. Robust real-time face detection[J]. International Journal of Computer Vision, 2004, 57(2): 137-54. doi: 10.1023/B:VISI.0000013087.49260.fb [13] Martinez B, Binefa X, Pantic M. Facial component detection in thermal imagery[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-workshops, 2010: 48-54. [14] WONG W K, LIM H S, BINTI ISHAK N I N, et al. Fuzzy logic based viola Jones fever detection method in thermal imaging system[C]//Fuzzy Logic Based Viola Jones Fever Detection Method in Thermal Imaging System, ICETCSIT-2013, 2013: 23-26. [15] Michael Kass, Andrew Witkin, Demetri Terzopoulos. Snakes: active contour models[J]. International Journal of Computer Vision, 1988, 1(4): 321-331. doi: 10.1007/BF00133570 [16] Cohen L, Kimmel R. Global minimum for active contour models: a minimal path approach[J]. International Journal of Computer Vision, 1997, 24(1): 57-78. doi: 10.1023/A:1007922224810 [17] KOPACZKA M, NESTLER J, MERHOF D. Face detection in thermal infrared images: a comparison of algorithm-and machine -learning-based approaches[C]//Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, 2017: 518-529. [18] FRIEDMAN J, HASTIE T, TIBSHIRANI R. Additive logistic regression: a statistical view of boosting: rejoinder [J]. The Annals of Statistics, 2000, 28(2): 337-407. [19] LEVINE M D, NAZIF A M. Dynamic measurement of computer generated image segmentations [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1985(2): 155-64. [20] BAILEY D G, JOHNSTON C T. Single pass connected components analysis[C]// Proceedings of Image and Vision Computing, 2007: 282-287. [21] Fitriyah H, Rachmadi A, Setyawan G E. Automatic measurement of human body temperature on thermal image using knowledge -based criteria[J]. Journal of Information Technology and Computer Science, 2017, 2(2): DOI: 10.25126/JITECS.20172235. [22] FITRIYAH H, WIDASARI E R, PUTRI R R M. Inner-Canthus localization of thermal images in face-view invariant[J]. International Journal on Advanced Science, Engineering and Information Technology, 2018, 8(6): 2570-2576. [23] Marzec M, Lamża A, Wróbel Z, et al. Fast eye localization from thermal images using neural networks[J]. Multimedia Tools and Applications, 2016: 1-14.