基于Harris图像拼接的全景视频稳像算法

王正家; 吴春林; 柯黎明; 陈文重; 刘城逍

基于Harris图像拼接的全景视频稳像算法

王正家^{1, 2,},
吴春林^{1, 2, ,},
柯黎明¹,
陈文重^{1, 2},
刘城逍¹

1.
湖北工业大学现代制造质量工程重点实验室, 湖北武汉 430068
2.
华中科技大学数字制造装备与技术国家重点实验室, 湖北武汉 430068

基金项目:

国家自然科学基金 51275158

详细信息

作者简介:
王正家（1970-），男，副教授，硕士生导师，主要从事电机智能控制、机器视觉等方法研究。E-mail：276318532@qq.com

通讯作者:
吴春林（1997-），男，硕士研究生，主要从事嵌入式图像处理、机器视觉等方面的研究。E-mail：1047601643@qq.com

中图分类号: TP391.4
计量
- 文章访问数: 77
- HTML全文浏览量: 24
- PDF下载量: 29
- 被引次数: 0
出版历程
- 收稿日期: 2021-12-08
- 修回日期: 2022-05-11
- 刊出日期: 2022-11-20

Panoramic Video Image Stabilization Algorithm Based on Harris Image Stitching

WANG Zhengjia^{1, 2
,},
WU Chunlin^{1, 2
, ,},
KE Liming¹,
CHEN Wenzhong^{1, 2},
LIU Chengxiao¹

1.
Key Laboratory of Modern Manufacturing Quality Engineering, Hubei University of Technology, Wuhan 430068, China
2.
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430068, China

摘要

摘要: 对于背景变换和抖动分量比较小的视频序列，传统稳像算法不能直接适用，本文提出一种基于Harris图像拼接的全景稳像算法。首先采用Prewitt算子提取出图像的边缘信息，在此基础上进行分区的Harris特征点检测；然后结合NCC（normalized cross correlation）算法与RANSAC（random sample consensus）算法实现图像间的特征点精确匹配，接着利用加权平均融合的方法进行图像融合；最后对融合后的全景图像进行剪裁，完成图像补偿，输出稳定的视频序列。实验结果表明：改进的Harris算法提高了算法效率以及正确特征点数量，并且本文稳像算法实时性较好，能够有效消除视频抖动并输出稳定的视频序列。
- 图像拼接 /
- 全景稳像算法 /
- Harris特征点检测 /
- 图像补偿
Abstract: Traditional image stabilization algorithms cannot be directly applicable for video sequences with small background transformation and jitter components. In this study, a panoramic image stabilization algorithm based on the Harris image mosaic is proposed. First, the Prewitt operator was used to extract the edge information of the image, and then, Harris feature points were detected. Subsequently, both NCC and RANSAC algorithms are combined to achieve the exact matching of feature points between images, and the weighted average fusion method is used for image fusion. Finally, the fused panoramic image is clipped to complete image compensation and output a stable video sequence. The experimental results show that the improved Harris algorithm improves the algorithm efficiency and number of correct feature points. Moreover, the image stabilization algorithm in this study demonstrated good real-time performance, which can effectively eliminate video jitter and output stable video sequences.
- image stitching /
- panoramic image stabilization algorithm /
- Harris feature point detection /
- image compensation

HTML全文

图 1 图像拼接算法流程

Figure 1. Image Mosaic algorithm process

下载: 全尺寸图片幻灯片

图 2 实验效果对比图

Figure 2. Comparison of experimental results

下载: 全尺寸图片幻灯片

图 3 传统Harris角点检测结果

Figure 3. Traditional Harris corner detection results

下载: 全尺寸图片幻灯片

图 4 本文算法检测结果

Figure 4. The algorithm of this article detection results

下载: 全尺寸图片幻灯片

图 5 待拼接图像序列

Figure 5. The sequence of images to be stitched

下载: 全尺寸图片幻灯片

图 6 拼接后实验结果

Figure 6. Results of the experiment after splicing

下载: 全尺寸图片幻灯片

图 7 三种稳像算法实验结果对比

Figure 7. Comparison of experimental results of three image stabilization algorithms

下载: 全尺寸图片幻灯片

表 1 特征点提取数量及时间

Table 1. Number and time of feature point extraction

Algorithm	Corner points	Time/s
Traditional Harris	1728	0.641
This paper Harris	1465	0.593

下载: 导出CSV

表 2 车辆行驶情况下电子稳像实验对比结果

Table 2. Comparison results of electronic stabilization experiments under the driving condition of the vehicle

Class of algorithm	Count	PSNR	SSIM	SFPT/ms	FPS
The original video	1	12.156	0.472	16.67	60
	2	11.875	0.468	16.67	60
	3	12.314	0.461	16.67	60
	Average	12.115	0.467	16.67	60
Yin^[8] algorithm	1	15.412	0.697	66.67	15
	2	15.348	0.713	71.43	14
	3	15.335	0.691	90.91	11
	Average	15.365	0.700	76.34	13.33
Hamza^[9] algorithm	1	15.132	0.682	62.50	16
	2	14.983	0.675	50.00	20
	3	15.015	0.651	55.56	18
	Average	15.043	0.669	56.02	18
Our algorithm	1	15.623	0.711	31.25	32
	2	15.512	0.724	28.57	35
	3	15.231	0.708	33.33	30
	Average	15.455	0.714	31.05	32.33

下载: 导出CSV

参考文献(9)

[1]	蔡成涛, 梁小龙, 谭吉来, 等. 自适应最优块匹配视频稳像算法[J]. 系统工程与电子技术, 2013, 35(6): 1324-1329. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201306035.htm CAI Chengtao, LIANG Xiaolong, TAN Jilai, et al. Adaptive optimal block matching video stabilization algorithm[J]. Systems Engineering and Electronics, 2013, 35(6): 1324-1329. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYD201306035.htm
[2]	YANG Weiping, ZHANG Zhilong, ZHANG Yan, et al. Real-time digital image stabilization based on regional field image gray projection[J]. Journal of Systems Engineering and Electronics, 2016, 27(1): 224-231.
[3]	胡明昊, 任明武, 杨静宇. 一种快速实用的特征点匹配算法[J]. 计算机工程, 2004(9): 31-33. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC200409012.htm HU Minghao, REN Mingwu, YANG Jingyu. A Fast and practical feature point matching algorithm[J]. Computer Engineering, 2004(9): 31-33. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC200409012.htm
[4]	蒋双霜, 朱振峰, 常冬霞. 利用全景图的视频稳像算法[J]. 信号处理, 2018, 34(7): 852-859. https://www.cnki.com.cn/Article/CJFDTOTAL-XXCN201807012.htm JIANG Shuangshuang, ZHU Zhenfeng, CHANG Dongxia. Video stabilization algorithm using panorama[J]. Journal of Signal Processing, 2018, 34(7): 852-859. https://www.cnki.com.cn/Article/CJFDTOTAL-XXCN201807012.htm
[5]	吴威, 许廷发, 王亚伟, 等. 高精度全景补偿电子稳像[J]. 中国光学, 2013, 6(3): 378-385. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGA201303015.htm WU Wei, XU Tingfa, WANG Yawei, et al. High precision panoramic compensation electronic stabilization[J]. Chinese Optics, 2013, 6(3): 378-385. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGA201303015.htm
[6]	樊佩琦, 吴诚, 杨瑞宇, 等. 一种基于BRISK改进的SURF红外图像拼接方法[J]. 红外技术, 2018, 40(1): 27-33. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201801006.htm FAN Peiqi, WU Cheng, YANG Ruiyu, et al. A SURF infrared image stitching method based on BRISK[J]. Infrared Technology, 2018, 40(1): 27-33. https://www.cnki.com.cn/Article/CJFDTOTAL-HWJS201801006.htm
[7]	李竹林. Cohen-Sutherland直线剪裁算法改进[J]. 计算机技术与发展, 2017, 27(6): 32-35. https://www.cnki.com.cn/Article/CJFDTOTAL-WJFZ201706008.htm LI Zhulin. Improvement of cohen-sutherland linear cutting algorithm[J]. Computer Technology and Development, 2017, 27(6): 32-35. https://www.cnki.com.cn/Article/CJFDTOTAL-WJFZ201706008.htm
[8]	尹丽华, 李范鸣, 刘士建. 基于区域分割与融合的全景稳像算法[J]. 红外与激光工程, 2018, 47(9): 391-400. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201809056.htm YIN Lihua, LI Fanming, LIU Shijian. Panoramic image stabilization algorithm based on region segmentation and fusion[J]. Infrared and Laser Engineering, 2018, 47(9): 391-400. https://www.cnki.com.cn/Article/CJFDTOTAL-HWYJ201809056.htm
[9]	Hamza A, Hafiz R, Khan M M, et al. Stabilization of panoramic videos from mobile multi-camera platforms[J]. Image and Vision Computing, 2015, 37: 20-30.