【作者】 肖建平；

【导师】 邹雪城；

【作者基本信息】 华中科技大学 ， 微电子学与固体电子学， 2006， 博士

【摘要】 本文首先介绍图像超分辨处理的研究目的和意义,图像超分辨率算法的研究现状,在此基础上着重研究了适应于不同应用场合的单帧图像超分辨图像算法及多帧图像超分辨率算法的数学原理,验证方法,评价手段及软硬件实现方法。尽管单帧图像插值算法在原理上分析是一个典型的病态问题,但在源图像信噪比高、放大比例在一定范围的情况下仍可满足实际应用的需求,尤其是在实时应用条件下或图像处理芯片设计中仍被广泛采用。文章回顾了传统的线性插值方法,包括理想插值、最近邻插值、双线性插值、四点双三次插值、六点双三次插值的数学表示方法及各自的频率特性。针对线性插值函数中低次插值方法获得的超分辨图像效果不理想、高次插值方法复杂度高、以及基于统计理论的单帧图像超分辨率算法不便于硬件实现这些特点,提出了一种算法复杂度低于双三次插值算法,而处理效果优于双三次插值算法的自适应牛顿插值算法,对其进行验证。多帧图像超分辨率算法利用序列图像之间包含相似但不完全相同的互补图像信息重构高分辨率图像,通常包含频域超分辨率重构及空域超分辨率图像重构。空域重建算法将插值、迭代、滤波和重采样放在一起处理,可以采用更广泛的观测模型。为此,本文提出了一种空域多帧图像超分辨率处理算法——基于Delaunay三角剖分的多帧图像超分辨率算法。该算法在空间不规则采样点的Delaunay三角剖分的基础上,对各顶点进行梯度估计,将每一个三角块区域采用双变量多项式进行拟合,获得一个连续且连续可微的曲面,然后对拟合曲面重采样,可获得任意比例的高分辨率图像。为了能有效的验证多帧图像超分辨率算法的有效性,设计了一种可由理想高分辨率图像产生低分辨率图像序列的相机运动模型,这样可以获得低分辨图像序列在高分辨网格上的配准参数,不必考虑影响低分辨率图像配准的各种因素,使多帧图像超分辨率算法的验证更具针对性。通过以上方法产生了几组不同的低分辨率图像序列,进行了如下对比实验:具有相同帧数但分辨率不同的低分辨率源图像序列的超分辨率重构;具有相同分辨率但采用不同帧数的低分辨率序列图像的超分辨重构。实验结果表明,低分辨率源序列图像的质量对超分辨率结果将产生关键性的影响;参与运算的序列图像的帧数越多,超分辨处理所得的高分辨图像视觉效果越好,但随着帧数的增加,视觉效果改善并不明显。图像质量评价作为图像处理系统及图像处理算法的性能预测的重要手段,分为主观评价方法及客观评价方法。主观评价方法可操作性差,常在实验室采用。基于误差统计的图像质量客观评价传统方法如PSNR和MSE目前被广泛应用于各种因素引起的退化图像的质量评价,某些情况下,其判断结果基本与图像主观视觉效果一致。但在某些特定条件下,这类基于误差统计的评价方法对图像质量的评价完全失效。为此研究了一种基于图像结构的质量评价方法,并对PSNR判断失效的几个实验结果采用基于图像结构的图像质量方法进行验证,获得了与图像主观质量一致的评价结果。图像超分辨率的非实时处理已不能满足实际应用的需求,超分辨率处理的专用芯片应用也日益广泛。限于目前的研究基础,本文没有对多帧图像超分辨率的硬件实现进行深入研究。对于单帧图像超分辨算法的硬件实现,以LCD定标器为研究背景,在分析平板显示技术的研究现状和发展趋势的基础上,设计了LCD定标器缩放引擎的系统结构,分析了LCD定标器的时序约束关系,并提出了双线性插值算法及自适应牛顿插值算法的硬件实现结构。然后对两种结构分别用Verilog HDL进行描述,在相关软件下进行仿真。为了降低了ASIC设计的风险,通常需要对设计的系统进行FPGA验证。确定适合于LCD定标器验证的合适的数据源,提出了相应的数据格式转换方案,设计了用于LCD定标器验证的评估板,然后对综合好的目标代码下载至评估板的FPGA中,验证整个系统的功能。更多还原

【Abstract】 In this paper, the motive, significance and status of the research on image super-resolution processing are introduced at first, upon which, several single-frame and multi-frame image super-resolution algorithms, adapting for different applications, are proposed. Then fundamental theories in mathematics, verification methods and evaluation criterion of the algorithms are developed in turn. Finally, implementations of the algorithms in software and hardware are designed.Although the single-frame image super-resolution processing is a typical ill-posed problem in principle, it still meets the requirements of the applications, in which the source image has high signal-noise ratio and small scaling up ratio. Especially it is widely adopted in real-time image enlargement and the ASIC (Application Specific Integrated Circuit) design for image processing. In this thesis, some traditional linear interpolation algorithms and their frequency characteristics are reviewed, including ideal interpolation, nearest interpolation, bilinear interpolation, four-point and six-point bicubic interpolations. Much research has pointed out that the lower order interpolations achieve unpleasant visual effects, the higher order interpolations have higher complexity, and the algorithms based on statistics and set theories are unfit for implementation by hardware. To avoid the shortcomings of these methods, an adaptive interpolation algorithm based on Newton polynomial, with lower complexity and more pleasant visual effects than those of the bicubic interpolation, is proposed and verified.By multi-frame image super-resolution algorithms, the similar but incompletely uniform information contained in sequential images is used to restore a high resolution image. These algorithms are usually classified into the processing in frequency and spatial domain respectively. Since spatial domain restoration simultaneously involves interpolation, iteration, filtering and re-sampling etc., it can be described by more comprehensive imaging model. Therefore, a multi-frame image super-resolution algorithm based on Delaunay triangulation in spatial domain is proposed. Spatial irregular samplings points are triangulated by Delaunay method, then the gradient values on these vertexes are computed, and each triangular area is fitted into a continuous and continuously differentiable curved surface. High resolution image with discretional enlarged ratio is achieved by re-sampling the curved surface.To verifiy the validity of multi-frame image super-resolution algorithms, a camera motion model is designed. According to the model, a group of low resolution images are generated from ideal high resolution images. Thus the registration parameters of low resolution images are obtained on high resolution grid, neglecting other influencing factors on image registration. So the verification is more precise. The following contrast experiments have been conducted by several groups of low resolution images generated by the above method. Firstly, two groups of low resolution images are restored with the same frame number and different resolutions. Secondly, two other groups with the same resolution and different frame numbers are restored. The experimental results show that the quality of the source low resolution sequential images has decisive effect on restoration of high resolution images, and on the other hand, more source images are involved, more pleasant visual effect can be achieved in high resolution image restoration. However, exceeding certain frame number, the visual effect is improved slightly.Image quality assessment is an important means to evaluate the performance of image processing systems and algorithms. It involves image quality subjective assessment and objective assessment. The objective assessment is usually adopted in lab for its defective practicability. At present the classical objective assessment parameters, such as PSNR (Peak Signal-Noise Ratio) and MSE (Mean Square Error) based on error statistics, are widely adopted to assess the quality of degenerated images. In some cases, the evaluation results are coincident with results of the subjective assessment. However, objective assessment is ineffective in many applications. Therefore, a novel objective assessment method based on image structure information is proposed. Several images are assessed by PSNR and useless results are gained, but the assessment results by the method based on image structure information is in accordance with the subjective assessment as expected.The non-real-time processing of super-resolution images can not meet the requirements of actual applications, and the needs of ASICs for super-resolution image processing is increasing. Due to the limited research foundation, the hardware implementation of multi-frame image super-resolution has not been further researched, while for hardware implementation of single-frame image super-resolution, the following research has been done based on LCD Scaler. Firstly, by analyzing the research in quo and development trend of panel display technology, system architecture of LCD Scaler is designed. Then timing constrains of the Scaler are deduced, and the hardware structure of implementing the bilinear interpolation and the adaptive Newton interpolation algorithms is devised. The two algorithms are both implemented in Verilog HDL, and functional simulations are performed.In order to reduce the risk and cost of ASIC design, FPGA verification is usually adopted. The appropriate data as the system input are selected, and the corresponding scheme for data format conversion is proposed. Then the object code is downloaded into FPGA on the evaluation board. Finally, the whole system function is verified by FPGA.更多还原