【作者】 杨秀红；

【导师】 郭宝龙；

【作者基本信息】 西安电子科技大学 ， 测试计量技术及仪器， 2014， 博士

【摘要】 数字图像修复是指采用计算机程序或软件对图像中信息完全丢失的区域进行自动修正的一系列图像处理技术，其本质是根据不完全信息重建完全信息，并保证修复痕迹不易被察觉。该技术的主要目标是恢复破损的绘画/照片以及移除/替换选定的对象。另外，该技术在图像插值、图像放大及超分辨、无线传输中的错误隐藏等方面具有很高的应用价值。经过十多年的发展，数字图像修复技术已经形成了一套比较完整的理论体系，其研究主要包括：针对小面积破损的变分泛函\PDE(Partial Differential Equation，偏微分方程)扩散修复模型、针对大面积破损的基于样本的修补方法和针对小波域信息丢失的小波域图像修复模型等。尽管这些方法都取得了长足的发展，得到了较好的修复效果，但仍存在一些问题亟需解决，比如：利用变分泛函\PDE模型修复小尺度图像细节或进行纹理修复、在小波域修复模型中自适应地控制图像的几何正则性、补全大面积破损的图像，等等。针对以上问题，本文以结构张量为主线，结合其他新兴的理论、方法，围绕数字图像修复技术进行了一系列的研究。结构张量作为一种有效的图像分析工具，在图像处理和计算机视觉等领域中起着关键和重要的作用。本文将结构张量与其他理论、方法相结合，针对不同的应用需求，提出了效果更优的图像修复方法。论文主要工作及贡献如下：1.深入探讨了数字图像修复技术的基本原理与研究现状，对主要的修复方法进行了归类总结，并指出了各自的适用范围及优缺点；研究了结构张量和张量扩散的基本原理，详细分析了张量扩散所具有的各向异性及其在扩散过程中能够保留图像结构连续性的优良性能。在此基础上，系统研究了结构张量的总体框架，重点对其应用领域做了归纳分析，着重指出了可将结构张量和张量扩散应用于图像修复的不同方面，为后续研究奠定了基础。2.针对现有小波域图像修复模型在自适应正则和噪声抑制等方面存在的不足，提出了一种基于张量扩散的小波域图像修复模型(TDWI)。该混合模型将结构自适应各向异性正则与小波表示结合起来，在像素域中通过张量扩散来控制图像的几何正则性，而在小波域中修复丢失和被损坏的小波系数。同时，依据变分法推导该能量泛函对应的Euler-Lagrange方程，并据此分析TDWI模型的几何正则性能。由于在其正则项中采用了矩阵表示的结构张量，使得其扩散核的形状能够根据图像的局部特征自适应地变化，包括尖锐边缘、角及各向同性区域，因此TDWI模型能够更加自适应和准确地控制像素域的几何正则性，并对噪声具有更强的鲁棒性。最后，针对所建立的混合模型，采用了一种更加有效的迭代解法，进而给出了TDWI模型的数值实现方案。实验结果表明，针对一系列不同的丢失情况，TDWI模型具有更好的修复效果和更高的抗噪性能。3.针对传统整数阶扩散PDE在处理小尺度图像细节及结构特征方面存在的不足，根据噪声是否存在的不同情况，提出了两种基于分数阶张量扩散的数字图像修复模型(FTDII)。FTDII模型将分数阶微积分与张量扩散相结合，既继承了张量扩散的各向异性，又因分数阶微积分的特性而能更好地处理小尺度图像特征。同时，依据分数阶泛函理论，推导两种变分模型对应的Euler-Lagrange方程。在数值实现过程中，利用移位的Grümwald-Letnikov(the Shifted G-L)分数阶微积分定义，推导分数阶梯度在x+、x-、y+和y-四个方向的离散模板，进而根据推得的Euler-Lagrange方程设计出所提模型的数值算法。对各种测试图像的仿真结果表明，所提的两种FTDII模型对图像细节及结构特征具有更好的修复效果，所得修复结果的视觉质量优于传统的张量扩散修复模型。4.针对传统结构张量在提取图像低层(low-level)纹理特征方面存在的不足，提出了一种基于改进的非局部结构张量的纹理修复算法，以修复小面积破损的结构性纹理图像。根据结构性纹理所具有的特性，改进传统的结构张量。首先，我们采用分数阶结构张量(Fractional-order Structure Tensor，FST)取代传统的整数阶结构张量，从而能够更好地处理复杂的类分形纹理细节。其次，为了避免混叠现象导致的小尺度纹理细节信息的丢失，FST的采样率必须加倍。从频域分析的角度对其原因进行了解释说明，并采用移位的G-L定义推导计算整点处和半点处的分数阶导数值，以实现采样率加倍的操作。第三，针对纹理的非局部特性，利用张量数据的自相似冗余信息对过采样的FST(Oversampled FST，OFST)进行非局部滤波。最后，将改进的结构张量(Nonlocal regularized OFST，NOFST)插入各向异性PDE中，并设计出利用该PDE进行纹理修复的数值实现方案。实验结果表明，改进的结构张量能够很好地提取图像的低层纹理细节和结构特征；将其插入PDE中能够相当有效地修复小面积破损的结构性纹理图像。5.针对现有的基于样本的图像修复方法在搜索匹配过程中存在的不足，提出了一种结构测度约束下的基于加权分形的数字图像修复算法。首先，对选定的定义域块进行几何变换和同构变换，构造码本，利用图像的自相似性来“丰富”搜索匹配范围；然后，计算各向异性非线性结构张量，得到局部结构测度，据此构造已知点的归一化权系数；第三，在亮度变换过程中，为待修复块与码本块之间的误差能量函数引入两类约束条件，并通过最小化约束能量函数，导出新的亮度变换参数。引入的这两类约束为：一是待修复块与码本块在已知像素点上的加权一致性约束，权重为得到的归一化权系数；二是待修复块的邻域块与码本块在丢失像素点上的相似性约束。最后，采用约束能量最小的估计块来填补待修复块。实验表明，该方法能够很好地补全破损的几何结构，并使得新填充区域与源区域保持很好的一致性，其修复结果的主观质量和客观评价指标都得到了显著提高。更多还原

【Abstract】 Digital image inpainting technology refers to automatically modifying a damagedimage, portions of whose information is lost or corrupted completely, by using computerprogram or software. Its essence is to rebuild complete information based on incompleteone in a non-detectable way. The technology mainly aims at retouching the damageddrawings/photographs and removing the unwished targets. In addition, the technologyhas found broad applications in image super-resolution and zooming, image interpo-lation, error concealment in wireless transmission, and so on.After several years’ development, digital image inpainting technology has formed arelatively complete system. It includes variational functional or PDE (Partial Differen-tial Equation) based inpainting models which are suitable to inpaint small-sizedscratches, exemplar-based completion methods which are very good at completing thelarge objects and wavelet inpainting models which are targeting at the case of waveletinformation loss. Although they have gained great developments, there exist someproblems which need to be solved, for example, inpainting small-scale details or textureimages by using variational functional or PDE based models, adaptively controlling thegeometric regularity in wavelet inpainting model, completing large holes, etc.According to the above issues, the dissertation mainly does researches on digitalimage inpainting taking the structure tensor (ST) as the main line and incorporatingother newly emerging theories or methods. ST is an effective tool for image analysisand plays a key and important role in image processing and computer vision. Accordingto different repair requirements, the dissertation proposes better image inpaintingalgorithms by combining ST with other theories or methods. The main research workand contributions are as follows.1. In-depth investigation is conducted on the basic principle and research statusof digital image inpainting. The main inpainting methods are classified andsummarized, and the existing problems are expounded. The basic principles ofST and TD are studied and their anisotropism and good performance ofretaining the continuity of image structures in the diffusion process areanalysed in detail. Based on the above, the overall framework of ST isestablished systematically, which outlines its theory and applications in imageprocessing and specifically highlights its application in image inpainting, thusestablishes research foundations for the later chapters.2. The abilities of the existing wavelet inpainting models for adaptive regularization and restraining noise are poor. To solve the problem, a newwavelet inpainting model based on tensor diffusion (TDWI) is proposed. Thehybrid model is built by combining structure-adaptive anisotropicregularization with wavelet representation, which controls the geometricregularity in pixel domain while restores the missing or damaged waveletcoefficients in wavelet domain. Its associated Euler-Lagrange equation isdeduced to analyze its regularity performance. Due to using matrix-valued STin the regularization term, the shape of diffusion kernel changes adaptivelyaccording to the image features, including sharp edges, corners andhomogeneous regions. Therefore, the TDWI model controls the geometricregularity in the pixel domain more adaptively and accurately and gives betterrobustness to noise. In addition, for the established hybrid model, an effectiveand proper iterative numerical solution is applied to improve the computation,and then the numerical scheme for the TDWI model is given. Experimentalresults on a variety of loss scenarios are given to demonstrate the advantagesof our proposed model.3. Fine scale features cannot be recovered satisfactorily by the traditionalinteger-order inpainting models. In order to solve this problem, depending onwhether or not noise needs to be eliminated in the image, two novel imageinpainting models based on fractional-order tensor diffusion (FTDII) arepresented. The proposed models integrates fractional calculus with tensordiffusion, not only inheriting the anisotropism of tensor diffusion, but alsodealing with better image details owing to the characteristics of fractionalcalculus. Meanwhile, according to the fractional functional theory, theirassociated Euler-Lagrange equations are deduced. In the procedure ofnumerical implementation, the discrete templates of fractional derivative infour directions are deduced according to the shifted Grümwald-Letnikov (theShifted G-L) fractional calculas definition. And then, according to the derivedEuler-Lagrange equations, a numerical scheme for the FTDII models is given.According to the experimental results on various testing images, the proposedFTDII models demonstrate superior inpainting performance to the originalinteger-order ones based on classic calculus.4. The ability of the traditional ST of extracting low-level texture feature is poor,focusing on which a texture inpainting algorithm based on an improved nonlocal ST is proposed to repair structural texture image. According to thecharacteristics of this kind of image, an improved ST with three modificationsis put forward. Firstly, fractional-order structure tensor (FST) is employed toreplace the traditional ST to deal with better complex fractal-like texturedetails. Secondly, in order to avoid aliasing, the sampling rate of FST must bedoubled. The reason is expounded from the perspective of frequency-domainanalysis and the operation is implemented by computing the fractional-orderderivative image at both integer and half-integer positions according to theshifted G-L definition. Thirdly, in consideration of the nonlocal property, anonlocal filtering is performed on the resulting oversampled FST (OFST) byutilizing the redundant infromation of the tensor data. Lastly, the resultingnonlocal filtered OFST (NOFST) is inserted into the anisotropic PDE whosenumerical implementation scheme is given to perform texture inpainting. Theexperimental results show that, for structural texture images, the improved STperforms particularly well in extracting low-level texture and structure featuresand the proposed inpainting algorithm is not only quite efficient in recoveringnon-homogeneous structures, but particularly efficient in the restitution of thetexture regions.5. In order to overcome the defects of the existing exemplar-based methods in theprocedure of searching and matching, an image completion algorithm based onweighted fractal under the structure-measurement constraint is presented.Firstly, the selected domain blocks were transformed by geometricaltransformation and isomorphic transformation. Using the resulting blocks, thecodebook was constructed and then served as the ‘enriched’ searching scope.Secondly, by computing the anisotropic nonlinear ST (NLST), the localstructure measurement was obtained, according to which the normalizedweight map was constructed. Thirdly, during the luminance transformation, itstwo parameters are derived through minimizing a constrained energy functionbetween the target patch and each codebook patch. In the constrained energyfunction, two types of constraints are introduced: one is the weightedconsistency constraint between the codebook patch and the target patch overthe already known pixels where the weight patch is obtained from the weightmap, the other is the neighborhood similarity constraint between the codebookpatch and the weighted mean of the neighboring patches over the missing pixels. Lastly, the target patch is filled in using the estimated patch with theminimum constrained energy. The experimental results show that comparedwith the existing congeneric algorithms, the proposed one preserves better thecontinuity of the broken structure and forces the newly filled area to be moreconsistent with the source area. Therefore, the restored results are improvedboth subjectively and objectively.更多还原