【作者】 赵汉理；

【导师】 金小刚；

【作者基本信息】 浙江大学 ， 计算机科学与技术， 2009， 博士

【摘要】 抽象化艺术反映了人类丰富的想象力,在人类文明发展过程中占有举足轻重的地位。如今,卡通效果已经被广泛应用于电影、电视、游戏、网络、广告、科普示图以及医学成像等诸多领域。中国拥有世界上最大的动漫产业消费市场以及在线聊天群体,被抽象化之后的可视信息往往能够更加吸引人眼的注意力,是艺术家和观看者之间信息交流的桥梁。图像和视频抽象化是把真实的照片或录制好的视频转换成非真实感效果的重要技术,因而对这方面的研究不仅具有重要的理论意义,而且还具有现实的应用价值。本文主要研究图像和视频抽象化中的若干关键技术,具体包括基于颜色的图像风格迁移、基于特征流场的流线感抽象化、基于形状简化的抽象化、基于视觉感知的非均匀抽象化以及基于可编程图形硬件的加速优化算法。1.我们提出了一种实时的基于颜色的图像风格迁移方法。首先,我们将输入图像从RGB颜色空间转换到oRGB颜色空间。该oRGB颜色空间能够充分地分离出彼此独立的白-黑、红-绿以及黄-蓝等三个颜色通道。然后,我们利用基于统计分析的颜色校正技术对两个彩色通道进行迁移。同时,我们运用直方图匹配算法对亮度通道进行迁移。我们还可以对颜色迁移后的图像进行抽象化以增强视觉效果。最后,我们把oRGB颜色空间转换回RGB空间得到最终的处理图片。当将一幅目标图像的颜色风格迁移到一段视频时,由于目标图像的颜色统计值以及亮度直方图都是不变的,我们只需要对相关信息计算一次来减少处理时间。2.我们提出了一个实时的基于特征流场的流线感图像和视频抽象化框架。首先,我们将输入图像从RGB颜色空间转换到CIE Lab颜色空间或oRGB颜色空间并只对其中的亮度通道进行后续操作。接下来,我们用迭代双边滤波的方法逐渐构造出一个光滑、连贯且特征保持的边缘切向流场。利用这个特征流场,低对比度区域用基于流场的双边滤波器进行平滑操作,而高对比度区域用基于流场的高斯差分滤波器进行进一步加深。然后我们再将抽象化后的图像用软量子化方法使其进一步卡通化并改进其帧连续性。最后,我们将结果转换回RGB颜色空间得到最终抽象化图像。3.我们采用一个更加准确的视觉感知模型,并提出一个实时的非均匀图像和视频抽象化框架。为了同时减少空间和时间的视频噪音,我们将时间视为第三维,对视频应用一次三维的双边滤波。然后我们导出一张光滑的兴趣区域函数图,该兴趣区域函数图基于一个视觉注意力模型,能很好地反应人眼的注意力。最后,我们再利用这张兴趣区域函数图来指导自动的非均匀抽象化。4.我们提出了一个实时的基于形状简化的图像抽象化框架,该框架能同步地简化图像中的形状和颜色信息。我们迭代地对边缘切向曲线进行双边滤波以得到一个光滑连贯的特征流场。该特征流场指示了输入图像的显著特征方向。然后我们用受特征流场约束的平均曲率流滤波器来迭代地简化和收缩图像的整体形状。最后,我们用Shock滤波器以保护重要的边界信息。为了直观地控制抽象化的程度,我们可以迭代地和渐进地应用上述滤波过程。由于颜色信息包括红、绿、蓝三个彩色通道,为了得到较好的颜色简化效果,我们对该三个彩色通道独立地进行抽象化处理。5.本文所提出的各图像和视频抽象化算法都是专门为图形硬件的并行处理特性而设计的。因此,我们的方法具有高度的可并行性,能够在可编程图形硬件上实时实现。另外,本文的抽象化系统无需任何人工交互,初级用户也能方便地制作出各种生动的卡通效果。最后,自动且实时的抽象化技术使我们还能方便地对在线视频或图像进行实时处理。更多还原

【Abstract】 The art of abstraction reflects the abundant imagination of human beings and plays an important role in the development of human civilization. Nowadays, cartoon effects have been widely applied in a variety of areas including movie, television, game, network, advertisement, scientific illustration, and medical imaging. China has the biggest consumer market in the animation industry and the biggest online chatting group. The simplified or even exaggerate visual information after abstraction can often improve the human perception and plays as a communication bridge between artists and viewers. The image and video abstraction is an important technique to convert real photographs or recorded videos to non-photorealistic styles. Consequently the research on it has not only the theoretical significance but also the realistic application value. This paper mainly focuses on some key issues on the image and video abstraction, specifically, color-based photo style transfer, feature flow-based image and video abstraction, shape-simplifying image abstraction, perception-based progressive image and video abstraction, and GPU-based optimization algorithms.1. We propose a real-time color-based photo style transfer method. We first transform the input image from RGB color space to oRGB color space. The oRGB supports independent manipulation of the luminance, pure yellow-blue, and pure red-green color channels. Then the statistical color correction technique is performed on the two chrominance channels. We also apply the histogram matching on the luminance channel. We optionally add an abstraction effect on it to improve the visual perception. At last, the result image is produced by transforming the image back to the RGB color space. When transferring a target image’s style to a video, since the target image is unchanged, we can compute its statistical information only once to reduce the processing time.2. We present a real-time feature flow-based image and video abstraction framework. First the input image is converted from RGB color space to CIE-Lab or oRGB color space. Next we iteratively construct a smooth, coherent, and feature-preserved edge tangent flow field using a bilateral filter. Using this feature flow, low contrast regions are smoothed with the flow-based bilateral filter, while high contrast regions are further strengthened with the flow-based difference-of-Gaussian filter. Then the soft luminance quantization is adopted to further enhance the cartoon-like effect with good temporal coherence. Lastly, the output image is generated by converting the result back to RGB space.3. We adopt a more elaborated visual perception model and introduce a real-time progressive image and video abstraction framework. In order to reduce both spatial and temporal video noises, we view time as the third dimension and apply a 3D bilateral filter on the input video first. Then we derive a smooth regions-of-interest function based on a visual saliency model, which effectively describes the human attention. Lastly, we use the regions-of-interest function to control our automatic progressive image and video abstraction.4. We design a real-time framework for shape-simplifying image abstraction, which simultaneously simplifies both the shape and color information in the input image. We iteratively apply the bilateral filter on edge tangent curves to obtain a smooth and coherent feature flow field, which indicates the salient feature directions of the image. Then we iteratively simplify and shrink the whole shape of the image using the feature-flow-constrained mean curvature flow. Finally, we protect important shape edges with the Shock filter. In order to intuitively control the abstraction level, we can iteratively and progressively apply the above filtering process. Since the color includes red, green, and blue channels, to obtain better color simplifying effects, we perform the abstraction on these three chrominance channels independently.5. All the above image and video abstraction algorithms proposed are designed to suit for the parallel processing characteristic of graphics hardware. Therefore, our methods are highly parallel, enabling real-time implementation on the programmable graphics processing unit. Additionally, our abstraction systems do not require any user interaction and even naive users can easily generate various vivid cartoon effects. At last, the automatic and real-time abstraction techniques make them convenient to process online images and videos.更多还原