【作者】 周洋；

【导师】 蒋刚毅；

【作者基本信息】 宁波大学 ， 通信与信息系统， 2013， 博士

【摘要】 三维视频能提供真实感和交互性，可应用在自由视点视频、三维电视、沉浸式视频会议及视频监控等系统中。然而，通过易错信道传输三维视频数据会不可避免地出现帧丢失或部分片丢失，因此如何设计有效的差错控制技术是三维视频系统设计中最受关注的研究内容之一。本文重点研究如何利用双/多视点视频中存在的视点间相关性、同一视点内时域和空域相关性，并结合三维视频图像的内容特征进行高效的三维视频错误隐藏。本论文的主要工作如下：(1)提出了基于块活力度的立体视频错误隐藏方法。该方法设计了时域运动活力度、时域运动补偿活力度、视点间视差活力度和视点间视差补偿活力度四个参数来表征立体视频同一视点图像间的运动和纹理复杂度以及同一时刻左右视点图像间的视差和纹理复杂度。在错误隐藏过程中，算法根据时域运动活力度的大小来选择恰当的运动补偿法进行运动补偿预测块的错误掩盖；根据视差活力度的大小来选择恰当的视差补偿法进行视差补偿块的错误掩盖，从而使错误隐藏算法能根据视频图像的内容灵活地调整错误隐藏策略，在隐藏质量和算法复杂度之间达到了较好的平衡。(2)提出了基于图像区域的立体视频错误隐藏方法。根据视频编码中丢失块周围各宏块预测模式类型的不同，将图像块划分成平滑区域、规则运动区域、以及不规则运动或纹理丰富区域，然后分别采用直接拷贝、四分之一像素运动补偿和视差补偿、权重可调的双向插值法对不同的出错区域进行错误掩盖。实验结果表明修复的图像与原始图像具有良好的相似度，特别是对图像中的不规则运动或边缘纹理区域也取得了良好的主观质量，解决了传统错误隐藏算法不能有效地修复图像中存在复杂运动或边缘纹理丰富区域的问题。(3)提出了一种基于相关性分析的双目视频B帧整帧丢失分级错误隐藏方法。该方法面向分层B帧(Hierarchical B Picture, HBP)预测结构下的立体视频流，与先前方法的区别主要如下：1)对B帧的重要性等级进行划分，将丢失的B帧分为核心帧丢失、次核心帧丢失和普通帧丢失，避免了对解码中起不同作用的丢失B帧采用单一方法进行差错掩盖的缺陷；2)对多视点视频序列的运动矢量和视差矢量的相关性进行了分析，并将经视差补偿后的相应块的运动矢量作为丢失块的运动矢量估计值，提高了运动矢量估计的准确性；3)利用图像运动特征和分析得到的相关性大小对丢失帧进行区域划分，根据其区域特征采用不同的错误隐藏策略进行错误修复，提高了修复后图像的总体质量。此外，针对核心B帧丢失，提出了一种基于帧差相关性的立体视频错误隐藏方法。(4)提出了一个多视点视频分级错误隐藏技术框架。该框架可以对多视点视频各视点各帧图像的重要性等级进行划分，并对不同等级的出错图像采用不同的错误隐藏方法进行差错掩盖。针对基于HBP预测结构的多视频视频中的I帧出错，提出了一种基于区域划分的错误隐藏方法。该方法中利用时域帧差信息对丢失块进行区域划分，对帧差值小的平滑丢失块采用直接拷贝法，对帧差值大的运动剧烈或边缘信息丰富的丢失块采用多方向插值法。实验结果表明该方法较好地保留了I帧图像的边缘信息，相较于H.264校验模型中的双线性插值法，在宏块丢失率为20%的条件下，对于第0视点和第2视点序列最高能达到1.76dB和1.77dB的平均峰值信噪比(PSNR)增益。(5)提出了一种高容错立体视频编码方案，改善了立体视频流传输的抗差错性能。更多还原

【Abstract】 Three Dimensional (3D) video that can deliver real and immersive experiences will be appliedin free-viewpoint video,3DTV, immersive teleconference, surveillance systems, et al. However,transmission of3D coded video over error-prone channels can lead to the loss of a few slices orframes. How to develop a robust error control technique is one of the most concerns of designingthe3D video systems.For these reasons, this thesis mainly focused on exploiting the spatial-temporal correlation andinter-view correlation of the double/multi-view sequences, and considering the content of picturesto design efficient error concealment (EC) techniques for3D video transmission. In summary, themain works of this thesis are listed as follows:(1) A block-activity based EC algorithm for stereoscopic video is proposed. We design fourparameters which are named temporal motion activity (TCA), motion compensated temporalactivity, inter-view disparity activity (IDA) and disparity compensated inter-view activity todescribe the motion and texture complexity of the adjacent frames at the same view, and thedisparity and texture complexity of the inter-view frames at the same time, respectively.According to the TCA, our method can choose appropriate motion compensated operation toconceal the lost motion compensated prediction (MCP) block. Also, According to the TCA,our method can choose appropriate disparity compensated operation to conceal the lostdisparity compensated prediction (DCP) block. So the proposed algorithm can convenientlychooses conceal approach according to the content of picture and makes the better tradeoffbetween the quality of concealed picture and the complexity of algorithm.(2) A region-based EC method for stereoscopic video is proposed. Based on the statistical analysisof the encoding modes of the surrounding macroblocks (MBs) and of the spatiallycorresponding MB in the neighboring frame, our method classifies the lost MBs into threecategories:(1) smooth MBs,(2) regular motion MBs and (3) irregular motion MBs. Followingthe classification, corresponding operations, including direct replacement, quarter-pixel motionand disparity compensation, and bilateral error concealment with adjustable weights, areapplied to reconstruct the smooth MBs, regular motion MBs and irregular motion MBs,respectively. The experimental result show that the similarity of concealed picture and originalpicture is high, especially, our method also make better subjective quality for irregular motionor texture region of the picture and solve the problem of the previous error concealment methods can not restore the lost irregular motion or texture region efficiently.(3) Based on the analysis of the correlation, a scalable EC algorithm for the whole B-frame loss indouble-view coded video is proposed. Our scheme focuses on stereoscopic video stream withHBP predicted structure and differs in several aspects from the prior work. First, according tothe model of frame important level, our scheme divides the lost frames into three types:1) thebest key frame;2) key frame;3) ordinary frame, and use different methods to different lostframe. It avoids the drawback of using the same method to conceal the different B frameswhich have different importance in the decoding process. Second, the proposed algorithmanalyses the correlation of motion vector (MV) and disparity vector (DV) in the double-viewvideo sequence, and selectively chooses the MV of corresponding block which is compensatedby disparity as the MV of the lost block. It improves the accurate of the estimated MV of thelost block. Third, the proposed algorithm divides the picture into different regions by utilizingthe object motion characteristic of the picture and the correlation which is acquired by thecorrelation analysis, and uses different approach to conceal different lost region. It improvesthe whole quality of the restore picture. Moreover, we introduce a frame-difference based ECalgorithm for key B-frame loss.(4) A scalable EC framework for multiview video coding is proposed. With this framework, wecan compute the importance level of frames of the different view in multiview video codingand use different approaches to conceal error pictures of different important levels. We proposea region based I-frame EC approach for multiview video coding with HBP prediction structrue.By utilizing the difference of frames at the same view, the proposed method first divides thelost MBs into smooth MBs and motion MBs, the former belonged to region of small framedifference and the latter belonged to region of large frame difference. Then, the lost smoothblocks are restored by direct copy and the lost motion blocks are recovered by the orientationinterpolation. Experimental result shows that the proposed method preserves the edge areas ofthe lost blocks efficiently, and offers significant average peak signal noise gain (up to1.76dBand1.77dB for the0-view sequence and2-view sequence respectively at the MB loss rate of20%) compared to the bilinear interpolation (BI) method which is employed in the H.264joint model (JM).(5) A robust stereoscopic video coding scheme is proposed to improve the performance of errorresilience.更多还原