Visual Quality and Perceptual Coding

Video Coding/Screen Video Coding with Quality Assessment

Author: Sudeng Hu and C.-C. Jay Kuo

Due to rapidly growing video applications in areas such as wireless display and cloud computing , screen content coding has received much interest from academia and industry in recent years .. The High Efficiency Video Coding (HEVC) standard has achieved significant improvement in coding efficiency as compared with the state-of-the-art H.264/AVC standard. However, HEVC has been designed mainly for natural video captured by cameras. Screen content images and video, also known as compound images, hybrid images, and mixed-raster content material, typically contains computer-generated content such as text and graphics, sometimes in combination with natural or camera-captured material. Since the properties of screen content are quite different from those of natural content, and HEVC currently does not exploit these properties, there is still room for improvement in coding efficiency.

For screen content, it is our observation that directly encoding residual signals in the spatial domain may not be efficient enough. This is because, except for the edge, the remaining areas are still smooth and can be coded more effectively with a transform. In this paper, we propose a new scheme, called Edge Mode (EM), to encode these kinds of blocks. Based on the intra prediction direction, six possible edge positions inside a block are defined, and one of them will be selected via rate-distortion (RD) optimization. To reduce the encoding complexity, the proposed scheme can be further simplified by classifying intra modes into four categories. Then, MXN 2D DCT transforms or non-orthogonal 2D transforms are performed separately in sub-blocks. Finally, the new edge mode is integrated into HEVC to result in a more powerful coding scheme.

[1] Sudeng Hu, Lei Deng, and C.-C. Jay Kuo “A New Distortion/Content-Dependent Video Quality Index (DCVQI),” [...]

By |November 21st, 2013|Visual Quality and Perceptual Coding|Comments Off on Video Coding/Screen Video Coding with Quality Assessment|

Development of the objective metric for 3D Image Quality Assessment

Author: Hyunsuk Ko and C.-C. Jay Kuo

A new framework for objective quality evaluation of stereoscopic image pairs is the goal of this research. Quality assessment of stereoscopic image pairs is more complicated than that for 2D images since it is a multi-dimensional problem where the quality is affected by distortion types as well as the relation between the left and right views (e.g. different levels of distortion in two views). In this work, a new formula-based metric was introduced to provide results better than state-of-the art methods. However, the formula-based metric still has its limitation. For further improvement, we further propose a 2-stage fusion model. That is, we classify distortion types into groups and design a set of scorer to handle them separately. In Stage 1, each scorer generates its own score. In Stage 2, all intermediate scores are fused to predict the final quality index with nonlinear regression. Experimental results demonstrate that the proposed quality index outperforms several existing quality assessment methods by a significant margin over different databases.

Although research on advanced image/video quality index methods that are more consistent with the human visual experience has made a substantial amount of progress in the last decade, the study on 3D image/video quality is still in its early stage. The 3D quality assessment (QA) is a difficult problem since it is affected by 2D image quality, depth perception and visual discomfort such as eye strain or dizziness. It is particularly challenging when the stereoscopic image pair consists of two views with different quality levels (called asymmetric distortion). To address this problem, we need a deeper understanding of the human visual system (HVS), e.g. the binocular combination in stereovision, to build a robust index that [...]

By |November 21st, 2013|Visual Quality and Perceptual Coding|Comments Off on Development of the objective metric for 3D Image Quality Assessment|

Image/video quality assessment

Author: Yuchieh Lin, Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo
This is a joint work during my internship in Mediatek in summer 2013.

Peak signal-to-noise ratio (PSNR) has been widely used to assess the quality of distorted images or videos with respect to their original ones for a long history. Human visual experience is affected by several psycho-visual factors, but PSNR does not take these factors into account. Thus, image and video quality metrics (IQM and VQM) [1] are proposed to emulate perceptional visual quality.

For image quality assessment, we developed a framework [2] to integrate visual saliency model to existing IQMs, such as SSIM and FSIM. For video quality assessment, we are working on building a new video database. Through this database, we would develop an algorithm for video quality assessment

So far, image or video quality metrics are not extensively applied to practical usage. One important reason is that the performance and complexity of no-reference algorithms are still not satisfied. We want to develop and employ no-reference IQM and VQM to enhance existing applications. We are facing a trade-off between performance and complexity.

[1] Tsung-Jung Liu, Yu-Chieh Lin, Weisi Lin and C.-C. Jay Kuo, “Visual quality assessment: recent developments, coding applications and future trends,” APSIPA Transactions on Signal and Information Processing, 2013
[2] Joe Yuchieh Lin, Tsung Jung Liu, Weisi Lin and C.-C. Jay Kuo, “Visual-saliencyenhanced image quality assessment indices,” APSIPA Annual Summit and Conference, Kao-Hsiung, Taiwan, Oct. 29-Nov. 1, 2013.

By |November 21st, 2013|Visual Quality and Perceptual Coding|Comments Off on Image/video quality assessment|

Image Quality Assessment Using Multi-Method Fusion

Author: Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo

A new methodology for objective image quality assessment (IQA) with multi-method fusion (MMF) is presented. The research is motivated by the observation that there is no single method that can give the best performance in all situations. To achieve MMF, we adopt a regression approach. The new MMF score is set to be the nonlinear combination of scores from multiple methods with suitable weights obtained by a training process. In order to improve the regression results further, we divide distorted images into three to five groups based on the distortion types and perform regression within each group, which is called “context-dependent MMF” (CD-MMF). One task in CD-MMF is to determine the context automatically, which is achieved by a machine learning approach. To further reduce the complexity of MMF, we perform algorithms to select a small subset from the candidate method set. The result is very good even if only three quality assessment methods are included in the fusion process. The proposed MMF method using support vector regression is shown to outperform a large number of existing IQA methods by a significant margin when being tested in six representative databases.

Although the proposed MMF has excellent performance, one issue concerning context classification for CD-MMF needs to be resolved in the future. Since one image may consist of multiple distortion types, the strict classification of images into one specific context may lead to the wrong context category, and then affect the subsequent quality prediction. One possible and better solution to overcome this shortcoming is to use unsupervised classification for context determination. Another alternative is to attach beliefs to the classification of the context and weight the corresponding regressed predicted [...]

By |November 21st, 2013|Visual Quality and Perceptual Coding|Comments Off on Image Quality Assessment Using Multi-Method Fusion|