Author: Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo

Research Problem

A new methodology for objective image quality assessment (IQA) with multi-method fusion (MMF) is presented. The research is motivated by the observation that there is no single method that can give the best performance in all situations. To achieve MMF, we adopt a regression approach. The new MMF score is set to be the nonlinear combination of scores from multiple methods with suitable weights obtained by a training process. In order to improve the regression results further, we divide distorted images into three to five groups based on the distortion types and perform regression within each group, which is called “context-dependent MMF” (CD-MMF). One task in CD-MMF is to determine the context automatically, which is achieved by a machine learning approach. To further reduce the complexity of MMF, we perform algorithms to select a small subset from the candidate method set. The result is very good even if only three quality assessment methods are included in the fusion process. The proposed MMF method using support vector regression is shown to outperform a large number of existing IQA methods by a significant margin when being tested in six representative databases.

Future Challenges

Although the proposed MMF has excellent performance, one issue concerning context classification for CD-MMF needs to be resolved in the future. Since one image may consist of multiple distortion types, the strict classification of images into one specific context may lead to the wrong context category, and then affect the subsequent quality prediction. One possible and better solution to overcome this shortcoming is to use unsupervised classification for context determination. Another alternative is to attach beliefs to the classification of the context and weight the corresponding regressed predicted quality indices. The two approaches mentioned above should be able to help to further improve the overall performance of MMF.

References

  • [1] Tsung-Jung Liu, Yu-Chieh Lin, Weisi Lin, and C.-C. Jay Kuo, “Visual Quality Assessment: Recent Developments, Coding Applications and Future Trends,” APSIPA Transactions on Signal and Information Processing, vol. 2, e4, July 2013.
  • [2] Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo, “Image quality assessment using multi-method fusion,” IEEE Transactions on Image Processing, vol. 22, no. 5, pp. 1793-1807, May 2013.
  • [3] Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo, “A fusion approach to video quality assessment based on temporal decomposition,” Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Dec. 2012.
  • [4] Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo, “Recent Developments and Future Trends in Visual Quality Assessment,” Proc. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Oct. 2011.
  • [5] Tsung-Jung Liu, Weisi Lin, and C.-C. Jay Kuo, “A multi-metric fusion approach to visual quality assessment,” Proc. IEEE the 3rd International Workshop on Quality of Multimedia Experience (QoMEX), Sep. 2011.