News

We are so happy to welcome a new MCL member, Youngrae Kim joining MCL this semester. Here is a quick interview with Youngrae: 1. Could you briefly introduce yourself and your research interests?

My name is Youngrae Kim, and I am a first-year PhD student in Computer Engineering at USC. Before starting my PhD, I completed my undergraduate studies at Hongik University and earned my master’s degree at the Korea Advanced Institute of Science and Technology (KAIST). My research interests lie in computer vision, machine learning, and green learning. Outside of academics, I enjoy hiking, working out, and traveling.

What is your impression of MCL and USC?

For me, MCL is an impressive research institute where there’s a strong sense of trust, not only in academic collaborations but also on a personal level. The community is welcoming, and I’ve felt a great sense of support from everyone here. I’m excited to be advised by Professor Kuo, one of the leading researchers in computer vision. The atmosphere at USC is also fantastic—people seem genuinely happy, enjoying life while working hard. I’m very glad to have joined USC, particularly MCL.

What are your future expectations and plans in MCL?

During my PhD, I plan to focus on Federated Learning for the medical domain. I expect to encounter various challenges in applying this technology to real-world scenarios, and I aim to address these through my research, ultimately proposing a comprehensive thesis that has practical applications. I also look forward to collaborating with peers in different areas, which I believe will broaden my academic perspective.

Congratulations to Zhanxuan Mei for passing his defense. Zhanxuan’s thesis is titled “Explainable and Lightweight Techniques for Blind Visual Quality Assessment and Saliency Detection.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Ulrich Neumann (Outside Member). The Committee members praised the quality of Zhanxuan’s work very much. The MCL News team invited Zhanxuan for a short talk on his thesis and PhD experience. Here is the summary. We thank Zhanxuan for his kind sharing and wish him all the best on his next journey. A high-level abstract of Zhanxuan’s thesis is given below:

Thesis:

Explainable and Lightweight Techniques for Blind Visual Quality Assessment and Saliency Detection

The thesis contains four main research topics:

We begin by presenting our proposed GreenBIQA method, a novel approach to BIQA characterized by its compact model size, low computational complexity, and high performance. Building on the foundation of GreenBIQA, we extend its application to BVQA through the development of GreenBVQA. To further enhance the performance of GreenBIQA, we introduce a lightweight and efficient image saliency detection method, termed GreenSaliency. Ultimately, we integrate GreenSaliency with GreenBIQA, culminating in the development of the Green Saliency-guided BIQA method (GSBIQA).

PhD Experience Sharing:

The PhD journey at MCL has been an unforgettable experience. Over the course of this long and challenging path, I navigated through the unprecedented COVID era, multiple rounds of rigorous exams, and a diverse range of responsibilities, including teaching assistantships and intensive research projects. Each challenge presented an opportunity to grow, expand my perspectives, and enhance my skill set. I have honed my communication skills, developed the ability to tackle real-world problems through collaborative projects, and cultivated teamwork skills by engaging in discussions and joint efforts with exceptionally talented peers. These valuable experiences and abilities [...]

Objective image quality assessment (IQA) plays a crucial role in various multimedia applications and is generally categorized into three distinct types: Full-Reference IQA (FR-IQA), Reduced-Reference IQA (RR-IQA), and No-Reference IQA (NR-IQA). FR-IQA involves a direct comparison between a distorted image and its original reference image to evaluate quality. RR-IQA, in contrast, relies on partial information from reference images to assess the quality of the target images. NR-IQA, also known as blind image quality assessment (BIQA), is indispensable in situations where reference images are unavailable, such as at the receiver’s end or for user-generated content on social media platforms. The increasing prevalence of such platforms has driven a significant rise in the demand for BIQA. BIQA is critical for estimating the perceptual quality of images without reference, making it particularly relevant in the context of user-generated content and mobile applications, where reference images are typically not accessible.

The challenge of BIQA lies in its need to handle a wide variety of content and the presence of multiple types of distortions. Although many BIQA methods leverage deep neural networks (DNNs) and incorporate saliency detectors to improve performance, their large model sizes pose significant limitations for deployment on resource-constrained devices.

To overcome these challenges, we propose a novel non-deep-learning BIQA method, termed Green Saliency-guided Blind Image Quality Assessment (GSBIQA). GSBIQA is distinguished by its compact model size, low computational requirements, and strong performance. The method integrates a lightweight saliency detection module that aids in data cropping and decision ensemble processes, generating features that effectively mimic the human attention mechanism. The GSBIQA framework is composed of five key processes: 1) green saliency detection, 2) saliency-guided data cropping, 3) GreenBIQA feature extraction, 4) local patch prediction, and 5) saliency-guided decision ensemble. [...]

Objective image quality assessment (IQA) is pivotal in various multimedia applications. It can be categorized into three distinct types: Full-Reference IQA (FR-IQA), Reduced-Reference IQA (RR-IQA), and No-Reference IQA (NR-IQA). FR-IQA directly compares a distorted image against a reference or original image to assess quality. RR-IQA, on the other hand, uses partial information from the reference images to evaluate the quality of the target images. NR-IQA, also known as blind image quality assessment (BIQA), becomes essential in scenarios where reference images are unavailable, such as at the receiver’s end or for user-generated content on social media. The demand for BIQA has surged with the increasing popularity of such platforms. BIQA is an essential task that estimates the perceptual quality of images without reference. This field is increasingly relevant due to the rise in user-generated content and mobile applications where reference images are typically unavailable.

The challenge in BIQA lies in the diversity of content and the presence of mixed distortion types. While many BIQA methods employ deep neural networks (DNNs) and incorporate saliency detectors to enhance performance, their large model sizes limit deployment on resource-constrained devices.

To address this challenge, we introduce a novel and non-deep-learning BIQA method with a lightweight saliency detection module, called Green Saliency-guided Blind Image Quality Assessment (GSBIQA). It is characterized by its minimal model size, reduced computational demands, and robust performance. The lightweight saliency detector in GSBIQA facilitates data cropping and decision ensemble and generates useful features in BIQA that emulate the attention mechanism. The GSBIQA method is structured around five key processes: 1) green saliency detection, 2) saliency-guided data cropping, 3) Green BIQA feature extraction, 4) local patch prediction, and 5) saliency-guided decision ensemble. Experimental results show that the performance of [...]

Single image super-resolution (SISR) is an intensively studied topic in image processing. It aims at recovering a high-resolution (HR) image from its low-resolution (LR) counterpart. SISR finds wide real-world applications such as remote sensing, medical imaging, and biometric identification. Besides, it attracts attention due to its connection with other tasks (e.g., image registration, compression, and synthesis). To deal with such ill-posed problem, we recently proposed two methods, LSR[1] and LSR++[2], by providing reasonable performance and effectively reduced complexity.

LSR consists of three cascaded modules:

Unsupervised Representation Learning by creating a pool of rich and diversified representations in the neighborhood of a target pixel.

Supervised Feature Learning by Relative Feature Test (RFT [3]) to select a subset from the representation pool that is most relevant to the underlying super-resolution task automatically, and

Supervised Decision Learning by predicting the residual of the target pixel based on the selected features through regression via classical machine learning, and effectively fusioning the predictions for more stable results.

LSR++ is promoted based on LSR, with emphasis on sample alignment, a more promising sample preparation process which is suitable for all patch-based computer vision problems. As illustrated in Fig 1, based on gradient histograms of patches along the eight reference directions (Fig.1.a), patch alignment utilizes patch rotations and flipping to meet the standard templates of gradient histograms, where D_max is the direction with the largest cumulative gradient magnitude, and D_max_orth_b and D_max_orth_s refer to the orthogonal directions to D_max with big and small cumulative gradient magnitude, respectively. By modifying the set of (D_max, D_max_orth_b, and D_max_orth_s) of a patch, patch alignment can regularize the edge pattern with the patch by directions perpendicular the edge (D_max) and directions along the edge (D_max_orth_b, D_max_orth_s). The process of patch [...]