News

Demosaicing is a crucial step in the process of converting raw image data captured by sensors with Bayer color filter arrays into a full-color image that humans can perceive. Bayer arrays are a type of color filter array commonly used in digital image sensors, named after Bryce Bayer, who patented the design. These arrays consist of a grid of photosites, each covered by a color filter—typically red, green, or blue. Please see Figure 1.

Demosaicing in real-world scenarios poses a significant challenge due to the emergence of artifacts caused by factors such as sensor noise, motion blur, and edge artifacts. Sensor noise, introduced during image acquisition, can manifest as color noise, luminance noise, and texture loss in demosaiced images.

Another hurdle in demosaicing involves the necessity for ground truth data. Obtaining precise RGB images for training machine learning-based demosaicing methods is time-consuming, requiring meticulous alignment and synchronization across multiple color channels. To overcome this challenge, our strategy involves leveraging synthetic data generation, data augmentation, and domain adaptation techniques to augment the training dataset and mitigate the constraints of limited training data. We also consider semi-supervised learning as a viable approach for demosaicing.

In our investigation of the semi-supervised representation-learning module, we delve into a novel form of representation termed “oriented line segments (OLS),” illustrated in Figure 2. The OLS introduces two key hyperparameters: the length of a line segment, denoted as (2l+1), and the angle formed between two consecutive line segments.

The computational complexity of advanced demosaicing algorithms presents another challenge, particularly for real-time applications on resource-constrained devices like mobile phones or embedded systems. To address this, we propose the utilization of regression-based methods. This involves extracting and processing small image patches during demosaicing to learn the ground [...]

Single image super-resolution (SISR) is an intensively studied topic in image processing. It aims at recovering a high-resolution (HR) image from its low- resolution (LR) counterpart. SISR finds wide real-world applications such as remote sensing, medical imaging, and biometric identification. Besides, it attracts attention due to its connection with other tasks (e.g., image registration, compression, and synthesis).To deal with such ill-posed problem, we recently proposed two methods, LSR[1] and LSR++[2], by providing reasonable performance and effectively reduced complexity. LSR consists of three cascaded modules:
1) Unsupervised Representation Learning by creating a pool of rich and diversified representations in the neighborhood of a target pixel,
2) Supervised Feature Learning by Relative Feature Test (RFT [3]) to select a subset from the representation pool that is most relevant to the underlying super-resolution task automatically, and
3) Supervised Decision Learning by predicting the residual of the target pixel based on the selected features through regression via classical machine learning, and effectively fusioning the predictions for more stable results. LSR++ is promoted based on LSR, with emphasis on sample alignment, a more promising sample preparation process which is suitable for all patch-based computer vision problems. As illustrated in Fig 1, based on gradient histograms of patches along the eight reference directions (Fig.1.a), patch alignment utilizes patch rotations and flipping to meet the standard templates of gradient histograms, where D_max is the direction with the largest cumulative gradient magnitude, and D_max_orth_b and D_max_orth_s refer to the orthogonal directions to D_max with big and small cumulative gradient magnitude, respectively. By modifying the set of (D_max, D_max_orth_b, and D_max_orth_s) of a patch, patch alignment can regularize the edge pattern with the patch by directions perpendicular the edge (D_max) and directions along [...]

With the rapid development of point cloud applications, we have witnessed the prosperity of point cloud coding techniques in recent years. These point cloud codecs yield various compression artifacts, posing challenges to the point cloud quality assessment. Current PCQA metrics cannot handle the complicated compression distortion effectively. To overcome the challenge, we attend the ICIP 2023 point cloud visual quality assessment (PCVQA) grand challenge[1], and our BPQA model[2] achieved a competitive result over the BASICS[3] dataset.

Our proposed BPQA model consists of three modules. First, it selects points of various salience degrees based on the color information. Second, it projects the local neighborhood of selected points along one of the three orthogonal axes to yield a five-channel map (namely, RGB, depth, and pairwise-point-distance-mean channels). Third, it extracts features using the channel-wise Saab transform (c/w Saab) and the relevant feature test (RFT) and trains an XGBoost regressor to predict the Mean Opinion Score (MOS). BPQA offers competitive performance in no-reference quality assessment tasks of the ICIP 2023 PCVQA Challenge.

Reference
[1]https://sites.google.com/view/icip2023-pcvqa-grand-challenge/
[2]Q. Zhou, A. Feng, T.-S. Yang, S. Liu, and C.-C. J. Kuo, “Bpqa: A blind point cloud quality assessment method,” in 2023 IEEE International Conference on image processing (ICIP). IEEE, 2023.
[3] A. Ak, E. Zerman, M. Quach, A. Chetouani, A. Smolic, G. Valenzise, and P. L. Callet, “Basics: Broad quality assessment of static point clouds in compression scenarios,” ArXiv, vol. abs/2302.04796, 2023

-By Qingyang Zhou

Congratulations to Zohreh Azizi for passing her defense on Aug 29th. Zohreh’s thesis is titled “Advanced Technologies for Learning-based Image/Video Enhancement, Image Generation and Attribute Editing.” Her Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Aiichiro Nakano (Outside Member). We thank Zohreh for her kind sharing and wish her all the best in the next journey.

“My thesis proposes novel methodologies in four main areas related to visual data:

1. Low-light image enhancement: We present a new method, called NATLE, which attempts to strike a balance between noise removal and natural texture preservation through a low-complexity solution.

2. Low-light video enhancement: We also present a self-supervised adaptive low-light video enhancement method, called SALVE. The combination of traditional retinex-based image enhancement and learning-based ridge regression in SALVE leads to a robust, adaptive and computationally inexpensive solution. Our user study shows that 87% of participants prefer SALVE over prior work.

3. Image generation: Then, we present a generative modeling approach based on successive subspace learning (SSL). The resulting method, called the progressive attribute-guided extendable robust image generative (PAGER) model, has advantages in mathematical transparency, progressive content generation, lower training time, robust performance with fewer training samples, and extendibility to conditional image generation.

4. Facial Attribute Editing: Finally, we present a facial attribute editing method based on Gaussian Mixture Model (GMM). Our proposed method, named AttGMM, has a great advantage in lowering the computational cost.

t’s hard to believe my four-year PhD journey at MCL has reached its end. When you are inside the process, sometimes it’s not easy to keep up your hope. But each time you decide not to give up, you pave your path one more step closer to success.

I have grown into a much more confident person thorough [...]

USC Fall 2023 starts on the coming Monday, August 21, and MCL will resume the lab weekly activities on campus at the same time. Hope everyone have a great Fall semester!

Image credit:

Campus Benefits