Congratulations to Hongyu Fu for Passing His Defense

Congratulations to Hongyu Fu for passing his defense today. Hongyu’s thesis is titled “Efficient Machine Learning Techniques for Low- and High-Dimensional Data Sources.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Aiichiro Nakano (Outside Member). The Committee members were very pleased with Hongyu’s high-quality work and professional presentation skills. MCL News team invited Hongyu for a short talk on her thesis and PhD experience, and here is the summary. We thankHongyu for his kind sharing, and wish him all the best in the next journey. A high-level abstract of Hongyu’s thesis is given below:

“This thesis concentrates on the development of efficient machine learning methodologies for both low and high-dimensional data. It presents a novel, feature-based machine learning technique, noted the Subspace Learning Machine (SLM), specifically designed for low-dimensional data. SLM combines the efficiency of decision trees and the effectiveness of multi-layer perceptrons to solve classification and regression problems with high performance and low model complexity. For high-dimensional data, the thesis proposes an efficient feed-forward machine learning framework and an adaptive SLM design with soft partitioning for image classification. These methods offer lightweight, adaptive models with low computational requirements and high performance.”
Hongyu shared his Ph.D. experience at MCL as follows :
“My PhD experience at the USC Media Communications Lab under the guidance of Prof. Kuo was both challenging and rewarding, providing me with a solid foundation in machine learning and multimedia data processing. The PhD training at MCL with the weekly report and seminar series was quite unique,  we practiced clear summarization ability, writing skills, and sharpened our presentation and oral communication abilities extensively. Prof. Kuo was extremely energetic, and working with constantly high efficiency, with more than 10 students in the [...]

By |November 19th, 2023|News|Comments Off on Congratulations to Hongyu Fu for Passing His Defense|

Congratulations to Max Chen for Passing His Defense

Congratulations to Max Chen for passing his defense today. Max’ thesis is titled “A Green Learning Approach to Deepfake Detection and Camouflage and Splicing Object Localization.” His Dissertation Committee includes Jay Kuo (Chair), Shrikanth Narayanan, and Aiichiro Nakano (Outside Member). The Committee members highly praised the quality of his work. MCL News team invited Max for a short talk on her thesis and PhD experience, and here is the summary. We thank Max for his kind sharing, and wish him all the best in the next journey.

“In the current technological era, the advancement of AI models has not only driven innovation but also heightened concerns over environmental sustainability due to increased energy and water usage. For context, the water consumption equivalent to a 500ml bottle is tied to 10 to 50 responses from a model like GPT-3, and projections suggest that by 2027, AI could be using an estimated 85 to 134 TWh per year, potentially surpassing the water withdrawal of half of the United Kingdom. In light of these challenges, there is an urgent call for AI solutions that are environmentally friendly, characterized by lower energy consumption through fewer floating-point operations (FLOPs), more compact designs, and the ability to run independently on mobile devices without depending on server-based infrastructures.

This thesis introduces a novel approach for Camouflaged Object Detection, termed “GreenCOD.” GreenCOD combines the power of Extreme Gradient Boosting (XGBoost) with deep features. Contemporary research often focuses on devising intricate DNN architectures to enhance the performance of Camouflaged Object Detection. However, these methods are typically computationally intensive and show marginal differences between models. Our GreenCOD model stands out by employing gradient boosting for detection tasks. With its efficient design, it requires fewer parameters and FLOPs [...]

By |November 12th, 2023|News|Comments Off on Congratulations to Max Chen for Passing His Defense|
  • Permalink Gallery

    MCL Research on Subspace Learning Machine with Soft Partitioning (SLM/SP)

MCL Research on Subspace Learning Machine with Soft Partitioning (SLM/SP)

Feature extraction and decision-making are two modules in cascade in the classical pattern recognition (PR) or machine learning (ML) paradigm. We recently proposed a novel learning diagram named Subspace Learning Machine (SLM) which considers this learning paradigm and focus on specific modules for classification-oriented decision making. SLM can be viewed as a generalized version of Decision Tree (DT). The linear combination of multiple features can be written as the inner product of a projection
vector and a feature vector. The effectiveness of SLM depends on the selection of good projection vectors, e.g. when the projection vector is a one-hot
vector, SLM is nothing but DT.

Both SLM and DT apply a hard split to a feature using a threshold at a decision node. To overcome the cons of hard feature space partitioning, we propose a new SLM method that adopts soft partitioning and denote it with SLM/SP in this proposed work. A comparison between hard decision and soft decision is illustrated in Fig 1. SLM/SP adopts the soft decision tree (SDT) data structure and a novel topology is proposed with inner nodes of SDT for data routing, leaf nodes of SDT for local decision making, and edge between parent and child nodes for representation learning. Specific modules are designed for the nodes and edges, respectively. The training of a SLM/SP tree starts by learning an adaptive tree structure via local greedy exploration between subspace partitioning and feature subspace learning. The tree structure is finalized once the stopping criteria are met for all
leaf nodes, and all module parameters are updated globally.

The overall frame working using Successive Subspace Learning and SLM/SP for image classification is as shown in Fig 2. The structure of the SLM/SP tree [...]

By |November 5th, 2023|News|Comments Off on MCL Research on Subspace Learning Machine with Soft Partitioning (SLM/SP)|

MCL Research on Green Image Demosaicing

Demosaicing is a crucial step in the process of converting raw image data captured by sensors with Bayer color filter arrays into a full-color image that humans can perceive. Bayer arrays are a type of color filter array commonly used in digital image sensors, named after Bryce Bayer, who patented the design. These arrays consist of a grid of photosites, each covered by a color filter—typically red, green, or blue. Please see Figure 1.

Demosaicing in real-world scenarios poses a significant challenge due to the emergence of artifacts caused by factors such as sensor noise, motion blur, and edge artifacts. Sensor noise, introduced during image acquisition, can manifest as color noise, luminance noise, and texture loss in demosaiced images.

Another hurdle in demosaicing involves the necessity for ground truth data. Obtaining precise RGB images for training machine learning-based demosaicing methods is time-consuming, requiring meticulous alignment and synchronization across multiple color channels. To overcome this challenge, our strategy involves leveraging synthetic data generation, data augmentation, and domain adaptation techniques to augment the training dataset and mitigate the constraints of limited training data. We also consider semi-supervised learning as a viable approach for demosaicing.

In our investigation of the semi-supervised representation-learning module, we delve into a novel form of representation termed “oriented line segments (OLS),” illustrated in Figure 2. The OLS introduces two key hyperparameters: the length of a line segment, denoted as (2l+1), and the angle formed between two consecutive line segments.

The computational complexity of advanced demosaicing algorithms presents another challenge, particularly for real-time applications on resource-constrained devices like mobile phones or embedded systems. To address this, we propose the utilization of regression-based methods. This involves extracting and processing small image patches during demosaicing to learn the ground [...]

By |October 31st, 2023|News|Comments Off on MCL Research on Green Image Demosaicing|

Congratulations to Joe Wang for Passing His Defense

Congratulations to Joe Wang for passing his defense. Joe’s thesis is titled “Green Knowledge Graph Completion and Scalable Generative Content Delivery.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Robin Jia (Outside Member). MCL News team invited Joe for a short talk on her thesis and PhD experience, and here is the summary. We thank Joe for his kind sharing, and wish her all the best in the next journey.

Knowledge graphs (KGs) and Generative AI (GenAI) models have powerful reasoning capabilities and are crucial for building advanced artificial intelligence (AI) systems. In my thesis, we focus on four fundamental research to improve the efficiency, scalability, and explainability of the existing methods. They are:
1. Improving KG Embeddings with Entity Types: Entity types describe the high-level taxonomy and categorization of entities in KGs. They are often ignored in KG embedding learning. Thus, we propose a new methodology to incorporate entity types to improve KG embeddings. Specifically, our method can represent entities and types in the same embedding space with a constant number of additional model parameters. In addition, our method has a huge advantage in computation efficiency during inference.
2. KG Completion with Classifiers: KG embeddings have limited expressiveness in modeling relations. Thus, we study using binary classifiers to represent relations in the KG completion task. There are several advantages to modeling missing links as a binary classification problem, including having access to more powerful classifiers and data augmentation.
3. Green KG Completion: KG completion methods often require higher embedding dimensions for good performance. Thus, we investigate applying feature transformation and univariate feature selection to reduce the feature dimensions in KG completion methods. The KGs are first partitioned into several groups to extract [...]

By |October 23rd, 2023|News|Comments Off on Congratulations to Joe Wang for Passing His Defense|

MCL Research on Green Image Super-resolution

Single image super-resolution (SISR) is an intensively studied topic in image processing. It aims at recovering a high-resolution (HR) image from its low- resolution (LR) counterpart. SISR finds wide real-world applications such as remote sensing, medical imaging, and biometric identification. Besides, it attracts attention due to its connection with other tasks (e.g., image registration, compression, and synthesis).To deal with such ill-posed problem, we recently proposed two methods, LSR[1] and LSR++[2], by providing reasonable performance and effectively reduced complexity. LSR consists of three cascaded modules:
1) Unsupervised Representation Learning by creating a pool of rich and diversified representations in the neighborhood of a target pixel,
2) Supervised Feature Learning by Relative Feature Test (RFT [3]) to select a subset from the representation pool that is most relevant to the underlying super-resolution task automatically, and
3) Supervised Decision Learning by predicting the residual of the target pixel based on the selected features through regression via classical machine learning, and effectively fusioning the predictions for more stable results. LSR++ is promoted based on LSR, with emphasis on sample alignment, a more promising sample preparation process which is suitable for all patch-based computer vision problems. As illustrated in Fig 1, based on gradient histograms of patches along the eight reference directions (Fig.1.a), patch alignment utilizes patch rotations and flipping to meet the standard templates of gradient histograms, where D_max is the direction with the largest cumulative gradient magnitude, and D_max_orth_b and D_max_orth_s refer to the orthogonal directions to D_max with big and small cumulative gradient magnitude, respectively. By modifying the set of (D_max, D_max_orth_b, and D_max_orth_s) of a patch, patch alignment can regularize the edge pattern with the patch by directions perpendicular the edge (D_max) and directions along [...]

By |October 23rd, 2023|News|Comments Off on MCL Research on Green Image Super-resolution|

MCL Research on Enhanced Image-to-Image Translation

The objective of image-to-image (I2I) translation involves learning a mapping from a source domain to
a target domain. Specifically, it aims at transforming images of the source style to those of the target
style with content consistency. While there is a domain gap, it can be mitigated by aligning the
distributions of the source and the target domains. Nevertheless, disparities between class distributions
of the source and target domains result in semantic distortion (see Figure 1); namely, different
semantics of correspondent regions between input and output. The semantic distortion could potentially
impact the efficacy of downstream tasks, such as semantic segmentation or object classification.
In this work, we propose a novel contrastive learning-based method that alleviates semantic
distortion by ensuring semantic consistency between input and output images. This is achieved by
enhancing the inter-dependence of structure and texture features between input and output by
maximizing their mutual information. In addition, we exploit multiscale predictions to boost the
I2I translation performance by employing global context and local detail information jointly to
predict translated images of superior quality, especially for high-resolution images. Hard negative
sampling is also applied to reduce semantic distortion by sampling informative negative samples.
For brevity, we refer to our method as SemST. Experiments conducted on I2I translation across
various datasets demonstrate the state-of-the-art performance of the SemST method. Additionally,
utilizing refined synthetic images in different UDA tasks confirms its potential for enhancing the
performance of UDA.

By |October 8th, 2023|News|Comments Off on MCL Research on Enhanced Image-to-Image Translation|

MCL Research on Point Cloud Quality Assessment

With the rapid development of point cloud applications, we have witnessed the prosperity of point cloud coding techniques in recent years. These point cloud codecs yield various compression artifacts, posing challenges to the point cloud quality assessment. Current PCQA metrics cannot handle the complicated compression distortion effectively. To overcome the challenge, we attend the ICIP 2023 point cloud visual quality assessment (PCVQA) grand challenge[1], and our BPQA model[2] achieved a competitive result over the BASICS[3] dataset.

Our proposed BPQA model consists of three modules. First, it selects points of various salience degrees based on the color information. Second, it projects the local neighborhood of selected points along one of the three orthogonal axes to yield a five-channel map (namely, RGB, depth, and pairwise-point-distance-mean channels). Third, it extracts features using the channel-wise Saab transform (c/w Saab) and the relevant feature test (RFT) and trains an XGBoost regressor to predict the Mean Opinion Score (MOS). BPQA offers competitive performance in no-reference quality assessment tasks of the ICIP 2023 PCVQA Challenge.

[2]Q. Zhou, A. Feng, T.-S. Yang, S. Liu, and C.-C. J. Kuo, “Bpqa: A blind point cloud quality assessment method,” in 2023 IEEE International Conference on image processing (ICIP). IEEE, 2023.
[3] A. Ak, E. Zerman, M. Quach, A. Chetouani, A. Smolic, G. Valenzise, and P. L. Callet, “Basics: Broad quality assessment of static point clouds in compression scenarios,” ArXiv, vol. abs/2302.04796, 2023

-By Qingyang Zhou

By |October 3rd, 2023|News|Comments Off on MCL Research on Point Cloud Quality Assessment|

Congratulations to Zhiruo Zhou for Passing Her Defense

Congratulations to Zhiruo Zhou for passing her defense on September 25, 2023. Zhiruo’s thesis
is titled “Green Unsupervised Single Object Tracking: Technologies and Performance
Evaluation.” Her Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Stefanos
Nikolaidis (Outside Member). Zhiruo received several questions and suggestions from the
Committee members. Zhiruo answered the questions professionally.
Congratulations to Zhiruo for this milestone moment in life. MCL News team invited Zhiruo for a
short talk on her thesis and PhD experience, and here is the summary. We thank Zhiruo for her
kind sharing, and wish her all the best in the next journey.
“Video object tracking is one of the fundamental problems in computer vision and has diverse
real-world applications such as video surveillance and robotic vision. Given the ground-truth
bounding box of the object in the first frame of a test video, a single object tracker (SOT)
predicts the object box in all subsequent frames. Supervised trackers trained on labeled data
dominate the single object tracking field for superior tracking accuracy. The labeling cost and
the huge computational complexity hinder their applications on edge devices. Unsupervised
learning methods have also been investigated to reduce the labeling cost, but their complexity
remains high.
In my dissertation, I investigate the feasibility of lightweight high-performance tracking with
algorithmic transparency and no offline pre-training. I present our design of the green object
tracker that exploits spatial and temporal correlations at different granularities for more robust
tracking. It has been examined on a variety of benchmarks against recent state-of-the-arts, with
the inference complexity that is between 0.1%-10% of neural-network-based trackers. The
tracker is called ‘green’ due to its low computational complexity in both training and inference
stages, leading to a low [...]

By |September 25th, 2023|News|Comments Off on Congratulations to Zhiruo Zhou for Passing Her Defense|

Congratulations to Zohreh Azizi for Passing Her Defense

Congratulations to Zohreh Azizi for passing her defense on Aug 29th. Zohreh’s thesis is titled “Advanced Technologies for Learning-based Image/Video Enhancement, Image Generation and Attribute Editing.” Her Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Aiichiro Nakano (Outside Member). We thank Zohreh for her kind sharing and wish her all the best in the next journey.

“My thesis proposes novel methodologies in four main areas related to visual data:

1. Low-light image enhancement: We present a new method, called NATLE, which attempts to strike a balance between noise removal and natural texture preservation through a low-complexity solution.

2. Low-light video enhancement: We also present a self-supervised adaptive low-light video enhancement method, called SALVE. The combination of traditional retinex-based image enhancement and learning-based ridge regression in SALVE leads to a robust, adaptive and computationally inexpensive solution. Our user study shows that 87% of participants prefer SALVE over prior work.

3. Image generation: Then, we present a generative modeling approach based on successive subspace learning (SSL). The resulting method, called the progressive attribute-guided extendable robust image generative (PAGER) model, has advantages in mathematical transparency, progressive content generation, lower training time, robust performance with fewer training samples, and extendibility to conditional image generation.

4. Facial Attribute Editing: Finally, we present a facial attribute editing method based on Gaussian Mixture Model (GMM). Our proposed method, named AttGMM, has a great advantage in lowering the computational cost.

t’s hard to believe my four-year PhD journey at MCL has reached its end. When you are inside the process, sometimes it’s not easy to keep up your hope. But each time you decide not to give up, you pave your path one more step closer to success.

I have grown into a much more confident person thorough [...]

By |September 14th, 2023|News|Comments Off on Congratulations to Zohreh Azizi for Passing Her Defense|