News

In the ever-evolving realm of artificial intelligence (AI) and machine learning (ML), deep learning (DL) has reigned supreme for the past decade. However, its enigmatic nature and computational intricacies have prompted a quest for alternatives. Enter green learning (GL), a novel approach committed to constructing AI systems that are not just powerful but also interpretable, reliable, and sustainable.
GL is structured around three key modules: unsupervised representation learning, supervised feature learning, and supervised decision learning. Our primary focus is on the second module, which tackles the shortcomings of DL.

In the initial stages of GL, a diverse set of representations is crafted without any guiding supervision. These representations then undergo a discriminant feature test (DFT) in the subsequent module, where they are ranked based on their ability to discriminate. The selected discriminant representations become features. While the unsupervised nature of the first module might make GL representations seem less competitive than their DL counterparts, a remedy is proposed.

To address this, a novel approach emerges—creating new features through linear combinations of selected features. This method shows promise in obtaining more discriminant features, introducing a challenge of finding optimal weights for these combinations. Previous search algorithms like probabilistic search, adaptive particle swarm optimization (APSO) search, and stochastic gradient descent (SGD) search have been explored, but they still come with computational expenses.

Enter the least-squares normal transform (LNT), a game-changer. LNT is a supervised method designed to efficiently generate discriminant complementary features. These new features, termed complementary features, complement the original input features, referred to as raw features. The significance of this work lies in two key contributions: the introduction of LNT as an efficient tool for generating discriminant complementary features and its practical application, showcasing its prowess in [...]

Congratulations to Hongyu Fu for passing his defense today. Hongyu’s thesis is titled “Efficient Machine Learning Techniques for Low- and High-Dimensional Data Sources.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Aiichiro Nakano (Outside Member). The Committee members were very pleased with Hongyu’s high-quality work and professional presentation skills. MCL News team invited Hongyu for a short talk on her thesis and PhD experience, and here is the summary. We thankHongyu for his kind sharing, and wish him all the best in the next journey. A high-level abstract of Hongyu’s thesis is given below:

“This thesis concentrates on the development of efficient machine learning methodologies for both low and high-dimensional data. It presents a novel, feature-based machine learning technique, noted the Subspace Learning Machine (SLM), specifically designed for low-dimensional data. SLM combines the efficiency of decision trees and the effectiveness of multi-layer perceptrons to solve classification and regression problems with high performance and low model complexity. For high-dimensional data, the thesis proposes an efficient feed-forward machine learning framework and an adaptive SLM design with soft partitioning for image classification. These methods offer lightweight, adaptive models with low computational requirements and high performance.”
Hongyu shared his Ph.D. experience at MCL as follows :
“My PhD experience at the USC Media Communications Lab under the guidance of Prof. Kuo was both challenging and rewarding, providing me with a solid foundation in machine learning and multimedia data processing. The PhD training at MCL with the weekly report and seminar series was quite unique,  we practiced clear summarization ability, writing skills, and sharpened our presentation and oral communication abilities extensively. Prof. Kuo was extremely energetic, and working with constantly high efficiency, with more than 10 students in the [...]

Feature extraction and decision-making are two modules in cascade in the classical pattern recognition (PR) or machine learning (ML) paradigm. We recently proposed a novel learning diagram named Subspace Learning Machine (SLM) which considers this learning paradigm and focus on specific modules for classification-oriented decision making. SLM can be viewed as a generalized version of Decision Tree (DT). The linear combination of multiple features can be written as the inner product of a projection
vector and a feature vector. The effectiveness of SLM depends on the selection of good projection vectors, e.g. when the projection vector is a one-hot
vector, SLM is nothing but DT.

Both SLM and DT apply a hard split to a feature using a threshold at a decision node. To overcome the cons of hard feature space partitioning, we propose a new SLM method that adopts soft partitioning and denote it with SLM/SP in this proposed work. A comparison between hard decision and soft decision is illustrated in Fig 1. SLM/SP adopts the soft decision tree (SDT) data structure and a novel topology is proposed with inner nodes of SDT for data routing, leaf nodes of SDT for local decision making, and edge between parent and child nodes for representation learning. Specific modules are designed for the nodes and edges, respectively. The training of a SLM/SP tree starts by learning an adaptive tree structure via local greedy exploration between subspace partitioning and feature subspace learning. The tree structure is finalized once the stopping criteria are met for all
leaf nodes, and all module parameters are updated globally.

The overall frame working using Successive Subspace Learning and SLM/SP for image classification is as shown in Fig 2. The structure of the SLM/SP tree [...]

Congratulations to Joe Wang for passing his defense. Joe’s thesis is titled “Green Knowledge Graph Completion and Scalable Generative Content Delivery.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Robin Jia (Outside Member). MCL News team invited Joe for a short talk on her thesis and PhD experience, and here is the summary. We thank Joe for his kind sharing, and wish her all the best in the next journey.

Knowledge graphs (KGs) and Generative AI (GenAI) models have powerful reasoning capabilities and are crucial for building advanced artificial intelligence (AI) systems. In my thesis, we focus on four fundamental research to improve the efficiency, scalability, and explainability of the existing methods. They are:
1. Improving KG Embeddings with Entity Types: Entity types describe the high-level taxonomy and categorization of entities in KGs. They are often ignored in KG embedding learning. Thus, we propose a new methodology to incorporate entity types to improve KG embeddings. Specifically, our method can represent entities and types in the same embedding space with a constant number of additional model parameters. In addition, our method has a huge advantage in computation efficiency during inference.
2. KG Completion with Classifiers: KG embeddings have limited expressiveness in modeling relations. Thus, we study using binary classifiers to represent relations in the KG completion task. There are several advantages to modeling missing links as a binary classification problem, including having access to more powerful classifiers and data augmentation.
3. Green KG Completion: KG completion methods often require higher embedding dimensions for good performance. Thus, we investigate applying feature transformation and univariate feature selection to reduce the feature dimensions in KG completion methods. The KGs are first partitioned into several groups to extract [...]

The objective of image-to-image (I2I) translation involves learning a mapping from a source domain to
a target domain. Specifically, it aims at transforming images of the source style to those of the target
style with content consistency. While there is a domain gap, it can be mitigated by aligning the
distributions of the source and the target domains. Nevertheless, disparities between class distributions
of the source and target domains result in semantic distortion (see Figure 1); namely, different
semantics of correspondent regions between input and output. The semantic distortion could potentially
impact the efficacy of downstream tasks, such as semantic segmentation or object classification.
In this work, we propose a novel contrastive learning-based method that alleviates semantic
distortion by ensuring semantic consistency between input and output images. This is achieved by
enhancing the inter-dependence of structure and texture features between input and output by
maximizing their mutual information. In addition, we exploit multiscale predictions to boost the
I2I translation performance by employing global context and local detail information jointly to
predict translated images of superior quality, especially for high-resolution images. Hard negative
sampling is also applied to reduce semantic distortion by sampling informative negative samples.
For brevity, we refer to our method as SemST. Experiments conducted on I2I translation across
various datasets demonstrate the state-of-the-art performance of the SemST method. Additionally,
utilizing refined synthetic images in different UDA tasks confirms its potential for enhancing the
performance of UDA.