News

The Keller Colloquium is a distinguished lecture series for the CMS (Computing and Mathematical Sciences) department at the California Institute of Technology (Caltech). The CMS department includes Applied Math, Control, and Computer Science programs. A committee of students and faculty chooses the speakers across these areas. Professor Kuo was invited to lecture during the Fall Quarter of the 2022-23 academic year.

Professor Kuo visited Caltech on October 31 (Monday). He met Professor P. P. Vaidyanathan and Professor Thomas Yizhao Hou before his seminar at 4 pm. The title of his lecture was “Green Learning: Methodology, Examples, and Outlook,” with the following abstract:

“The rapid advances in artificial intelligence in the last decade are primarily attributed to the wide applications of deep learning (DL). Yet, the high carbon footprint yielded by larger DL networks is a concern to sustainability. Green learning (GL) has been proposed as an alternative to address this concern. GL is characterized by low carbon footprints, small model sizes, low computational complexity, and mathematical transparency.  It offers energy-effective solutions in cloud centers and mobile/edge devices.  It has three main modules: 1) unsupervised representation learning, 2) supervised feature learning, and 3) decision learning. GL has been successfully applied to a few applications. This talk provides an overview of the GL solution, its demonstrated examples, and its technical outlook. The connection between GL and DL will also be discussed.”

The Lecture was well attended. People showed interest in this new machine-learning paradigm.

Knowledge graphs (KGs) store human knowledge in a graph-structured format, where nodes and edges denote entities and relations, respectively. Most KGs, such as wikidata [1], suffer from the incompleteness problem; namely, a large number of factual triples are missing, leading to performance degradation in downstream applications. Thus, there is growing interest in developing KG completion (KGC) methods to solve the incompleteness problem by inferring undiscovered factual triples based on existing ones.

Prior KGC work focuses on learning embeddings for entities and relations through a simple score function. Yet, a higher-dimensional embedding space is usually required for a better reasoning capability, which leads to larger model size and hinders applicability to real-world problems (e.g., large-scale KGs or mobile/edge computing). A lightweight modularized KGC solution, called GreenKGC, is proposed to address this issue. GreenKGC consists of three modules: representation learning, feature pruning, and decision learning, to extract discriminant KG features and make accurate predictions on missing relationships using classifiers and negative sampling. The overview of the model is shown in Fig. 1.

Experimental results demonstrate several advantages of GreenKGC. First, it only requires a low-dimensional space (e.g. d = 8) to achieve achieve competitive or even better performance against high-dimensional models with much smaller model sizes. Second, as compared with other classification-based methods, it requires a shorter inference time and provides better performance. Third, the feature pruning module is 20x faster than knowledge distillation methods in training powerful low-dimensional features. A comparison of GreenKGC and other KGC models under different dimensions is given in Fig. 2.

-By Yun Cheng Wang

Reference:

[1] Vrandečić, Denny, and Markus Krötzsch. “Wikidata: a free collaborative knowledgebase.” Communications of the ACM 57.10 (2014): 78-85.

Xuejing Lei studied on Nicolaus Copernicus, the first mathematician in MCL genealogy, and shared her study with MCL members in the pre-seminar sharing on September 12, 2022. Nicolaus Copernicus was born in 1473 in Toruń, Royal Prussia, Poland and died in 1543 aged 70. He was a Renaissance polymath, who made contributions in a wide variety of fields including astronomy, canon law, economics, mathematics, etc. He is best known for Heliocentrism, Quantity theory of money and Gresham–Copernicus law.

Nicolaus Copernicus was born in a powerful family. His father was a well-to-do merchant who dealt in copper, and died about 1483. His mother was the daughter of a wealthy Toruń patrician and city councilor, deceased after 1495. After his father’s death, his maternal uncle, Lucas Watzenrode the Younger, took the little boy under his wing and saw to his education and career. Lucas formed close relations with three successive Polish monarchs Watzenrode and many rulers. He came to be considered the most powerful man in Warmia, and his wealth, connections and influence allowed him to secure Copernicus’s education and career as a canon at Frombork Cathedral.

Nicolaus Copernicus’s study in University of Kraków (now Jagiellonian University) gave him a thorough grounding in the mathematical astronomy and initiated his analysis of logical contradictions in the two “official” systems of astronomy — Aristotle’s theory of homocentric spheres, and Ptolemy’s mechanism of eccentrics and epicycles. He then went to Italy and studied in University of Bologna for 4 years and University of Padua for 2 years, and obtained his Doctoral degree of Canon Law in University of Ferrara in 1503.

Although Nicolaus Copernicus was best known to his contemporaries as a doctor and the Canon of Frauenburg Cathedral, he is best [...]

We are so happy to welcome a new graduate member of MCL, Haiyi Li. Here is an interview with Haiyi:

Could you briefly introduce yourself and your research interests?

My name is Haiyi Li. I am currently a Ph.D student in Electrical Engineering at USC. I received my bachelor’s degree in Automation Engineering from the University of Electronic Science and Technology of China in 2022. I enjoy swimming and outdoor sports in my spare time. My research interests include image processing and machine learning.

What is your impression about MCL and USC?

MCL is a fantastic place to exchange thoughts and obtain inspiration. Professor Kuo is a knowledgeable and passionate advisor, providing us with a lot of new ideas. And he is also a patient and responsible instructor, offering me some research directions to dig deeper. Also, students in MCL are really nice and intelligent. I feel inspired when discussing questions with them. USC is a vibrant campus. I am impressed by the strong academic resources and diverse environment here.

What is your future expectation and plan in MCL?

I plan to have more inspiring discussions with Professor Kuo and senior students of MCL and lay a sound foundation for my research. And I will focus on some specific image processing research directions to conduct some hands-on projects to make models more reasonable with better performance. I hope I am able to equip myself with mature research ability and insightful ideas.

Congratulations to MCL Alumnus, Dr. Yueru Chen, and Director, Professor Jay Kuo, for receiving the 2022 Best Paper Award from the Journal of Visual Communication and Image Representation for their work:

Yueru Chen and C.-C. Jay Kuo, “PixelHop: a successive subspace learning (SSL) method for object recognition,” Journal of Visual Communication and Image Representation, Vol. 70, July 2020, 102749.

The PixelHop paper proposed a successive subspace learning (SSL) framework for unsupervised feature representation. It lays a key foundation for green learning. Professor Kuo said, “Deep learning has been very dominating in the computer vision and image analysis field in the last 10 years. It was not easy for Yueru to pursue a totally different research direction in her PhD research. I am glad to see that her effort on developing an interpretable and modularized learning system has been gradually recognized by the community.”

MCL has received three best paper awards (2018, 2021, 2022) and two best paper award runner-ups (2019, 2020) from the Journal of Visual Communication and Image Representation in the last five years. The other four papers are listed below.

The 2021 Best Paper Award of the Journal of Visual Communication and Image Representation.

C.-C. Jay Kuo, Min Zhang, Siyang Li, Jiali Duan and Yueru Chen, “Interpretable convolutional neural networks via feedforward design,” the Journal of Visual Communication and Image Representation, Vol. 60, pp. 346-359, April 2019.

The 2020 Best Paper Award Runner-up of the Journal of Visual Communication and Image Representation.

C.-C. Jay Kuo and Yueru Chen, “On data-driven Saak transform,” the Journal of Visual Communication and Image Representation, Vol. 50, pp. 237-246, January 2018.

The 2019 Best Paper Award Runner-up of the Journal of Visual Communication and Image Representation.

Ronald Salloum, Yuzhou Ren [...]