weiwang

Supervised learning is the main stream in pattern recognition, computer vision and natural language processing nowadays due to the great success of deep learning. On one hand, the performance of a learning system should improve as the number of training samples increases. On the other hand, some learning systems may benefit more than others from a large number of training samples. For example, deep neural networks (DNNs) often work better than classical learning systems that contain feature extraction and classification two stages. How the quantity of labeled samples affects the performance of learning systems is an important question in the data-driven era.

In fact, humans can learn effectively in a weakly supervised setting. In contrast, deep learning networks often need more labeled data to achieve good performance. What makes weak supervision and strong supervision different? There is little study on the design of supervision-scalable leaning systems. Is it possible to design a supervision-scalable learning system? Recently, MCL researchers attempt to shed light on these questions by choosing the object recognition problem as an illustrative example [1]. The design of two learning systems are presented that demonstrate an excellent scalable performance with respect to various supervision degrees. The first one adopts the classical histogram of oriented gradients (HOG) features, while the second one named improved PixelHop (IPHop) uses successive-subspace-learning (SSL) features [2]. The scalable learning system consists of three modules: representation learning, feature learning, and decision learning. In the second and the third modules, different designs are proposed to be adaptive to different supervision levels. Specifically, variance thresholding based feature selection and kNN classifier are used when the training size is small, while when the training size becomes larger, Discriminant Feature Test (DFT) [...]

The problem of facial expression recognition (FER) attempts to understand human emotion through facial image analysis. The technique can be applied to driver status monitoring, affective computing, and serious games. Solutions to FER can be categorized into two types: conventional methods and deep-learning-based (DL-based) methods. While conventional methods use hand-extracted features, DL-based methods conduct end-to-end optimization of certain networks whose performance highly depends on training data, the network architecture and the cost function. DL-based methods have become popular in recent years because of their higher performance. Yet, they demand a large model size. Although there has been research on reducing the number of parameters of DL models, it does not solve the computational complexity problem completely.

In this research, we are interested in a lightweight FER solution and name it ExpressionHop. ExpressionHop has low computational and memory complexity so that it is most suitable for mobile and edge computing environments. As shown in Figure 1, ExpressionHop consists of four modules: 1) cropping patches out based on facial landmarks, 2) applying filter banks to each patch to generate a rich set of joint spatial-spectral features, 3) conducting the discriminant feature test (DFT) to select features of higher discriminant power, and 4) performing the final classification task with a classifier. We conduct performance benchmarking on ExpressionHop, traditional and deep learning methods on several commonly used FER datasets such as JAFFE, CK+ and KDEF. Experimental results in table 1 show that ExpressionHop achieves comparable or better classification accuracy. Yet, its model size only has 30K parameters, which is significantly lower than those of deep learning methods.

As to the future research directions, there are several extensions to be pursued. First, it is desired to extend ExpressionHop to non-frontal [...]

MCL Director, Professor C.-C. Jay Kuo, was elected as one of 19 Academia Sinica’s 33rd Academicians. The news was announced on July 7, 2022. Professor Kuo was cited for his contributions to the fields of “multimedia computing” and “data science and engineering”.

Academia Sinica means ’Chinese Academy’. Its Chinese name is 中央硏究院. It was founded in 1928 in Nanjing and relocated to in Nangang, Taipei, in 1949. It is the national academy of the Republic of China (Taiwan). Academia Sinica supports research activities in a wide variety of disciplines, ranging from mathematical & physical sciences, to life sciences, and to humanities and social sciences. As an educational institute, it provides PhD training and scholarship through its English-language Taiwan International Graduate Program.

Professor Kuo acknowledged MCL alumni for his achievements, saying that “This honor is not only a recognition of me but also the outstanding performance of MCL alumni all over the world.” Furthermore, Professor Kuo was thankful to the strong support of the University of Southern California, the Viterbi School of Engineering, and the Ming-Hsieh Department of Electrical and Computer Engineering in the last three decades. He said, “Without the strongest support of the university, school and department, this would not happen at all.”

In Summer 2022, we have a new MCL member, Hardik Prajapati, joining our big family. Here is a short interview with Hardik with our great welcome.

1. Could you briefly introduce yourself and your research interests?

Hello, I am Hardik Prajapati. I am currently pursuing my Masters in Electrical and Computer Engineering at USC.I did my undergraduate in Instrumentation and Control Engineering from Nirma University, Ahmedabad(India). I like hiking and doing adventure activities. My research interests lie in the field of Computer vision, Machine Learning and Point Cloud Processing.

2. What is your impression about MCL and USC?

It’s an honor to work at MCL which has been actively functional since 1989. The best thing that I like about MCL is the direction of research that Professor Kuo has been instrumental in setting up. To summarize my impression about MCL I would say “Success meets MCL at a good tradeoff”. USC has a fun vibe going around.

3. What is your future expectation and plan in MCL?

I really do not wish to get ahead of myself and rather focus on having a productive and enjoyable summer. I am very hopeful in having rich and interesting conversations, making new friends and getting inspired by Professor Kuo and seniors at the lab.

Congratulations to Mozhdeh for passing her defense exam on May 12, 2022. Her PhD dissertation is titled with “Data-Efficient Image and Vision-Language Synthesis and Classification”. Her Dissertation Committee members were Jay Kuo (Chair), Keith Jenkins, and Aiichiro Nakano (Outside Member). All committee members were very pleased with the quality of Mozhdeh’s research and her well-polished presentation. We are glad to invite Mozhdeh here sharing the overview of her PhD study. We wish Mozhdeh all the best for her future career and life!

“Image classification and image synthesis are two fundamental yet challenging tasks in computer vision and pattern recognition and have drawn significant research attention over the last several decades. Image classification models learn to predict the probability of an image belonging to different classes. On the other hand, image synthesis models learn the probability distribution of data conditioned on some specific input. With the emergence of Deep Learning (DL) techniques and the availability of large annotated datasets and computational power, classification and generation models could achieve great success, however, in domains with a limited amount of data, leveraging such methods is challenging. Therefore, having data-efficient models requires further attention. In this thesis, we focus on learning-based data-efficient image and vision-and-language classification and image synthesis tasks.

In the first part, initially, we propose a method for data-efficient feature learning from images and video frames and then, use it to propose data-efficient models for face recognition, face gender classification, and also medical vision-and-language classification. We analyze each proposed model carefully and demonstrate their advantages over existing models.

In the second part, we offer a one-shot mask-guided model that allows controlling manipulations of a single image by inverting a quasi-robust classifier equipped with strong regularizers. We also demonstrate the [...]