News

The problem of facial expression recognition (FER) attempts to understand human emotion through facial image analysis. The technique can be applied to driver status monitoring, affective computing, and serious games. Solutions to FER can be categorized into two types: conventional methods and deep-learning-based (DL-based) methods. While conventional methods use hand-extracted features, DL-based methods conduct end-to-end optimization of certain networks whose performance highly depends on training data, the network architecture and the cost function. DL-based methods have become popular in recent years because of their higher performance. Yet, they demand a large model size. Although there has been research on reducing the number of parameters of DL models, it does not solve the computational complexity problem completely.

In this research, we are interested in a lightweight FER solution and name it ExpressionHop. ExpressionHop has low computational and memory complexity so that it is most suitable for mobile and edge computing environments. As shown in Figure 1, ExpressionHop consists of four modules: 1) cropping patches out based on facial landmarks, 2) applying filter banks to each patch to generate a rich set of joint spatial-spectral features, 3) conducting the discriminant feature test (DFT) to select features of higher discriminant power, and 4) performing the final classification task with a classifier. We conduct performance benchmarking on ExpressionHop, traditional and deep learning methods on several commonly used FER datasets such as JAFFE, CK+ and KDEF. Experimental results in table 1 show that ExpressionHop achieves comparable or better classification accuracy. Yet, its model size only has 30K parameters, which is significantly lower than those of deep learning methods.

As to the future research directions, there are several extensions to be pursued. First, it is desired to extend ExpressionHop to non-frontal [...]

MCL Director, Professor C.-C. Jay Kuo, was elected as one of 19 Academia Sinica’s 33rd Academicians. The news was announced on July 7, 2022. Professor Kuo was cited for his contributions to the fields of “multimedia computing” and “data science and engineering”.

Academia Sinica means ’Chinese Academy’. Its Chinese name is 中央硏究院. It was founded in 1928 in Nanjing and relocated to in Nangang, Taipei, in 1949. It is the national academy of the Republic of China (Taiwan). Academia Sinica supports research activities in a wide variety of disciplines, ranging from mathematical & physical sciences, to life sciences, and to humanities and social sciences. As an educational institute, it provides PhD training and scholarship through its English-language Taiwan International Graduate Program.

Professor Kuo acknowledged MCL alumni for his achievements, saying that “This honor is not only a recognition of me but also the outstanding performance of MCL alumni all over the world.” Furthermore, Professor Kuo was thankful to the strong support of the University of Southern California, the Viterbi School of Engineering, and the Ming-Hsieh Department of Electrical and Computer Engineering in the last three decades. He said, “Without the strongest support of the university, school and department, this would not happen at all.”

In Summer 2022, we have a new MCL member, Hardik Prajapati, joining our big family. Here is a short interview with Hardik with our great welcome.

1. Could you briefly introduce yourself and your research interests?

Hello, I am Hardik Prajapati. I am currently pursuing my Masters in Electrical and Computer Engineering at USC.I did my undergraduate in Instrumentation and Control Engineering from Nirma University, Ahmedabad(India). I like hiking and doing adventure activities. My research interests lie in the field of Computer vision, Machine Learning and Point Cloud Processing.

2. What is your impression about MCL and USC?

It’s an honor to work at MCL which has been actively functional since 1989. The best thing that I like about MCL is the direction of research that Professor Kuo has been instrumental in setting up. To summarize my impression about MCL I would say “Success meets MCL at a good tradeoff”. USC has a fun vibe going around.

3. What is your future expectation and plan in MCL?

I really do not wish to get ahead of myself and rather focus on having a productive and enjoyable summer. I am very hopeful in having rich and interesting conversations, making new friends and getting inspired by Professor Kuo and seniors at the lab.

Congratulations to Mozhdeh for passing her defense exam on May 12, 2022. Her PhD dissertation is titled with “Data-Efficient Image and Vision-Language Synthesis and Classification”. Her Dissertation Committee members were Jay Kuo (Chair), Keith Jenkins, and Aiichiro Nakano (Outside Member). All committee members were very pleased with the quality of Mozhdeh’s research and her well-polished presentation. We are glad to invite Mozhdeh here sharing the overview of her PhD study. We wish Mozhdeh all the best for her future career and life!

“Image classification and image synthesis are two fundamental yet challenging tasks in computer vision and pattern recognition and have drawn significant research attention over the last several decades. Image classification models learn to predict the probability of an image belonging to different classes. On the other hand, image synthesis models learn the probability distribution of data conditioned on some specific input. With the emergence of Deep Learning (DL) techniques and the availability of large annotated datasets and computational power, classification and generation models could achieve great success, however, in domains with a limited amount of data, leveraging such methods is challenging. Therefore, having data-efficient models requires further attention. In this thesis, we focus on learning-based data-efficient image and vision-and-language classification and image synthesis tasks.

In the first part, initially, we propose a method for data-efficient feature learning from images and video frames and then, use it to propose data-efficient models for face recognition, face gender classification, and also medical vision-and-language classification. We analyze each proposed model carefully and demonstrate their advantages over existing models.

In the second part, we offer a one-shot mask-guided model that allows controlling manipulations of a single image by inverting a quasi-robust classifier equipped with strong regularizers. We also demonstrate the [...]

In Summer 2022, we have a new MCL member, Jintang Xue, joining our big family. Here is a short interview with Jintang with our great welcome.

1. Could you briefly introduce yourself and your research interests?

My name is Jintang Xue. I am currently pursuing my Master’s degree in Electrical Engineering at USC. I got my Bachelor’s degree from Shanghai University, majoring in communication engineering. My research interests include machine learning and computer vision. I will work on point cloud classification this summer at MCL. I think it is an excellent opportunity to dive deeper into this field.

2. What is your impression about MCL and USC?

This is my first year at USC. The campus is beautiful. The people here are all very kind. I learned a lot from the valuable courses provided by USC, especially EE 569. MCL is a warm family. People in MCL are friendly, hardworking, and intelligent. They are willing to help each other. I am glad to work with them.

3. What is your future expectation and plan in MCL?

I want to make friends and learn from the members of MCL. I find point clouds very interesting. I will work hard on this topic to gain more knowledge and improve my programming skill. I hope I can contribute to MCL. I believe the experience at MCL is valuable in my life.