News

The title of his Ph.D. thesis proposal is “Theory and Applications of Adversarial and Structured Knowledge Learning”. His qualifying exam committee consisted of C.-C. Jay Kuo (Chair), Keith Michael Chugg, Keith Jenkins, Rahul Jain and Stefanos Nikolaidis.

Abstract of thesis proposal:

Deep learning has brought impressive improvements for many tasks, thanks to end-to-end data-driven optimization. However, people have little control over the system during training and limited understanding about the structure of knowledge being learned. In this thesis proposal, we study theory and applications of adversarial and structured knowledge learning: 1) learning adversarial knowledge with human interaction or by incorporating human-in-the-loop; 2) learning structured knowledge by modelling contexts and users’ preferences.

In the first category, our research topics include human-robot adversarial learning; Human-guided curriculum reinforcement learning and PortraitGAN for simultaneous emotion and modality manipulations. In the second category, a real-world compatible recommendation problem was tackled with structural graph representation and deep metric learning. The two categories are also related in the sense that structured knowledge often help lay a solid foundation, on which adversarial knowledge can be learned more successfully. Additionally, we contribute technically by open-sourcing relevant platforms.

Yeji Shen is a PhD candidate in Multimedia Communication Lab (MCL) in USC, supervised by Prof. C.-C. Jay Kuo. He received his Bachelor’s degree in Computer Science from Peking University, Beijing, China in June 2016. Since August 2016, he has been pursuing his PhD degree in MCL. His research interests include Machine Learning, Computer Vision and Artificial Intelligence. During this summer, he did an internship at Facebook. Here is a short interview with Yeji.

1. How does the study in USC and MCL help you?

First of all, in MCL, I learned to have a reasonable understanding of the research topics that I’ve been focusing on, like active learning, 3D vision and some semi-supervised learning. Such understanding is pretty helpful and valuable for both job interviews and the actual working experience. Second, I got to have a reasonable level of presentation skills, which I believe is very important in the future career. Third, a tough mind. Life is challenging. Only those with a tough mind can get through all those difficulties and obtain happiness.

2. How was it like working at Facebook?

The internship this year was a remote one. Different from normal working style, interns needed to work at home with the equipment sent by the company. (Of course, I need to mail them back.) Compared to a normal internship, the main pros are: 1) No need to physically move to the bay area. And thus the fee for house rent was saved. 2) Commuting time was saved. However, it is also clear that some cons are: 1) Harder to communicate. 2) Less interaction with team members. 3) It just didn’t feel good when the remote working style lasts for too long. Still, the overall feeling was not bad.

3. [...]

Jiali Duan is a PhD candidate supervised by Prof. C-.C. Jay Kuo at MCL. He interned at Amazon A9 during Summer. Here is a short interview with Jiali.

1. How does the study in USC and MCL help you? (technically and psychologically)

First, MCL helped me lay a solid foundation for research and communication. Academic and technical exchange of

ideas happen almost daily for applied scientist intern at Amazon, making it a necessity to communicate concisely and logically. Thankfully, I got trained for this aspect at MCL by sticking to weekly report and regular personal meetings. Second, the ability to prototype new ideas is intensely tested during internship. Original research here at MCL prepare us well for this aspect.

2. How was it like working at Amazon A9?

I had two internship experience at A9, a physical one in last year and a remote one this year. Generally, the physical one is much better. Last year, I was working at A9 Palo Alto. The company is located at University St, within walking distance to StandFord. Due to its location, there’re many nice restaurants and the neighborhood is very safe. The internship program was also very kind to provide free baseball/football match tickets, Santa Cruz tour and Seattle headquarter tour.

3. Do you have any suggestions for current graduate students?

I know that some companies provide more software development positions than research positions and some allow limited number of interview trials from the same person. So, be ready when you try. In terms of suggestions, be prepared for questions that reach beyond your scope of knowledge, which may require certain amount of improvision.

Face attributes classification is an important topic in biometrics. The ancillary information of faces such as gender, age and ethnicity is referred to as soft biometrics in forensics. The face gender classification problem has been extensively studied for more than two decades. Before the resurgence of deep neural networks (DNNs) around 7-8 years ago, the problem was treated using the standard pattern recognition paradigm. It consists of two cascaded modules: 1) unsupervised feature extraction and 2) supervised classification via common machine learning tools such as support vector machine (SVM) and random forest (RF) classifiers.

We have seen a fast progress on this topic due to the application of deep learning (DL) technology in recent years. Cloud-based face verification, recognition and attributes classification technologies have become mature, and they have been used in many real world biometric systems. Convolution neural networks (CNNs) offer high performance accuracy. Yet, they rely on large learning models consisting of several hundreds of thousands or even millions of model parameters. The superior performance is contributed by factors such as higher input image resolutions, more and more training images and abundant computational/memory resources.

Edge/mobile computing in a resource-constrained environment cannot meet the above-mentioned conditions. The technology of our interest finds applications in rescue missions and/or field operational settings in remote locations. The accompanying face inference tasks are expected to execute inside a poor computing and communication infrastructure. It is essential to have a smaller learning model size, lower training and inference complexity, and lower input image resolution. The last requirement arises from the need to image individuals at farther standoff distances, which results in faces with fewer pixels.

In this research, MCL worked closely with ARL researchers in developing a new interpretable non-parametric machine [...]

MCL received a grant from the US Army Research Laboratory (ARL) recently for joint research on theory and applications of artificial intelligence (AI) and machine learning (ML) technologies. Through this support, MCL researchers will collaborate with ARL researchers to conduct fundamental and applied research in the next 4 years.

The fundamental research targets at explainable neural networks. Cybenko and Hornik et al. proved that the multi-layer perceptron (MLP) is a universal approximator in late 80s, which appears to be the most important theoretic result even up to now. However, for a given dataset, they do not provide a constructive procedure to build the MLP. We will investigate a systematic way to specify an MLP architecture and determine its model parameters. Resource-rich and resource-scarce networks refer to those have abundant and fewer model parameters, respectively. The objective of the stress test study is to understand the behavior of the transition of a network from a resource-rich one to a resource-scarce one. We would like to understand the behavior transition in several areas, including model sensitivity to different weight initializations, classification accuracy, overfitting, etc.

The applied research targets at spatial-temporal attention and semantic scene understanding via successive subspace learning (SSL). To extract key spatial-temporal information from visual data, facilitates video processing and understanding down-stream tasks. For example, an image contains one or several objects. Object detection and recognition is critical to image understanding. Also, motion provides cues for object tracking in video understanding. The sponsored research will also focus on two issues on multi-modality data, say, those obtained by RGB, depth and infrared sensors: 1) representation of multi-domain data and 2) understanding of multi-domain data.

—- by Dr. C.-C. Jay Kuo