News

Deep learning has shown its effectiveness in various computer vision tasks. However, a large amount of labeled data is usually needed for deep learning approaches. Active learning can help reduce the labeling efforts by choosing the most informative samples to label and thus achieves a comparable performance with less labeled data.

There are two major types of active learning strategy: uncertainty based and diversity based.

The core idea of uncertainty based methods is to label those samples that are most uncertain to the existing model trained on current labeled set. For example, an image with a prediction of 50 percent cat is empirically considered to be more valuable than an image with a prediction of 99 percent cat, where the former has larger uncertainty. Besides uncertainty metrics from information theory like entropy, Beluch et al. [1] proposes to use an ensemble to estimate the uncertainty of unlabeled images and achieves good results in ImageNet dataset.

In contrast, diversity based methods rely on an assumption that a more diverse set of images chosen as the training set can lead to better performance. Sener et al. [2] formalizes the active learning problem into a core-set problem and achieves competitive performance in CIFAR-10 dataset. Mixed-integer programming is used to solve their objective function.

Our current research focuses on balancing the two factors (uncertainty and diversity) in a explainable way.

References:
[1] Beluch, William H., et al. “The power of ensembles for active learning in image classification.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[2] Sener, Ozan, and Silvio Savarese. “Active learning for convolutional neural networks: A core-set approach.” (2018).

Author: Yeji Shen

The deep learning technologies such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have great impacts on modern machine learning due to their impressive performance in many application fields that involve learning, modeling, and processing of complex sensing data. Yet, the working principle of deep learning remains mysterious. Furthermore, it has several well-known weaknesses: 1) vulnerability to adversarial attacks, 2) demanding heavy supervision, 3) generalizability from one domain to the other. Professor Kuo and his PhD students at Media Communications Lab (MCL) have been working on explainable deep learning since 2014 and published a sequence of pioneering papers on this topic.

Explanation of nonlinear activation, convolutional filters and discriminability of trained features of CNNs [1]-[3]. The role of CNN’s nonlinear activation function is well explained in [1] at the first time. That is, the nonlinear activation operation is used to resolve the sign confusion problem due to the cascade of convolutional operations in multiple layers. This work received the 2018 best paper award from the Journal of Visual Communication and Image Representation. The convolutional filters is viewed as a rectified correlations on a sphere (RECOS) and CNN’s operation is interpreted as a multi-layer RECOS transform in [2].  The discriminability of trained features of a CNN at different convolution layers is analyzed using two quantitative metrics in [3] – the Gaussian confusion measure (GCM) and the cluster purity measure (CPM), The analysis is validated by experimental results.

Saak transform and its application to adversarial attacks [4]-[5]. Being inspired by deep learning, we  develop  a new mathematical transform called the Saak (Subspace approximation with augmented kernels) transform in [4]. The Saak and inverse Saak transforms provide signal analysis and synthesis tools, respectively. CNNs are known to [...]

Welcome New MCL Member Haoxuan You! Here is an short interview with Haoxuan:

1. Could you briefly introduce yourself and your research interests?

My name is Haoxuan You. I received my bachelor’s degree at Xidian University in China last July and spent my gap year at Tsinghua University as a research assistant. Now I am having my six-month visiting at USC. Generally speaking, my research interest lies in computer vision and machine learning. I hope to model the 3D environment by effectively 3D data processing.

2. What is your impression about MCL and USC?

The first impression of USC to me is the sunshine and beautiful campus. Then I found MCL a warm and big family with very kind people. The seminar lunch, the group meeting all give me a good opportunity to get involved in and help me to broaden my horizon. I feel really lucky to be with these lovely friends.

3. What is your future expectation and plan in MCL?

I hope to have a better understanding of how to be a good researcher and independent thinker from Prof. Kuo and other MCL members. And I wish to do some more valuable work such as the interpretable network in 3D data processing and 3D object detection.

Congratulations to Yueru Chen for passing her Qualifying Exam on Jan. 23, 2019! The title of her Ph.D. thesis proposal is “OBJECT CLASSIFICATION BASED ON NEURAL-NETWORK-INSPIRED IMAGE TRANSFORMS”. Her Qualifying Exam Committee includes: Jay Kuo (Chair), Antonio Ortega, Shri Narayanan, Keith Chugg and Ulrich Neumann (Outside Member).

Abstract of thesis proposal:

Convolutional neural networks (CNNs) have recently demonstrated impressive performance in image classification and change the way building feature extractors from carefully handcrafted design to automatically deep learned from a large labeled dataset. However, a great majority of current CNN literature are application-oriented, and there is no clear understanding and theoretical foundation to explain the outstanding performance and indicate the way to improve. In this thesis proposal, we focus on solving the image classification problem based on the neural-network-inspired Saak (Subspace approximation with augmented kernels) transform and Saab (Subspace approximation with adjusted bias) transform.

Based on the lossy Saak transform, we firstly proposed an efficient, scalable and robust approach to the handwritten digits recognition problem. We conduct a comparative study on the performance of the LeNet-5 and the Saak-transform-based solutions in terms of scalability and robustness as well as the efficiency of lossless and lossy Saak transform under a comparable accuracy level. We also develop an ensemble method that fuses the output decision vectors of Saab-transform-based decision system (i.e. the FF-CNN model) to solve the image classification problem. To enhance the performance of the ensemble system, it is critical to increasing the diversity of FF-CNN models. To achieve this objective, we introduce diversities by adopting three strategies: 1) different parameter settings in convolutional layers, 2) flexible feature subsets fed into the Fully-connected (FC) layers, and 3) multiple image embeddings of the same input source. Furthermore, we partition [...]

Congratulations to Fenxiao (Jessica) Chen for Passing His Qualifying Exam on 01/22/19! Her thesis proposal is titled with “GRAPH EMBEDDING WITH DEEP LEARNING TECHNIQUES”. Her Qualifying Exam Committee include: Jay Kuo (Chair), Sandy Sawchuk, Panos Georgiou, Viktor Prasanna and Xiong Ren (Outside Member).

Abstract of thesis proposal:

Graph representation learning is an important task nowadays due to the fact that most real-world data naturally comes in the form of graphs in many applications. Graph data often come in high-dimensional irregular form which makes them more difficult to analyze than the traditional low-dimensional data. Graph embedding has been widely used to convert graph data into a lower dimensional space while preserving the intrinsic properties of the original data.

In this thesis proposal, we specifically study two graph embedding problems: 1) Developing effective and graph embedding techniques can provide researcher with deeper understanding of the collected data more efficiently; 2) Use the embedded information to conduct applications such as node classification and link prediction.

To find an efficient way to learn and encode graph into a low dimensional embedding. We first present a novel Deepwalk-assisted Graph PCA (DGPCA) method is proposed for processing language network data represented by graphs. This method can generate a precise text representation for nodes (or vertices) in language networks. Unlike other existing work, our learned low dimensional vector representations add flexibility in exploring vertices neighborhood information while reducing noise contained in the original data. To demonstrate the effectiveness, we use DGPCA to classify vertices that contain text information in three language networks. Experimentally, DGPCA is shown to perform well on the language datasets in comparison to several state-of-the-art benchmarking methods.

To solve the node prediction problem, we present A novel graph-to-tree conversion mechanism called the deep [...]