XuejingLei

With the rise of visualization, animation and autonomous driving applications, the demand for 3D point cloud analysis and understanding has rapidly increased. Point Cloud is a kind of data obtained from lidar scanning which contains abundant 3D information. Our research directions about point cloud in autonomous driving are object detection, segmentation and classification.

Due to its unstructured and unordered properties, people usually transfer point cloud into other data types such as mesh, voxel and multi-view. But the transformation must cause information lost. Recently, several deep-learning-solutions such as PointNet/Pointnet++ [1, 2] tailored to point clouds provide a more efficient and flexible way to handle 3D data. Some successful results for object classification and parts and semantic scene segmentation have been demonstrated. However, object and scene understanding with Convolutional Neural Networks (CNNs) on 3D volumetric data is still limited due to its high memory requirement and computational cost. This brings a challenge for autonomous driving since it requires real-time and concise processing of the observed scenes and objects.

An interpretable CNN design based on the feedforward (FF) methodology [3] without any backpropagation (BP) was recently proposed by the Media Communications Lab at USC. The FF design offers a complementary approach to CNN filter weights selection. We are now designing a feed-forward (FF) network for both object classification and indoor scene segmentation. The advantages of the FF design methodology are multiple folds. It is completely interpretable. It demands much less training complexity and training data. Furthermore, it can be generalized to weakly supervised or unsupervised learning scenarios in a straightforward manner. The latter is extremely important in real world application scenarios since data labeling is very tedious and expensive.

References:

R. Qi, H. Su, K. Mo, and L. J. Guibas. [...]

Deep learning has shown its effectiveness in various computer vision tasks. However, a large amount of labeled data is usually needed for deep learning approaches. Active learning can help reduce the labeling efforts by choosing the most informative samples to label and thus achieves a comparable performance with less labeled data.

There are two major types of active learning strategy: uncertainty based and diversity based.

The core idea of uncertainty based methods is to label those samples that are most uncertain to the existing model trained on current labeled set. For example, an image with a prediction of 50 percent cat is empirically considered to be more valuable than an image with a prediction of 99 percent cat, where the former has larger uncertainty. Besides uncertainty metrics from information theory like entropy, Beluch et al. [1] proposes to use an ensemble to estimate the uncertainty of unlabeled images and achieves good results in ImageNet dataset.

In contrast, diversity based methods rely on an assumption that a more diverse set of images chosen as the training set can lead to better performance. Sener et al. [2] formalizes the active learning problem into a core-set problem and achieves competitive performance in CIFAR-10 dataset. Mixed-integer programming is used to solve their objective function.

Our current research focuses on balancing the two factors (uncertainty and diversity) in a explainable way.

References:
[1] Beluch, William H., et al. “The power of ensembles for active learning in image classification.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[2] Sener, Ozan, and Silvio Savarese. “Active learning for convolutional neural networks: A core-set approach.” (2018).

Author: Yeji Shen

The deep learning technologies such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have great impacts on modern machine learning due to their impressive performance in many application fields that involve learning, modeling, and processing of complex sensing data. Yet, the working principle of deep learning remains mysterious. Furthermore, it has several well-known weaknesses: 1) vulnerability to adversarial attacks, 2) demanding heavy supervision, 3) generalizability from one domain to the other. Professor Kuo and his PhD students at Media Communications Lab (MCL) have been working on explainable deep learning since 2014 and published a sequence of pioneering papers on this topic.

Explanation of nonlinear activation, convolutional filters and discriminability of trained features of CNNs [1]-[3]. The role of CNN’s nonlinear activation function is well explained in [1] at the first time. That is, the nonlinear activation operation is used to resolve the sign confusion problem due to the cascade of convolutional operations in multiple layers. This work received the 2018 best paper award from the Journal of Visual Communication and Image Representation. The convolutional filters is viewed as a rectified correlations on a sphere (RECOS) and CNN’s operation is interpreted as a multi-layer RECOS transform in [2].  The discriminability of trained features of a CNN at different convolution layers is analyzed using two quantitative metrics in [3] – the Gaussian confusion measure (GCM) and the cluster purity measure (CPM), The analysis is validated by experimental results.

Saak transform and its application to adversarial attacks [4]-[5]. Being inspired by deep learning, we  develop  a new mathematical transform called the Saak (Subspace approximation with augmented kernels) transform in [4]. The Saak and inverse Saak transforms provide signal analysis and synthesis tools, respectively. CNNs are known to [...]

After a short stay in Athens for ICIP 2018, Professor Kuo flew to Israel and visited Technion, Israel Institute of Technology.  He delivered the Viterbi Special Guest Lecture titled with “Unveil Convolutional Neural Networks and Go Beyond” on October 11. The talk was very well received.

The Technion – Israel Institute of Technology – is a public research university in Haifa, Israel. The university was established in 1912 during the Ottoman Empire, which was more than 35 years before the State of Israel. The Technion is the oldest university in Israel. It is ranked the best university in Israel and in the whole of the Middle East.

There is a close tie between USC and Technion through Dr. Andrew J. Viterbi. Dr. Viterbi received a Technion Honorary Doctorate in 2000. He has been a Distinguished Visiting Professor of Electrical Engineering at the Technion since then. Dr. Viterbi announced a $50 million gift to secure and enhance the Technion-Israel Institute of Technology’s leadership position in electrical and computer engineering in Israel and globally in 2015. He is a member of the Technion Board of Governors.

Professor Kuo said, “It was my great honor to have this opportunity to be a bridge between the USC Viterbi School of Engineering and the Technion Viterbi Faculty of Electrical Engineering. It is very meaningful to have more interactions and faculty/student exchanges between these two world top universities.” Professor Kuo’s visit was sponsored by the Technion Rubiner/Viterbi Fund. He used the same office of Dr. Andrew Viterbi during his stay. Professor Kuo’s visit was hosted by Professor Josh Zeevi, who is a world renowned expert in vision and image sciences.

1. Could you briefly introduce yourself and your research interests?

I’m pursuing my Master Degree in Electrical Engineering at USC with specialization in Image Processing and Machine Learning. I joined MCL in August 2018. My current research interests lie in the field of Machine Learning for Image Processing applications. I have worked on many applications of image processing such as Object detection, Vehicle tracking and Behavioral cloning using Deep models. I’m very interested in Natural Language Processing as well.

2. What is your impression about MCL and USC?

USC has been an amazing environment for my growth. Though I have been here only for a year, I have gained so much of knowledge and I’m very grateful to USC for that. The people at MCL are forward-thinking and very intelligent. The research at MCL is cutting edge and I am very happy and excited to be a part of the lab.

3. What is your future expectation and plan in MCL?

I want to interact more with Professor Kuo and all the students of the lab. I want to improve my research skills and expand my knowledge.