Congratulations to Hao Xu for passing his defense on January 23, 2017. His Ph.D. thesis is entitled “Understanding Deep Learning from Its System Architectures, Feature Representations to Applications”.

Abstract of thesis:

Deep learning plays key roles in various aspects of the modern computer vision research. Our research focused on analyzing, adopting, and developing better CNN architectures which outperform the previous methods. To begin with, a car detection method using deformable part models consisting of composite feature sets (DPM/CF) is proposed. It recognizes cars of various types and from multiple viewing angles. The DPM/CF system consists of two stages. In the first stage, an HOG template is used to detect the bounding box of the entire car of a certain type and viewed from a certain angle (called a t/a pair), which yields a region of interest (ROI). In the second stage, we detect each salient part in a given t/a-specific ROI using either the HOG or the CNN feature. An optimization procedure based on latent logistic regression is adopted to choose the most discriminative location/size and the most suitable feature set for each part automatically. It is observed that the DPM/CF detector can strike a balance between detection performance and training complexity, through selecting the capable and simple feature from the composite feature set. Extensive experimental results are given to demonstrate the superior performance of the proposed DPM/CF method.

The CNN features used in DPM/CF demonstrate strong performance in detecting objects from images. To analyze the strength and weakness of the CNN feature representation, two quantitative metrics are proposed for the automatic evaluation of trained features at different convolution layers. The Gaussian confusion measure (GCM) is used to identify the discriminative ability of an individual feature, while the cluster purity measure (CPM) is used to analyze the discriminative ability of a group of features. Experiments have confirmed these metrics accurately reflect the discriminative ability of the trained Convolution Neural Network (CNN) features. Further studies utilizing the metrics as tools reveal important insights into the CNN, such as understanding the behavior of trained CNN features and the good detection performance of some object classes that were considered difficult in the past. The trained feature representation is compared between different CNN structures to validate the superiority of deeper networks.

With the insights learned from the CNN analysis, a novel attention model using the encoder-decoder framework is proposed for semi-supervised semantic segmentation. The proposed attention model utilizes both global and local features extracted from the encoder to generate class-dependent attention map. The attention map is then decoded by a carefully designed decoder to generate the segmentation result. Since our attention model can be trained separately from the decoder, the encoder is trained solely by class labels. This procedure reduces both the training complexity and the requirement of expensive image segmentation to be conducted by humans manually. Furthermore, when the number of object classes grows, our decoder has the potential to segment untrained object classes. Extensive experiments are conducted to analyze the effectiveness of the proposed attention model on the PASCAL VOC 2012 dataset and the extensibility of the proposed decoder from the Microsoft COCO dataset to the PASCAL VOC2012 datasets. The proposed solution is benchmarked with several state-of-the-art methods.

We are so glad to have him share his Ph.D. experience with us. Here is his sharing.

Ph.D. experience:

Working on my PhD is a unique experience, and I value every day of it. I learned how to be humble on the successes, be calm at the failures, and keep the fighting morale at all time. I am really lucky to be a PhD student of Prof. Kuo, who is willing to share wisdom, passion and endurance to all his students. Prof. Kuo taught so many good things to me, such as logic thinking, hardworking, and be patient. These characters will benefit me not only in the professional career but also personal life. My peers in the MCL are also great friends and mentors. Their kind help and warm support are what kept me fighting. For the younger MCL students, it is a precious opportunity to be a part of the MCL. The days ahead of you will be glorious and rewarding. Keep fighting and you will achieve anything you want. Fight on!!

Congratulations again to Hao and we wish him all the best in his future career.