News

Congratulations to Heming Zhang for passing her defense!

Let us hear what she wants to say about her defense and an abstract of her thesis.

Deep learning techniques utilize networks with multiple layers cascaded to map the inputs to desired outputs. To map the entire inputs to desired outputs, useful information should be extracted through the layers. During the mapping, feature extraction and prediction are jointly performed. We do not have direct control for feature extraction. Consequently, some useful information, especially local information, is also discarded in the process.

In this thesis, we specifically study local-aware deep learning techniques from four different aspects: 1) Local-aware network architecture 2) Local-aware proposal generation 3) Local-aware region analysis 4) Local-aware supervision

Specifically, we design a multi-modal attention mechanism for generative visual dialogue system, which simultaneously attends to multi-modal inputs and utilizes extracted local information to generate dialogue responses. We propose a proposal network for fast face detection system for mobile devices, which detects salient facial parts and uses them as local cues for detection of entire faces. We extract representative fashion features by analyzing local regions, which contain local fashion details of humans’ interests. We develop a fashion outfit compatibility learning method, which models each outfit as a graph and learns outfit compatibility using both global and local supervisions on the graphs.

I would like to thank Prof. Kuo and all the lab members for their help. I have learned a lot through my PhD journey and I want to share some feelings and experiences. One essential part of the doctoral training is mental training, from which I have become more persistent, self-disciplined and motivated. As this journey may take several years, maintaining a balanced life is very important. I wish the best to all the lab members and [...]

By |January 26th, 2020|News|Comments Off on Congratulations to Heming Zhang for passing her defense!|

MCL Research on Behavior Analysis of Stressed CNNs

CNNs have demonstrated effectiveness in many applications. However, few efforts have been made to understand CNNs. To better explain the behaviors of convolutional neural networks (CNNs), we adopt an experimental methodology with simple datasets and networks in this research. Our study includes three steps: 1) design a sequence of experiments; 2) observe and analyze network behaviors; and 3) present conjectures as learned lessons from the study. In particular, we wish to examine the behaviors under limited resources, including limited amount of labeled data and limited network size. First, we examine the effect of limited labeled data. Semi-supervised learning deals with the case where limited labeled data and abundant unlabeled data are available. Co-training is one of the techniques. In this part, we also focus on how CNNs behave under co-training. Second, to facilitate easier analysis of the roles of individual layers, we adopt a very simple LeNet-5-like network in our experiments. We adjust the number of filters in each layer and analyze the effect. In particular, we wish to show how differently networks with limited resources (i.e., when the number of filters is very small) and networks with rich resources behave in the following four aspects of CNNs:

Scalability: How does the network respond to datasets of different sizes?
Non-convexity: Is the performance of a network stable against different initializations of the network parameters?
Overfit: Is there a big gap between training and test accuracies?
Robustness: Is the classification result sensitive to small perturbation to the input?

 

An important contribution of our work is the investigation into the resource-sparse networks. Most works on CNN adopted networks with very rich resources. In our work, we also look into how the networks behave under [...]

By |January 12th, 2020|News|Comments Off on MCL Research on Behavior Analysis of Stressed CNNs|
  • Permalink Gallery

    MCL Research on Fashion Compatibility Recommendation (Jiali Duan)

MCL Research on Fashion Compatibility Recommendation (Jiali Duan)

In the task of fashion compatibility prediction, the goal is to pick an item from a candidate list to complement a partial outfit in the most appealing manner. Existing fashion compatibility recommendation work comprehends clothing images in a single metric space and lacks detailed understanding of users’ preferences in different contexts. To address this problem, we propose a novel Metric-Aware Explainable Graph Network (MAEG). In MAEG, we leverage a Latent Semantic Extraction Network (LSEN) to obtain representations of items in the metric-aware latent semantic space. Then, we develop a graph filtering network and Pairwise Preference Attention (PPA) module to model the interactions between users’ preferences and contextual information. With MAEG, we can provide recommendation to users as well as explain how each item and factor contribute to the final prediction. Extensive experiments on two large-scale real-world datasets reveal that MAEG not only outperforms the state-of-the-art methods, but also provides interpretable insights by highlighting the role of semantic attributes and contextual relationships among items.

By |January 5th, 2020|News|Comments Off on MCL Research on Fashion Compatibility Recommendation (Jiali Duan)|

Merry Christmas and Happy New Year

2019 has been a fruitful year for MCL. Some members graduated with impressive research work and began a new chapter of life. Some new students joined the MCL family and explored the joy of research. MCL members have made great efforts on their research and published quality research papers on top journals and conferences.

Wish all MCL members a happy new year!

 

Image credits:

Image 1: http://www.sohu.com, resized; Image 2: http://www.sohu.com, resized.

By |December 29th, 2019|News|Comments Off on Merry Christmas and Happy New Year|

Professor Kuo Delivered Invited Lecture at Kyoto University

MCL Director, Professor Kuo, gave an invited speech at Kyoto University on December 19, 2019. The title of his speech was “From Feedforward-Designed Convolutional Neural Networks (FF-CNNs) to Successive Subspace Learning (SSL)”.  Professor Kuo’s visit to Kyoto University was hosted by Professor Tatsuya Kawahara. The lecture was also an event of IEEE SPS Kansai Chapter.

The abstract of his speech is given below. “Given a convolutional neural network (CNN) architecture, its network parameters are typically determined by backpropagation (BP). The underlying mechanism remains to be a black-box after a large amount of theoretical investigation. In this talk, I will first describe a new interpretable feedforward (FF) design with the LeNet-5 as an example. The FF-designed CNN is a data-centric approach that derives network parameters based on training data statistics layer by layer in a one-pass feedforward manner. To build the convolutional layers, we develop a new signal transform, called the Saab (Subspace approximation with adjusted bias) transform. The bias in filter weights is chosen to annihilate nonlinearity of the activation function. To build the fully-connected (FC) layers, we adopt a label-guided linear least squared regression (LSR) method. To generalize the FF design idea furthermore, we present the notion of “successive subspace learning (SSL)” and present a couple of concrete methods for image and point cloud classification. Experimental results are given to demonstrate the competitive performance of the SSL-based systems. Similarities and differences between SSL and deep learning (DL) are compared.”

By |December 22nd, 2019|News|Comments Off on Professor Kuo Delivered Invited Lecture at Kyoto University|
  • Permalink Gallery

    Professor Kuo Delivered Keynote at 5G/B5G Mobile Broadband Com Forum in KaoHsiung, Taiwan

Professor Kuo Delivered Keynote at 5G/B5G Mobile Broadband Com Forum in KaoHsiung, Taiwan

MCL Director, Professor Kuo, gave a keynote speech at 5the G/B5G Mobile Broadband Communication Forum in KaoHsiung, Taiwan, on December 14, 2019. The title of his keynote was “On Efficient and Explainable AI/ML Techniques for 5G/B5G Broadband Communication Systems”.  Here is the abstract of his speech.

“There have been many successful artificial intelligence and machine learning (AI/ML) stories in the last decade due to the resurgence of neural-network-based deep learning. We may ask whether the amazing success is primarily attributed to “deep learning” or “big data”. If it is attributed to deep learning, we need to understand these tools fully and open the black-box for further advancement. If it is attributed to big data, we need to find a way to position ourselves since data collection and labeling are expensive. With respect to the first question, I argue that AI is not necessarily dependent upon deep learning. We are working on an alternative approach, called Successive Subspace Learning (SSL), to solve the big data problem at USC. SSL is mathematically transparent and expandable depending on the training data sizes and classes. It has great potential. With respect to the second question, my view is that big data is helpful but not sufficient by itself. It is essential to integrate the “rule-based” and the “data-driven” approaches for generalizability and robustness. It leads to the conclusion that AI/ML is not about competing with humans in terms of intelligence but an advanced form of “automation”, which leverages both data and domain knowledge to achieve domain-specific tasks more effectively. Finally, I will give a couple of examples on how to apply AI/ML techniques to 5G/B5G broadband communication systems.”

By |December 15th, 2019|News|Comments Off on Professor Kuo Delivered Keynote at 5G/B5G Mobile Broadband Com Forum in KaoHsiung, Taiwan|
  • Permalink Gallery

    Professor Kuo Delivered Keynote at Grand Cloud Conference in Seoul

Professor Kuo Delivered Keynote at Grand Cloud Conference in Seoul

MCL Director, Professor Kuo, was invited by Korean National IT Industry Promotion Agency (NIPA) to deliver a keynote speech at the Grand Cloud Conference, Seoul, Korea on Dec. 3, 2019. The title of Professor Kuo’s talk was “AI Cloud Strategy for Government and Industry”. Here is the abstract.

There have been many successful artificial intelligence (AI) stories in the last decade due to big data analytics and the resurgence of neural-network-based deep learning. For clarity, I would like to give my own definition about AI. That is, AI is not about competing with humans in terms of intelligence but an advanced form of “automation”, which leverages both data and domain knowledge to achieve domain-specific tasks more effectively. Then, I will discuss the AI deployment strategy. It is centered around “smart environments” such as smart homes, smart cities, etc. We can leverage sensors, Internet of Things (IoT) and 5G infrastructures to acquire a large amount of data for better decision making. After that, I will mention three R&D opportunities: 1) explainable AI, 2) integrated rule-based/data-driven methodology and 3) federated learning. For the first topic, AI is not necessarily dependent upon deep learning, which is a black box. We are working on an alternative approach, called Successive Subspace Learning (SSL), to solve the big data problem at USC. SSL is mathematically transparent and expandable depending on the training data sizes and classes. It has great potential. For the second topic, I want to emphasize that big data is helpful but not sufficient by itself alone. It is more powerful to have an integration of the “rule-based” and the “data-driven” approaches. The integration allows better generalizability and robustness. The third topic is concerned with data-driven model training without [...]

By |December 8th, 2019|News|Comments Off on Professor Kuo Delivered Keynote at Grand Cloud Conference in Seoul|

MCL’s Thanksgiving Luncheon

MCL had the annual Thanksgiving Luncheon at Kirin Buffet on November 28, 2019. The Thanksgiving Luncheon has been a tradition of MCL for about 20 years. It’s a good chance for the whole group to gather and have a lunch together as a warm and happy family. All of us enjoyed the dilicious food and the wonderful time chatting with each other. It’s also a good opportunity to have a rest after a busy semester. Thank Professor Kuo for holding this event and thank Bin for organizing it.

Happy Thanksgiving to everyone!

By |November 28th, 2019|News|Comments Off on MCL’s Thanksgiving Luncheon|

MCL Work Won APSIPA Sadaoki Furui Paper Award

Dr. Sachin Chachada, a former MCL alumnus, and Professor C.-C. Jay Kuo received the 2019 Sadaoki Furui Paper Award at the open ceremony of the 2019 APSIPA ASC held in Lanzhou, China, on November 19 for their paper below:
Sachin Chachada and C.-C. Jay Kuo, “Environmental sound recognition: a survey,” Published online: 15 December 2014, e14, APSIPA Trans. on Signal and Information Processing.
The paper has been cited by 107 in Google Scholar, the number of downloads in 2019 (to end of Sept.) is 1523. The abstract of the paper is given below.
Although research in audio recognition has traditionally focused on speech and music signals, the problem of environmental sound recognition (ESR) has received more attention in recent years. Research on ESR has significantly increased in the past decade. Recent work has focused on the appraisal of non-stationary aspects of environmental sounds, and several new features predicated on non-stationary characteristics have been proposed. These features strive to maximize their information content pertaining to signal’s temporal and spectral characteristics. Furthermore, sequential learning methods have been used to capture the long-term variation of environmental sounds. In this survey, we will offer a qualitative and elucidatory survey on recent developments. It includes four parts: (i) basic environmental sound-processing schemes, (ii) stationary ESR techniques, (iii) non-stationary ESR techniques, and (iv) performance comparison of selected methods. Finally, concluding remarks and future research and development trends in the ESR field will be given.

By |November 24th, 2019|News|Comments Off on MCL Work Won APSIPA Sadaoki Furui Paper Award|

Congratulations to Ye Wang for Passing His PhD Defense

Congratulations to Ye Wang for passing his defense on November 12, 2019. His Ph.D. thesis is entitled “Video Object Segmentation and Tracking with Deep Learning Techniques”.

Abstract of the thesis:

Video object segmentation (VOS) aims to segment foreground objects from complex background scenes in video sequences. It is a challenging problem because of the complex nature of videos: occlusions, motion blur, deforming shapes and truncations etc. There are two main categories in existing VOS methods: semi-supervised and unsupervised. Semi-supervised VOS algorithms require manually annotated object regions in the first frame and then automatically segment the specified object in the remaining frames throughout the video sequence. Unsupervised VOS algorithms segment the most conspicuous and eye-attracting objects without prior knowledge of these objects in the video.

This thesis describes how we build an intelligent system to perform video object segmentation and tracking. We discuss the challenges in this field and then propose deep learning algorithms to significantly improve the performances in this thesis. First, this thesis addresses unsupervised video object segmentation by designing pseudo ground truth and online adaptation. Second, a novel unsupervised video object segmentation approach via distractor-aware online adaptation is proposed to deal with challenging videos when multiple objects occur and interact in a given video clip. Finally, this thesis also presents a two-stage approach, track and then segment, to perform semi-supervised video object segmentation with only bounding box annotations. Besides, we provide some interesting problems which can be addressed in the future.

Ph.D. experience:

I am very fortunate to join Media Communications Lab (MCL) led by Professor C.-C. Jay Kuo in Fall 2015, who gave me the chance and inspiration to explore the new direction and guided me through my whole PhD study. His immense knowledge has greatly [...]

By |November 17th, 2019|News|Comments Off on Congratulations to Ye Wang for Passing His PhD Defense|