News

Humans can accumulate and maintain the knowledge learned from previous tasks and use it seamlessly in learning new tasks and solving new problems — learning new concepts over time is a core characteristic of human learning. Therefore, it is desirable to have a computer vision system that can learn incrementally about new classes when training data for them becomes available, as this is a necessary step towards the ultimate goal of building real intelligent machines that learn like humans.

Despite the recent success of deep learning in computer vision for a broad range of tasks, classical training paradigm of deep models is ill-equipped for incremental learning (IL). Most deep neural networks can only be trained in batch mode in which the complete dataset is given and all classes are known prior to training. However, the real world is dynamic and new categories of interest can emerge over time. Re-training a model from scratch whenever a new class is encountered is prohibitively expensive due to the huge data storage requirements and computational cost.  Directly fine-tuning the existing model on only the data of new classes using stochastic gradient descent (SGD) optimization is not a better approach either, as this might lead to the notorious catastrophic forgetting effect, which refers to the severe performance degradation on old tasks.

Existing IL approaches attempting to overcome catastrophic forgetting tend to produce a model that is biased towards either the old classes or new classes, unless with the help of exemplars of the old data. To address this issue, we propose a class-incremental learning paradigm called Deep Model Consolidation (DMC), which works well even when the original training data is not available. The idea is to train a model on the [...]

Tianhorng Chang, an MCL alumnus, entered the PhD/EE program at USC in Spring 1989, passed his screening exam on December 4, 1989, and became the third PhD student of Dr. Kuo in Spring 1990. Tianhorng took his qualifying exam on May 7, 1992. He was originally scheduled to take his defense in 1993. However, he could not do it but passed away in 1994 because of cancer.

Tianhorng Chang completed 46 units at USC and transferred in another 30 units from his previous MSEE program at the University of Florida. His graduate GPA at USC was 3.93. His seminal paper titled with “Texture Analysis and Classification with Tree-Structured Wavelet Transform” published by IEEE Trans. on Image Processing (1993) received 2000 citations. Due to Tianhorng’s near-complete curriculum and impactful research, USC granted Tianhorng’s posthumous PhD degree in 2018 August.

Tianhorng received his BSEE degree from the National Tsinghua University, Taiwan, in 1983. A ceremony to award the posthumous PhD degree to Tianhorng’s family was held in the National Tsinghua University on November 18, 2018. Many of his undergraduate classmates and MCL alumni in Taiwan attended the award ceremony. During the ceremony, Professor Kuo and several participants shared their memory on Tianhorng. A video clip featuring short speeches given by Professor Richard Leahy (the current Chair of USC/EE), Professor Sandeep Gupta (the former Chair of USC/EE) and Ms. Diane Demetras was also played. Tianhorng’s younger brother received the posthumous PhD degree on behalf of his family. The whole ceremony took about 1 hour.

Professor Kuo said, “Hope this award ceremony brings some comfort to Tianhorng’s family and Tianhorng’s story will encourage a younger generation to pursuit advanced research as long as they have strong passion.”

Image inpainting is the task to reconstruct the missing region in an image with plausible contents based on its surrounding context, which is a common topic of low-level computer vision. Making use of this technique, people could restore damaged images or remove unwanted objects from images or videos. In this task, our goal is to not only fill in the plausible contexts with realistic details but also make the inpainted area coherent with the contexts as well as the boundaries.

In spite of recent progress of deep generative models, generating high-resolution images remains a difficult task. This is mainly because modeling the distribution of pixels is difficult and the trained models easily introduce blurry components and artifacts when the dimensionality becomes high. Following [1] which proposes to synthesize an image based on joint optimization of image context and texture constraints, we divide the task into inference and translation as two separate steps and model each step with a deep neural network [2]. We also use simple heuristics to guide the propagation of local textures from the boundary to the hole. Meanwhile, we introduce a novel block-wise procedural training scheme to stabilize the training and propose a new strategy called adversarial annealing to reduce the artifacts [3].

On the other hand, we observe that existing methods based on generative models don’t exploit the segmentation information to constrain the object shapes, which usually lead to blurry results on the boundary. To tackle this problem, we propose to introduce the semantic segmentation information, which disentangles the inter-class difference and intra-class variation for image inpainting [4]. This leads to much clearer recovered boundary between semantically different regions and better texture within semantically consistent segments. Experiments on multiple public datasets show [...]

Dr. Xinfeng Zhang and Dr. Chao Yang are currently Postdoctoral Research Fellows at the MCL. They will complete their one-year stay and go back to China at the end of October.

Dr. Xinfeng Zhang received his PhD degree from Institute of Computing Technology, Chinese Academy of Sciences while Dr. Chao Yang received his PhD degree from Shanghai University. They are both experts in video coding. They joined the MCL in November 2017 and participated in two industrial projects: “Perceptual Video Coding based on Visual Attention Mechanism” (sponsored by Huawei) and “Joint Image Coding and Visual Understanding” (sponsored by Netflix, Tencent and Mediatek). They have done an excellent job in both projects, which leads to two journal papers under review.

MCL Director, Dr. C.-C. Jay Kuo, said that “It is our great pleasure to have Dr. Zhang and Dr. Yang to be around in our lab for the last year. They have made very important contributions. I do wish them the very best in their future career development.”

Dr. Xinfeng Zhang said that “It is a wonderful year for me in MCL, which is a prestigious research lab but also a family with love. I appreciate Prof. Kuo very much for the professional advices in my research, the strong support for my faculty job applications and the sincere guidance for life and career.  Moreover, I am very pleased to know all of the MCL members and become friends with you. Especially, thank Prof. Li, Dr. Yang, Haiqiang for the good research cooperation, and thank Bing Li and Bin Wang for the helps in my life of USC. Thanks, and best wishes for our MCL members!”

And Dr. Chao Yang said that “It’s a great honor to be here working [...]

The 25th IEEE International Conference on Image Processing (ICIP) was held in the Megaron Athens International Conference Centre, Athens, Greece, from October 7-10, 2018. ICIP is the world’s largest and most comprehensive technical conference focused on image and video processing and computer vision. The theme of ICIP 2018 will be “Imaging beyond imagination”. The conference features world-class speakers, tutorials, exhibits, and a vision technology showcase.

MCL Director, Professor C.-C. Jay Kuo, delivered the Plenary Speech entitled with “Unveil Convolutional Neural Networks (CNNs) and Go Beyond” on Oct. 8, 2018. Professor Kuo has worked on theoretical understanding of CNNs since 2015 and published a sequence of papers on this topic. This speech contained main results of his research endeavor.

Specifically, Professor Kuo described a new interpretable feedforward CNN design methodology. It does not demand any backpropagation. This design adopts a data-centric approach and derives network parameters of the current layer based on data statistics from the output of the previous layer. This process continues layer after layer in one pass. The feedforward design leads to a CNN that has a classification performance close to the one designed by backpropagation. Yet, it is more robust with respect to adversarial attacks. Above all, it is mathematically transparent.

Professor Kuo has a paper entitled with “Interpretable Convolutional Neural Networks via Feedforward Design” as the main reference. It can be downloaded from the arXiv: https://arxiv.org/abs/1810.02786.