News

MCL Research on Source-Distribution-Aimed Generative Model

There are typically two types of statistical models in mechine learning, discriminative models and generative models. Different from discriminative models that aim at drawing decision boundaries, generative models target at modeling the data distribution in the whole space. Generative models tackle a more difficult task than discriminative model because it needs to model complicated distributions. For example, generative models should capture correlations such as “Things look like boats are likely to appear near things that look like water” while discriminative model differentiates “boat” from “not boat”.

Image generative models have become popular in recent years since Generative Adversarial Network (GANs), can generate realistic natural images. They, however, have no clear relationship to probability distributions and suffer from difficult training process and mode dropping problem. Although difficult training process and mode dropping problems may be alleviated by using different loss functions [1], the underlying relationship to probability distributions remains vague in GANs. It encourages us to develop a SOurce-Distribution-Aimed (SODA) generative model that aims at providing clear probability distribution functions to describe data distribution.
There are two main modules in our SODA generative model. One is finding proper source data representations and the other is determining the source data distribution in each representation. One proper representation for source data is joint spatial-spectral representation proposed by Kuo, et.al. [2, 3]. By transforming between spectral domain and spatial domain, a rich set of spectral and spatial representations can be obtained. Spectral representations are vectors of Saab coefficients while spatial representations are pixels in an image or Saab coefficients that are arranged based on their pixel order in spatial domain. Spectral representation at the last stage give a global view of an image while the spatial representations describe details in [...]

By |April 27th, 2020|Computer Vision and Scene Analysis, News, Research|Comments Off on MCL Research on Source-Distribution-Aimed Generative Model|
Read More

Congratulations to Yuhang Song for Passing His Defense

Abstract of thesis:

The world around us is highly structured. Images not only contain various object categories with complex scenes but also include relationships between different objects or between humans and objects.  In recent years, deep learning has made a lot of achievements to the computer vision community, in both visual recognition and image generation tasks. In this thesis, we mainly leverage structure information to enhance the visual generation and understanding of these computer vision tasks.

On the visual generation side, image inpainting is the task to reconstruct the missing region in an image with plausible contents based on its surrounding context. In order to overcome the difficulty to directly learn the distribution of high-dimensional image data, we first divide the task into inference and translation as two separate steps and leverage the semantic information to help refine the textures. Second, we propose to introduce the semantic segmentation information, which disentangles the inter-class difference and intra-class variation to improve the quality of the generated images. On the visual understanding side, we study the problem of novel human-object interaction (HOI) detection, which is to recognize the relationship between humans and objects in images. We formulate it as a domain generalization problem and propose a unified framework of domain generalization to learn object-invariant features for predicate prediction, aiming at improving the generalization ability of the model to unseen scenarios. Finally, we provide some interesting research directions which can be addressed in the future.

 

Ph.D. experience:

I would like to express my gratitude to my advisor Professor C.-C. Jay Kuo for the continuous support of my Ph.D. study during these years. He has given me the freedom to pursue various projects without objection, and he has also provided insightful discussions about the [...]

By |April 19th, 2020|News|Comments Off on Congratulations to Yuhang Song for Passing His Defense|
Read More

Professor Kuo Received TCMC Impact Award

MCL Director, Professor Dr. C.-C. Jay Kuo, has been selected to receive the 2020 IEEE TCMC Impact Award, for “outstanding contributions to multimedia computing technologies in terms of research & development and as an inspiring educator.”

The TCMC Impact Award is granted to Kuo by the IEEE Computer Society Technical Committee on Multimedia Computing (TCMC). TCMC offers three awards annually – Impact Award, Rising Star Award and Service Award. The recipient of 2020 TCMC Service Award is Dr. Sethuraman Panchanathan who is nominated by President Trump as the next Director of the National Science Foundation. TCMC plans to present the three awards at the banquet of the IEEE 3rd International Conference on Multimedia Information Processing and Retrieval (IEEE MIPR2020) which will be held on August 6-8, 2020, in Shenzhen, China.

Dr. Kuo is a world-renowned technical leader in multimedia computing technologies, systems and applications with an enduring impact on both academic and industry realms. He said, “We have witnessed the rapid development and deployment of multimedia technologies in the last 30 years. They have great influences on our daily lives, e.g., image/video capturing by smart phone cameras and news and entertainment video streaming. It has been exciting to be part of this technology breakthrough. Also, I am truly honored by the recognition of the 2020 IEEE TCMC Impact Award.”

Dr. Kuo often travels around the world and meets MCL alumni in different countries. The two photos showed his re-union events with MCL alumni in Northern California (2019 July) and Taipei (2019 September).

By |April 12th, 2020|News|Comments Off on Professor Kuo Received TCMC Impact Award|
Read More

MCL Research on Image Super-resolution

Image super-resolution (SR) is a classic problem in computer vision (CV), which aims at recovering a high-resolution image from a low-resolution image. As a type of supervised generative problem, image SR attracts wide attention due to its strong connection with other CV topics, such as object recognition, object alignment, texture synthesis and so on. Besides, it has extensive applications in real world, for example, medical diagnosis, remote sensing, biometric information identification, etc.

For the state-of-the-art approaches for SR, typically there are two mainstreams: 1) example-based learning methods, and 2) Deep Learning (CNN-based) methods. Example-based methods either exploit external low-high resolution exemplar pairs [1], or learn internal similarity of the same image with different resolution scales [2]. In order to tackle model overfitting and generativity, some dictionary strategies are normally applied for encoding (e.g. Sparse coding, SC). However, features used in example-based methods are usually traditional gradient-related or just handcraft, which may affect model efficiency. While CNN-based SR methods (e.g. SRCNN [3]) does not really distinguish between feature extraction and decision making. Lots of basic CNN models/blocks are applied to SR problem, e.g. GAN, residual learning, attention network, and provide superior SR results. Nevertheless, the non-explainable process and exhaustive training cost are serious drawbacks of CNN-based methods.

By taking advantage of reasonable feature extraction [4], we utilize spatial-spectral compatible features to express exemplar pairs. In addition, we formulate a Successive-Subspace-Learning-based (SSL-based) method to partition data into subspaces by feature statistics, and apply regression in each subspace for better local approximation. Moreover, some adaptation is also manipulated for better data fitting. In the future, we aim at providing such a SSL-based explainable method with high efficiency for SR problem.

— By Wei Wang

 

Reference:

[1] Timofte, Radu, Vincent De Smet, and [...]

By |April 6th, 2020|Computer Vision and Scene Analysis, News, Research|Comments Off on MCL Research on Image Super-resolution|
Read More

Professor Kuo Highlighted by MIT LIDS Magazine

MCL Director, Professor Kuo, was recently highlighted in an article of the MIT LIDS (Laboratory for Information and Decision Systems) Magazine. The title is “Multimedia and Mentoring”. Professor Kuo is a world-renowned scholar in multimedia computing and applications. His another major accomplishment is his PhD student mentorship. Professor Kuo has guided more than 150 students to their PhD degrees at USC for the last 30 years. This article describes his mentorship philosophy and practice as well as the evolution of his research activities at USC. For the full article, please click here (https://lidsmag.lids.mit.edu/multimedia_and_mentoring.html).

Professor Kuo is listed as the top advisor in the Mathematics Genealogy Project in the number of supervised PhD students. His educational achievements have won a wide array of recognitions such as the 2014 USC Northrop Grumman Excellence in Teaching Award, the 2016 USC Associates Award for Excellence in Teaching, the 2016 IEEE Computer Society Taylor L. Booth Education Award, the 2016 IEEE Circuits and Systems Society John Choma Education Award, the 2016 IS&T Raymond C. Bowman Award, the 2017 IEEE Leon K. Kirchmayer Graduate Teaching Award, the 2017 IEEE Signal Processing Society Carl Friedrich Gauss Education Award, and the 2018 USC Provost’s Mentoring Award.

By |March 30th, 2020|News|Comments Off on Professor Kuo Highlighted by MIT LIDS Magazine|
Read More

MCL Research on Statistics-based Attention

Object detection and recognition has always been one of the key challenges in the field of computer vision for finding and identifying objects in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes and scales or even when they are translated or rotated. Many approaches to the task have been implemented over multiple decades, including handcrafted features, machine learning algorithms and deep learning.

Recently breakthrough results have been made via Deep Learning with loads of labelled data for supervised training, while Deep Learning is notorious for lacking in scalability and interpretability. With the advantage of recent proposed scalable and interpretable Pixelhop model [1] and pixelhop++ [2], a new object detection pipeline can be proposed via Object Proposal-> Feature Extraction using SSL ->  Classification, thus a machine learning based Statistic-based attention is the key to generate object proposals.

Apart from Deep Learning, object proposal via visual saliency on single images such as DRFI [3] can be a good start for a machine learning based object proposal. To further take advantage of the statistics from training data, we formulate the weakly supervised object proposal problem into object search with features capable of matching, such as SURF [4]. In the future, we aim to improve these results by further exploration with a query-retrieval based saliency proposal method along with adapted bag of word features.

 

-By Hongyu Fu

[1] Yueru Chen and C-C Jay Kuo, “Pixelhop: A successive subspace learning (ssl) method for object recognition,” Journal of Visual Communication and Image Representation, p. 102749, 2020.

[2]Yueru Chen , Mozhdeh Rouhsedaghat , Suya You, Raghuveer Rao and C.-C. Jay Kuo, [...]

By |March 22nd, 2020|News, Research|Comments Off on MCL Research on Statistics-based Attention|
Read More

MCL Research on SSL-based Graph Learning

In this research, we proposed an effective and explainable graph vertex classification method, called GraphHop. Unlike the graph convolutional network (GCN) that is based on the end-to-end optimization, the GraphHop method generates an effective feature set for each vertex in an unsupervised and feedforward manner. GraphHop determines the local-to-global attributes of each vertex through successive one-hop information exchange, called the GraphHop unit. The GraphHop method is mathematically transparent. It can be explained using the recently developed “successive subspace learning (SSL)” framework [1, 2], which is mathematically transparent. Unlike GCN that is based on the end-to-end optimization of an objective function using back propagation, GraphHop generates an effective feature set for each vertex in an unsupervised and feedforward manner. Since no backpropagation is required in the feature learning process, the training complexity of GraphHop is significantly lower. By following the traditional pattern recognition paradigm, the GraphHop method decouples the feature extraction task and the classification task into two separate modules, where the feature extraction module is completely un-supervised. In the feature extraction module, GraphHop determines the local-to-global attributes of each vertex through successive one-hop information exchange, called the GraphHop unit. To control the rapid increase of the dimension of vertex attributes, the Saab transform is adopted for dimension reduction inside the GraphHop unit. Multiple Graph-Hop units are cascaded to obtain the higher order proximity information of a vertex. In the classification module, vertex attributes of multiple GraphHop units are extracted and ensembled for the classification task. There are many machine learning tools to be considered. In the experiments, we choose the random forest classifier because of its good performance and low complexity. To demonstrate the effectiveness of the GraphHop method, we apply it to three real-world [...]

By |March 17th, 2020|News, Research|Comments Off on MCL Research on SSL-based Graph Learning|
Read More

MCL Research Presented at WACV 2020

MCL member, Junting Zhang presented her paper at 2020 Winter Conference on Applications of Computer Vision (WACV ’20), in Snowmass village, Colorado. The title of Junting’s paper is “Class-incremental Learning via Deep Model Consolidation”, with Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, C.-C. Jay Kuo as co-authors. Here is a brief summary of Junting’s paper:

“Deep neural networks (DNNs) often suffer from “catastrophic forgetting” during incremental learning (IL) — an abrupt degradation of performance on the original set of classes when the training objective is adapted to a newly added set of classes. Existing IL approaches tend to produce a model that is biased towards either the old classes or new classes, unless with the help of exemplars of the old data. To address this issue, we propose a class-incremental learning paradigm called Deep Model Consolidation (DMC), which works well even when the original training data is not available. The idea is to first train a separate model only for the new classes, and then combine the two individual models trained on data of two distinct set of classes (old classes and new classes) via a novel double distillation training objective. The two existing models are consolidated by exploiting publicly available unlabeled auxiliary data. This overcomes the potential difficulties due to the unavailability of original training data. Compared to the state-of-the-art techniques, DMC demonstrates significantly better performance in image classification (CIFAR-100 and CUB-200) and object detection (PASCAL VOC 2007) in the single-headed IL setting.”

Junting was also invited to attend the WACV 2020 Doctoral Consortium (WACVDC) to present her research and progress to date. She also shared this experience with us:

“It was a great opportunity to interact with experienced researchers in [...]

By |March 8th, 2020|News|Comments Off on MCL Research Presented at WACV 2020|
Read More

Welcome Professor Junsong Yuan’s Visit to USC/MCL

Prof. Junsong Yuan visited USC/MCL on Feb. 25, and delivered a talk on “Beyond Deep Recognition: Discovering Visual Patterns in Big Visual Data ”. Thanks to the success of deep learning, many computer vision tasks nowadays are formulated as regression problems, which, however, often relies on large amounts of annotated training data to make the high-dimensional regression successful. In this talk, Prof. Yuan discussed a complementary yet overlooked problem beyond deep visual recognition and regression. He addressed why and how to discover visual patterns in images and videos that are not annotated, e.g., unsupervised and weakly-supervised visual learning and pattern discovery, and explore how to utilize them to better model, search, and interpret big visual data. Applications in visual search, object detection, action recognition, and video analytics were also explored. 

Junsong Yuan is an Associate Professor and Director of Visual Computing Lab of CSE Department, State University of New York at Buffalo. Before that he was an Associate Professor at Nanyang Technological University (NTU), Singapore. He received his PhD from Northwestern University and M.Eng. from National University of Singapore. He is currently Associate Editor of IEEE Trans. on Image Processing (T-IP) and Machine Vision and Applications (MVA), and Senior Area Editor of Journal of Visual Communication and Image Representation (JVCI), and served as program co-chair for ICME 2018 and area chair for CVPR/ACM MM/WACV/ACCV/ICIP/ICPR etc. He received Best Paper Award from IEEE Trans. on Multimedia, Nanyang Assistant Professorship from NTU, and Outstanding EECS Ph.D. Thesis award from Northwestern University. He is a Fellow of International Association of Pattern Recognition (IAPR).

By |March 2nd, 2020|News|Comments Off on Welcome Professor Junsong Yuan’s Visit to USC/MCL|
Read More

Welcome New MCL Member Hamza Ghani

We are so glad to welcome our new MCL member, Hamza Ghani! Here is a short interview with Tian:

1. Could you briefly introduce yourself and your research interests?

My name is Hamza Ghani and I am from Austin, Texas. I am a graduate student here at USC pursuing a Masters in Electrical Engineering. I went to UT Austin for my undergrad which was in ECE focusing on computer engineering. I also currently work full-time as a Data Scientist while pursuing my Masters. My research interests include: Machine Learning, Graphs and GANs.

2. What is your impression about MCL and USC?

All the members I’ve met in MCL are very knowledgeable in several topics. I am definitely learning a lot by interacting with everyone.  Additionally, everyone I’ve worked with in MCL has been great/enjoyable to work with. I want to thank Professor Kuo for giving me a chance to join the MCL lab and I don’t think my USC experience would be the same without MCL. USC has been great so far, the campus is really nice and it’s easy to make friends even outside of my major.

3. What is your future expectation and plan in MCL?

My current goal is to successfully complete the project my team is currently working on regarding model compression. Overall I want to keep learning through research work, publish papers and make connections with my peers in the lab. 

By |February 25th, 2020|News|Comments Off on Welcome New MCL Member Hamza Ghani|
Read More