News

Image super-resolution (SR) is a classic image reconstruction problem in computer vision (CV), which aims at recovering a high-resolution image from a low-resolution image. As a type of supervised generative problem, image SR attracts wide attention due to its strong connection with other CV topics, such as object recognition, object alignment, texture synthesis and so on. Besides, it has extensive applications in real world, for example, medical diagnosis, remote sensing, biometric information identification, etc.

For the state-of-the-art approaches for SR, typically there are two mainstreams: 1) example-based learning methods, and 2) Deep Learning (CNN-based) methods. Example-based methods either exploit external low-high resolution exemplar pairs [1], or learn internal similarity of the same image with different resolution scales [2]. However, features used in example-based methods are usually traditional gradient-related or just handcraft, which may affect model performance. While CNN-based SR methods (e.g. SRCNN [3]) does not really distinguish between feature extraction and decision making. Lots of basic CNN models/blocks are applied to SR problem, e.g. GAN, residual learning, attention network, and provide superior SR results. Nevertheless, the non-explainable process and exhaustive training cost are serious drawbacks of CNN-based methods.

By taking advantage of reasonable feature extraction [4], we utilize spatial-spectral compatible cw-Saab features to express exemplar pairs. In addition, we formulate a Successive-Subspace-Learning-based (SSL-based) method to gradually partition data into subspaces by feature statistics, and apply regression in each subspace for better local approximation. By visualization the samples in representative subspaces, we find obvious sample similarity in pixel domain. This demonstrates the efficiency of our method in splitting samples into subspaces with semantic meaning. In the future, we aim at providing such a SSL-based explainable method with high efficiency for SR problem.

—  By Wei Wang

Reference:

[1] Timofte, Radu, [...]

Point cloud registration refers to the process of aligning two point clouds. The two point clouds to be aligned are commonly called source and target. The goal is to find a spatial transformation (3D rotation and translation) that needs to be applied to the source to optimally align it with the target.  Registration has become popular with the proliferation of 3D scanning devices like LiDAR and their applications in autonomous driving, robotics, graphics, mapping, etc.

Point clouds need to be registered in order to merge data from different sensors to obtain a globally consistent view, mapping a new observation to known data, etc. Registration is challenging due to several reasons. The source and the target point clouds may have different sampling densities and different number of points. Point clouds may contain outliers and/or be corrupted by noise. Sometimes, only partial views are available.

The problem of registration (or alignment) has been studied for a long while. Prior to point cloud processing, the focus has been on aligning lines, parametric curves and surfaces. The classical Iterative Closest Point (ICP) algorithm alternates between finding corresponding points and estimating the optimal rotation and translation. ICP just uses the spatial coordinates of points to establish point correspondences. More recently there has been a trend to use deep learning, feature based methods for registration. Two such popular methods include PointNetLK and Deep Closest Point (DCP). PointNetLK and DCP treat registration as a supervised learning problem and train end-to-end networks using deep learning. The supervision is in terms of class labels and ground truth rotation matrix and translation vector. We propose a method called ‘Salient Points Analysis (SPA)’ [1] for registration.  In contrast with the recent deep learning methods, our SPA method [...]

The title of his Ph.D. thesis proposal is “Theory and Applications of Adversarial and Structured Knowledge Learning”. His qualifying exam committee consisted of C.-C. Jay Kuo (Chair), Keith Michael Chugg, Keith Jenkins, Rahul Jain and Stefanos Nikolaidis.

Abstract of thesis proposal:

Deep learning has brought impressive improvements for many tasks, thanks to end-to-end data-driven optimization. However, people have little control over the system during training and limited understanding about the structure of knowledge being learned. In this thesis proposal, we study theory and applications of adversarial and structured knowledge learning: 1) learning adversarial knowledge with human interaction or by incorporating human-in-the-loop; 2) learning structured knowledge by modelling contexts and users’ preferences.

In the first category, our research topics include human-robot adversarial learning; Human-guided curriculum reinforcement learning and PortraitGAN for simultaneous emotion and modality manipulations. In the second category, a real-world compatible recommendation problem was tackled with structural graph representation and deep metric learning. The two categories are also related in the sense that structured knowledge often help lay a solid foundation, on which adversarial knowledge can be learned more successfully. Additionally, we contribute technically by open-sourcing relevant platforms.

Yeji Shen is a PhD candidate in Multimedia Communication Lab (MCL) in USC, supervised by Prof. C.-C. Jay Kuo. He received his Bachelor’s degree in Computer Science from Peking University, Beijing, China in June 2016. Since August 2016, he has been pursuing his PhD degree in MCL. His research interests include Machine Learning, Computer Vision and Artificial Intelligence. During this summer, he did an internship at Facebook. Here is a short interview with Yeji.

1. How does the study in USC and MCL help you?

First of all, in MCL, I learned to have a reasonable understanding of the research topics that I’ve been focusing on, like active learning, 3D vision and some semi-supervised learning. Such understanding is pretty helpful and valuable for both job interviews and the actual working experience. Second, I got to have a reasonable level of presentation skills, which I believe is very important in the future career. Third, a tough mind. Life is challenging. Only those with a tough mind can get through all those difficulties and obtain happiness.

2. How was it like working at Facebook?

The internship this year was a remote one. Different from normal working style, interns needed to work at home with the equipment sent by the company. (Of course, I need to mail them back.) Compared to a normal internship, the main pros are: 1) No need to physically move to the bay area. And thus the fee for house rent was saved. 2) Commuting time was saved. However, it is also clear that some cons are: 1) Harder to communicate. 2) Less interaction with team members. 3) It just didn’t feel good when the remote working style lasts for too long. Still, the overall feeling was not bad.

3. [...]

Jiali Duan is a PhD candidate supervised by Prof. C-.C. Jay Kuo at MCL. He interned at Amazon A9 during Summer. Here is a short interview with Jiali.

1. How does the study in USC and MCL help you? (technically and psychologically)

First, MCL helped me lay a solid foundation for research and communication. Academic and technical exchange of

ideas happen almost daily for applied scientist intern at Amazon, making it a necessity to communicate concisely and logically. Thankfully, I got trained for this aspect at MCL by sticking to weekly report and regular personal meetings. Second, the ability to prototype new ideas is intensely tested during internship. Original research here at MCL prepare us well for this aspect.

2. How was it like working at Amazon A9?

I had two internship experience at A9, a physical one in last year and a remote one this year. Generally, the physical one is much better. Last year, I was working at A9 Palo Alto. The company is located at University St, within walking distance to StandFord. Due to its location, there’re many nice restaurants and the neighborhood is very safe. The internship program was also very kind to provide free baseball/football match tickets, Santa Cruz tour and Seattle headquarter tour.

3. Do you have any suggestions for current graduate students?

I know that some companies provide more software development positions than research positions and some allow limited number of interview trials from the same person. So, be ready when you try. In terms of suggestions, be prepared for questions that reach beyond your scope of knowledge, which may require certain amount of improvision.