News

3D point cloud segmentation requires the understanding of both the global geometric structure and the fine-grained details of each point. According to the segmentation granularity, 3D point cloud segmentation methods can be classified into three categories: semantic segmentation (scene level), instance segmentation (object level) and part segmentation (part level). Our research focuses on solving the semantic segmentation problem. Given a point cloud, the goal of semantic segmentation is to separate it into several subsets according to the semantic meanings of points. There are two common scenes, urban scene and indoor scene. Efficient semantic segmentation of large-scale 3D point clouds is a fundamental and essential capability for real-time intelligent systems, such as autonomous driving and augmented reality. We mainly try to do the large-scale 3D indoor scene semantic segmentation more efficiently. The representative large-scale public is the indoor S3DIS dataset [1].

A key challenge is that the raw point clouds acquired by depth sensors are typically irregularly sampled, unstructured and unordered.  Recently, the pioneering work PointNet [2] was proposed for directly processing 3D point clouds by learning per-point features using shared multilayer perceptrons (MLPs) and max pooling. Its following works try to capture wider context information for each point. Although these approaches achieve impressive results for object recognition and semantic segmentation, almost all of them are limited to extremely small 3D point clouds and cannot be directly extended to larger scale without preprocessing such as block partition.

We design a different data preprocessing method to learn large scale data directly. Each room in the dataset is treated as an input sample and feed into an unsupervised feature extractor to obtain point-wise features. The unsupervised feature extractor is developed upon our previous work PointHop [3]. It is extremely [...]

Video object tracking is one of the fundamental computer vision problems and has found rich applications in video surveillance, autonomous navigation, robotics vision, etc. In the setting of online single object tracking (SOT), a tracker is given a bounding box on the target object at the first frame and then predicts its boxes for all remaining frames. Online tracking methods can be categorized into two categories, unsupervised and supervised. Traditional trackers are unsupervised. Recent deep-learning-based (DL-based) trackers demand supervision. Unsupervised trackers are attractive since they do not need annotated boxes to train supervised trackers. The performance of trackers can be measured in terms of accuracy (higher success rate), robustness (automatic recovery from tracking loss), and speed (higher FPS).

We examine the design of an unsupervised high-performance tracker and name it UHP-SOT (Unsupervised High-Performance Single Object Tracker) in this work. UHP-SOT consists of three modules: 1) appearance model update, 2) background motion modeling, and 3) trajectory-based box prediction. Previous unsupervised trackers pay attention to efficient and effective appearance model update. Built upon this foundation, an unsupervised discriminative-correlation-filters-based (DCF-based) tracker STRCF [1] is adopted by UHP-SOT as the baseline in the first module. Yet, the use of the first module alone has shortcomings such as failure in tracking loss recovery and being weak in box size adaptation. We propose ideas for background motion modeling and trajectory-based box prediction to address the mentioned problems. The baseline tracker gets initialized at the first frame. For the following frames, UHP-SOT gets proposals from all three modules and chooses one of them as the final prediction based on a fusion strategy, as shown in Fig. 1. Fig. 2 shows example results on sequences from the OTB-2015 [2] benchmark. Our tracker runs [...]

Image classification has been studied for many years as a fundamental problem in computer vision. With  the development of convolutional neural networks (CNNs) and the availability of larger scale datasets, we see a rapid success in the classification using deep learning for both low- and high-resolution images. Although being effective, one major challenge associated with deep learning is that its underlying mechanism is not transparent.

Being inspired by deep learning, the successive subspace learning (SSL) methodology was proposed by Kuo et.al. in a sequence of papers. Different from deep learning, SSL-based methods learn feature representations in an unsupervised feedforward manner using multi-stage principle component analysis (PCA). Joint spatial-spectral representations are obtained at different scales through multi-stage transforms. Three variants of the PCA transform were developed. They are the Saak transform [1], the Saab transform [2], and the channel-wise (c/w) Saab transform [4]. Two SSL-based image classification pipelines, PixelHop [3] and PixelHop++ [4], were designed based on the Saab transform and c/w Saab transform respectively. Both follow the  traditional  pattern  recognition  paradigm  and  partition  the classification  problem  into  two  cascaded  modules: 1) feature extraction and 2) classification. Every step in PixelHop/PixelHop++ is explainable, and the whole solution is mathematically transparent.

To further improve the performance, we propose a SSL-based two-stage sequential image classification pipeline, named E-PixelHop method. The motivation is that for a multi-class classification problem, it is easier to distinguish between classes of dissimilarity than those of similarity. For example, one should distinguish between cats and cars better than between cats and dogs. Along this line, one can build a hierarchical relation among multiple classes based on their semantic meaning to improve classification performance. Instead of manually constructing the hierarchical learning structure before classification, E-PixelHop resolves [...]

As a fundamental Computer vision problem, image denoising aims at reducing noise images to improve resolutions.  As a sub-topic of image restoration, image denoising not only has wide applications in practical problems, but also can be important pre-processing procedures for other CV or NLP problems.

Traditionally, algorithms by patch-wise denoising, like Non-local Mean and BM3D, usually assume the noise are Gaussian noise try to reduce noise by the randomness of noise and signal preservation across similar patches. After CNN architecture introduced to CV field, similar to other image restoration problems like super-resolution, deblurring, and dehazing, denoising problem also developed out CNN-based methods, with two main streams: one focus more on pixel-wise restoration and the other cares more about overall pleasure. Besides, combining different image restoration problems together, that building a more general model which can work on multiple image restoration problems gradually obtained more attention.

With better performance achieved in denoising problem, more and more algorithms suffer from the model size and reference speed. We would like to introduce SSL principle to tackle denoising problem with comparable performance while with higher efficiency in the future.

Image credit: Dabov, Kostadin, et al. “Image denoising by sparse 3-D transform-domain collaborative filtering.” IEEE Transactions on image processing 16.8 (2007): 2080-2095.

In Summer 2021, we have a new MCL member, Peida Han, joining our big family. Here is a short interview with Peida with our great welcome.

1. Could you briefly introduce yourself and your research interests?

My name is Peida Han, and I am a first year master student in Computer Science (artificial intelligence) at USC. I received my Bachelor’s degree in Computer Science and Engineering from the Ohio State University in 2016. I previously worked on some machine learning based projects such as an autonomous aerial system on drones in my undergrad. With strong interest in image processing for practical real life applications, I am currently working on the Breast Cancer Image segmentation project in MCL.

2. What is your impression about MCL and USC?

My impression about MCL is that the lab members are friendly and motivating. I feel everyone is approachable and the whole group like to help each other out. In addition, everyone is dedicated to their work and I am inspired to work hard and learn from them. USC provides valuable resources from the perspectives of both academia and industry. My impression of USC is that students have access to resources easily and professors have high standards of course quality, and there are many other valuable resources. I am glad to be a part of the Trojan family.

3. What is your future expectation and plan in MCL?

My expectation in MCL is to explore my potentials in pure research, especially in the image processing field. And I am glad I can be involved in the research of image segmentation and hope that could be helpful to society. I learnt great a lot from Prof. Kuo and I hope I can contribute my own efforts in MCL. It [...]