ZhiruoZhou

Point Cloud Compression (PCC) has received a lot of attention in recent years due to its wide applications such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). Video-based PCC (V-PCC) and geometry-based PCC (G-PCC) are two distinct technologies developed by MPEG 3DG[1][2]. Deep-learning-based (DL-based) PCC is a strong competitor to them. Most DL methods generalize the DL-based image coding pipeline to the point cloud data [3][4]. They outperform G-PCC in the current MPEG 3DG standard in the dense point cloud compression. Yet, their performances are still inferior to that of V-PCC in the coding of dynamic point clouds.

We propose to design a learning-based PCC solution that could outperform those DL-based methods with lower complexity and less memory consumption. Our method uses geometry projection to generate 2D images and apply vector quantization-based 2D image codec to compress the projected map. For a point cloud sequence, we can do the projection in three steps. First, split the sequence into blocks by doing the octree partition. Second, project each 3D block into a plane and pack all the planes into a map. Third, encode/decode the 2D map and reconstruct the 3D point cloud sequence. They are demonstrated in Fig.1. We do the non-uniform sampling for the projected planes and pack all the planes to generate one depth map and one texture map in the reconstruction process. The two maps are shown in Fig.2.

Presently, we utilize the x264/x265 codec to code the maps. In the future, we will adopt a vector quantization-based image codec to compress the two maps.

— Qingyang Zhou

Reference

[1] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivoku ́ca, S. Lasserre, Z. Li et [...]

Odometry is the process of using motion sensors to estimate the change in position of an object over time. It has been widely studied in the context of mobile robots, autonomous vehicles, drones, and other moving agents. Traditional odometry based on motion sensors such as Inertial Measurement Unit (IMU) and magnetometers is prone to error accumulation over time, known as odometry drift. Visual odometry makes use of camera images and/or point cloud scans collected over time to determine the position and orientation of the moving object. Several visual odometry systems that integrate monocular, stereo vision, point clouds and IMU have been developed for object localization.
We propose an unsupervised learning method for visual odometry from LiDAR point clouds called Green Point Cloud Odometry (GPCO). GPCO follows the traditional scan matching based approach to solve the odometry problem by incrementally estimating the motion between two consecutive point cloud scans. The GPCO method can be divided into four steps. First, a geometry-aware sampling method selects a small subset of points from the input point clouds. To do so, the eigen features of points in a local neighborhood are considered followed by random point sampling. Next, the 3D view surrounding the moving object is partitioned into four parts representing the front, rear, left and right-side view. The view-partitioning step divides the sampled points into four disjoint sets. The features of the sampled points are derived using the PointHop++ [1] method. Matching points between two consecutive point clouds are found in each view using nearest neighbor rule in the feature space. Finally, the motion between the two scans is estimated using Singular Value Decomposition (SVD). The motion is updated to the estimates from previous times and the process [...]

Given a point cloud set, the goal of semantic segmentation is to label every point as one of the semantic categories. Semantic segmentation of large-scale point clouds finds a wide range of real-world applications such as autonomous driving in an out-door environment and robotic navigation in an in- or out-door environment. As compared with the point cloud classification problem that often targets at small-scale objects, a high-performance point cloud semantic segmentation method demands a good understanding of the complex global structure as well as the local neighborhood of each point. Meanwhile, efficiency measured by computational complexity and memory complexity is important for practical real-time systems.

State-of-the-art point cloud classification and segmentation methods are based on deep learning. Raw point clouds captured by the LiDAR sensors are irregular and unordered. They cannot be directly processed by deep learning networks designed for 2D images. This problem was addressed by the pioneering work on the PointNet. PointNet and its follow-ups achieved impressive performance in small-scale point cloud classification and segmentation tasks, but they can’t be generalized to handle large-scale point cloud directly due to the memory and time constraints.

An efficient solution to semantic segmentation of large-scale indoor scene point clouds is proposed in this work. It is named GSIP (Green Segmentation of Indoor Point clouds) [1], and its performance is evaluated on a representative large-scale benchmark — the Stanford 3D Indoor Segmentation (S3DIS) dataset. GSIP has two novel components: 1) a room-style data pre-processing method that selects a proper subset of points for further processing, and 2) a new feature extractor which is extended from PointHop. For the former, sampled points of each room form an input unit. For the latter, the weaknesses of PointHop’s feature extraction when [...]

We are so happy to welcome a new graduate member of MCL, Mahtab Movahhedrad. Here is an interview with Mahtab:

Could you briefly introduce yourself and your research interests?

My name is Mahtab Movahhedrad. I’m from Tabriz, Iran. I started my Ph.D. at USC in spring 2021. I received my bachelor’s and my master’s degree from the University of Tabriz and, Tehran polytechnic respectively. I have been engaged in many different fields of study such as integrated electronics, photonics, metamaterials, and Fourier optics. Now, I plan to gear my Ph.D. Studies more towards signal processing and computer vision and do research in the same area.

What is your impression about MCL and USC?

I have been exploring my passion for computer vision and multimedia for some time now and I think MCL is the best research group in USC when it comes to this field. My motivation to join as a Ph.D. student to MCL is its diverse and outstanding research group in Multimedia. There are about 30 PhD students and several post-doctoral research fellows/visiting scholars from all over the world working in the lab. Doing my PhD at USC, among erudite professors and interested students would create a challenging and at the same time amicable atmosphere for me to broaden my knowledge and collaborate with other scientists.

What is your future expectation and plan in MCL?

My future expectation from the MCL is to learn about green AI and the chance to do research in this field. This method has the potential to accelerate global efforts to protect the environment and conserve resources. I find the concept quite interesting and can’t wait to explore more. I also hope to make lasting connections and effective collaborations with individuals [...]

The 2nd Ph.D. alumnus of MCL, Professor Kyoung Mu Lee of Seoul National University (SNU), has been appointed as the Editor-in-Chief (EiC) of the prestigious IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) from 2022 to 2024. TPAMI publishes articles on areas of computer vision and image understanding, all traditional areas of pattern analysis and recognition, and selected areas of machine intelligence. It is one of the premier journals in all of computer science. Its excellence, combined with its focus on computer vision and machine learning, positions it as one of IEEE’s flagship journals. Its impact factor is 16.39 in 2020.

Kyoung Mu Lee received his Ph. D. degree from the University of Southern California in 1993. He is currently a professor in the department of ECE and the director of the Interdisciplinary Graduate Program in Artificial Intelligence at Seoul National University (SNU). His primary research areas are Computer Vision and Machine Learning. He has served as an Editorial board member of many journals, including an Associate Editor in Chief (AEiC) of the IEEE TPAMI, Area Editor of the Computer Vision and Image Understanding (CVIU), and Associate Editor (AE) of the IEEE TPAMI, the Machine Vision Application (MVA), the IPSJ Transactions on Computer Vision and Applications (CVA), and the IEEE Signal Processing Letter. He is an Advisory Board Member of the Computer Vision Foundation (CVF) and an Editorial Advisory Board Member for Academic Press/Elsevier. He has served as a General co-Chair of the prestigious ICCV2019, ACM MM2018, ACCV2018, and an Area Chair of CVPR, ICCV, and ECCV many times. He was a Distinguished Lecturer of the Asia-Pacific Signal and Information Processing Association (APSIPA) for 2012-2013. He is currently serving as the president of the [...]