News

Usually, the early learning-based point cloud classification methods were developed under the assumption that all point clouds in the dataset are well aligned with the canonical axes. In such scenarios, the 3D Cartesian point coordinates were to learn features. As a consequence, when input point clouds were not aligned, the classification performance dropped significantly. The same assumption holds true in the PointHop and PointHop++ methods proposed by MCL.

In our work SO(3)-Invariant PointHop (or S3I-PointHop in short), we analyze the reason for failure of PointHop due to pose variations, and solve the problem by replacing its pose dependent modules with rotation invariant counterparts. Furthermore, we significantly simplify the PointHop pipeline by using only one single hop along with multiple spatial aggregation techniques. We begin by aligning the point cloud to its three principal axes. This offers a coarse alignment and comes with several ambiguities such as due to eigen vector sign and object asymmetries. The feature extraction process consists of constructing local and global point features. The geometric features are derived from distances and angles in a local point cloud neighborhood. Similarly, the covariance features are found by performing eigen decomposition of the local covariance matrix. The geometric and covariance features form the set of local features. The global features comprise of omni-directional octant features of points in the 3D space similar to PointHop. Later, the Saab transform is conducted.

For aggregating the local and global point features into a global shape feature, conical and spherical aggregation is proposed. For conical aggregation, along each positive and negative principal axes, cones with tip at the origin and at unit distance along the axis are constructed. Then, only the features of points lying inside the respective cones are [...]

Videos captured under low light conditions are often noisy and of poor visibility. Low-light video enhancement aims to improve viewers’ experience by increasing brightness, suppressing noise, and amplifying detailed texture.  The performance of computer vision tasks such as object tracking and face recognition can be severely affected under low-light noisy environments.  Hence, low-light video enhancement is needed to ensure the robustness of computer vision systems. Besides, the technology is highly demanded in consumer electronics such as video capturing by smart phones.

A self-supervised adaptive low-light video enhancement (SALVE) method is proposed in this work. SALVE first conducts an effective Retinex-based low-light image enhancement on a few key frames of an input low-light video. Next, it learns mappings from the low- to enhanced-light frames via Ridge regression.  Finally, it uses these mappings to enhance the remaining frames in the input video. SALVE is a hybrid method that combines components from a traditional Retinex-based image enhancement method and a learning-based method. The former component leads to a robust solution which is easily adaptive to new real-world environments. The latter component offers a fast, computationally inexpensive and temporally consistent solution. We conduct extensive experiments to show the superior performance of SALVE. Our user study shows that 87% of participants prefer SALVE over prior work.

First figure shows an overview of the proposed SALVE method.  For intra-coded frames (I frames), it estimates an illumination component and a reflectance component using the NATLE method.  For inter-coded frames (P/B frames), it predicts these components using a ridge regression learned from the last raw and enhanced I frame pairs.

Second figure shows a quantitative comparison table between our low-light video enhancement method and prior work. To further demonstrate the effectiveness of our method, we [...]

At the beginning of 2023, We wish all MCL members a more wonderful year with everlasting passion and courage!

Image credit:

VectorStock.com/43400056

Alamy Stock Photo

Congratulations to Min Zhang for passing her defense on Dec 9, 2022. Her PhD dissertation is titled with “Explainable and Green Solutions to Point Cloud Classification and Segmentation”. Her dissertation Committee members include Prof. C.-C. Jay Kuo (Chair), Keith Jenkins, and Prof. Stefanos Nikolaidis (Outside member). Min’s presentation was highly praised by the Committee. We invite Min Zhang here to share an abstract of her thesis and her defense experience. We wish Min Zhang all the best for her future career and life!

Point cloud processing is a fundamental but challenging research topic in the field of 3D computer vision, we specifically study two point cloud processing related problems — point cloud classification and point cloud segmentation. Given a point cloud as the input, the goal of classification is to label every point cloud as one of the object categories and the goal of segmentation is to label every point as one of the semantic categories. State-of-the-art point cloud classification and segmentation methods are based on deep neural networks. Although deep-learning-based methods provide good performance, their working principle is not transparent. Furthermore, they demand huge computational resources (e.g., long training time even with GPUs). Since it is challenging to deploy them in mobile or terminal devices, their applicability to real world problems is hindered. To address these shortcomings, we design explainable and green solutions to point cloud classification and segmentation.

We first propose an explainable machine learning method, PointHop, for point cloud classification and further improve its model complexity and performance in PointHop++. Then, we extend the PointHop method to do explainable and green point cloud segmentation. Specifically, an unsupervised feedforward feature (UFF) learning scheme for joint classification and part segmentation of 3D point clouds and [...]

Despite prolific work on evaluating generative models, little research has been done on the quality evaluation of an individual generated sample. To address this problem, a lightweight generated sample quality evaluation (LGSQE) method is proposed in this work. In the training stage of LGSQE, a binary classifier is trained on real and synthetic samples, where real and synthetic data are labeled by 0 and 1, respectively. In the inference stage, the classifier assign soft labels (ranging from 0 to 1) to each generated sample. The value of soft label indicates the quality level; namely,the quality is better if its soft label is closer to 0. LGSQE can serve as a post-processing module for quality control. Furthermore, LGSQE can be used to evaluate the performance of generative models, such as accuracy, AUC, precision and recall, by aggregating sample-level quality. Experiments are conducted on CIFAR-10 and MNIST to demonstrate that LGSQE can preserve the same performance rank order as that predicted by the Frechet Inception Distance (FID) but with significantly lower complexity.

Fig. 1 shows the pipeline of the proposed method. The LGSQE method consists of three cascaded modules:

Module 1: Representation Learning. effective local and global representations of images are learned based upon PixelHop framework.

Module 2: Discriminant Feature Test (DFT). Use DFT to choose powerful features from large numbers of representations obtained from Module 1 against a particular task.

Module 3: Binary Classification for Evaluation. We partition the real/generated data into training and testing sets. A binary classifier is trained on the union of real and generated training samples, which are labeled with “0” and “1”, respectively. The classifier assigns a soft score, to each testing sample as the sample quality index.

Fig. 2 shows the evaluation of generated [...]