The subspace technique has been widely used in signal/image processing, pattern recognition, computer vision, etc. It may have different definitions in different contexts. A subspace may denote a dominant feature space where less relevant features are dropped. One example is the principal component analysis (PCA). A subspace may also refer to a certain object class such as the subspace of digit “0” in the MNIST dataset. Generally speaking, subspace methods offer a powerful and popular tool for signal analysis, modeling and processing. They exploit statistical properties of a class of underlying signals so as to determine a smaller yet significant subspace for further processing.
However, existing subspace methods are conducted in a single stage. We may wonder whether there is any advantage to perform subspace methods in multiple stages. Research on generalizing from one-stage subspace methods to multi-stage subspace methods is actually rare. Two PCA stages are cascaded in a straightforward manner in the PCAnet[1]. Being motivated by multiple convolutional layers in convolutional neural networks (CNNs), Prof. Kuo proposed a new machine learning paradigm, called successive subspace learning (SSL). It has multiple subspace modules in cascade by mimicking the feedforward CNN operations, and the parameters of subspace transformation are learned from the training data. Although there is a strong similarity between the feedforward paths of CNNs and the SSL approach, they are fundamentally different in the machine learning model formulation, the training process and complexity.
To illustrate the SSL approach furthermore, Yueru Chen and Prof. Kuo proposed a PixelHop method based on SSL for image-based object recognition. It consists of three steps: 1) local-to-global attributes of images are extracted through multi-hop information exchange, 2) subspace-based dimensionality reduction (SDR) is adopted to new image representation from each PixelHop unit, and 3) feature vectors obtained from multiple PointHop units are concatenated and fed to a classifier. Extensive experiments are conducted on the MNIST, the Fashion MNIST and the CIFAR-10 datasets to demonstrate the advantages of the proposed PixelHop method.
By Yueru Chen
