Feature extraction and decision-making are two modules in cascade in the classical pattern recognition (PR) or machine learning (ML) paradigm. We recently proposed a novel learning diagram named Subspace Learning Machine (SLM) which considers this learning paradigm and focus on specific modules for classification-oriented decision making. SLM can be viewed as a generalized version of Decision Tree (DT). The linear combination of multiple features can be written as the inner product of a projection
vector and a feature vector. The effectiveness of SLM depends on the selection of good projection vectors, e.g. when the projection vector is a one-hot
vector, SLM is nothing but DT.

Both SLM and DT apply a hard split to a feature using a threshold at a decision node. To overcome the cons of hard feature space partitioning, we propose a new SLM method that adopts soft partitioning and denote it with SLM/SP in this proposed work. A comparison between hard decision and soft decision is illustrated in Fig 1. SLM/SP adopts the soft decision tree (SDT) data structure and a novel topology is proposed with inner nodes of SDT for data routing, leaf nodes of SDT for local decision making, and edge between parent and child nodes for representation learning. Specific modules are designed for the nodes and edges, respectively. The training of a SLM/SP tree starts by learning an adaptive tree structure via local greedy exploration between subspace partitioning and feature subspace learning. The tree structure is finalized once the stopping criteria are met for all
leaf nodes, and all module parameters are updated globally.

The overall frame working using Successive Subspace Learning and SLM/SP for image classification is as shown in Fig 2. The structure of the SLM/SP tree is adaptive to various learning capabilities with specific module designs for the nodes and egdes in the tree topology. For example, when utilizing identity function for the edge modules, the SLM/SP tree reduces to a decision-learning model. For the efficient setting, we proposed a novel local feature learning block named Subspace Learning enhanced Block (SLAB) for edge modules. SLAB utilize the depthwise separable convolution for efficient convolution design, squeeze-and-excitement block for enhancing the local representational power of feature map, and linear feature subspace learning for preserving the information on manifold of interest. We leverage global average pooling and single hidden layer MLP for inner node modules and linear classifier for leaf node modules to achieve the most efficient and high performance models.

The proposed methodology enables efficient training, high classification accuracy, and a small model size. It is shown by experimental results that an SLM/SP tree offers a lightweight and high performance classification solution.