Supervised feature learning is a critical component in machine learning, particularly within the green learning (GL) paradigm, which seeks to create lightweight, efficient models for edge intelligence.

Supervised feature learning involves creating new, more discriminant features from an existing set of features, typically produced during an earlier stage of unsupervised representation learning. The objective is to enhance the discriminative power of the features, thereby improving the model’s accuracy and robustness in decision-making tasks such as classification.

The process typically begins with a rich set of representations obtained from the unsupervised module. These representations are then rank-ordered according to their discriminant power using a discriminant feature test (DFT). However, if the initial set of features lacks sufficient discriminant power, new features can be derived through linear combinations of the existing ones. This method transforms a multi-class classification problem into multiple binary classification problems, then applies linear regression to generate new features that are more discriminant than the original ones. These new features are shown to improve the performance of the classifier, demonstrating the effectiveness of the supervised feature learning process within the GL framework.
We proposed the Least-Squares Normal Transform (LNT) for generating new discriminant features. This method transforms a multi-class classification problem into multiple binary classification problems, then applies linear regression to generate new features that are more discriminant than the original ones. These new features are shown to improve the performance of the classifier, demonstrating the effectiveness of the supervised feature learning process within the GL framework.

References:

X. Wang, V. K. Mishra, and C.-C. J. Kuo, “Enhancing edge intelligence with highly discriminant lnt features,” in 2023 IEEE International Conference on Big Data (BigData). IEEE, 2023, pp. 3880–3887