Machine learning has witnessed a rapid increase in the amount of training/testing data, feature dimensions and class number due to the arrival of the big data era. In many applications, the systems are expected to deal with a very large number of classes and a huge amount of training/test data. These impose major challenges in: 1) classification accuracy, 2) model complexity in terms of the number of model parameters, and 3)  computational complexity in terms of training and testing costs.  Although deep-learning-based (DL-based) systems can provide good performance in many application contexts, their model sizes are large and training complexities are high.

A popular machine learning tool is the XGBoost [1], which is able to achieve excellent performance on many tasks. XGBoost is a boosting algorithm, which combines multiple weak classifiers to form a more powerful classifier. The XGBoost adds a tree in each iteration, which models the residue from the last iteration. We can simply sum the output from each tree to obtain the final prediction. For classification, XGBoost supports both binary classification and the multiclass prediction. However, multiclass classification tasks with XGBoost can take a very long time if the class number is huge. Hence, in our research, we aim to speed up the multiclass XGBoost.

Image credits:

[1] Chen, T., Guestrin, C., 2016.   Xgboost:  A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794.