The previous post mentioned an efficient tree ensemble algorithm, the Xgboost model, but its efficiency is still low when dealing with data that has a large feature dimension and a large number of samples. Compared with models such as GBDT and Xgboost, the Lightgbm model open-sourced by Microsoft uses a variety of techniques to reduce the amount of data and the number of features, which greatly speeds up training while keeping accuracy. ...
Today it occurred to me that I have not updated my blog for a long time; lately I have been busy with written tests and interviews, and with my graduation project. Now that things have come to a pause today, I might as well organize the Xgboost model I reviewed a few days ago and post an article. Tree ensemble Ensemble algorithms are a broad and very important class of algorithms in machine learning, generally divided into Bagging and Boosting. Ensembling decision trees is of course a customary technique. For tree ensembles, the representative of the Bagging method is Random Forest, which randomly groups attributes and data for decision tree learning and finally aggregates the results; the representative of the Boosting method is AdaBoost and GBDT (gradient boosted decision trees). These algorithms are all quite important and cannot be summed up in a sentence or two, so I will not expand on them. Within tree Boosting, Tianqi Chen proposed a new model, Xgboost. This model makes use of the second derivative of the objective function and also adds a regularization term to prevent the model from overfitting. Below is an interpretation of the Xgboost model. ...