LightGBM
Original author(s) | Guolin Ke[1] / Microsoft Research |
---|---|
Developer(s) | Microsoft and LightGBM contributors[2] |
Initial release | 2016 |
Stable release | v4.3.0[3]
/ January 15, 2024 |
Repository | github |
Written in | C++, Python, R, C |
Operating system | Windows, macOS, Linux |
Type | Machine learning, gradient boosting framework |
License | MIT License |
Website | lightgbm |
LightGBM, short for light gradient-boosting machine, is a
Overview
The LightGBM framework supports different algorithms including GBT,
LightGBM works on
Gradient-based one-side sampling
Gradient-based one-side sampling (GOSS) is a method that leverages the fact that there is no native weight for data instance in GBDT. Since data instances with different gradients play different roles in the computation of information gain, the instances with larger gradients will contribute more to the information gain. So to retain the accuracy of the information, GOSS keeps the instances with large gradients and randomly drops the instances with small gradients.[13]
Exclusive feature bundling
Exclusive feature bundling (EFB) is a near-lossless method to reduce the number of effective features. In a sparse feature space many features are nearly exclusive, implying they rarely take nonzero values simultaneously. One-hot encoded features are a perfect example of exclusive features. EFB bundles these features, reducing dimensionality to improve efficiency while maintaining a high level of accuracy. The bundle of exclusive features into a single feature is called an exclusive feature bundle.[13]
See also
References
- ^ "Guolin Ke". GitHub.
- ^ "microsoft/LightGBM". GitHub. 7 July 2022.
- ^ "Releases · microsoft/LightGBM". GitHub.
- ^ Brownlee, Jason (March 31, 2020). "Gradient Boosting with Scikit-Learn, XGBoost, LightGBM, and CatBoost".
- PMID 32686721– via www.nature.com.
- ^ "Understanding LightGBM Parameters (and How to Tune Them)". neptune.ai. May 6, 2020.
- ^ "An Overview of LightGBM". avanwyk. May 16, 2018.
- ^ "Parameters — LightGBM 3.0.0.99 documentation". lightgbm.readthedocs.io.
- ^ The Gradient Boosters IV: LightGBM – Deep & Shallow
- ^ XGBoost, LightGBM, and Other Kaggle Competition Favorites | by Andre Ye | Sep, 2020 | Towards Data Science
- CiteSeerX 10.1.1.89.7734.
- ^ "Features — LightGBM 3.1.0.99 documentation". lightgbm.readthedocs.io.
- ^ a b c Ke, Guolin; Meng, Qi; Finley, Thomas; Wang, Taifeng; Chen, Wei; Ma, Weidong; Ye, Qiwei; Liu, Tie-Yan (2017). "LightGBM: A Highly Efficient Gradient Boosting Decision Tree". Advances in Neural Information Processing Systems. 30.
- ^ "lightgbm: LightGBM Python Package". 7 July 2022 – via PyPI.
- ^ "Microsoft.ML.Trainers.LightGbm Namespace". docs.microsoft.com.
- ^ "microsoft/LightGBM". October 6, 2020 – via GitHub.
Further reading
- Guolin Ke; Qi Meng; Thomas Finely; Taifeng Wang; Wei Chen; Weidong Ma; Qiwei Ye; Tie-Yan Liu (2017). "LightGBM: A Highly Efficient Gradient Boosting Decision Tree" (PDF). Neural Information Processing System.
- Quinto, Butch (2020). Next-Generation Machine Learning with Spark – Covers XGBoost, LightGBM, Spark NLP, Distributed Deep Learning with Keras, and More. ISBN 978-1-4842-5668-8.
- van Wyk, Andrich (2023). Machine Learning with LightGBM and Python. ISBN 978-1800564749.