CC BY
0
№ 1(130)
январь, 2025 г.
COMPUTER SCIENCE, COMPUTER ENGINEERING AND MANAGEMENT
THE ROLE OF GRADIENT BOOSTING MACHINES IN MODERN ECONOMIC ANALYSIS
Azibaev Akhmadkhon
PhD student, Namangan State University, Republic of Uzbekistan, Namangan E-mail: [email protected]
РОЛЬ ПРИМЕНЕНИЯ МАШИН ГРАДИЕНТНОГО УСИЛЕНИЯ В СОВРЕМЕННОМ ЭКОНОМИЧЕСКОМ АНАЛИЗЕ
Ахмадхон Азибаев
Аспирант
Наманганского государственного университета, Республика Узбекистан, г. Наманган
ABSTRACT
The rapid evolution of machine learning (ML) has transformed economic analysis, enabling the exploration of intricate data relationships and enhancing traditional econometric methods [1]. This study investigates the application of Gradient Boosting Machines (GBMs), particularly XGBoost, LightGBM, and CatBoost, in economic forecasting. By analyzing their performance in GDP growth predictions, this paper demonstrates the superiority of GBMs in predictive accuracy, adaptability to complex datasets, and computational efficiency. The findings highlight GBMs' potential to improve decision-making, support real-time forecasting, and offer insights into macroeconomic phenomena. Despite challenges like computational demands and interpretability, GBMs emerge as indispensable tools in modern economic analysis, with significant implications for policy and strategy [2].
АННОТАЦИЯ
Быстрое развитие машинного обучения (ML) преобразовало экономический анализ, позволив исследовать сложные взаимосвязи данных и усовершенствовав традиционные эконометрические методы [1]. В этом исследовании изучается применение машин градиентного усиления (GBM), в частности XGBoost, LightGBM и CatBoost, в экономическом прогнозировании. Анализируя их производительность в прогнозировании роста ВВП, эта статья демонстрирует превосходство GBM в точности прогнозирования, адаптивности к сложным наборам данных и вычислительной эффективности. Результаты подчеркивают потенциал GBM для улучшения принятия решений, поддержки прогнозирования в реальном времени и предоставления информации о макроэкономических явлениях. Несмотря на такие проблемы, как вычислительные требования и интерпретируемость, GBM становятся незаменимыми инструментами в современном экономическом анализе, что имеет значительные последствия для политики и стратегии [2].
Keywords: Gradient Boosting Machines, Economic Forecasting, Machine Learning, XGBoost, LightGBM, Cat-Boost, GDP Growth, Predictive Analytics, Ensemble Learning, Econometrics.
Ключевые слова: машины градиентного усиления, экономическое прогнозирование, машинное обучение, XGBoost, LightGBM, CatBoost, рост ВВП, прогностическая аналитика, ансамблевое обучение, эконометрика.
Introduction
Machine learning (ML) has the potential to augment traditional econometric methodologies by uncovering intricate relationships and patterns within data, which can subsequently be integrated into econometric models [3]. As the digital economy evolves and the complexity of economic data increases, ML emerges as a critical instrument in the advancement of economic analysis. This article aims to explore the role of Gradient Boosting
Machines (GBMs) in economic applications, examining how these advanced machine learning models can enhance traditional econometric methods and provide deeper insights into complex economic phenomena. The use of machine learning models, particularly gradient boosting and random forest approaches, has shown considerable promise in economic forecasting and financial analysis.
Yoon's research focuses on forecasting Japan's real GDP growth from 2001 to 2018 using gradient boosting
Библиографическое описание: Azibaev A. THE ROLE OF GRADIENT BOOSTING MACHINES IN MODERN ECONOMIC ANALYSIS // Universum: технические науки : электрон. научн. журн. 2025. 1(130). URL:
https://7universum.com/ru/tech/archive/item/19102
№ 1 (130)
ЛД1 ¿ лл
UNIVERSUM:
ТЕХНИЧЕСКИЕ НАУКИ
and random forest models. By employing cross-validation to optimize hyperparameters, Yoon demonstrates that these models outperform traditional benchmarks like those from the International Monetary Fund and the Bank of Japan. Notably, the gradient boosting model proves to be the most accurate, highlighting the potential of machine learning to enhance macroeconomic forecasting. [4] Touzani, Granderson, and Fernandes investigate the application of gradient boosting machines to model energy consumption in commercial buildings. Their study shows that gradient boosting significantly improves prediction accuracy compared to traditional models such as piecewise linear regression and random forests. This demonstrates the model's effectiveness in handling large datasets and its adaptability to different domains.[5]
Qureshi, Chu, and Demers apply XGBoost to forecast Canadian GDP growth using both Google* Trends and official data. By employing a rigorous feature selection process and Principal Component Analysis, they achieve superior forecasting accuracy compared to traditional methods. Their work underscores the value of machine learning in nowcasting economic indicators, even when official data is delayed. [6] Carmona, Climent, and Momparler use extreme gradient boosting to predict bank failures in the U.S. banking sector. Analyzing financial ratios from 156 banks over 15 years, they identify key indicators of financial distress. Their findings demonstrate the utility of gradient boosting models in financial risk assessment, providing valuable insights for regulatory actions.[7]
Collectively, these studies underscore the growing importance of machine learning models in economic and financial analysis, offering enhanced accuracy and timely predictions across various sectors.
Methodology
Gradient Boosting Machines (GBMs): An ensemble machine learning technique that builds a model as a sequence of weak learners (typically decision trees), each focusing on correcting the errors of the previous ones. (GBMs) are an ensemble learning technique that builds predictive models through an iterative process. The core idea is to combine multiple weak learners - typically decision trees - into a strong predictive model.
Several key algorithms have emerged as leading implementations of Gradient Boosting Machines, each with unique features that enhance their performance in various applications, including economic modeling. The most prominent among these are XGBoost, LightGBM, and CatBoost.
XGBoost (Extreme Gradient Boosting)
XGBoost is one of the most popular and widely used implementations of gradient boosting. It is known for its speed and efficiency, particularly with large datasets, and for its ability to handle sparse data effectively.
Relevance in Economic Modeling: XGBoost's scalability and accuracy make it ideal for economic forecasting and risk assessment, where large datasets and complex interactions are common. It also includes
январь, 2025 г.
regularization parameters to control overfitting, a crucial feature for reliable economic predictions.
LightGBM (Light Gradient Boosting Machine)
Developed by Microsoft, LightGBM is designed to be faster and more memory-efficient than traditional gradient boosting algorithms. It achieves this by using a histogram-based approach and by growing trees leaf-wise rather than level-wise, which helps in capturing more complex patterns in the data.
Relevance in Economic Modeling: LightGBM is particularly well-suited for handling large-scale economic datasets with many features. Its efficiency and ability to handle categorical features natively make it a powerful tool for high-frequency trading, credit scoring, and other data-intensive economic applications.
CatBoost (Categorical Boosting)
Developed by Yandex, CatBoost is specifically optimized for datasets with categorical features, a common characteristic in economic data. It uses a novel algorithm to convert categorical variables into numerical values without requiring extensive preprocessing.
Relevance in Economic Modeling: CatBoost's ability to handle categorical data directly and its focus on reducing overfitting make it particularly useful in economic scenarios where data often includes categories such as industry sectors, regions, or policy types. It is highly effective in modeling complex economic systems and generating interpretable models.
These algorithms have become essential tools in modern economic modeling due to their ability to capture non-linear relationships, handle large and complex datasets, and produce highly accurate predictions. Their adoption in economic research is driven by the increasing complexity of economic data and the need for models that can provide deep insights and reliable forecasts.
Gradient Boosting Machines (GBMs): Mathematical Concept
Gradient Boosting Machines (GBMs) are a powerful ensemble learning method used for regression and classification problems. They build a strong predictive model by combining weak learners (usually decision trees). Here's the mathematical breakdown:
The goal of GBMs is to minimize a specified loss function L (y, F (x)), where:
у - True output
F(x) - Predicted output, which is a sum of weak learners.
The model is iteratively updated as:
M
F(X) = X ymhm(x),
m=1
Where:
hm (x) - Weak learner at iteration m
Ym - Weight for the weak learner.
№ 1 (130)
ЛД1 i лл
UNIVERSUM:
ТЕХНИЧЕСКИЕ НАУКИ
январь, 2025 г.
Gradient Descent Optimization
At each iteration m, GBMs optimize the loss by adding a new weak learner hm (x) to the model. The optimization steps are as follows:
Step 1: Compute Residuals
The negative gradient of the loss function with respect to the current model's prediction gives the residuals:
(m) = dL(y,F(xi)) n [ dF(Xi) ]
These residuals represent the direction to minimize the loss function.
Step 2: Fit Weak Learner
Fit a weak learner hm(x) to predict these residuals:
hm(x) « ri(m)
Step 3: Update Model
Update the model by adding the weak learner multiplied by a learning rate v:
Fm(x)
^m—1
(x) + vymh (x)
where (learning rate) controls the contribution of each learner.
Step 4: Optimize the Coefficients
Find ym that minimizes the loss:
N
ym = argmin ; Kji
, ^m—1
(xi) + vhm(xi).
Y i—i
i=1
Key Parameters
• Loss Function L(y,F(x)): Can be Mean Squared Error (MSE) for regression or Log-loss for classification.
• Learning Rate v: A small value to prevent over-fitting and ensure smoother convergence.
• Number of Trees M: Controls the ensemble's complexity.
• Tree Depth: Affects the flexibility of each weak learner.
Algorithm Steps
1. Initialize the model with a constant prediction:
¡V
Fo (x) = arg min ^ L (y^y)
2. For m=1,2,...,M Compute residuals ri(m'' Fit a weak learner hm (x) to ri(m'' Find ym
Update the model: Fm(x) = Fm—i(x) + vymhm(x) Output the final model FM (x)
Results
The application of Gradient Boosting Machines (GBMs) to economic forecasting demonstrated superior predictive accuracy compared to traditional econometric models. Key performance metrics, including Mean Absolute Error (MAE), Mean Squared Error (MSE), and R-squared (R2), consistently showed significant improvements when employing GBMs:
a) XGBoost - Delivered the highest accuracy for GDP growth forecasting, with an MAE of 0.15% and R2 of 0.92.
b) LightGBM - Achieved comparable performance with faster computation times, reporting an MAE of 0.18% and R2 of 0.90.
c) CatBoost - Excelled in handling categorical data, especially for models incorporating sectoral or regional economic indicators, yielding an MAE of 0.17% and R2 of 0.91.
Discussion
The results confirm that GBMs outperform traditional econometric models in terms of both predictive accuracy and adaptability. Traditional models, such as linear regression and ARIMA, struggle with non-linear relationships and complex interactions inherent in economic data. GBMs address these limitations by leveraging ensemble learning, gradient descent optimization, and flexible loss functions.
For example, XGBoost's regularization techniques mitigate overfitting, a common challenge in traditional econometric methods, while LightGBM's efficiency makes it suitable for handling large, dynamic datasets. CatBoost's specialized treatment of categorical variables reduces the need for extensive feature engineering, streamlining the modeling process.
Conclusion
Gradient Boosting Machines (GBMs) represent a significant advancement in economic forecasting methodologies, combining high predictive accuracy with adaptability to complex datasets. Their application to GDP growth forecasting highlights their potential to enhance traditional econometric models, offering deeper insights into economic phenomena.
The comparative analysis of XGBoost, LightGBM, and CatBoost underscores the versatility of GBMs, each excelling in specific contexts—whether optimizing for computational efficiency, handling large-scale data, or managing categorical features. These models are particularly valuable in addressing the increasing complexity of economic data in a digitalized global economy.
Future research should focus on integrating GBMs with domain-specific knowledge and advanced feature engineering techniques to further improve interpretabil-ity and application in policy-making. Additionally, exploring hybrid approaches that combine GBMs with traditional econometric frameworks could provide a comprehensive toolkit for addressing diverse economic challenges.
1=1
• 7universum.com
UNIVERSUM:
ТЕХНИЧЕСКИЕ НАУКИ_январь. 2025 г.
References:
1. Athey, S. (2018). The impact of machine learning on economics. The economics of artificial intelligence: An agenda, 507-547.
2. Salman, M., Das, P.K., & Mohanty, S.K. (2024). A Systematic Review on Recent Advancements in Deep Learning and Mathematical Modeling for Efficient Detection of Glioblastoma. IEEE Transactions on Instrumentation and Measurement.
3. Iskhakov, F., Rust, J., & Schjerning, B. (2020). Machine learning and structural econometrics: contrasts and synergies. The Econometrics Journal, 23(3), S81-S124.
4. Yoon, J. (2021). Forecasting of real GDP growth using machine learning models: Gradient boosting and random forest approach. Computational Economics, 57(1), 247-265.
5. Touzani, S., Granderson, J., & Fernandes, S. (2018). Gradient boosting machine for modeling the energy consumption of commercial buildings. Energy and Buildings, 158, 1533-1543.
6. Qureshi, S., Chu, B.M., & Demers, F.S. (2020). Forecasting Canadian GDP Growth Using XGBoost (No. 20-14). Carleton University, Department of Economics.
7. Carmona, P., Climent, F., & Momparler, A. (2019). Predicting failure in the US banking sector: An extreme gradient boosting approach. International Review of Economics & Finance, 61, 304-323.
8. Azibaev, A. (2024). FORECASTING GDP GROWTH AND GDP PER CAPITA IN UZBEKISTAN BY THE ORDINARY LEAST SQUARES (OLS) REGRESSION ANALYSIS. Scientific and Technical Journal of Namangan Institute of Engineering and Technology, 9(2), 284-290.
9. Ugli, A.A. G. (2024). Analytical and numerical expressions of the golden rule of capital accumulation. Илм-фан ва инновацион ривожланиш/Наука и инновационное развитие, 7(4), 15-26.
10. Lee, H.C., Yoon, S.B., Yang, S.M., Kim, W.H., Ryu, H.G., Jung, C.W., ... & Lee, K.H. (2018). Prediction of acute kidney injury after liver transplantation: machine learning approaches vs. logistic regression model. Journal of clinical medicine, 7(11), 428.
11. Otto, M., & Thornton, J. (2023). Forecasting gross domestic product (gdp) and gdp growth: an exploration of improved prediction using machine learning algorithms. Qo 'qon universiteti xabarnomasi, 9-14.
*(По требованию Роскомнадзора информируем, что иностранное лицо, владеющее информационными ресурсами Google является нарушителем законодательства Российской Федерации - прим. ред.)
№ 1 (130)
