Comparative framework for electricity demand forecasting using machine learning and rolling temporal validation
Article Sidebar
Main Article Content
Abstract
Accurate load forecasting is essential for power system planning and operation, particularly under pronounced temporal variability and temporal drift. This study presents a reproducible comparative framework for machine learning models based on rolling-origin expanding validation, multihorizon evaluation, and an operational relative tolerance metric denoted as Tol. Four representative models are evaluated: EvoXGB, a sequential residual XGBoost ensemble; XGB; TabNet; and FT-Transformer. These models are applied to hourly active power forecasting in distribution substations within an Ecuadorian power system. To ensure a fair comparison when models exhibit differences in prediction coverage or temporal misalignment, the framework incorporates an explicit comparability audit based on temporal alignment and a common evaluation mask denoted as COMMONMASK, complemented the longest common contiguous block for the zoomed time-series visualization. For the representative substation, with metrics computed on the common set, XGB achieves the best performance, with R2= 0.993 for the short horizon and R2= 0.983 for the medium horizon, and RMSE values of 21.16 and 30.84 kW, respectively. EvoXGB remains competitive, whereas TabNet and FT-Transformer exhibit greater degradation in the medium horizon. The 90/10 holdout verification shows the expected performance decline associated with temporal drift while preserving the comparative ranking. Overall, the proposed framework provides a traceable benchmark for substation load forecasting and supports future extensions toward adaptive and hybrid forecasting approaches.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The Universidad Politécnica Salesiana of Ecuador preserves the copyrights of the published works and will favor the reuse of the works. The works are published in the electronic edition of the journal under a Creative Commons Attribution/Noncommercial-No Derivative Works 4.0 Ecuador license: they can be copied, used, disseminated, transmitted and publicly displayed.
The undersigned author partially transfers the copyrights of this work to the Universidad Politécnica Salesiana of Ecuador for printed editions.
It is also stated that they have respected the ethical principles of research and are free from any conflict of interest. The author(s) certify that this work has not been published, nor is it under consideration for publication in any other journal or editorial work.
The author (s) are responsible for their content and have contributed to the conception, design and completion of the work, analysis and interpretation of data, and to have participated in the writing of the text and its revisions, as well as in the approval of the version which is finally referred to as an attachment.
References
[1] T. Hong and S. Fan, “Probabilistic electric load forecasting: A tutorial review,” International Journal of Forecasting, vol. 32, no. 3, pp. 914–938, 2016. [Online]. Available: https://doi.org/10.1016/j.ijforecast.2015.11.011
[2] IEA, “Renewables 2023: Analysis and forecast to 2028,” International Energy Agency, Paris, France, Tech. Rep., 2023, accessed: May 15, 2026. [Online]. Available: https://upsalesiana.ec/ing36ar2r2
[3] M. G. Pinheiro, S. C. Madeira, and A. P. Francisco, “Short-term electricity load forecasting—a systematic approach from system level to secondary substations,” Applied Energy, vol. 332, p. 120493, Feb. 2023. [Online]. Available: https://doi.org/10.1016/j.apenergy.2022.120493
[4] S. Akhtar, S. Shahzad, A. Zaheer, H. S. Ullah, H. Kilic, R. Gono, M. Jasiński, and Z. Leonowicz, “Short-term load forecasting models: A review of challenges, progress, and the road ahead,” Energies, vol. 16, no. 10, p. 4060, May 2023. [Online]. Available: https://doi.org/10.3390/en16104060
[5] F. Rodrigues, C. Cardeira, J. M. F. Calado, and R. Melicio, “Short-term load forecasting of electricity demand for the residential sector based on modelling techniques: A systematic review,” Energies, vol. 16, no. 10, p. 4098, May 2023. [Online]. Available: https://doi.org/10.3390/en16104098
[6] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’16. ACM, Aug. 2016, pp. 785–794. [Online]. Available: https://doi.org/10.1145/2939672.2939785
[7] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: a highly efficient gradient boosting decision tree,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017, p. 3149–3157. [Online]. Available: https://upsalesiana.ec/ing36ar2r4
[8] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “Catboost: unbiased boosting with categorical features,” in NIPS’18: Proceedings of the 32nd International Conference on Neural Information Processing Systems. arXiv, 2017, pp. 6639–6649. [Online]. Available: https://doi.org/10.48550/arXiv.1706.09516
[9] C. Bentéjac, A. Csörgő, and G. Martínez-Muñoz, “A comparative analysis of gradient boosting algorithms,” Artificial Intelligence Review, vol. 54, no. 3, pp. 1937–1967, Aug. 2020. [Online]. Available: https://doi.org/10.1007/s10462-020-09896-5
[10] Z. Mustaffa and M. H. Sulaiman, “Advanced forecasting of building energy loads with XGBoost and metaheuristic algorithms integration,” Energy Storage and Saving, vol. 4, no. 4, pp. 421–438, Dec. 2025. [Online]. Available: https://doi.org/10.1016/j.enss.2025.03.005
[11] T.-N. Tran and Q.-D. Nguyen, “Research on the influence of genetic algorithm parameters on XGBoost in load forecasting,” Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 18 849–18 854, Dec. 2024. [Online]. Available: https://doi.org/10.48084/etasr.8863
[12] B. Liang, W. Qin, and Z. Liao, “A differential evolutionary-based xgboost for solving classification of physical fitness test data of college students,” Mathematics, vol. 13, no. 9, p. 1405, Apr. 2025. [Online]. Available: https://doi.org/10.3390/math13091405
[13] S. Arik and T. Pfister, “TabNet: Attentive interpretable tabular learning,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 8, pp. 6679–6687, May 2021. [Online]. Available: https://doi.org/10.1609/aaai.v35i8.16826
[14] Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko, “Revisiting deep learning models for tabular data,” in NIPS’21: Proceedings of the 35th International Conference on Neural Information Processing Systems. arXiv, 2021, pp. 18 932–18 943. [Online]. Available: https://doi.org/10.48550/arXiv.2106.11959
[15] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci, “Deep neural networks and tabular data: A survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 6, pp. 7499–7519, 2024. [Online]. Available: https://doi.org/10.1109/TNNLS.2022.3229161
[16] L. Grinsztajn, E. Oyallon, and G. Varoquaux, “Why do tree-based models still outperform deep learning on tabular data?” in NIPS’22: Proceedings of the 36th International Conference on Neural Information Processing Systems. arXiv, 2022, pp. 507–520. [Online]. Available: https://doi.org/10.48550/arXiv.2207.08815
[17] R. Shwartz-Ziv and A. Armon, “Tabular data: Deep learning is not all you need,” Information Fusion, vol. 81, pp. 84–90, May 2022. [Online]. Available: https://doi.org/10.1016/j.inffus.2021.11.011
[18] V. Cerqueira, L. Torgo, and I. Mozetič, “Evaluating time series forecasting models: an empirical study on performance estimation methods,” Machine Learning, vol. 109, no. 11, pp. 1997–2028, Oct. 2020. [Online]. Available: https://doi.org/10.1007/s10994-020-05910-7
[19] R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and Practice, 3rd ed. Melbourne, Australia: OTexts, 2021, accessed: May 15, 2026. [Online]. Available: https://upsalesiana.ec/ing36ar2r16
[20] C. Bergmeir and J. M. Benítez, “On the use of cross-validation for time series predictor evaluation,” Information Sciences, vol. 191, pp. 192–213, May 2012. [Online]. Available: https://doi.org/10.1016/j.ins.2011.12.028
[21] M. Q. Raza and A. Khosravi, “A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings,” Renewable and Sustainable Energy Reviews, vol. 50, pp. 1352–1372, Oct. 2015. [Online]. Available: https://doi.org/10.1016/j.rser.2015.04.065
[22] C. Borges, Y. Penya, I. Fernández, J. Prieto, and O. Bretos, “Assessing tolerance-based robust short-term load forecasting in buildings,” Energies, vol. 6, no. 4, pp. 2110–2129, Apr. 2013. [Online]. Available: https://doi.org/10.3390/en6042110
[23] W. Guamán, P. Benalcázar, J. Córdova-García, and M. Torres, Machine Learning-Based Projections of Long-Term Electricity Consumption: The Case Study of Ecuador. Springer Nature Switzerland, 2025, pp. 174–187. [Online]. Available: https://doi.org/10.1007/978-3-031-83432-5_12
[24] G. Pesántez, W. Guamán, J. Córdova, M. Torres, and P. Benalcázar, “Reinforcement learning for efficient power systems planning: A review of operational and expansion strategies,” Energies, vol. 17, no. 9, p. 2167, May 2024. [Online]. Available: https://doi.org/10.3390/en17092167
[25] J. C. Castillo. (2026) Forecasting-rollingenergy. GitHub repository. [Online]. Available: https://upsalesiana.ec/ing36ar2r27