Digital Twin-Enabled Deep Reinforcement Learning for PV-ESS Energy Management in Active Distribution Networks

PhongMinh le

doi:10.31258/ijeepse.9.2.61-78

PhongMinh le Thu Duc College of Technology, Ho Chi Minh City, Viet Nam

DOI: https://doi.org/10.31258/ijeepse.9.2.61-78

Abstract viewed: 0 times

pdf downloaded: 0 times

Keywords: Active distribution network, Deep reinforcement learning, Digital twin, Energy storage system, Uncertainty-aware energy management.

Abstract

This paper proposes a real-time energy management framework for photovoltaic (PV) and energy storage system (ESS) integrated distribution networks using deep reinforcement learning (DRL) and Digital Twin (DT) technology. The objective is to improve operational efficiency under uncertain conditions while maintaining voltage stability and reducing battery degradation. The proposed method is validated on the IEEE 33-bus system with high PV penetration. Simulation results demonstrate that the proposed approach achieves a 22.2% reduction in operational cost, significantly improves voltage profiles, and reduces performance variability under uncertainty compared to rule-based control and optimal power flow methods. The integration of DRL and Digital Twin enables adaptive and robust decision-making, making the framework suitable for modern distribution networks with high renewable penetration.

References

Z. Liu, Y. Zhang, and H. Li, “Digital twin for power systems: State-of-the-art and future trends,” Renew. Sustain. Energy Rev., vol. 152, p. 111675, 2021, doi: 10.1016/j.rser.2021.111675.

K. Zhou, S. Yang, and Z. Shao, “Digital twin-based optimization for energy systems,” Appl. Energy, vol. 285, p. 116419, 2021, doi: 10.1016/j.apenergy.2020.116419.

U. I. Musa and S. Ghosh, “Advancing Digital Twin through the Integration of new AI Algorithms,” Int. Res. J. Eng. Technol, pp. 922–932, 2023, [Online]. Available: https://www.researchgate.net/publication/369693333

N. B. Mohamed, G. Kaddoum, and M. Z. Hassan, “Digital Twin-Driven Continual Deep Reinforcement Learning for Coexistence of Multiple Radio Access Technology IoT Links with Nonlinear Receivers,” IEEE Trans. Mach. Learn. Commun. Netw, vol. 4, pp. 591–611, 2026, doi: 10.1109/TMLCN.2026.3669837.

M. R. Kabir, D. Halder, and S. Ray, “Digital Twins for IoT-Driven Energy Systems: A Survey,” IEEE Access, vol. 12, pp. 177123–177143, 2024, doi: 10.1109/ACCESS.2024.3506660.

C. Nneoma Ugwu, F. Chukwudi Ogenyi, J. Nnenna Ugwu, P. C. Okechukwu Ugwu, and M. Ben Okon, “Integration of AI-driven digital twins for real-time optimization of renewable energy grids,” Front. Energy Res, vol. 14, pp. 1–13, 2026, doi: 10.3389/fenrg.2026.1748233.

N. Kumari, A. Sharma, B. Tran, N. Chilamkurti, and D. Alahakoon, “A comprehensive review of digital twin technology for grid-connected microgrid systems,” Energies, vol. 16, no. 14, p. 5525, 2023, doi: 10.3390/en16145525.

J. M. Guerrero, J. C. Vasquez, J. Matas, L. G. Vicuña, and M. Castilla, “Hierarchical control of droop-controlled AC and DC microgrids—A general approach toward standardization,” IEEE Trans. Ind. Electron, vol. 58, no. 1, pp. 158–172, 2011, doi: 10.1109/TIE.2010.2066534.

A. Parisio, E. Rikos, and L. Glielmo, “A model predictive control approach to microgrid operation optimization,” IEEE Trans. Control Syst. Technol, vol. 22, no. 5, pp. 1813–1827, 2014, doi: 10.1109/TCST.2013.2295737.

M. Glavic, R. Moreno, T. Bi, and M. Shahidehpour, “Reinforcement learning for electric power system decision and control: Past considerations and perspectives,” IEEE Trans. Smart Grid, vol. 12, no. 6, pp. 5128–5142, 2021, doi: 10.1109/TSG.2021.3070934.

Z. Zhang, J. Wang, and B. Chen, “Deep reinforcement learning for energy management in microgrids,” Appl. Energy, vol. 288, p. 116618, 2021, doi: 10.1016/j.apenergy.2021.116618.

Y. Wang, X. Liu, J. Zhao, and M. Shahidehpour, “Digital Twin-Driven Resilient Operation of Active Distribution Networks With High Renewable Penetration,” IEEE Trans. Smart Grid, vol. 15, no. 2, pp. 1648–1661, 2024, doi: 10.1109/TSG.2023.3334869.

A. A. Ardebili, M. Zappatore, A. I. H. A. Ramadan, A. Longo, and A. Ficarella, “Digital Twins of smart energy systems: a systematic literature review on enablers, design, management and computational challenges,” Energy Informatics, vol. 7, no. 1, Art. no. 94, 2024, doi: 10.1186/s42162-024-00385-5.

W. F. Mbasso, A. Harrison, I. Dagal, P. Jangir, M. Khishe, H. Kotb, M. S. Shaikh, A. Smerat, E. F. Donfack, “Digital twins in renewable energy systems: A comprehensive review of concepts, applications, and future directions,” Int. J. Electr. Power Energy Syst, vol. 61, Art. no. 101814, 2025, doi: 10.1016/j.esr.2025.101814.

B. Xu, A. Oudalov, A. Ulbig, G. Andersson, and D. S. Kirschen, “Modeling of Lithium-Ion Battery Degradation for Cell Life Assessment,” IEEE Trans. Smart Grid, vol. 9, no. 2, pp. 1131–1140, 2018, doi: 10.1109/TSG.2016.2578950.

B. Xu, Y. Shi, D. S. Kirschen, and B. Zhang, “Optimal Battery Participation in Frequency Regulation Markets Considering Degradation Cost,” IEEE Trans. Power Syst, vol. 33, no. 5, pp. 4908–4918, 2018, doi: 10.1109/TPWRS.2018.2799437.

J. Chen, X. Wang, M. Zhou, and Y. Fu, “Battery Degradation-Aware Deep Reinforcement Learning for Energy Storage Scheduling in Distribution Networks,” IEEE Trans. Sustain. Energy, vol. 16, no. 1, pp. 215–227, 2025, doi: 10.1109/TSTE.2024.3478912.

T. N. Ton, P. M. Le, and T. M. Le, “Multi-objective optimization of distributed generation placement and sizing in active distribution networks considering harmonic distortion,” Int. J. Electr. Comput. Eng, vol. 16, no. 2, pp. 598–607, 2026, doi: 10.11591/ijece.v16i2.pp598-607.

T.N.Ton, P.M. Le, L.H.Pham, and T. M. Le, “Multi-Objective Optimization of Electric Distribution Systems with integrated distributed Generation using Deep Reinforcement Learning,” Eng. Technol. Appl. Sci. Res, vol. 15, no. 2, pp. 22166–22171, 2025.