Enhancing eco-driving strategies: integrating IDM prior knowledge into the TD3 reinforcement learning framework for connected vehicles AITranslate
Abstract AITranslate
Developing low-carbon transportation is crucial for ecological conservation and energy security. This paper addresses eco-driving challenges in connected vehicles by proposing a model-guided deep reinforcement learning framework to enhance the practicality and reliability of eco-driving strategies. Integrating model-guided and data-driven approaches, the method first designs a speed guidance algorithm based on the Intelligent Driver Model (IDM), which flexibly adjusts driving conservatism through parameter tuning. Subsequently, a reinforcement learning framework based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) is proposed. The reference speed generated by the IDM is embedded into the state space as prior knowledge to guide the agent toward rapid convergence during training. This strategy significantly optimizes vehicle energy consumption while strictly ensuring driving safety and having minimal impact on travel efficiency. Experimental results demonstrate that compared to baseline methods and traditional deep reinforcement learning strategies, the proposed approach achieves superior performance in energy savings, driving smoothness, and traffic continuity. It achieves an average energy savings rate of 7.96% with only a 0.44% increase in travel time, validating its effectiveness and practicality for eco-driving tasks.
KeyWords AITranslate
1.J. J. Cai, Y. G. Liu, Y. J. Zhang, S. Q. Shen, Z. Z. Lei, and Z. Chen, "Hierarchical Cooperative Eco-Driving Optimization for Multidimensional Mixed Vehicles at Signalized Intersections," Energy 325 (2025): 136174, https://doi.org/10.1016/j.energy.2025.136174.
2.Y. J. Pan, Y. Xi, W. P. Fang, Y. S. Liu, Y. L. Zhang, and W. S. Zhang, "An Eco-driving Strategy for Electric Buses at Signalized Intersection with a Bus Stop Based on Energy Consumption Prediction," Energy 317 (2025): 134672, https://doi.org/10.1016/j.energy.2025.134672.
3.C. X. Liu, Y. Chen, R. Z. Xu, H. J. Ruan, C. Wang, and X. Y. Li, "Co-Optimization of Energy Management and Eco-Driving Considering Fuel Cell Degradation via Improved Hierarchical Model Predictive Control," Green Energy and Intelligent Transportation 3, no. 6 (2024): 100176, https://doi.org/10.1016/j.geits.2024.100176.
4.Y. H. Hu, Y. B. Wang, J. Q. Guo, et al., "Eco-Driving of Connected Autonomous Vehicles in Urban Traffic Networks of Mixed Autonomy with Cut-in and Escape Lane-Changes of Manually-Driven Vehicles," Transportation research Part C: Emerging technologies 169 (2024): 104889, https://doi.org/10.1016/j.trc.2024.104889.
5.J. Li, X. D. Wu, M. Xu, and Y. G. Liu, "Deep Reinforcement Learning and Reward Shaping Based Eco-Driving Control for Automated HEVs among Signalized Intersections," Energy 251 (2022):123924, https://doi.org/10.1016/j.energy.2022.123924.
6.Z. X. Zhu, S. Gupta, A. Gupta, and M. Canova, "A Deep Reinforcement Learning Framework for Eco-Driving in Connected and Automated Hybrid Electric Vehicles," IEEE Transactions on Vehicular Technology 73, no. 2 (2024): 1713–1725, https://doi.org/10.1109/TVT.2023.3318552.
7.S. Y. Bao, P. Sun, J. X. Zhu, Q. Ji, and J. H. Liu, "Improved Multi-Dimensional Dynamic Programming Energy Management Strategy for a Vehicle Power-Split Hybrid Powertrain," Energy 256 (2022): 124682, https://doi.org/10.1016/j.energy.2022.124682.
8.A. Irshayyid, J. Chen, and G. J. Xiong, "A Review on Reinforcement Learning-Based Highway Autonomous Vehicle Control," Green Energy and Intelligent Transportation 3, no. 4 (2024): 100156, https://doi.org/10.1016/j.geits.2024.100156.
9.Y. T. Wu, Y. G. Liu, J. Peng, Z. Chen, Y. Liu, and Y. Zhang, "Enhanced Hierarchical Eco-Driving Control for Electric Vehicles via Global Velocity Optimization With Road Slope Adaptation," IEEE Transactions on Transportation Electrification 11, no. 4 (2025): 10322–10335, https://doi.org/10.1109/TTE.2025.3562715.
10.W. W. Xin, E. Y. Xu, W. G. Zheng, H. B. Feng, and J. R. Qin, "Optimal Energy Management of Fuel Cell Hybrid Electric Vehicle Based on Model Predictive Control and Online Mass Estimation," Energy Reports 8 (2022): 4964–4974, https://doi.org/10.1016/j.egyr.2022.03.194.
11.Z. W. Yang, Z. D. Zheng, J. W. Kim, and H. Rakha, "Eco-Driving Strategies Using Reinforcement Learning for Mixed Traffic in the Vicinity of Signalized Intersections," Transportation Research Part C: Emerging Technologies 165 (2024): 104683, https://doi.org/10.1016/j.trc.2024.104683.
12.S. H. Chen, Y. Huang, J. Zhang, X. S. Yu, Y. F. Lu, and D. J. Xuan, "Research on a Novel Multi-Agent Deep Reinforcement Learning Eco-Driving Framework," Energy 326 (2025): 136308, https://doi.org/10.1016/j.energy.2025.136308.
13.D. W. Li, F. Zhu, J. P. Wu, Y. D. Wong, and T. Y. Chen, "Managing Mixed Traffic at Signalized Intersections: An Adaptive Signal Control and CAV Coordination System Based on Deep Reinforcement Learning," Expert Systems with Applications 238, Part C (2024): 121959, https://doi.org/10.1016/j.eswa.2023.121959.
14.M. Y. Chen, B. B. Li, Y. G. Bian, et al., "Intersection Signal-Vehicle Coupled Coordination with Mixed Autonomy Vehicles," IEEE Transactions on Transportation Electrification 11, no. 1 (2025): 668–681, https://doi.org/10.1109/TTE.2024.3394595.
15.Y. N. Ye, Z. J. Xu, C. Y. Wang, and W. Z. Zhao, "Energy-Efficient Roundabout Crossing Strategy for Autonomous Electric Vehicles Based on Reinforcement Learning with Expert Demonstration," IEEE Transactions on Transportation Electrification 11, no. 6 (2025): 13802–13814, https://doi.org/10.1109/TTE.2025.3602960.
16.H. Tong, L. Chu, Z. Chen, Y. G. Liu, Y. J. Zhang, and J. C. Hu, "Multi-Objective Autonomous Eco-Driving Strategy: A Pathway to Future Green Mobility," Green Energy and Intelligent Transportation 4, no. 4 (2025): 100279, https://doi.org/10.1016/j.geits.2025.100279.
17.H. W. He, X. F. Meng, Y. Wang, et al., "Deep Reinforcement Learning Based Energy Management Strategies for Electrified Vehicles: Recent Advances and Perspectives," Renewable and Sustainable Energy Reviews 192 (2024): 114248, https://doi.org/10.1016/j.rser.2023.114248.
18.X. Y. Shi, J. Zhang, X. Jiang, J. Chen, W. Hao, and B. Wang, "Learning Eco-Driving Strategies from Human Driving Trajectories," Physica A: Statistical Mechanics and its Applications 633 (2024): 129353, https://doi.org/10.1016/j.physa.2023.129353.
19.X. L. Zhang, X. Jiang, N. Li, Z. Y. Yang, Z. Xiong, and J. Zhang, "Eco-Driving for Intelligent Electric Vehicles at Signalized Intersection: A Proximal Policy Optimization Approach," in Proceedings of ISCTT 2021-6th International Conference on Information Science, Computer Technology and Transportation, 1–7, VDE VERLAG, 2021.
20.W. Q. Chen, J. K. Peng, T. H. Ren, H. L. Zhang, H. W. He, and C. Y. Ma, "Integrated Velocity Optimization and Energy Management for FCHEV: An Eco-Driving Approach Based on Deep Reinforcement Learning," Energy Conversion and Management 296 (2023): 117685, https://doi.org/10.1016/j.enconman.2023.117685.
21.Y. Huang, H. Q. Hu, J. Q. Tan, C. L. Lu, and D. J. Xuan, "Deep Reinforcement Learning Based Energy Management Strategy for Range Extend Fuel Cell Hybrid Electric Vehicle," Energy Conversion and Management 277 (2023): 116678, https://doi.org/10.1016/j.enconman.2023.116678.
22.X. D. Wu, J. Li, C. R. Su, J. W. Fan, and M. Xu, "A Deep Reinforcement Learning Based Hierarchical Eco-Driving Strategy for Connected and Automated HEVs," IEEE Transactions on Vehicular Technology 72, no. 11 (2023): 13901–13916, https://doi.org/10.1109/TVT.2023.3283617.
23.H. Lee, K. Kim, N. Kim, and S. W. Cha, "Energy Efficient Speed Planning of Electric Vehicles for Car-Following Scenario Using Model-Based Reinforcement Learning," Applied Energy 313 (2022): 118460, https://doi.org/10.1016/j.apenergy.2021.118460.
24.Q. C. Su, R. C. Huang, and H. W. He, "Heterogeneous Multi-Agent Deep Reinforcement Learning for Eco-Driving of Hybrid Electric Tracked Vehicles: A Heuristic Training Framework," Journal of Power Sources 601 (2024): 234292, https://doi.org/10.1016/j.jpowsour.2024.234292.
25.J. Li, X. D. Wu, J. W. Fan, Y. G. Liu, and M. Xu, "Overcoming Driving Challenges in Complex Urban Traffic: A Multi-Objective Eco-Driving Strategy via Safety Model Based Reinforcement Learning," Energy 284 (2023): 128517, https://doi.org/10.1016/j.energy.2023.128517.
26.D. Coraci, S. Brandi, T. Z. Hong, and A. Capozzoli, "Online Transfer Learning Strategy for Enhancing the Scalability and Deployment of Deep Reinforcement Learning Control in Smart Buildings," Applied Energy 333 (2023): 120598, https://doi.org/10.1016/j.apenergy.2022.120598.
27.C. X. Ling, J. K. Peng, Y. Fan, Z. X. Wang, S. C. Yu, and C. C. Wu, "Safety-Awareness Enhanced Eco-Driving Strategy for Dual-Motor Electric Vehicle in Highway Scenarios Based on Improved Proximal Policy Optimization Algorithm," Energy 340 (2025): 139177, https://doi.org/10.1016/j.energy.2025.139177.
28.K. H. Mo, P. G. Ye, X. J. Ren, S. W. Wang, W. J. Li, and J. Li, "Security and Privacy Issues in Deep Reinforcement Learning: Threats and Countermeasures," ACM Computing Surveys 56, no. 6 (2024): 1–39, https://doi.org/10.1145/3640312.
29.C. T. Zhang, W. H. Huang, X. Y. Zhou, C. Lv, and C. Sun, "Expert-Demonstration-Augmented Reinforcement Learning for Lane-Change-Aware Eco-Driving Traversing Consecutive Traffic Lights," Energy 286 (2024): 129472, https://doi.org/10.1016/j.energy.2023.129472.
30.J. Q. Liu, C. Y. Wang, W. Z. Zhao, "An Eco-driving Strategy for Autonomous Electric Vehicles Crossing Continuous Speed-Limit Signalized Intersections," Energy 294 (2024): 130829, https://doi.org/10.1016/j.energy.2024.130829.
31.M. L. Li, L. Yin, M. Yan, J. D. Wu, H. W. He, and C. C. Jia, "Hierarchical Intelligent Energy-Saving Control Strategy for Fuel Cell Hybrid Electric Buses Based on Traffic Flow Predictions," Energy 304 (2024): 132144, https://doi.org/10.1016/j.energy.2024.132144.
Basic Information:
DOI：10.23919/CHAIN.2026.000008
Chinese Library Classification Number:
Citation Information:
Developing low-carbon transportation is crucial for ecological conservation and energy security. This paper addresses eco-driving challenges in connected vehicles by proposing a model-guided deep reinforcement learning framework to enhance the practicality and reliability of eco-driving strategies. Integrating model-guided and data-driven approaches, the method first designs a speed guidance algorithm based on the Intelligent Driver Model (IDM), which flexibly adjusts driving conservatism through parameter tuning. Subsequently, a reinforcement learning framework based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) is proposed. The reference speed generated by the IDM is embedded into the state space as prior knowledge to guide the agent toward rapid convergence during training. This strategy significantly optimizes vehicle energy consumption while strictly ensuring driving safety and having minimal impact on travel efficiency. Experimental results demonstrate that compared to baseline methods and traditional deep reinforcement learning strategies, the proposed approach achieves superior performance in energy savings, driving smoothness, and traffic continuity. It achieves an average energy savings rate of 7.96% with only a 0.44% increase in travel time, validating its effectiveness and practicality for eco-driving tasks.
quote
| GB/T 7714-2015 | [1] Mei Yan, Shengjie Chen, Hongwen He, et al. Enhancing eco-driving strategies: integrating IDM prior knowledge into the TD3 reinforcement learning framework for connected vehicles[J]. Chain, 2026, 3(2): 245-266. DOI:10.23919/CHAIN.2026.000008. |
| MLA | [1] Mei Yan, et al., "Enhancing eco-driving strategies: integrating IDM prior knowledge into the TD3 reinforcement learning framework for connected vehicles." Chain, vol. 3, no. 2, 2026, pp. 245-266, https://doi.org/10.23919/CHAIN.2026.000008. |
| APA | [1] Mei Yan, Shengjie Chen, Hongwen He, Yunlong Wang, & Menglin Li. (2026). Enhancing eco-driving strategies: integrating IDM prior knowledge into the TD3 reinforcement learning framework for connected vehicles. Chain, 3(2), 245-266. https://doi.org/10.23919/CHAIN.2026.000008 |
| IEEE | [1] Mei Yan, Shengjie Chen, Hongwen He, Yunlong Wang, and Menglin Li, "Enhancing eco-driving strategies: integrating IDM prior knowledge into the TD3 reinforcement learning framework for connected vehicles," Chain, vol. 3, no. 2, pp. 245-266, 2026, doi: 10.23919/CHAIN.2026.000008. keywords: {connected vehicles;driving efficiency;eco-driving;energy consumption;reinforcement learning} |