|
|
|
| Research on real-time optimization decision of energy storage system based on improved Q-learning algorithm |
| Wang Bo, Zheng Wendi |
| School of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350116 |
|
|
|
|
Abstract With the increasing proportion of clean distributed energy in the power grid, the stable operation of the power grid is challenged by the power prediction error, a control method is required for energy storage system to make a rapid charge and discharge decision to achieve the best effect of peak shaving and load shifting in the real-time scheduling cycle. Considering the constraints such as unit ramp rate and capacity limit of the energy storage system, an improved Q-learning algorithm is proposed, using the penalty term lead the algorithm to obtain optimal decisions by training in phases using the offline data, which has a fast convergence performance and high accuracy; In the real-time scheduling cycle, when the prediction of load power is not accurate, energy storage system don’t need the iterative operation, it just needs to correct the state values and judge its state, then obtain its optimal action value by using the greedy strategy based on the trained Q value table. The simulation results show that the method proposed in this paper has a faster convergence performance compared with the traditional Q learning algorithm, and the trained Q value table can be used for energy storage system to make optimal decisions in the real-time scheduling cycle.
|
|
Received: 07 September 2017
Published: 07 February 2018
|
|
|
|
| Cite this article: |
|
Wang Bo,Zheng Wendi. Research on real-time optimization decision of energy storage system based on improved Q-learning algorithm[J]. Electrical Engineering, 2018, 19(2): 54-60.
|
|
|
|
| URL: |
|
http://dqjs.cesmedia.cn/EN/Y2018/V19/I2/54
|
[1] 李乐. 微网的经济运行研究[D]. 北京: 华北电力大学, 2011.
[2] 张明, 朴政国. 含储能的分布式光伏并网系统对配电网调峰的研究[J]. 电气技术, 2016, 17(12): 11-14, 19.
[3] 孙建龙, 窦晓波, 张子仲, 等. 直流对等式微电网混合储能系统协调控制策略[J]. 电工技术学报, 2016, 31(4): 194-202.
[4] 李锐, 李鹏. 储能系统在孤岛微网中应用[J]. 电气技术, 2014, 15(6): 15-18.
[5] 李建林, 马会萌, 惠东. 储能技术融合分布式可再生能源的现状及发展趋势[J]. 电工技术学报, 2016, 31(14): 1-10, 20.
[6] 刘皓明, 陆丹, 杨波, 等. 可平抑高渗透分布式光伏发电功率波动的储能电站调度策略[J]. 高电压技术, 2015, 41(10): 3213-3223.
[7] Chaouachi A, Kamel R M, Andoulsi R, et al. Multiobjective intelligent energy management for a microgrid[J]. IEEE Transactions on Industrial Elec- tronics, 2013, 60(4): 1688-1699.
[8] 宋晓英, 王艳松. 基于协同进化遗传算法的微网经济环保调度[J]. 电力系统保护与控制, 2014(5): 85-89.
[9] 李秀磊, 耿光飞, 季玉琦, 等. 考虑实际运行的配电网电池储能系统的优化配置研究[J]. 电力系统保护与控制, 2017, 45(9): 88-94.
[10] Lee T Y. Operating schedule of battery energy storage system in a Time-of-Use rate industrial user with wind turbine generators: a multipass iteration particle swarm optimization approach[J]. IEEE Transactions on Energy Conversion[J]. Sept, 2007, 22(3): 774-782.
[11] 李树雷, 展海艳, 薛松, 等. 考虑间歇性分布式电源并网的分布式储能系统最优控制策略[J]. 华东电力, 2013, 41(11): 2338-2343.
[12] 鲍冠南, 陆超, 袁志昌, 等. 基于动态规划的电池储能系统削峰填谷实时优化[J]. 电力系统自动化, 2012, 36(12): 11-16.
[13] 高雪莹, 唐昊, 苗刚中, 等. 储能系统能量调度与需求响应联合优化控制[J]. 系统仿真学报, 2016, 28(5): 1165-1172. |
|
|
|
2018, Vol. 19 
