Optimization Control of Building Facilities Using Deep Rein-forcement Learning Comparison with Dynamic Programming

¹ I.I.S., The University of Tokyo, 4-6-1 Komaba, Meguro-Ku, Tokyo, Japan
² Mikoto Strategy Co, Kudan Minami, Chiyoda-Ku, Tokyo, Japan

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

In recent years, Artificial Intelligence (AI) has been rapidly advancing worldwide and is being applied in many fields. While its utilization is also expected in the building energy, we need to quantitatively evaluate its effectiveness for full-scale practical implementation. Therefore, in this study, we examined the effect of controlling a photovoltaic power generation system including storage batteries using AI on its power cost reduction by comparing it with a theoretically optimal solution obtained by mathematical programming. More specifically, a comparison was conducted between charge/discharge control using Dynamic Programming (DP) and Soft Actor-Critic (SAC). SAC is a reinforcement learning algorithm known for its high performance in problems with continuous action spaces. In contrast, DP is a method that can obtain a mathematically optimal solution under discretized variable settings based on the principle of optimality. The results showed that SAC-based control achieved a 7.0% to 11.2% reduction in electricity cost compared to the case without utilizing the energy storage system. However, when compared with the theoretical optimum obtained by DP, a maximum cost reduction gap of up to 18% was observed. This may be attributed to the fact that DP considers future demand, whereas SAC focuses solely on the present.