The effect of seasonal weather changes on the performance of databased models of the thermodynamic behaviour of buildings

Several studies have indicated that Model Predictive Control (MPC) of space heating systems can utilize the thermal mass of residential buildings as short-term thermal storage for various demand response purposes. Realization of this potential relies heavily on the accuracy of the model used to represent the thermodynamics of the building. Such models, whether they are grey box or black box, are calibrated using relevant data obtained from initial measurements, and the performance of the calibrated model is validated using data from a subsequent period. However, many studies use validation periods with weather conditions similar to those of the calibration period. Only a few studies investigate whether the calibrated model performs satisfactory when subjected to significantly different conditions. This paper presents data from a simulation-based study on the effect of seasonal weather changes on the performance of a black-box model. The study was conducted using 11 years of Danish weather data (2008-2018). The results indicate that the performance of the black-box model deteriorate as the weather data conditions become increasingly different from those used in the initial model calibration. Further, the results show that calibration in heating season leads to satisfactory model performance through the heating season, but lower performance in transitional seasons (especially spring). Results also show that calibration in February led to highest model performance through heating season, while calibration in March led to satisfactory model performance in the whole heating and fall season.


Introduction
Previous studies investigated the potential of using blackbox models of the thermodynamic of buildings for Model Predictive Control (MPC) of space heating, see e.g. [1][2][3], to mention a few. Here, the purpose of the MPC was to utilise the thermal mass of buildings as thermal storage to change the temporal heating patterns e.g. to even out the heating demand profile of district heating systems. The performance of the MPC for this purpose relies on the accuracy of the model-based predictions. Previous simulation-based studies have used a calibration and validation period with similar weather and model conditions to evaluate the accuracy of a model, see e.g. [1][2][3][4][5]. In reality, these conditions will change over time, both in relation to physical changes of the building and seasonal changes in weather conditions. Only a few studies evaluates how changes in weather conditions affect model performance. One study changed weather conditions by changing the location of the EnergyPlus model representing the 'real' building [6]. The study concluded that even when changing the weather conditions, the black-box model was satisfyingly close to the 'measured' values from the EnergyPlus model. Another study simulated a whole year and counted the number of necessary recalibrations on a black-box model [4]. This study found that only few recalibrations were necessary compared to the number of evaluated days, although the number of recalibrations was higher for a 2 nd -order model than for a 4 th or higher order model. In ref. [7], the calibration of a grey-box model was based on five different situations, and a cross-validation was made for each situation with the four remaining ones. The results showed that a model calibrated with summer conditions performed satisfactory in winter conditions. However, the weather data was from Florida, where the difference in weather conditions for a summer and winter situation is lower than in northern Europe, for example. Another study evaluated the influence of the choice of calibration period by choosing various calibration periods, (winter, mid-season and summer) in Nantes, France, and compared the performance of a grey-box model from these periods with the performance in various validation periods [8]. Here, it was concluded that the model performed best within the same season as the calibration period.
The above-mentioned studies show quite different results, and not all results are representative for a northern European climate. The purpose of this paper is therefore to extend current literature with an evaluation of the robustness of a black-box model in relation to seasonal weather changes in a northern European (temperate) climate. This analysis focuses on the weather conditions that have the largest impact on energy consumption in buildings, namely the external air temperature and solar radiation. The objective of the analysis is to identify whether recalibration of the blackbox model is necessary due to changing seasons, and if so, to identify which period of the year that is most suited for calibration of a model suitable for MPC of space heating systems.

Method
This section describes the method used to evaluate the performance of a black-box model for MPC of space heating systems. A detailed EnergyPlus [9] model was used as an analogue for a real building, while the control of the heating system was implemented in MATLAB. Co-simulation between EnergyPlus and MATLAB was facilitated by the building controls virtual test bed (BCVTB) [10]. First, a pseudo-random binary sequence (PRBS) signal was generated and used to control temperature set point for a full-year simulation of the EnergyPlus case building. A full-length PRBS signal was used to ensure white noise properties [11]. The simulation was repeated with historical weather data for 11 consecutive years (2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018). A black-box model was then calibrated each month using data from the first 15.9 days of the month because this equals two full-length PRBS signals of order 7 and 8, respectively. Models calibrated using a PRBS signal will in the following sections be referred to as PRBS-models. The calibrated models were validated by comparing their simulation output for the last 10.6 days every month, which corresponds to the full-length 8 th -order PRBS signal, with the EnergyPlus simulation output from the same period.
In practice, an excitation with a PRBS signal can lead to discomfort for the user, as the temperature will fluctuate randomly during the day (Fig. 3). Therefore, the experiment was repeated using a rule-based on/off signal that does not fluctuate as much as the PRBS signal.
This rule-based on/off signal reflects the typical behaviour of an MPC, inspired by [12]. As opposed to the PRBS signal, the MPC similar signal only boosts the heating system for a couple of hours during the night, as illustrated in Fig. 4. Models calibrated using this night boosting signal will in the following sections be referred to as NB-models. The purpose of testing this alternative signal was to evaluate whether it can substitute the use of the more intrusive PRBS signal. The PRBS signal is considered a more thorough test of the capabilities of the calibrated models, therefore NB-models was validated with the night boosting signal, as well as a PRBS signal.

Case building
The case building is an actual 81 m 2 single-family residential building, which comply with the regulations of the Danish Building Standard [13]. The entire house was modelled as a single zone with no internal walls. The building has three environmentally exposed facades, illustrated in Fig. 1, and one adiabatic façade facing a building equal to the one simulated. The properties of the constructions were provided by the building owner, and are given in Table 1.

South Façade
North Façade West Façade Furthermore, for the sake of simplicity, all windows were modelled with the EnergyPlus class SimpleGlazingSystem with a U-value of 0.74 W/m 2 K and solar heat gain coefficient of 0.5. On the northern façade, in Fig. 1, the window on the left is a nontransparent door with U-value of 0.67 W/m 2 K. The building is naturally ventilated with a ventilation rate of 0.3 l/s per m 2 and infiltration rate of 0.0024 l/s per m 2 , both modelled with the EnergyPlus BLAST algorithm for infiltration [14].
Weather data for a location in the city of Skanderborg (longitude: 10.0, latitude: 56.05) for the 11 simulated years (2008-2018) were obtained from ref. [15] -a service that provides EnergyPlus weather files for custom locations throughout Denmark. Fig. 2 illustrates how the output from the EnergyPlus example simulation behaves as expected. For simplicity, the example uses a constant temperature set point at 22 °C. From the example, it is seen that the heating system is tuned off when the sun exposure is high. The indoor temperature is in general kept around the desired 22 °C, except for two peaks caused by the solar heat gain. The natural ventilation system follows the wind speed and the temperature difference between indoor and outdoor as expected due to the BLAST algorithm.   [16] should be enough excitation to get a high performing model. The signal was created using the MATLAB function 'idinput' with a range from 20 to 24 °C and a switching time of 1, meaning that the signal can change value each time step (time steps of the signal was one hour). Fig. 3 shows an example of three days of the signal. The alternative to the PRBS signal, i.e. the rule-based on/off signal, had a fixed excitation for four hours, from 2 a.m. to 6 a.m. each night, as shown in Fig. 4. The on/off signal is a simplification compared to an actual MPC signal. A linear state space model representing the thermodynamics of the building was formulated: (1) Where k is the time index of the simulation of the control model (in this case an hourly time step was used),

Results
Root Mean Square Error (RMSE) was used to evaluate how well the black-box model was able to perform 24hour predictions of the indoor temperature output from the EnergyPlus model. The mean and standard deviation of RMSE for the 11 years of simulation with system excitation by a PRBS signal and a night boosting signal is shown in Table 2 and Table 3, respectively. Both are validated on dataset with PRBS excitation to ensure a fair comparison between the two groups of models (PRBS and NB) and thereby ensure that the lower excitation of the NB-signal did not skew results. Each column in these tables represents a new calibration period, and each row represents a new validation period. The fit in the upper left corner thereby shows how a model calibrated in the beginning of January fits the EnergyPlus output data from the validation period in the end of January. One row below shows the RMSE between a calibration made in January and a validation made in the last days of February and so on. The results in Table 2 and Table 3 show that the black-box model is not always robust to seasonal changes of weather, for example when calibrating in a heating season (January) and validating with a transitional season (April, spring). However, it is seen that when calibrating within the heating season (for this particular building, November-March, where a heating demand occurs more than 80 % of the time, see Table 4), the RMSEs for the validation within these months has a mean temperature error below 1°C most of the time, except for validations in March. Results in Table 2 and Table 3 show that for both PRBS-models and NB-models, calibration with February data leads to models with most accurate representation of the temperature in the building during the heating season. On the other hand, it is seen that calibration in the month of March leads to models that perform slightly worse in the heating season than February models, but perform better in transitional periods. In transitional seasons (for this building, September, October, April and May) where there is still a demand for space heating (see Table 4), more caution should be taken using the model, especially in spring, as it may be necessary to rely on more frequent recalibration of the model (at least once a month) to ensure its ability to accurately describe temperature conditions within the building.
One reason for this may be that these periods are characterized by a larger variety of weather conditions, especially solar heat gain, which leads to larger variations in heating demand (see Table 4). It is seen that spring is characterized by slightly colder weather and slightly higher solar radiation compared to fall.    Table 5 compares model performance of PRBSmodels validated on dataset with PRBS excitation, NBmodels validated on dataset with night boosting excitation and NB-models validated on dataset with PRBS excitation. The comparison shows that the performance of NB-models, using night boosting or PRBS validation data, is similar to the performance of PRBS-models, although PRBS-models seems slightly better when calibrating and validating within the same month. The evaluation of the NB-models indicates slightly higher average fit for the models compared to the PRBS-models using the PRBS data for validation, when calibrating in heating season. From comparing results of Table 2 and Table 3 it is seen that both PRBS-models and NB-models performs in par, when evaluated on the PRBS validation data. The performance metrics presented in Table 5 indicate that the NB-NB evaluation is characterized by the lowest average RMSE, while there is less of a clear picture when one compares the PRBS and NB-models both evaluated on PRBS data. The models calibrated and validated in two different seasons had, in general, the poorest performance. From Table 5 it is seen that the average RMSE on models in the heating season when calibrated in transitional season is 25-172% higher than when calibrating within the heating season, depending on the model type, see example from NB-models validated on datasets from night boosting excitation, in eq. 3.

ሺʹǤͳͷ െ ͲǤͻሻ
ͲǤͻ ή ͳͲͲ ൌ ͳʹ Further, it is seen that models in the transitional season when calibrated in the heating season is 38-97% higher that when calibrating in the transitional season. From this, it seems that the black-box model is not robust to weather changes, and recalibrations between seasons is recommended.

Discussion
The analysis presented in this paper is based on the overall assumption that the RMSE of a model is a suitable indicator for the performance of the model when used for MPC. A topic of future research is to investigate whether this assumption is true, or whether other methods and metrics should be employed to determine the appropriateness of a model for MPC purposes. Furthermore, the length of the calibration period used in this analysis was chosen to be similar to what has been used in previous studies, such that it could be composed of full-length PRBS signals. Future research should investigate the necessary length of a calibration period to obtain a satisfactory model, and whether this length is affected by the choice of excitation signal.
The results indicate that it is possible to obtain a suitable model using the night-boosting signal used in this study. This means that a heating system using MPC might excite the building sufficiently to generate the data needed for frequent recalibration of the model without the occupants noticing any differences in the indoor climate. The results thereby indicate that it is possible to avoid the use of the inarguably more fluctuating PRBS signals which risk becoming an annoyance for the occupants.
All models, whether it is PRBS-models evaluated using PRBS signal, NB-models evaluated with a night boosting signal or NB-models evaluated with a PRBS signal, show similar results when calibrating and validating within the same season. This opens up the possibility of recalibrating the model in transitional seasons (spring and fall), where there still can be a relatively high heating requirement, as seen in Table 4. The gains of using MPC signals for recalibrating a model and the potential of recalibrating with such a signal in transitional seasons could be further investigated.
Finally, this study considered the performance of a black-box model. Future studies could be to conduct a similar analysis for grey-box models to evaluate whether a physics-based model structure is less vulnerable to seasonal changes in weather conditions than a black-box model.

Conclusion
The purpose of this paper was to evaluate the robustness of black-box models intended for MPC of the heating system to seasonal changes in weather conditions. Blackbox models were identified for a case building using simulated performance data from the first half of every month of a year. The performance of these models were evaluated by comparing simulation output from the blackbox model with data from the last half of each month from the EnergyPus simulation. The analysis used two different excitation signals to generate the data used to calibrate and validate the models: a highly fluctuating PRBS signal designed to generate informative data , and an arguably more occupant-friendly night boosting signal which was considered to be an approximation of the expected performance of an MPC for space heating. For both of these signals, the results suggest that models calibrated in the heating season leads to a satisfactory performance of models through almost the entire heating season. While models calibrated in February leads to the best model performance through the heating season, models calibrated in March perform better across both heating and transitional seasons. The results also show that when seasonal weather changes occur (especially spring), the black-box model loses accuracy and a recalibration is necessary to maintain a reasonable MPC performance.
Because of this, it should be considered whether it is even feasible to operate the building using MPC for demand response purposes in spring since the heating demand in these transitional periods is also varying. The difference in results when exciting the building with a PRBS signal and a night boosting signal, respectively, is very small. Consequently, it should be further investigated whether an MPC-similar signal can be used for identification of a model suitable for MPC purposes, as this signal is less likely to lead to discomfort for the users.