Proposed method for probabilistic risk analysis using building performance simulations and stochastic parameters

. As parts of the world continue the work of mitigating the impact of climate change, many countries strive for continued reductions in energy demand from buildings by implementing more stringent building regulations. Consequently, the importance of accurate and efficient building performance simulations to predict the energy use of a building design increases. As observed in earlier studies, there are performance gaps between the predicted annual energy demand from building energy performance simulations based on deterministic methods compared to the monitored annual energy use of a building. This paper presents a preliminary method developed using probabilistic methods for risk analysis and building performance simulations to predict the energy performance of buildings using stochastic parameters. The method is used to calculate the probability for the energy performance of a building design to fulfil the energy requirements. The consequences are quantified using an example of energy performance contracting to evaluate the inherent risk of a building’s design. The method was demonstrated in a case study and validated by comparing the results in energy performance and probability of failure against measured data from 26 single-family houses.


Background
As parts of the world continue the work of mitigating the impact of climate change, many countries strive for continued reductions in energy demand from buildings by implementing more stringent building regulations [1]. Consequently, the importance of accurate building performance simulations (BPS) increases [2]. As observed in earlier studies [3]- [8], there is evidently a performance gap between the predicted annual energy use from BPS based on traditional deterministic methods compared to the monitored annual energy use of a building. Overcoming the performance gap is one of the main challenges that remains in BPSs [9].
There are many causes for the performance gap, as shown in earlier research [3]. To a varying degree, the expertise of the modeler impacts the result, but even BPS performed by highly experienced and well-trained specialists will have a performance gap when performed according to traditional deterministic methods. Excluding the modeler's impact on the simulation results, there will still be a performance gap because of the inherent limitations with the deterministic method where the uncertainties of the real-world are modelled with certainty. These uncertainties originate from different causes [7], [10]. Examples of this include uncertain design choices, component properties, limited data, the operator's interpretation of information and time constraints, limitations in the simulation software used and deviation in operation and occupancy usage of the finished building [7], [10]. The classification of uncertainties presented by Rezaee, R. et. al. [11] clarifies the different types of uncertainties, as shown in Table 1. Table 1. Classification of uncertainties in building performance simulations [14].

Numerical Computational and numerical imperfections
Modelling Modellers interpretation and abstraction of reality Scenario Boundary conditions, scenario for building use

Design Decided
Parameters with a specific design decision, physical uncertainty quantifying variability in material properties

Undecided
Uncertainty in the evolution of the design The Swedish building regulation [12] requires that the energy use of a building be simulated during the design phase using a dynamic BPS software and deterministic methods to predict the energy use for space heating, domestic hot water use and electricity use by building services to indicate the code compliance of the building design. It is also required that a margin of safety is added to the simulated results of the building's energy use, to take the uncertainties into account. This measure is taken to reduce the risk of the measured energy use of the building not fulfilling the requirements. Unfortunately, the Swedish building regulation does not quantify the size of the margin of safety to be added. Previous research has tried to quantify the margin of safety but have been unsuccessful [13]. The regulation also requires that the completed building be monitored uninterrupted over a 12-month period within the first two years of operation. The measured values for this 12-month period of use should then be normalized and compared to the requirement in the regulation to determine if the building fulfils the regulation. However, the real-world use of a building is stochastic. Significant deviations between the real-world use and the predicted results based on predetermined conditions set in the deterministic model must be explained. If the explanation is due to the performance of the building, then the building does not meet the regulations. However, if the explanation is extreme user behaviour, and the building model can predict this increased energy use when the extreme use is included in the model, then the building still fulfils the regulations. Because of these differences in the building's monitored energy use compared to the simulated energy use, the Swedish building regulation requires that the predicted energy performance from the BPS be adjusted with a margin of safety.
Because the Swedish building regulations focus on the real-world performance of the buildings, the necessity of accurate results and either a quantified margin of safety, or a method for including uncertainties and quantifying risk is high. Using a probabilistic method -with stochastic input data using Monte Carlo-method and dynamic BPS software -makes it possible to simulate and quantify the response of a building design as a probability distribution for the predicted energy performance [2], [10]. The results could then be used to indicate the probability of a building design fulfilling a requirement [14].
The consequences of not fulfilling the critical condition (i.e. energy requirement) depends on the contract involved in the project. In Sweden, it is becoming more common that construction companies sign a so-called "Energy Performance Contract" or Guaranteed Energy Use agreement. These agreements say that if the building's actual energy use exceeds a certain value, then economic penalties will incur. An example of this is "Energy agreement 12" from [15] which states, depending on the agreement, that; the construction company pays all extra energy costs to the building owner; the construction company pays a fine; the construction company must reduce the energy use to the agreed value at no cost to the building own, etc. This is now becoming an economic risk for construction companies who sign such contracts and there are no cost/time-efficient methods available which can analyse the risk that the building will exceed the agreed upon energy use.

Aim
This paper presents a preliminary method developed for probabilistic risk analysis of the energy performance for a building design. To evaluate the inherent risk of a proposed building design, the method aims to quantify the probability of fulfilling the requirements -using probabilistic BPS -and the associated consequences.

Limitation
The probabilistic risk analysis method developed in this paper is meant to be used in the later stages of the design process to evaluate the economic risk for a building design not fulfilling the energy requirements.
This study was performed on a building level and within a Swedish context, e.g. based on Swedish regulations, buildings, climate, and other input data. However, the method and conclusions are applicable elsewhere. Since cooling is uncommon in Swedish residential buildings and not available in the case study buildings, this parameter is excluded from the method described.
As mentioned in the introduction, there are several types of parameter uncertainty. The types included and evaluated in this study are classified as "scenario" uncertainties -based on user behaviour uncertainty such as domestic hot water use -or "decided design" uncertainties -where a specific property is decided but the real-world performance of the component when installed in a building is uncertain. The explorative phase, where it is decided which parameters to include in the evaluation is not part of this first stage of proposed method.

Methods used in study
The study began by investigating the current state-of-theart, as described under the section background. Based on the compiled knowledge and the identified limitations of current methods, ideas on how to improve or solve these problems were identified.
Developing the method: The next step was to conceptually define the method for probabilistic risk analysis of the energy performance for a building design. This method is a combination of two paths, used to determine both the probability and the consequence. The path for determining the probability of failure is based the method for probabilistic energy calculations presented by Burke et al. [16] and further developed to enable the identification of probability. For the path of determining the consequences a method for defining and evaluating the consequences was developed. To achieve the goal, the paths were divided into smaller steps for which methods and solutions for each step were either identified or developed. For each step, the developed method is described and using equations and figures to further explain the process.
Demonstrating the method: To demonstrate and evaluate the developed method and to demonstrate how to use the output, a case study was performed -applying the method to a case study building and evaluating the building design -to produce data for the quantitative analysis.
Validating the method: The results from the case study were validated by comparing them to measured data from the newly developed neighbourhood comprised of 26 single-family houses.

Developed method
The methodology developed in this study begins with defining the initial conditions: stakeholder values, objectives and conditions, followed by defining the baseline building model and the starting point. The deterministic parameters are then determined and entered to the model followed by sampling the input matrix for the stochastic parameters from the probabilistic distributions. Next, the additional design options to be evaluated are defined. When these initial steps have been performed the simulations can then be performed, the output is collected, converted and analyzed in the following steps. Adding the consequences, the risk could be calculated, and the design options could be assessed. Thus, the inherent risk of a building design not fulfilling the requirements can be evaluated.

Input data -Stochastic parameters, iterations and probability distributions
The probabilistic method requires a large data set to enable the statistical analysis. This data is produced by an iterative process of simulations of the building model using randomized input data for the stochastic parameters (x) and constant values for the deterministic parameters for the iterations (i). The Monte-Carlo technique was used for the random sampling from the stochastic parameter's cumulative distribution function to determine the numerical value for each parameter in each iteration, represented by the n-by-m sampling matrix X, see equation 1. (1) To determine the distribution of each stochastic parameter, performance data (observations) or estimations are compiled and analysed. If performance data is available, the distributions are converted to a probability distribution. If data is not available, an estimation based on experience is used to decide on the most relevant alternative for a distribution. A limitation in the current method is that the probability distributions were simplified to two types of distributions, uniform and triangular. The triangular could be either standard or skewed. This simplificationto discrete distributions -is used to resolve the issue with unrealistic numerical values for input data, e.g. negative energy use.
Uniform distribution: The uniform distribution (PU) is a distribution with an even probability density function (PDF) that have the same probability for all possible observations within the interval [a, b], see  Probability density function:

Simulations and equations
Building performance modelling and simulation: The evaluated building is modelled in the BPS software using a three-dimensional model-builder and the building characteristics. All parameters relevant for the energy performance of a building, that are not included as a stochastic parameter, are assigned a constant numerical value, as with a traditional deterministic method. For sampling of the input data for the stochastic parameters (x), the Monte Carlo method is used, as described under section Input data. For the evaluated building design, the input data (X) for each iteration (i) and stochastic parameters (x) are used in in the BPS and the output -the energy use for each iteration (Ei) for the building, consisting of space heating (ESH,i), domestic hot water (EDHW,i) and electricity used by building services (EEL,i)are compiled, see equation 6.
Energy performance: The energy performance is defined by the contract of the project. An example of how the energy performance (EP) should be calculated and evaluated against the contracted level (EPCode) based on the Swedish Passive house standard definition is shown below, see equation 7. When used with other certification systems, this step needs to be altered according to that specific systems definition of energy performance.
Probability: The probability density function for the energy performance is calculated using a goodness-of-fit test to determine a distribution fitted to the data, between different distributions and the compiled energy performance (EPi) for all iterations, see equation 8.0-8.3.

‫ݔ(ܲ‬
The mean value, ߤ, is calculated by: Based on this, the variance, ߪ ଶ , is calculated by: Then the standard deviation, ߪ, is calculated by: Code compliance and probability of failure: The probability of failure (PF) is calculated using the probability density function (P(xEP)) by integrating the share exceeding the energy requirement (EPCode), see equation 9.

Consequences:
The conditional consequence (Q) given that we do not comply with the building code is based on the iterations with an simulated EP above the requirement and is calculated based on the average operational cost increase for the energy use, based on the energy cost per kilowatt-hour (CE), see equations 10 and 11.

Demonstration of method -Case study
The case study was applied to the same building that has been previously used in the project [16], and using the same parameter distributions. The same simulation tool -IDA ICE [17] -and method for probabilistic energy calculations developed by Burke et al. were used to replicate the probability of fulfilling the regulation and to collect quantitative data. However, how the method was applied during this study differs from the previous study, because of new possibilities and limitations available from new features added to the software in the new versions (4.8 and 5.0).
The evaluated object is presented in in the methods section and in Fig 5. The parameter uncertainty included in the simulations using the probabilistic method for BPS is presented below in Table 3. The number of parameters (x), and iterations (i) used in the case study were: x = 15, i = 1000.

Conditions
The conditions used for the probability of failure in the evaluation were based on the requirements in the Swedish passive house standard FEBY09 [18]. The energy performance of a building is not allowed to exceed 60 kWh•m -2 •a -1 .
The conditions used for the consequences included in the case study are; increased operational costs due to energy use exceeding requirements. The costs were estimated based on the concept of energy performance contracting described in "Sveby avtal 12" [15], which is a standardized contract developed by the building industry regarding the energy performance of buildings in Sweden. The contract defines a fine (CF) -€ per kilowatt-hourfor the monitored energy use (EP) that exceeds the contracted level (EPCode). The fine is then to be payed until the building contractor either corrects the buildingthrough measures that improve the performance of the building -so that the measured energy is reduced to the contracted level, alternatively pay the fine until end of the warranty period (10 years), see equation 11. In this evaluation the fine (CF) was assumed to be 0.2 € per kilowatthour.

Input data -parameters, iterations, and probability distributions
The building model is based on both deterministic and stochastic parameter input data. In addition to the 15 stochastic parameters presented later, the other parameters required in the building model were kept as deterministic constant design values.  Table 2.

Stochastic parameters:
The stochastic parameters included in the evaluation were kept to the same 15 parameters as in the previous study [16], both to simplify the work and to be able to compare the results. The type of uncertainty that was included in the parameter distributions were based on the "scenario" and "design -decided" as defined earlier in the introduction section. The assigned range and shape of the distributions for each stochastic parameter were also kept identical to the previous study. The stochastic parameters and their assigned distribution are presented in Table 3.

Validation of method
To validate the method, the energy performance and probability of failure results from the simulations in the case study were compared against measured data from the newly developed neighbourhood comprised of 26 singlefamily houses. The houses were built as passive houses according to the Swedish certification FEBY09 [18] with identical floorplans, building envelopes, and building services. To test the method the results was compared against measured data for the total energy use -consisting of space heating, domestic hot water use, and electricity used for building services -in each of the 26 single-family houses. By comparing these, the accuracy of the method in predicting the probability for a building design to fulfilling the energy requirements is evaluated.

Results from Demonstration
The building model used in this case study was based on the original building model used in the previous study for the case study building. The building model was simulated for 1000 iterations using the sample input data, from the distributions and the Monte Carlo method. There were instances when simulations failed, for unknown reasons, and these simulations were thus excluded from the analysis. A total of three simulations failed, out of the 1000 performed. The output from the simulations were compiled and converted to energy performance and compared to the requirements. The results were compiled, and the statistical analysis was performed in MATLAB R2019a. Using MATLAB, the PDF and CDF were determined, and a curve fitting was performed to estimate a function for the data distribution -using a normal distribution and the Statistics and Machine Learning Toolbox. The results are presented as a probability density function in Fig 6 and the probability of failure with the current design was calculated to 1.6 percent. presented as PDF and fitted as a normal distribution. The probability of failure was calculated to 1.6 percent.
Based on the results from the simulations and the conditions defined for the case study the consequence was calculated according to equation 13. The average consequence per house was 355 € and for the whole project with 26 houses the total consequence was 9 200 €. Combining the consequence for the whole project and probability of failure the risk level was calculated to 147. However, without an alternative design options to compare with, there is no relative number to compare and further evaluate this result.

Proof of concept
To test the method and evaluate the results presented above, the simulated results were compared to the measured data from the case study buildings. The probability of failure was tested against measured data from the case study buildings. The measured values for the energy use from the 26 case study buildings are presented in Fig. 7. As can be seen, there is a correlation between the simulated and measured values regarding both range and shape of the probability distribution. However, the simulated results do not include the extreme outliers that are found in the measured data. This is as expected, since the parameter distributions were discrete, i.e. defined in a limited space to avoid unrealistic input values. As an example, negative domestic hot water use, which could happen if the parameter distribution was continuous. This result is additional confirmation of earlier research by Burke et al. [16] since an alternative method was used to replicate previous research with similar results. Unfortunately, there is no data available to compare and test the consequences and the results from the risk analysis against.
In a previous study when the 26 houses were evaluated, an abnormal user behaviour was identified in some houses [19]. Excluding the outlier data point in the measured data the curves based on the simulated and measured data converges, as expected. The remaining difference could likely be explained by the uncertainties currently not included in the simulations. Another likely cause is the household electricity, which in a recent study [20] has been showed to be lower than what is included in the standard data. Comparing the results from the demonstration (PF: 1.6 %) to the measured data (PF: 18 %), and the measured data excluding outlier (PF: 11 %), see Fig. 7. Thus, the current data and stochastic variation results in an underestimation of the probability of failure. Another uncertainty in the results is the impact from the number of iterations used for the simulations and the limited data set for the measured data. Fig. 7. The results from the BPSs (n=996) and the measured data from the 26 single-family houses imported into MATLAB and data presented as CDF and fitted as a normal distribution. Results, probability of failure: simulated, 1.6 %, measured data 18 % and adjusted measured data, 11 %.

Stakeholder impact
The example of consequences was based on the viewpoint of a building contractor and their risk for liabilities because of a too narrow margin of safety for the critical conditions. The consequences could be defined differently if using the viewpoint of other stakeholders, i.e. the building owners increased operational costs or -on a national level -the regulatory body's need for increased energy production and distribution capacity to compensate for the increased energy demand when the building stock does not fulfil the energy requirements.

Conclusions and future work
The study has conceptually demonstrated how a preliminary method for probabilistic risk analysis using dynamic building performance simulations and stochastic parameters could be implemented. The developed method was demonstrated using a case study and the results of the method evaluated and validated by comparison to measured data. The first part -regarding energy performance and probability of failure -was tested against measured data from the case study buildings. However, for the second part regarding risk analysis and comparing design options, there are no results or measured data to test against. Since the evaluation was performed post-construction, and all reference buildings were built identically, no data for an alternative design with reduced risk is available. Thus, the possibility of evaluating the risk analysis and how the results could be implemented in the buildings and thus validated was not possible using this case study.
Further development is required to improve the quality and the availability of data for other relevant stochastic parameters and how to implement these parameters in the method. The new tools available improve the availability to the building sector and the possibility for the sector to implement this type of analysis during the building design process. However, there is still a need for improvements in the method regarding decision-support, what uncertainties and sensitivity analysis does the stakeholders require and how to present the results to improve the robustness for the basis of their decisions.
Based on the knowledge gathered and the results discussed in this paper, the continued work of this research project aims to develop and test the method for improved multi-objective decision-support during the design phase. Also, how the visualization of the output could be improved to better support the decision-makers during the design phase to reduce the risk in projects. The results also need to be validated against monitored data. Applying the methodology to other typologies is also needed, to evaluate the possibilities and limitations in more complex building models and with additional stochastic parameters. A continuation of the method aims to include the explorative phase, where the decision of which parameters to include in the evolution is described and how alternative design options could be evaluated. A more detailed description of the method will be published in future work.
The project was funded by the Development Fund of the Swedish Construction Industry (SBUF) and NCC AB.