A Characterisation Framework for Parametric Building Performance Simulation Tools

Implementing Building Performance Simulation (BPS) in a parametric design framework is a prevalent way of facilitating environmental assessments in early design stages. However, no up-to-date overviews of potential approaches are available, and no characterisation frameworks adapted for parametric BPS tools are present. In this study, such a framework was developed and its use demonstrated through an investigation of eleven available BPS tools for the parametric design framework Grasshopper®. It was found that the framework was able to successfully differentiate tools based on the level of BPS expertise integrated in the tools, and the allocation of responsibility for data entry and interpretation. A contrast was found between streamlined tools, and tools which provide more versatility. The characterisation framework, and the resulting overview of approaches, can be used to guide the future development of parametric environmental analysis frameworks for buildings


Introduction
To achieve a sustainable built environment, the energy consumption and emissions associated with buildings need to be reduced, while simultaneously considering human well-being and economic development (Clarke and Hensen, 2015). Factors of ecological, social, and economic sustainability can (in conjuction with other methods such as Life Cycle Assessment (LCA)) be evaluated through Building Performance Simulation (BPS), which refers to the computational numerical modelling of buildings. BPS was initially performed for assessing thermal performance and has gradually grown to include the coupled heat and mass transfer through the building envelope, airflows within the building, daylighting, and thermal and visual comfort (Spitler, 2006), as well as locally generated renewable energies (Attia et al., 2013). The large variety of performance indicators, as well as a potential discrepancy between the architect's vision and building performance, often leads to conflicting design targets (Ne-gendahl, 2015). Despite the fact that considering these tradeoffs and predicting the behaviour of a building during its design is more efficient and economical than implementing changes to fix problems in the use phase, BPS is largely implemented in the final phases of design in current practice, to design the HVAC system and demonstrate code-compliance (Hensen and Lamberts, 2011). Thus, research into BPS tools used by architects as design guidance, as opposed to by specialised engineers as design evaluation, is motivated (Attia et al., 2013). To perform a BPS, the geometric building model developed by the architect in a Computer Aided Design (CAD) environment must be transposed into a calculation model, either through a direct connection to a BPS engine within the CAD tool, through export for use with a dedicated BPS tool, or through re-modelling in the BPS environment. Negendahl (2015) sees the "integrated design approach", where the design is driven forward by multidisciplinary teams of architects, engineers, clients, and other stakeholders, as the most promising approach to achieve high performing designs in the early design stage. He concludes that integrated dynamic models which combine design tools, a parametric design framework, and a BPS tool, better support multidisciplinary collaboration than exporting data using formats such as IFC, and performing simulations in external simulation software. Parametric design involves describing the architectural geometry as parametric equations, which in turn "express a set of quantities as explicit functions of a number of independent variables, known as 'parameters'" (Weisstein and Stover, nd). Further, non-geometric information such as materials of building components can be varied parametrically (Hollberg, 2016). Parametric design is a means of dealing with the uncertainties inherent in the early design stage, such as the quick changes to the design, and decisions which are left until later design stages (Meex et al., 2018). The advantages of parametric models include quick generation of alternative designs, comparison and evaluation, as well as optimisation. The most commonly used parametric design frameworks adopted in current practice are Grasshopper® (GH) (Nisztuk and Myszkowski, 2018), for Rhinoceros 3D® (Rhino) (Robert McNeel & Associates, 2022), and Dynamo for Autodesk Revit® (Autodesk, 2022). As Dynamo largely interacts with the Building Information Modelling (BIM) model within Revit®, it is more relevant when studying BIM/BPS interaction rather than purely parametric BPS tools. Hence, BPS plug-ins for GH is the main topic of this study.

Previous studies
There are several studies discussing capabilities and grouping of tools for BPS in early design stages. Attia et al. (2012) identify requirements and selection criteria for BPS tools, namely usability, knowledge integration, accuracy, interoperability, and process integration. Østergård et al. (2016) provide an extensive review of building performance simulation approaches and software, and mention some parametric approaches relevant at the time, as well as links to LCA. Touloupaki and Theodosiou (2017) provide a literature review of works discussing parametric tools and conclude that issues of interoperability and user-friendliness are the main barriers to implementation of BPS in early design stages in current practice. Han et al. (2018) review simulation tools for green buildings with no explicit focus on parametric tools. Batish and Agrawal (2019) review data-driven approaches for building energy prediction which can integrate with parametric tools. Gassar et al. (2021) provide an overview of studies on performance optimisation. None of these works specifically provide an overview of approaches to BPS within parametric design frameworks. Further, there is no characterisation framework which can support the comparison of parametric BPS tools through systematic characterisation and categorisation. This knowledge gap motivates the present study.

Objective, purpose and research questions
The objective of the research project including the present study is to find approaches to parametric BPS which can inspire the development of design integrated frameworks and tools for holistic environmental assessment of buildings in the early design stage. This calls for a qualitative assessment of the steps taken in each approach, and not of the precision of assessment.
The purpose of this study is, firstly, to develop a framework for characterisation and qualitative evaluation of parametric BPS tools. Secondly, for purposes of exemplification, to provide an inventory of currently available parametric BPS tools and to apply the characterisation framework to some relevant tools. This is done to investigate if the proposed framework can identify alternative approaches to performing BPS in a parametric design environment.
The purpose is fulfilled through responding to the following research questions: • How can parametric BPS tools be systematically categorised and characterised? • What is the state of currently available parametric BPS tools in Grasshopper®?
• What are alternative approaches to parametric BPS tools in terms of workflow and scope of BPS?

Scope and delimitations
The following scope and delimitations have been defined for the study: • Only tools which perform BPS (energy, solar, daylight, comfort) are studied, • Only tools readily available and updated within the last five years are studied, • Only tools for Grasshopper® are considered, • Tools are investigated from the perspective of architects active in the early design stage, • Tools are qualitatively evaluated based on potential for integration in architectural design workflows, not quantitatively, based on e.g. precision, • Tools are only investigated based on sample scripts and test cases, not in real design processes.

Characterisation methodology
First, a framework for classification, characterisation of parametric BPS tools based on their potential for implementation in the early design stage was developed, and a user persona was developed to position the characterisation from the intended users of the tools, which is architects in the early design stage. For purposes of exemplification, a test case was developed to show the principal workflow and functionality of the tools. Second, the characterisation framework was tested by performing an inventory of actively maintained BPS plug-ins for GH, and subjecting some of the tools from the inventory deemed promising to an investigation using the characterisation framework. Information was gathered for use with the characterisation framework both from documentation provided by the tool developers and through testing of the plug-ins using sample files packaged with the software, if available.

Classification and characterisation framework
Based on a classification framework for LCA tools integrated in BIM software, proposed by Wastiels and Decuypere (2019) and adapted for parametric LCA tools by Säwén et al. (2022), three classes of parametric BPS tools were identified, as shown in Figure 1. Using approach 1, geometry is modelled in GH, and simulations performed by the plug-in within the GH environment, where results are output and visualised. Using approach 2, the plugin instead interacts with an external simulation engine, which typically returns results to GH for output and visualisation. Using approach 3, the plug-in exports GH data to a spreadsheet or another data format, which can then be imported in an external software which handles simulation and result visualisation.
To characterise tools based on their features and capabilities, an LCA tool characterisation approach developed by Hildebrand and Bach (2018), and adapted for parametric LCA tools by Säwén et al. (2022), was modified to allow . for comparison of parametric BPS software. Hildebrand and Bach (2018) specify eight characterisation categories for LCA tools: origin, data source, required user's knowledge, accessibility, entry format, level, default settings, and LCA phases. To account for the requirements for parametric BPS software, these categories were adapted into nine categories, as shown and explained in Table 1.

User persona
As the end goal of this study is supporting the development of tools and frameworks which are relevant to architects in early design stages, a user persona (Johansson and Messeter, 2005) was developed to achieve a consistent perspective for the characterisation. The user persona was defined by the following set of assumptions and points of departure: • The user is an architect with basic skills in Rhino/GH. • The user is aware of the fundamental principles of BPS and the considered indicators. • The user is mainly interested in evaluating design alternatives/effects of design choices in early stages. • The user has limited time/effort to perform the analysis. The situation could be a meeting with a client or a design competition. • It is assumed that the core of the script has been modelled previously, that is, that the role of the user is to adapt an existing script to the project at hand.

Test case
To allow for fair and easy comparison, each plug-in characterised was used to model and perform an energy, daylight, and solar heat load simulation on a 1 m 3 cubic box, with one face containing a glass window, and the other surfaces modelled as two material layers stacked elements, external 50 mm mineral wool and internal 50 mm Cross Laminated Timber (CLT).

Tool inventory
Tools were identified through searching the Rhino plugin community food4Rhino provided by McNeel, the developer of Rhino (McNeel Europe, 2016). Searches were made using alternative combinations of the keywords: "energy", "daylight", "solar", and "building performance". Further tools were identified from the literature study, and from the knowledge of the authors.

Tool characterisation
The results of the tool inventory are shown in Table 2. Eleven tools were found. Each tool was classified as using one of the approaches show in Figure 1. Based on the inventory, four tools were selected for an exemplifying characterisation, based on showing maturity and potential for exemplifying different approaches to early-stage design process integration: BeDOT, ClimateStudio, ICE-Bear, and Ladybug Tools. The tools were classified and characterised using the developed characterisation framework, from the perspective of the user persona. is an tool which includes energy simulations, solar analysis and web-based daylight studies. The tool has been developed in a cooperation between Chalmers University of Technology and the consulting company Bengt Dahlgren AB (who provide access to the tool), both in Gothenburg, Sweden. This characterisation focuses on the energy performance simulation part developed initially by Fantin Do Amaral Silva and Bergel Gómez (2018), and its up-to-date version of the script (BeDOT v1.3).
BeDOT includes seven modules: 0. pre-processing, 1. geometry, 2. ground modelling, 3. solar radiation, 4. energy, 5. visualising and 6. post-processing and data export. The data flow and calculations follow this order, downstream. Ground modelling is excluded in the test case since it is not up to date. In the pre-processing module all inputs are retrieved. Geometry is imported from Rhino and the specification of properties, like internal heat gains, ventilation rates, Uvalues, and time schedules, are defined in an Excel file. The input is adapted and processed using components from Honeybee, Ladybug and DAYSIM to create thermal zones with specified properties to run solar analysis. The energy calculation is performed within a component written in Python and provides heating/cooling energy and power demands based on the calculation procedure in EN13790:2008 with hourly time steps. The results can then be visualised on the geometry or reported back to  the Excel file. The adaptability for the user is mainly constrained to the Excel file and to Rhino, whereas the GH script is mainly working as the simulation engine. A large benefit of moving the input specifications to an external Excel file is that the user is in full control of all data compiled in a commonly used environment and it is easy to save and document alternative settings and thereby evaluate alternative solutions. The data required in the Excel file is primarily on a system level, focusing on components, zones or building properties. This allows easy assessment of large changes to the building systems without including detailed material data. E.g., only one value needs to be adjusted to evaluate a change of the thermal inertia instead of specifying all included materials.
Since the tool is packaged as a GH script rather than condensed into a plug-in it can be overwhelming for the novice. Nevertheless, this provides a transparency to the tool, and it follows a clear and hierarchical logic. The primary interface for defining input data is Excel, a commonly used spreadsheet tool, while the geometry is drawn within Rhino, and the GH interface is used to get the simulation running. However, the input in Excel requires detailed information from the user, both about the building itself and about building technology which is not the focus of architects in early stages. In return one achieves better precision and detailing in the results. ClimateStudio ClimateStudio is a software developed by Solemma LLC (2022). It provides an extensive set of environmental analyses regarding daylight, sunlight, and thermal simulations. The tool is built on EnergyPlus and Radiance and is a plug-in for Rhino which is also available in GH.
The workflow in ClimateStudio is similar regardless if the aim is an energy, daylight or sunlight analysis. Properties are assigned to the geometry which is then combined into a thermal or daylight model before it is passed into the simulation engines together with climate data from Ener-gyPlus weather files (.epw). The construction of daylight models requires the geometry with assigned material properties as well as control point grids on the geometry where one wants to evaluate light conditions. The thermal model requires thermal zones which are constructed of geometry with assigned thermal properties and a program which defines the scheduling of thermal loads. ClimateStudio includes a component with predefined workflows for common simulations which can be used to get the core of the needed scripts instantly. The input required to perform simulations in ClimateStudio can be defined on various levels. Materials, loads  and schedules can either be entirely defined by the user or selected from an extensive library containing predefined settings. In the thermal model, the constructions can be built up by defining several material layers or chosen from predefined construction types in the database. This means that it is possible to either quickly model the case based on predefined values or to customise the model to capture a specific case. The simulations are easily adaptable regarding resolution, analysis period and weather conditions and the results can be plotted on the model itself as surface colouring or in charts by the provided visualisation components. Since ClimateStudio is offering high flexibility it is adaptable to early design stages where the knowledge of the building is limited but also has the potential to be applied further into the design process, meaning the required knowledge of the user depends on the scope of analysis.

ICEbear
ICEbear is a plug-in for building performance energy modelling developed at Aarhus University (idbuild.dk, 2022). It targets early design stages and was initially developed for GH, but also works with Sketchup and will be extended to Revit as well. Solar heat gain calculation is implemented in the energy calculation and there is a possibility to connect to daylight calculations performed in Honeybee or DIVA. The investigated version is v 01.00.00. The workflow in GH when using ICEbear is very simplified. The only input data needed is the geometry divided into roof, external walls, floor, windows, and floor facing ground. There is also a possibility to add external shading geometry. When the geometry is set, a separate user interface allows selecting the number of occupants, location, glazing system and a zone template containing the data needed for the thermal simulation. The simulation outputs simplified and detailed results about energy demand, thermal comfort, daylight, and air quality. The interface is adapted for quick modelling without the need for detailed information about the building. Instead, predefined templates for different type of use are provided. However, the tool contains the option to open these templates and edit the detailed settings for the simulation. The adaptable parameters are mainly concentrated on the geometry itself and the window and material properties, i.e., aspects which are closely related to architectural design in early stages. The tool is built in a way that non-experts easily and quickly can set up models and simulate the performance. It can be done by only drawing the geometry and then making four selections. However, for both the input and the output more detailed options are provided through detailed simulations settings and the results can be investigated further by opening the detailed results with hourly data and plots.

Ladybug Tools (Honeybee + Ladybug)
Ladybug Tools is an open-source modelling environment consisting of a set of plug-ins for environmental design.
In this characterisation, the plug-ins Ladybug and Honeybee are studied. The Ladybug plug-in itself is created for analysing weather data and provides sun path, thermal comfort, wind analysis, and a connection to climate data. Honeybee, is a plug-in establishing a link to EnergyPlus and Radiance, allowing for both energy and daylight simulations (Ladybug Tools, 2022a). The first version of the plug-in was developed by Sadeghipour Roudsari and Pak (2013) with the aim to connect the workflows included in environmental building design and to increase the general understanding of it (Ladybug Tools, 2022b). The workflow in Honeybee starts by converting Rhino geometry into Honeybee Faces which make up Honeybee Rooms and Honeybee Models. This is enough to run a daylight or sunlight analysis, while the energy simulations also requires definition of boundary conditions, thermal zones and programs assigned to the zones which contains the schedules for thermal loads. In addition, weather data and location are needed for all types of analysis and are retrieved from .epw files for energy and sunlight simulations. Daylight analyses requires specification of sky settings.
When the type of analysis is selected, and the analysis settings are defined the analysis is running via the simulation engine: Radiance for daylight, and EnergyPlus for energy performance. The results are then visualised using components which colour grids or meshes.
Honeybee includes several components to adjust the properties of Honeybee geometry. This could for instance be window-to-wall ratios and shading geometries. Another possibility is to assign properties to the geometry according to the orientation towards the cardinal directions. Honeybee contains various simulations related to daylight and sunlight, and there are possibilities to select different preset sky conditions compliant with different standards.
Since the analyses are based on control points defined by grids there are also possibilities to adjust the resolution of the simulations, and the simulation period. Results can also be average, cumulative, or peak values. Another way to adapt the plug-in for the required simulation is by utilising modifiers to change the properties of the materials and geometry that are related to the specific analysis. E.g., in daylight simulations one can modify the reflectance of materials. The energy section of the Honeybee library includes components to specify or apply existing building properties from a library. One can specify building and zone program to retrieve general thermal load data for the specific building type. For more detail constructions and their integrated material properties can be specified. HVAC properties can also be provided and edited through templates.
There is an extensive list of components included in the plug-in libraries and the required knowledge very much depends on how detailed one wants perform the simulation. The experience is that the detail of the plug-in can be used to define scripts applicable for non-experts in early stages as well as extended into the later stages by experienced users.

Discussion
The purpose of the characterisation framework was to systematically determine which aspects of the investigated tools were generally similar, and which aspects showed the largest differences. The application of the framework allowed a comparison of the approaches in the four tools investigated in detail. Similarities were found in term of the input data and the included analysis and simulation types.
The largest difference between the investigated BPS tools was the sequence of choices the user must make to go from model to simulation results. The ICEbear plug-in combined a quick workflow with allowing detailed input data, if desired, in an interface separate from GH. BeDOT also makes use of a separate interface in Excel with the detailed input data specifying the properties of materials, HVAC systems, and internal loads. Contrastingly, Ladybug Tools and ClimateStudio provide a wide set of simulation types and settings on several levels of detail. In this way, the user can tailor the script to the specific use case in an extensive way. The workflow is largely similar in Ladybug Tools and ClimateStudio.
BeDOT and ICEbear provide a strict scope for the assessment, ensuring consistent output and a robustness of the tool, while the flexible nature of Ladybug Tools and ClimateStudio makes them more dependent on the user's knowledge, but in return allows possible application in a wider set of situations.
The main takeaways of the investigated tools can be summarised as follows: • The input data should be limited to aspects and in-formation about the building relevant for, and known by, the architect in the early design stage. • The detail of the input should be at the same level as the design, complemented with reference values. • Provide a clear and intuitive order in the workflow of the user interface. • The assessments and calculation models should be transparent, yet not adaptable in its scope.
It is important to bear in mind that GH allows for a versatile workflow and that plug-ins can be used in different ways and adapted to different purposes and the outcome of this characterisation is based on a subjective experience.

Conclusion
In the present study, a characterisation framework for parametric Building Performance Simulation (BPS) tools aimed at the early design stage was developed and applied using a user persona approach in a tool characterisation. Eleven tools for the parametric design framework Grasshopper® (GH) were collected in an inventory, and four of these were further investigated using the characterisation framework to evaluate its usefulness in identifying alternative approaches to parametric BPS tools.
The proposed characterisation framework acknowledges three classes of parametric BPS tools based on their functionality: 1) modelling geometry, performing simulation using a plug-in, and visualising results directly in GH; 2) modelling geometry in GH, and using a plug-in to export building information directly to an external analysis engine where simulations are performed and results are visualised; and 3) modelling geometry in GH, and using a plug-in to export building information into a spreadsheet format which can then similarly be used in an external analysis engine. From the eleven plug-ins found in the tool inventory, most used the second approach, where GH is used to prepare data for use with an external engine. Tools can then be characterised according to nine categories: required knowledge, geometry input, adaptability, modelling level, output of results, intended application, building performance analysis, complexity of input, and type of simulation. This characterisation is useful to guide ongoing and future development of analysis tools, as it provides developers with information on what aspects of BPS tools are common to all tools, and which aspects show a greater variety. From the four tools selected for an exemplifying characterisation, two separate approaches could be identified, where BeDOT and ICEBear provide a limited and simplified analysis which requires little previous knowledge from the user, whereas Ladybug Tools and ClimateStudio provide more flexibility which requires more knowledge from the users but in return covers more potential situations.
The main contribution of this article is a proposed and demonstrated systematic characterisation framework for parametric BPS tools. This is useful to identify prevalent approaches, and allows investigating success factors for their application in the architectural engineering practice. The framework developed in this study can be applied to a more in-depth review of tools, the information from which can be used to develop a holistic analysis framework for the early architectural design stages which includes BPS. In further research, the usefulness of the two main approaches identified in this study should be compared and evaluated using e.g. prototyping and user tests, and further approaches could be identified to a wider-scope review.