Comparison and development of daylight simulation software – A case study

Working with daylight simulations on a regular basis, the authors discovered that it is increasingly challenging to achieve daylight criteria in new building projects. In addition to complex building designs, a new European Standard was published in 2018 named EN 17037:2018 Daylight in Buildings, which includes climate-based daylight simulations. These new criteria require a different type of simulation and presentation of results, compared to the simplified method of calculating average daylight factor, which is mostly used today. The authors have created, tested and validated scripts for performing climate-based daylighting simulations, using Rhinoceros and Grasshopper. The script designed for climatebased daylight simulations, showed too extensive differences between supposedly equivalent criteria for daylight factor and daylight autonomy, meaning it can’t be applied in construction projects as the only documentation of daylight availability. Further testing and investigation of the climate-based daylight script has to be done, in order to perform correct evaluations of realistic daylighting conditions.


Introduction
Working with daylight simulations on a regular basis, the authors discovered that it is increasingly challenging to achieve daylight criteria in new building projects. In addition to complex building designs, a new European Standard was published in 2018 named EN 17037:2018 Daylight in Buildings. This standard, as well as other certification systems such as BREEAM-NOR, LEED and WELL, include climate-based daylight requirements. These new criteria require a different type of simulation and presentation of results, compared to the simplified and traditional method of calculating the average daylight factor, mostly used in Scandinavia today.
The daylight factor is the oldest daylight availability metric and was defined by Moon and Spencer in 1942 [1] as the ratio of the internal illuminance at a point in a building to the unshaded, external horizontal illuminance under a CIE overcast sky [2]. This means that "the daylight factor is independent of building location and orientation, season, time of day, direct solar ingress and variable sky conditions" [3]. "The result is a daylight factor optimized building, admitting as much daylight as possible into the building, neglecting arising issues concerning comfort and energy use" [3].
"Climate-based daylight modelling is the prediction of illuminance on the working plane using realistic sun and sky conditions based on standardized climate data. Daylight Autonomy (sDA) is a measure of how often a minimum workplane illuminance threshold can be maintained by daylight alone" [4]. "It is a measure of daylight illuminance sufficiency for a given area, reporting a percentage of floor area that exceeds a specified illuminance for a specified percentage of the analysis period" [5]. EN 17037:2018 states ''that a space is considered to provide adequate daylight if a target and minimum illuminance level is achieved across a fraction of the reference plane within a space for at least half of the daylight hours'' [6]. The recommended values are illustrated in Fig. 1.  The two main objectives of the work were to develop a solution to validate daylight factor simulations between different software and to perform climate-based daylight simulations.

Methodology
Experiencing a shortage in the simulation software currently used for daylight simulations in Norway, a new software was considered. Rhinoceros with Grasshopper is a visual programming software, where the user creates scripts using simulation modules and plugins [7]. Honeybee is a plugin in Grasshopper, containing several daylight-analysis modules, including climate-based daylight simulations [8]. Being a visual programming software, Grasshopper-scripts have to be designed by the user. Therefore, it was essential to validate the scripts that were created before applying it to projects. The CIE standard 171:2006 Test cases to assess the accuracy of lighting computer programs, is used to validate programs simulating light, and contains tests for both artificial lighting and daylight [9]. Working with this standard, the experience was that it was aimed at program developers rather than users, requiring a high level of programming expertise. This is also mentioned in papers comparing and validating daylight modelling methods and programs [10,11]. Therefore, a different validation approach was selected, comparing results from Rhinoceros and Grasshopper with other daylight simulation software, validated by the developers. IDA ICE (EQUA Simulation AB, version 4.8) and Relux (Relux Informatik AG, version 2.0.0) were the two programs chosen for the comparison.

Test cases
To compare the programs, 16 test cases were made. The cases consist of four different room geometries, exposed to four different situations: One-sided illumination, twosided illumination, shading overhang, and shading obstruction. The variety of the cases was chosen to expose the script to different obstacles which are common and important in daylight simulations. The situations were also inspired by the test-cases in CIE 171:2006 [9]. The four room geometries are presented in  Table 1.

Validation of Script
Since the software cannot be used to compare climatebased daylight simulations, another common output had to be chosen. IDA ICE and Relux, are both mainly used for simulating daylight factor, making average daylight factor the natural choice. Two scripts were made, one simulating daylight factor, and the other simulating daylight autonomy. Based on the knowledge and validation of the framework and process of the first script, the second script was created to be able to perform climate-based daylight simulations. The second script was made using the framework of the first script, but changing the daylight simulation engine and extracting of the results. A quantitative study was performed, where the results from the first script were compared with results from the other simulation programs. The similarity or variation between the results was then evaluated and used in the evaluation process deciding whether the performance of the script was adequate or not.

Comparison of daylight factor and illuminance
EN 17037:2018 specifies two methods to conduct daylight calculations. Method 1, daylight factor depending on the location, and method 2, illuminance levels using climate data and adequate time step. Knowing that the script simulating daylight factor was validated, simulation results using both method 1 and 2 in EN 17037:2018 were evaluated and compared. The standard states both methods as adequate, making the recommended criteria for daylight factor and illuminance levels equivalent. Checking the compliance of the results using the two scripts was used as a reassurance of the validity of the climate-based daylighting script.

Software and scripts
Rhinoceros, including Grasshopper for visual programming, were chosen because of its 3D-modelling features and flexible user-interface. Ladybug tools plugin for Grasshopper were used, which is an open source plugin that utilizes several validated simulation engines such as Radiance, EnergyPlus, OpenStudio, Therm and OpenFOAM [8]. The two programs used for comparison were IDA ICE and Relux. IDA ICE is a simulation software for indoor climate and energy. Relux is a highperfomance program, used for both artificial and daylight simulations. Both of the programs use Radiance to perform daylight simulations, which uses raytracing calculations [12,13]. Radiance parameters used in the simulations are listed in Table 2.
The first Grasshopper-script used for the comparison simulated the average daylight factor like the two existing programs. The second script simulated climate-based daylight conditions.

Validation of the simulation script for average daylight factor
The simulation results for the first situation with onesided illumination, is presented in Fig. 4. The L-shaped room achieves the same average daylight factor, while the other three rooms show a variety of 0.1 % and 0.2 %. The largest deviation occurs for the Relux simulations of the quadratic room.
The results for the second situation with two-sided illumination, is presented in Fig. 5. Both the quadratic and polygonal room achieve the same average daylight factor, while the other two rooms deviate with 0.1 % and 0.2 %. The largest deviation occurs for the Relux simulations of the rectangular room.
The simulation results for the third situation with two-sided illumination and shading overhang, is presented in Fig. 6. The L-shaped room achieves the same average daylight factor, while the other three rooms have a variety of 0.1 % and 0.2 %. The largest deviation occurs for the Relux simulations of the polygonal room.
The simulation results for the fourth situation with two-sided illumination and shading obstruction, is presented in Fig. 7. Both the L-shaped and rectangular room achieve the same average daylight factor, while the two other rooms vary with 0.1 % and 0.2 %. The largest deviation occurs for the Relux simulations of the polygonal room.
Out of all the simulations, the maximum difference in average daylight factor was 0.2 %, where Relux differed from the two other software. There is no clear trend in the variations based on the room geometry or situation. The results vary between the software, rooms and situations. Variations between simulations run in the same software is also common, because of the random raytracing directions in each simulation. It should be noted that even though the variations between the software are small, Relux has the largest variation in each situation. The results from the Relux simulations are both lower and higher than the two other software, making it difficult to find a pattern if Relux is more or less conservative than IDA ICE and Rhinoceros/Grasshopper.

Test-cases placed in Oslo and Kiruna
After validating the script for calculating daylight factor, the climate-based daylight script was applied to the same test-cases used for validation. Simulations using Rhinoceros/Grasshopper where performed for two locations with different latitude, Oslo (59.90°) in Norway and Kiruna (67.82°) in Sweden. The results are presented in Fig. 8 and Fig. 9. The climate-based daylight conditions were performed and evaluated according to EN 17037:2018, method 1, which has recommended minimum and target sDA values. In addition, the daylight factor was evaluated according to method 2 in EN 17037:2018 for comparison.
The results illustrate a difference in the daylight conditions for the two locations. The results are overall lower for Kiruna than Oslo, being located further north. The difference in area achieving target illuminance of 300 lux for the two locations, varies from 3 % to 21 %, depending on the room geometry situation and orientation.
13 of the 16 test-cases located in Oslo achieve the target illuminance criteria for climate-based daylight conditions, while only 11 of the test-cases achieve the criteria when located in Kiruna. 8 of the 16 test-cases achieve the criteria for daylight factor. Fig. 9. shows that 13 of the 16 test-cases achieve the minimum illuminance criteria, for both locations. 10 of the 16 test-cases achieve the criteria for daylight factor. This indicates that there is a high probability of having achieved the minimum illuminance criterion when the target illuminance criterion is fulfilled, but not reversed. Thus, it is easier to achieve the minimum illuminance than the target illuminance, for higher latitudes. This correlates with the knowledge of the sun path having a lower sky height for higher latitudes, bringing light further into the room.
The results show higher difficulty reaching the criteria for method 2, using daylight factor. The difference between the results for the criteria for target daylight factor and target illuminance, is in average 21 % lower for Oslo. The greatest difference is 37.5 %. The results for Kiruna cannot be compared to the results for the target daylight factor, as these are location and latitude specific for Oslo.

Construction project: Bjørvika School
The climate-based daylight script was tested in an actual construction project of a school, located at Bjørvika in Oslo. The 3D-model was built using the Grasshopper-Archicad live connection to import surface and zone geometry directly from the architectural model to the Rhino interface. The generated model and surrounding buildings are illustrated in Fig. 10. Further, the surfaces were divided and given correct input data. Reflection factors and glass-transmission is stated previously in the paper. Wall thickness was set to 500 mm.
Simulations of sDA and daylight factor for three chosen rooms on the ground floor, with different orientations were conducted. The rooms were given 3.4 m high windows, placed in the whole façade width. The results were evaluated according to the recommended values for Oslo in EN 17037:2018, using the methods for both sDA and daylight factor [5]. Due to simulation time the Radiance parameters were adjusted down, resulting in the ambient bounces (ab) to be 5. Fig. 10. Illustration of Bjørvika School with surrounding buildings, located in Oslo. The school is designed by L2 Architects, and the project owner is Undervisningsbygg.    11. The percentagenumber on each figure represents the area of the plane, which receives illuminance levels above a minimum and target value, according to EN 17037:2018, for more than 50 % of the annual daylight hours in Oslo(sDA). The rooms to the left illustrate the area that achieves more than 100 lux within a year, while the room to the right illustrates the area that achieves more than 300 lux within a year. The legend illustrates DLA, which indicates the percentage of hours above the threshold in each measure point. One of the three simulated rooms achieve a DF of 0.8 %, while two of the rooms achieve a minimum illuminance of 100 lux in more than 95 % of the area. One of the three rooms achieve more than the target daylight factor of 2.4 % and target illuminance of 300 lux for more than 50 % of the area. This implies that only one of the simulated rooms, achieves adequate daylight availability according to EN 17037:2018. The greatest threshold difference is room 1 where DF 0.8% is 13.5% higher than sDA 100 lux and room 2 where DF 0.8% is 6.5% lower than sDA 100 lux.

Validation of script
The results presented in chapter 3, show a difference up to 0.2 % for the average daylight factor between the different software. Relux had the largest variations in results compared to IDA ICE and Rhinoceros/Grasshopper. In addition, the results show a higher compliance in the results between IDA ICE and Rhinoceros/Grasshopper. This means that even between two validated simulation software, such as IDA ICE and Relux, variations in results occur. Based on this observation, and the knowledge of variations in daylight simulations that also occur within a software, the difference in results is evaluated to be acceptable.

Application of climate-based script
When applying the climate-based daylighting script to the test-cases, the results showed a realistic difference for simulations performed at different locations, but a deviation in results evaluating the different criteria. To investigate this further, the script was tested in an ongoing project. The application of the script for climate-based daylight simulations to an actual building design, proved to be more challenging than anticipated. Bjørvika School has a complex building design as well as shading neighboring buildings. Generating the Rhinoceros-model directly from the ArchiCad-model is an advantage in work-flow, but created several unexpected pitfalls. Leaks in the model were discovered, leading to unrealistic amounts of daylight in the rooms. Adjusting and testing of the model and simulations were done, in order to obtain realistic results. Simulating both daylight autonomy and daylight factor, revealed a better compliance between the results for daylight autonomy and daylight factor for Bjørvika School, than for the test cases. The reason for this deviation could not be pinpointed and has to be investigated further. There can be several reasons for the deviating results. Uncertainties regarding the collection of illuminance levels for the daylight hours used in the simulations, is one of them. As seen in the visualization of the results in Fig. 11, the calculation grid has some uneven calculation areas, depending on the room geometry. These differences may also affect the calculation results. Another reason may be how the Radiance parameters and raytracing handle the two calculation methods.
The findings from the simulations performed in this paper, suggest that it is easier to achieve the criterion for daylight autonomy than the supposedly equivalent criterion for daylight factor in EN 17037:2018. This is surprising, since the criteria are presented as equivalent. This observation indicates that the script needs further testing.

Conclusion
Being able to perform climate-based daylight simulations, including real climatic data, increases the ability to provide a close to realistic approach to daylight performance.
The chosen approach included the creation of two scripts, one for simulating daylight factor and one for daylight autonomy. The first script was validated by comparing simulations results of average daylight factor, with two other validated simulation programs IDA ICE and Relux. The results for the 16 test-cases showed a maximum 0.2 % difference in average daylight factor between the programs, thus evaluated acceptable.
The results in daylight autonomy showed a realistic difference for simulations performed at different locations. A large deviation between the performance of the two equivalent criteria (daylight autonomy and daylight factor) from EN 17037:2018 was discovered for the test-cases. When applied to an actual construction project, the differences were smaller. The reasons for this finding are uncertain and need to be investigated further.
The script developed for climate-based daylight simulations shows too extensive deviations for it to be applied in construction projects as the only documentation of daylight availability, in its current state. Further testing and investigation of the climatebased daylight script has to be done, in order to perform correct evaluations of realistic daylighting conditions.