Simulation based assessment on representativeness of a new performance rating procedure for cold climate air source heat pumps

Cold climate air-to-air heat pumps (CCHP) offer a strong potential for energy use reductions in Canadian homes. Proper selection of the unit is critical in order to take advantage of the improved efficiency and increased heat capacity at low ambient temperatures and ability to modulate to meet a wide range of heating loads. A new performance rating procedure (CSA EXP07) was developed to better represent the seasonal energy efficiency of CCHP systems versus current test procedures that do not always accurately characterize the response to dynamic loads in a colder climate zones, thus resulting in inaccurate equipment rating. To validate the representativeness of the new performance rating procedure and quantify the potential over-or underestimation of energy savings using current performance rating procedures, a CCHP data-driven model is developed and simulated in a code-compliant single-detached Canadian home for different climate regions: Marine, Cold-Humid, Cold-Dry, Very-Cold and Subarctic. These energy models then serve as the basis for comparing the seasonal heating coefficient of performance of a CCHP system, which can be compared to the current and newly proposed performance rating procedure.


Introduction
The built environment accounts for nearly 17% of Canada's greenhouse gas emissions [1], and has a critical role to play in any national strategy to reduce carbon emissions. In recognition of the need for strong action, the Government of Canada has developed a Market Transformation Roadmap, which presents a framework to increase the adoption of high-performing heat pump systems in the Canadian built environment [2].
The magnitude of emission reductions is in part closely linked to the performance of the building space conditioning equipment. Traditional air-to-air heat pump single-stage systems often experience a significant degradation in heating capacity at colder ambient temperatures while cycling frequently at milder conditions, reducing their energy savings potential in the Canadian climate. Recent advances have seen the introduction of Cold Climate Air to Air Heat Pumps (CCHPs), which combine variable capacity compressors with larger outdoor heat exchangers and other cycle improvements [3] to greatly improve low temperature heating capacity, while efficiently modulating to meet building loads in milder conditions. However, despite these technological advances, these higher performing systems still represent a relatively small share of the Canadian market [4].
Appropriate labelling programs can support a greater adoption of CCHPs by better highlighting their performance benefits. However, current seasonal performance ratings in North America are typically provided only for a single climate region [5]. This climate region is not representative of most Canadian climates zones, making it difficult to appropriately estimate the performance of a system in a Canadian home. While conversion factors do exist to better estimate performance in the Canadian climate, these estimations are often unable to fully capture the unique features of newer CCHP units.
In response to the need for a more representative estimate of seasonal performance, the Canadian Standards Association (CSA) is developing a new testing and rating procedure to better capture performance of variable capacity heat pump systems. This new procedure, CSA EXP07 [6], proposes a load-based test that aim to more accurately characterize the performance of variable capacity heat pumps under dynamic load conditions.
To validate the representativeness of the new load-based test procedure, ideally heat pumps would be field trialled and the measured seasonal coefficient of performance compared with that predicted by the CSA EXP07. While field trials are ongoing, to provide a preliminary assessment, this paper will use a simulation-based approach to investigate the representativeness of the new CSA EXP07 performance rating procedure during heating operations. First, an energy model is developed and used as a base to estimate the seasonal coefficient of performance (SCOP) for a selection of market available CCHP systems in each of the climate zones defined in CSA EXP07. SCOP values determined via simulations are then compared to those calculated using current and proposed procedures. This paper will also review current methodologies used to simulate heat pump systems and examine how they compare to the method used in this paper.

AHRI Standard 210/240
AHRI Standard 210/240 is widely used in North America to evaluate the heating performance of heat pumps. The procedure to rate the performance of variable-speed systems consists of evaluating their performance at a predefined number of operating points at varying outdoor temperature and compressor speeds.
The principal characteristic that differs from the CSA EXP07 procedure is that it is not a load-based standard; thus, all the measurements are done under steady-state operation and do not capture every part-load condition nor other aspects like defrost. To capture the unit performance at temperatures below -8.3°C (17°F), an extrapolation is used from the two standard points -8.3°C (17°F) & 8.3°C (47°F). No cut-off temperature is also assumed potentially resulting in the rating procedure taking into account much more efficient heat pump operation than in reality.
The performance rating can be calculated for multiple regions, but manufacturers are required to publish the performance only for Region IV, which is not representative of many Canadian regions.

CSA EXP07
CSA EXP07 is a dynamic load-based and climatespecific testing and rating procedures for heat pumps and air conditioners. The proposed procedure aims to better capture defrost and part-load operation. The key element of this procedure is that the different systems are tested under manufacturer-supplied controls, allowing for a better estimation of in-field performance. These tests are done under a calculated building heat load line according to the heat pump rated capacity, where the unit, in a dynamic manner, tries to maintain the set condition as shown in Figure 1. Furthermore, to fully capture the performance below -8.3°C (17°F), additional test data is collected at colder temperatures. To make use of the additional data points, the performance is published for multiple climate categories: Subarctic, Very cold, Cold/dry, Cold/humid, Marine, Mixed, Hot/humid, and Hot/dry. The first five (5) of these are representative of the Canadian climate.

Simulation
A simulation approach is used to analyse the accuracy of the two procedures. To do so, an energy model was developed using a commonly used software: eQUEST 3.65 [7]. The advantage of this software is its ability to handle a large number of simulations in a low amount of time.

Building modelling
A model of a typical medium-sized bungalow was developed as a base for the different testing.
The model was simplified into a single zone with a total heated floor area of 140 m 2 . The total window-to-wall ratio is equal to 20% for each facade of the building. The building envelope represents new construction housing and is built according to the National Building Code of Canada (NBC) Part 9 [8] requirements for each of the climate zones.
Lighting and receptacles end-uses were calculated from a power density assumption of 5 W/m 2 each as suggested in the City of Vancouver's Energy Modelling Guidelines [9]. The domestic hot water (DHW) consumption is not considered in the current study. In North America, systems used to heat the DHW typically operate independently of other HVAC systems, as is the case in this study. Future work will examine air to water heat pumps that address both space and water heating. The different schedules used in this paper, follow the residential section of the National Energy Code of Canada for Buildings (NECB) [10].
To capture the different climate zones used in CSA EXP07, building performance was examined in five (5) different regions across Canada, using the appropriate CWEC weather file. These locations were selected to cover the major population location and capture different heating degree days. Locations includes Vancouver, BC; Toronto, ON; Montreal, QC; Winnipeg, MB and Yellowknife, NT.

Heat pump modelling
As part of CSA analysis, a series of market-available heat pumps have been tested under controlled conditions according to the CSA EXP07 procedure. To maintain similarity, the same CCHP models used during these tests are examined under this simulation methodology. A total of 18 units were tested under the CSA procedure, 14 of which were selected for this paper as per availability of performance data.
The system modelled consist of an air-to-air heat pump, which represents the most common type of heat pump integration in North America. Air is distributed via a centrally ducted system where the heating coil is within the duct. Since the model use a single zone, the HVAC system is controlled via this very zone. The system aims to maintain the zone temperature at around 21.1°C (70°F) during the day and at around 18.3°C (65°F) at night. Note that unlike many air-to-water heat pump systems, air-to-air heat pumps do not include any sort of storage capacity. The speed at which the heat pump operates is calculated using the part load ratio (PLR) which is calculated using an hourly average of the heat demand over the maximum capacity of the unit.
Defrost is modelled using the default methodology of eQUEST which is an electric resistance melting the frost. While the reverse defrost method is generally more common for air-air units in North America, a sensitivity analysis by the authors comparing these two methods in the simulation showed that the impact on annual energy use was negligible. Defrost is controlled by the elapsed operating time of the compressor whether frost has accumulated or not. However, defrost does not occur above the specified outdoor temperature (4.4°C, 40°F).
To capture the system performance, the typical approach done in eQUEST is via three (3) different curves that are explained in detail in the DOE-2 Refence Manual [11]. These curves include the maximum capacity output at different outdoor temperatures (CAPACITY CURVE), the maximum energy input ratio at different outdoor temperatures (EFFICIENCY CURVE) and the energy input ratio at different part loads (PART-LOAD CURVE). To develop these curves for the different units under study, performance data from manufacturers and from the Northeast Energy Efficiency Partnerships (NEEP) Cold Climate Air Source Heat Pump Product List [12] were used.
CAPACITY CURVE: Represented in eQUEST by a quadratic function, this curve was developed using the capacity at every outdoor temperature published by the manufacturer at a fixed indoor temperature of 19.4°C (67°F) or 21.1°C (70°F). The default DOE-2 capacity curve used to represent single-speed Air-source heat pumps (ASHPs) is compared to some of the curves developed for this paper in Figure 2. EFFICIENCY CURVE: Efficiency curves in eQUEST are represented by a quadratic function of the energy input ratio (EIR). EIR can be calculated using 1/COP, thus using the data published by the manufacturer at the same fixed indoor temperature as for the capacity curve it is possible to generate the efficiency curve of the heat pump in eQUEST. The Figure 3 compares the default DOE-2 efficiency curve used to represent ASHP to some of the curve developed for this paper. PART-LOAD CURVE: In this software the performance of the unit at part load is represented as EIR in function of the PLR. Usually, this curve is often represented by the part load factor in function of the part load ratio. To convert from one to the other the formula below is used: Software limitations do not allow to model these curves as a map using the outdoor temperature as an additional variable. It was decided that developing the performance curve at a fixed outdoor temperature of 8.3°C (47°F) would allow to best capture its performance. Using performance at -8.3°C (17°F) would not have yielded an accurate result as it is expected that close to -8.3°C (17°F) the unit will be working close to maximum compressor speed (full load).
To develop these curves, the NEEP database was used to obtain the part-load performance using the capacity and COP at minimum, rated and maximum compressor speed. From these points, the PLF(PLR) curve is generated and used to generate the cubic function of EIR(PLR).
While some other part-load performance curves for variable-capacity units are available in the literature, these are more suitable to represent a generic CCHP and do not capture the unique performance of all makes and models. For example, the performance curve derived in Filliard et al. [13] is compared with data for four CCHP units compared in this study. While two units display similar performance trends, the other two operate quite differently as shown in Figure 4.

Heat pump sizing
The sizing methodology used by both performance rating procedures makes use of a load line approach. The load line is developed using the rated cooling capacity at 35°C (95°F) and determine the heating demand at each temperature.
As modifying the building energy simulation model to represent a calculated load for each heat pump under test would be challenging, instead, the rated cooling capacity of the different CCHPs are adjusted to match the fixed load line of the model for each climate zone as shown in Figure  5. The resulting rated cooling capacity matching the model load line is close to 1.5 tons (5.3 kW, 18 000 BTU/h). The rated heating capacity of the heat pump is then adjusted by the same ratio. The simulated performance of the different heat pump systems are then compared to a model using electric resistance to meet the heating loads. The different sizing of the electric resistance heat and design temperature for each city is summarized in the Table 1.

Results
To assess the SCOP of each system, simulations were performed in eQUEST 3.65 using the appropriate Bin file.
Results from the Montreal simulation are compiled in Table 2. CSA EXP07's Very Cold representative climate accurately captures the CCHP performance, with results closely matching those obtained via simulation. On the other hand, SCOP estimates for AHRI Region IV do not represent well the performance in Montreal. Even when using the recommended conversion factor of 1.15 [14] to estimate the Region V SCOP from the published Region IV, it still does not reflect the SCOP observed in Montreal.

Sensitivity analysis
Sensitivity analysis were completed to capture the impact of small modifications on the SCOP.
Climate Zone: Different climate zones were examined to capture the accuracy of CSA EXP07 throughout various regions. As expected, the representativeness of the current AHRI procedure tends to decrease for colder regions, primarily because of the poor match of the Region IV climate zone to those regions examined in the simulations.   Figure 7, bin temperatures for Winnipeg are better captured by the Subarctic bin temperatures. Extremely cold regions like Yellowknife also showed a worse representation than other typical Canadian regions. The main explanation is that Yellowknife represents a colder region than the average one represented by the Subarctic region. This results in a lower accuracy when comparing the bin temperature at colder temperature as shown in Figure 8. To capture performance more accurately in extremely cold locations, an additional representative climate zone could be added to CSA EXP07. In comparing the Marine region simulated SCOP with that predicted by the CSA EXP07 and AHRI Region IV and V (Figure 9), the CSA EXP07 demonstrated a greater difference than the AHRI Region IV for some of the units. While the AHRI Region VI would be used to represent Marine climates in Canada, since the data for Region VI is not published and no conversion factor was readily available, the simulation results were only compared to Region IV. Comparing the bin temperature profiles of the test procedure climate regions to that of Vancouver it is seen that AHRI Region IV poorly captures the climate in Vancouver, while Region VI and CSA EXP07 Marine climate zones better represent it ( Figure 10). A possible explanation for the ultimate better representative SCOP using AHRI Region IV is that the different curves used in the simulation were not adjusted to the wet bulb temperature, resulting in potentially inaccurately simulating the performance in humid regions like Vancouver. The CSA EXP07 procedure does include different testing methods to capture the unique humidity characteristic of the Marine region. It is therefore plausible that in reality the SCOPs for Vancouver are in fact better represented by the CSA EXP07 test procedure than by the simulation; however, additional work would be needed to confirm this hypothesis via experimental test benches to develop more detailed performance curves accounting for both the wet bulb and dry bulb temperatures. Overall, SCOP obtained using CSA EXP07 are accurate to an acceptable level throughout the different regions tested, with an overall average difference of 18% and standard deviation of 12%. Of the 14 units evaluated, the CSA EXP07 procedure has a tendency to be more conservative than the current AHRI procedure, which may  help to ensure that heat pumps properly installed in buildings perform at least as well as estimated during testing. Table 3 shows the accuracy of the different regions. System Oversize & Undersize: Both procedures suggest the use of a 1.15 oversize factor on the heat pump cooling capacity to determine the load line to be used for testing. As explained previously, the cooling/heating ratio varies from one CCHP unit to another and to amplify this variation, three (3) factors were imposed to the heating capacity while keeping the cooling capacity the same. The three (3) factors used to multiply the calculated heating capacity are: 0.6, 1 and 1.4. This would represent installing an under and oversized heat pumps for the recommended balance point temperature of the CSA EXP07 load line.
Results show that using a factor of 0.6 considerably alters simulated performance because of an increased use of auxiliary electric heating during colder periods. Oversizing the system with a factor of 1.4 resulted in minimal performance difference vs. the current sizing, primarily because the simulation approach does not capture the impact of increased cycling on system energy use (i.e., operating behaviour of a heat pump turning after an off cycle).
From these results, it is possible to conclude that using the rated cooling capacity to estimate the heating building load line does not have a too large of an impact to render the calculated SCOP inaccurate and thus is an acceptable methodology to be representative. Table 4 shows the different impact when adjusting the heating size factors on the simulated SCOP. Simulated performance curve: The initial simulations presented above were done using manufacturer performance data. To evaluate the impact of selected performance curves on SCOP estimates, four (4) additional methodologies were also compared. 1. DOE-2 Default ASHP Curves: The Department of Energy (DOE) has developed a set of curves used to represent the different characteristics of ASHP, with more information available in the DOE-2 Refence Manual. In typical building energy modelling software, these curves are widely used to model most types of heat pump systems, including CCHP.
2. DOE-2 Variable speed system Curves: DOE also provides curves that can be used to represent variablecapacity units like CCHPs. These curves may often be overlooked as the system modeller must be aware that they exist and be unaware that there is a difference in part load performance in comparison to single stage systems of which the default curves are based upon.
3. CSA EXP07 Data: Published data under the CSA EXP07 test procedure can be used to develop a set of performance curves representing variable-capacity ASHPs. Data being published at part load; the part load performance is then included within the efficiency curve. Thus, the sizing of the unit would need to match the CSA EXP07 calculated load. For the capacity curve, it can be generated using the provided data combined with the unit maximum capacity at each temperature bin. Using all data points at maximum compressor speed from CSA with an interpolation between the maximum capacity of the unit at 8.3°C (47°F) reported by AHRI, it is possible to generate a fairly accurate hypothetical maximum capacity curve, as shown in Figure 11. It should be noted though, that the true operational maximum capacity of the heat pump may be lower at the warmer ambient temperatures, which would need to be considered when modelling these systems with this approach.  Readers may wonder if published data under CSA EXP07 may be used to generate performance curves to be used in simulation software. Although CSA EXP07 data can be used for this purpose, other methodologies offer better accuracy and are easier to use. For highly detailed CCHP modelling, it is recommended to opt for a combination of manufacturer data and data from the NEEP database or experimental test data. For models that do not require an exhaustive accuracy, deviating from the standard DOE-2 curves in favour of performance curves developed using the NEEP database is a worthy alternative, as shown by the noticeable improvement in model accuracy. Additional efforts could serve to develop a new set of generic CCHP curves to replace the existing ones developed by DOE.

Procedure discrepancy/difference
As the performance rating procedures intend to provide an estimate on the expected seasonal energy savings possible with heat pumps, it is seen from the simulated results that there is some discrepancy. The following section describes some of the assumptions made in the performance rating procedures, which can help identify improvements required to generate a more representative result.
Performance below -8.3°C (17°F): Current testing and rating procedures like AHRI use linear extrapolation of the capacity at 8.3°C (47°F) and -8.3°C (17°F) to determine performance outside of this outdoor temperature range. In some instances, this can lead to a higher performance than reality as shown in Figure 12. Additionally, the current AHRI rating procedure does not include the cut-off temperature of the heat pump. Therefore, the procedure could assume a certain heat pump capacity at a given outdoor temperature, when in fact the heat pump would be unable to operate under this condition. The impact of this varies depending on the climate region: In AHRI Region IV, relatively few bin hours occur below -8.3°C (17°F), but in the colder Region V, this number is significantly higher (19% of total operations), potentially leading to larger discrepancies in calculations. Steady-state: AHRI develops its performance rating based on four (4) individual measurement points under steady-state operation (Full, Low, Int & Norm compressor speed). This procedure limits the ability to capture how the heat pump dynamically adapts to a wider variety of partload conditions and neglects the important performance implications of other operating modes such as defrost. Manufacturers may also be able to optimize performance around the three required rating points, to the detriment of overall performance. These limitations may lead to poor estimates of actual seasonal performance when compared to simulated and lab-measured SCOP values.
Bin temperature: The bin temperature used for the published performance (AHRI Region IV) does not fully represent typical Canadian climates. Similarly, the simplified conversions factor used to convert the performance from AHRI region IV to AHRI region V do not seem to appropriately capture system performance in comparison to simulation and CSA tested values. The use a poorly representative bin temperature that is based on milder regions to represent the Canadian climate result in inadequate prediction of performance.

Discussion
Building simulation is a useful tool to estimate a building's energy use and the performance of different spaces conditioning systems. Still, it is only an approximation of reality and it is worth mentioning the limitation of the software to accurately represent real-world performance to validate representativeness. In eQUEST, many simplifications are made to accelerate calculation. For example, the program runs on an hourly time step, such that the impacts of shorter-term behaviour like on/off cycling is not fully captured. While the parasitic losses are captured in the part load performance curves, performance testing is often required to gain a thorough understanding on the degradation of performance to develop these curves. As CCHPs are relatively new to the market, performance curves that do exist do not necessarily represent all units in the market. This part load performance assumption needs to be acknowledged in this analysis and part of the limitation in the simulation-based study.
Some decisions were also made to simplify the analysis or because of limitation by the software. As explained previously, the performance curves derived did not include wet bulb temperatures, as it was assumed not to have a significant impact on the results. However, as highlighted when looking at the SCOP results for Vancouver, the addition of a wet bulb temperature dependence may have provided more accurate estimates of the CCHP performance.

Conclusion
This paper presents a simulation-based assessment of the energy performance of various cold climate heat pumps and compares it to current (AHRI 210/240) and newly proposed (CSA EXP07) procedures. A simplified model of a singlefamily bungalow was developed in eQUEST for different Canadian cities based on the NBC envelope requirement. Using different modelling methodologies, 14 CCHP units were analysed via simulation and the SCOP compared to the results obtained by the two procedures. Results demonstrate that the new CSA EXP07 load-based testing and rating procedure offers a strong potential to more accurately capture the performance of CCHP throughout the different Canadian climate zones by providing additional information on units for colder climate zones that the current procedure does not publish.
The different simulations showed that commonly used heat pump performance curves do not properly capture the unique performance of the studied CCHPs. Developing a general curve for CCHP systems would be helpful to better characterize their performance through energy modelling and estimate their potential energy and cost savings. While the CSA EXP07 performance rating procedure provides additional performance data commonly not published by manufacturers, there is still insufficient data to develop accurate performance curves in building energy models. Combining the information published by AHRI or the NEEP database with CSA EXP07 data, is an option that could be used to develop improved performance curves; however, should be investigated further.
The current study represents an initial analysis regarding the capability of the suggested procedure CSA EXP07 and the impact on labelling a more representative performance for the Canadian climate. Future work would be to explore the HSPF2 [15] developed by AHRI, that will be adopted in 2023 by the United States and compare the results to ongoing field testing.