In-Use Performance of Air-to-Water Heat Pumps: are the Standards robust?

This paper presents an energy performance comparison of air to water heat pump manufacturer quoted values, calculated seasonal performance values from standards and actual in-use values for space heating mode only. Within this, the paper determines whether an in-use factor would be suitable to correct performance predictions where differences exist. The paper also presents an in-use Carnot efficiency. Over five weeks of high-resolution data during winter and shoulder season conditions were collected and analysed for three different systems. The average in-use Carnot efficiency was found to vary largely from system to system (0.19 to 0.42). All heat pumps were found to underperform on average with respect to their standardised energy performance certificate values. The difference between standardised values and in-use values was between 11% and 53% in relative terms. The gap in performance is thought to be due to oversized or under-utilised systems. Standardised methodologies appear unable to capture potential under-performance in these types of systems. An in-use factor appeared to be an inappropriate approach in reflecting this difference. Further investigations are required to incorporate these risks into standards.


Introduction
According to the European Heat Pump Association (EHPA) statistics dashboard, there are over 14 million heat pumps installed in total in Europe and over 1.8 million air-to-water heat pumps (AWHPs) installed for heating purposes only [1]. In Ireland, there are over 36,000 heat pumps installed and, according to 2017 figures, there were around 18,000 AWHPs installed in that year alone. More importantly, the growth rate in year on year sales of heat pumps more generally has been over 11% for the three years leading up to 2017 [2], and was greater than that level in 2019, with Irish growth rates appearing to exceed that of European levels in the same year [1]. The main drawback of using a heat pump is that the primary energy factor for each location can affect the renewable status of the device itself. Nowak et al. reported that seasonal performance factors (SPFs) needed to be around 2.875 at a minimum if heat pumps were to be counted as renewables [3]. The EHPA state 2.53 as a requirement [1], while a value of 2.5 is a requirement in some countries [4]. With this in mind, there is a clear goal for operating AWHPs to exceed these average SPFs in-use. Over the past ten years an extensive library of published AWHP in-use performance studies has emerged in the literature [5]- [10]. The average SPFs reported in these studies is around 2.7 with a large range of reported values of between 1.2 to 4.5 depending on the system boundary and location. Given the clear spread in reported inefficiencies from field trials there is a need to reconcile energy performance between manufacturers quoted 2 Materials and Methods

Buildings and heat pumps
The three heat pumps investigated were installed in three A-rated dwellings. The floor area of these buildings is between 182m 2 and 288m 2 , with each built and occupied from 2015 onwards. Two buildings had heating systems which served underfloor heating (AWHP-1 and AWHP-3) and one building (AWHP-2) had radiators. In this regard, these buildings and their respective heating systems represented typical modern northern EU homes. One building (using AWHP-3) was a deep retrofit. The rated outputs of these heat pumps are 5.2kW (AWHP-1, low temperature applications, 35°C), 5.3kW (AWHP-2, medium temperature applications, 55°C) and 8.5kW (AWHP-3, for temperature applications, 35°C) respectively.

Performance metrics and system boundaries
The theoretical performance of all AWHPs can be calculated in a very simple manner by using the Carnot Co-efficient Of Performance (COP) shown in Equation 1.
Where, is the source temperature (which is the external ambient air temperature for AWHPs) and ∆ − is the difference between source ( ) and sink ( ) temperatures, known as the lift temperature, described in Equation 2.
The sink temperatures we refer to here depends on the system adopted and the set points observed, these can be fixed or weather compensated. Typically, manufacturers report the COP at fixed flow temperatures or for a temperature application (e.g. low, medium, high temperature application), calculated at the test points indicated in EN 14511 [11]. Equation 3 describes the heating capacity, , in Watts, W, according EN 14511-3.
Where, q is the volumetric flow rate in m 3 /s, ρ is the density of water or air in kg/m 3 , C p is the specific heat capacity of the heat transfer medium at constant pressure in J/kg K and − is the difference between inlet and outlet temperatures (which is applicable for waterside or airside calculations), expressed in Kelvin, K, which depends on the system. Equation 4 describes the COP. The COP is defined in EN14511-1 [12] as the ratio of heating capacity to effective power input.
is the heating capacity expressed in Watts and is the effective power input, also expressed in Watts, W. The Carnot COP is a useful benchmark in that it gives an indication of the maximum COP that a system can achieve for given boundary conditions. It also relates well to manufacturers test points. Figure 1 presents the relationship between COP and external air temperature ( ) for the Carnot COP and a linear fit of manufacturers quoted COP with respect to external air temperature for the three AWHPs investigated in this paper. The Carnot COP indicated in Figure 1 refers to two values, 35°C and 55°C respectively. Outside of presenting individual test points according to EN 14511, manufacturers will also quote a Seasonal COP (SCOP) according to EN 14825 [13] if the heat pump is used for space heating and may also quote a seasonal value for hot water performance according the EN 16147 [14]. The SCOP is a weighted COP, which includes different heat pump operational modes as well as backup heating requirements, and reflects the annualised performance of a heat pump at reference climates.

Fig. 1. Scatterplots of external air temperature Vs manufacturers quoted COP and COPCarnot for similar temperatures
The SCOP is often calculated with the assumption that the heat pump is designed for the load it is tested at, which often neglects the use of backup heating. However, most published field trials or studies report a seasonal performance factor (SPF) to reflect annualised performance. The SPF can be quoted at different boundaries. In this study, all SPFs that are calculated refer to the H 4 boundary unless otherwise stated in the text. Equation 5 describes the calculation of SPF at the H 4 boundary.
Where, Q HP is the heat output of the heat pump, is the heat output of backup heaters, is the work of the heat pumps compressor, , is the work of the source pump or fan, ℎ is the work of the back-up heater and , is the work of the distribution pump(s). Of the three heat pumps studied in this paper, two (AWHP-2, AWHP-3) had flow meters which allowed for a water side calculation (see Equation 3). It was assumed that water was present on the condenser side of the AWHP-2 as it was a split (C p = 4200J/kg K). For AWHP-3 it was assumed that glycol was present in the primary loop and so the specific heat of the water was adjusted accordingly (C p = 3442J/kg K). AWHP-1 had no integrated flow meter and so a water side calculation was not feasible. A sensible evaporator air side calculation for heating capacity was used. This was based on an empirically derived relationship between evaporator fan speed and air mass flow rate from in-situ field measurements of fan airflow rates at the dwelling. The sensible heat output only was considered as the temperature difference between the outside air and evaporator surface temperature. Table 1 presents a comparison of the approaches adopted in standards such as EN 14825 and compares them to the SPF boundaries reported in this study. The information conveyed in Table 1 shows how different in-use measurement boundaries (SPF H 1 etc.) can be compared to standardised values. Based on Table 1, SPF H 4 and the EN 15316 standard are at the same boundary, which makes them the most suitable metrics for comparative purposes.

Carnot efficiency and in-use factor
Based on a review of the literature presented in Section 1 it is evident that there often exists a deviation between manufacturers quoted performance and in-use performance. A challenge for the standards is to try and represent this in the form of an in-use factor. The work of SEPEMO proposes that this could be reflected in the form of a Carnot efficiency for the same conditions. Equation 6 describes the Carnot efficiency for space heating only (ԑ C,SPF ). , = 1 (6) Where, SPF H1 is the calculated SPF at the H 1 boundary for heating only and SPF C refers to the Carnot SPF, which is the Carnot COP (from Equation 1) calculated using in-use data. The Carnot efficiency described by SEPEMO was found to vary between 0.3 to 0.5 for small electric heat pumps, and 0.5 to 0.7 for large very efficient electric heat pumps [15]. The efficiency metric proposed in this paper is an adjustment of the Carnot efficiency to reflect in-use performance at a different boundary. Equation 7 describes the proposed in-use Carnot efficiency, which is the Carnot efficiency calculated at the H 4 boundary. Where, SPF H4 is the calculated SPF at the H 4 boundary for heating only and SPF C refers to the Carnot SPF, which is the Carnot COP calculated using in-use data. By including additional energy from components other than the compressor, this efficiency metric will attempt to reflect system performance as a function of the theoretical maximum.

Measurements
Four different sources of data were collected for each AWHP. The first source was data from the manufacturers integrated data logging systems, which gathered data on; operational modes, set-points, fan operational points, temperatures at various points in each system as well as flowrates. The temperature sensors that were used were factory installed sensors and had varying accuracies, most of which had an accuracy of ±0.5°C, both flowmeters were rated an accuracy of ±2% of measured values. The second source of data for this study came from Efergy energy monitors, which were installed at the electrical board of each house. The reported uncertainty for Efergy devices is less than 2% for household applications [16]. Energy data was measured using a current transducer (CT) for the heat pump and the immersion backup heater separately. For AWHP-1 two CTs were used; one was for the outside unit (e.g. compressor and fans) and for the inside unit (e.g. pumps). For AWHP-2 and AWHP-3 the heat pump CT captured all energy associated with the compressor, fans and pumps, as well as auxiliary systems. The third source of data for this study was from internal Netatmo Indoor Air Quality (IAQ) monitors. Each site also had a Netatmo weather station installed outside the dwelling.
A full list of accuracy information on these devices can be found at [17]. The fourth source of data for this study was data taken from the nearest national weather stations and was accessed from Met Éireann's website [18]. Energy measurements were conducted for a continuous period between February 2020 to December 2020. However, system level data was extracted from some systems in small one-week periods (e.g. AWHP-2). At least five weeks of shoulder and winter season data was collected for each system. SPFs were calculated using an hourly averaged dataset that was statistically aggregated from an empirical dataset that sampled at 1-minute intervals. Figure 2 illustrates the performance of each heat pump with respect to external air temperature. Data for AWHP-1 indicated that outside conditions ranged between 1.8°C and 17.5°C. During this period SPFs were between 0.2 and 3.7, and were on average 2.5. Data for AWHP-2 indicated that outside conditions ranged between 1.7°C and 11.5°C. During this period SPFs were between 1.6 and 4.9, and were on average 3.0. Data for AWHP-3 indicated that outside conditions ranged between -2.8°C and 15.1°C. During this period SPFs were between 0.8 and 11.9, and were on average 5.0. The flow temperatures that were observed in each system were between 24.5°C and 32°C for AWHP-1 (mean: 29°C), between 26°C and 44°C for AWHP-2 (mean: 38°C) and between 21°C and 60°C for AWHP-3 (mean: 31°C). The lack of strong relationships between external air temperature and SPF for AWHP-1 and AWHP-2 indicates a potential that these systems are oversized for their application. The building that uses AWHP-1 incorporates two heat pumps into one home. One exhaust air heat pump in a ducted system and the second heat pump is described in this paper. Both heat pumps are attempting to maintain comfortable conditions and it is likely that this is leading to low demand for heat. AWHP-2 has a fixed time schedule, which only allows operation of space heating for a few hours in the morning and the evening. This could explain its underperformance.  Figure 3 shows histograms for the measured in-use Carnot efficiency. Based on this figure we can see that AWHP-1 and AWHP-2 have a distribution of in-use Carnot efficiencies that is at the lower end proposed by SEPEMO. No strong correlations were observed between the in-use Carnot efficiency and either lift and external air temperature variables. The average in-use Carnot efficiency was 0.19 for AWHP-1, 0.33 for AWHP-2, and 0.42 for AWHP-3. It is unlikely this inuse value could be used successfully in reflecting the true performance of systems given the large deviations in values. The application of an in-use factor to reflect true performance would therefore seem difficult to apply, given its crude nature. Energy Assessment Procedure (DEAP) software which uses EN 15316 as its basis for calculation of system SPFs, and were entered by the relevant Building Energy Rating (BER) Assessor for each dwelling. Based on this, what is evident is that there are gaps between the estimated system efficiencies used to represent the performance of AWHPs in compliance tools and in-use efficiency. Differences of between 11% and 53% were observed in relative terms, between EN 15316 and its representative boundary SPF H 4 . (High frequency (>=90%), low frequency <=30%, mid-range is between these two. (High frequency (>=90%), low frequency <=25%, mid-range is between these two. The differences that have been observed could be explained by a number of factors including; 1) differences between standardised climate conditions and actual conditions, 2) incorrect sizing of heat pumps or 3) excess defrosting. Based on the available system level data, defrost mode made up less than 1% of the data, which would suggest that defrosting is not the main cause for the differences observed. Differences between standardised climates and real climates is likely, however, a good spread in external temperature was observed for the case studies which is typical for conditions in Ireland. What would seem more likely in this case is that AWHP-1 and AWHP-2 are either oversized or under-utilised. System level data shown in Figure 4 for AWHP-1 indicates that the variable speed compressor used in this system operates below its lower limit frequency (e.g. 30%) for 44% of the time. This is likely to lead to compressor cycling [8], where cycling can lead to 25% reduction in seasonal efficiency [19] which is in part due to the COP on start-up being 50% lower for some heat pumps [20]. This cycling can effectively remove the benefit of the variable speed compressor. Comparing this to Figure 5, AWHP-3 operates in its mid-range frequency more frequently, with frequencies less than or equal to 25% for only 1% of the time. It should be noted that missing data indicated in Figure 4 and 5 is due to compressor being off (0%) or due to dropouts in data logging systems during the monitoring period. AWHP-2 operates on a fixed time schedule for periods in the morning and evening which may represent the worst times of the day to external conditions. AWHP-3 performs well with respect standardised values. Based on this, it would seem that the standardised methodologies used in EN 15316 are accurate for systems that are appropriately sized. The lack of consideration for oversized systems or systems that may be under-utilised for their buildings is a concern. The lack of consideration for this factor in standardised methodologies could lead to systems underperforming relative to expectations by 53%. As the causes for under-performance are potentially related to heat pump sizing, it may be difficult to reflect this by using an in-use factor or multiplier.

Conclusions
This paper presented the difference between standardised values and in-use performance values for three AWHPs. Large differences in efficiency were observed in relative terms for two of three systems observed, one system performed well compared to standardised values. The work in this paper reveals that existing standardised methodologies are not likely to capture the effects of oversized or under-utilised AWHP systems, which could lead to a performance gap of over 50%. Where systems are appropriately sized standards would appear robust enough to capture system performance accurately. The work indicated above also highlighted that an in-use factor may not be appropriate in reflecting this gap. More work is needed to improve standardised methodologies in considering sizing issues.