Feasibility study of the UX indoor framework in two public office buildings: A standardized approach to scoring indoor user experience with qualitative and quantitative techniques

. A human-centered strategy is crucial for effective energy retrofits, as the subjective experience of occupants directly affects the energy performance of buildings. To address this, we introduce UX indoor , a standardized scoring framework for assessing the User eXperience (UX) of Indoor Environmental Quality (IEQ). UX indoor is a key component of the PRIME Energy-Indoor project, which aims to guide integrated building energy retrofits based on real energy usage and IEQ data. The framework utilizes questionnaires, handheld instruments, and IoT environmental sensors in a unitary assessment framework. This feasibility study presents the results of the pilot-testing of the framework in two public buildings in Thessaloniki, Greece over a six-month period from September 2022 to February 2023. The study discusses the advantages of the proposed system as well as its limitations.


Introduction
We stand at a crucial juncture where buildings and cities must reduce energy consumption and carbon emissions to combat the effects of climate change.Buildings globally account for one third of energy consumption and greenhouse gas emissions [1].The EU has prioritized the decarbonization of its aging building stock, through the Renovation Wave Strategy which aims to double renovation rates over the next decade, starting with public buildings.
While the emphasis of building retrofitting is on energy use and carbon emissions, its crucial to consider the human occupant as well.The anthropocentric approach to energy retrofits recognizes the complex relationship between energy consumption, Indoor Environmental Quality (IEQ) and human behavior [2][3][4][5].IEQ can be broken down to four basic components: (i) thermal comfort, (ii) visual comfort, (iii) acoustic comfort and (iv) indoor air quality [2].
The interaction between the different components of IEQ can have a significant impact on how a building is operated by its occupants and ultimately on how much energy it consumes.For example, noisy or polluted outdoor environments may discourage natural ventilation [6,7].Frequent glare issues may force occupants to permanently deploy indoor shading and use artificial lighting instead [8].Poor HVAC design may be the source of heating or cooling discomfort and even contribute to the sick building syndrome [9].These are some design issues that can remain unnoticed until a building is retrofitted and occupied.
Researchers and designers increasingly recognize that while pursuing energy efficiency, indoor environmental quality (IEQ) should not be compromised, given its ties to human health, well-being, and productivity [10].The notion of a "standardized" human occupant with constant preferences, perceptions and needs was challenged by the sudden changes in building use caused by the COVID-19 pandemic [11].The idea of adaptable, high quality IEQ is now considered as an integral part of energy-efficient design and has been embedded in several sustainability certification systems, such as LEED, BREEAM, DGNB, LBC and WELL [2].
However, IEQ evaluation and its role in building energy retrofit is still a topic under research [3,12].For example, it has been found that IEQ can account for as little as 10% of the total score in the aforementioned certification systems [2].A 2013 review [13] found that there is a lack of consensus on data acquisition protocols and interpretation and scoring of IEQ results.For the above reasons, a relatively recent review [12] has argued for a standardized IEQ evaluation procedure that is adaptable and combines both qualitative and quantitative methods.
Here, we present the results of a feasibility study of a standardized framework of IEQ scoring that is applied to two public office buildings in Thessaloniki, Greece.The framework, called UX indoor , emphasizes the subjective User eXperience (UX) of IEQ by integrating occupant surveys, point-in-time measurements and longitudinal measurements using IoT sensors under a unitary scoring system.
Our aim is to develop a standardized and quickly reproducible framework that can assist the decision-E3S Web of Conferences 436, 01003 (2023) https://doi.org/10.1051/e3sconf/202343601003ICED2023 making processes when comparing the IEQ performance of multiple buildings with a view to energy retrofitting.While the study does not improve data collection and analysis methods it demonstrates that a simplified framework of IEQ assessment combining qualitative and quantitative techniques can be developed.Nevertheless, the pilot-testing of the framework highlights practical limitations and gaps of knowledge that need to be addressed for UX indoor to mature further (see Sections 5 and 6).
This feasibility study is part of the ongoing "PRIME Energy-Indoor" project, which aims to prioritize, monitor, and validate energy retrofits using real energy consumption and IEQ data.The UX indoor score combined with normalized energy consumption values, will eventually be used to determine which buildings, within a large building portfolio, should be prioritized for energy retrofits by the PRIME system.

Methodology
The UX indoor system combines occupant surveys, point-intime measurements using handheld instruments, and longitudinal measurements with IoT devices to assess IEQ.Fig. 1. shows how these techniques are incorporated into the UX indoor scoring system.Each of the four IEQ parameters receives a separate score, which is then used to calculate a final score for the building.All scores are normalized on the 1-5 scale (1: poor performance, 5: excellent performance) (Tables 1 and 2).Subjective IEQ experience: The occupant survey is conducted using an online semi-structured questionnaire, designed primarily for long-term building users.The questionnaire focuses on tertiary buildings and consists of sections related to Thermal Comfort, Air Quality, Lighting, Noise and Personal questions.Each section except the last is further split into three sub-sections: "Satisfaction", "Control" and "Frequent Issues" containing structured questions (yes/no and 5-point Likert scale).This is the base "template" of 21 core questions that can be adjusted for different use cases.The questionnaire is digitally distributed to long-term building users through the appropriate secretary or building manager.Data collection is anonymous and complies with the EU's GDPR through a publicly available privacy notice.User responses are then reclassified using a simple scoring scheme (Table 1).

Spatial distribution of IEQ:
In-situ point-in-time measurements are conducted during normal operating hours using handheld instruments, including a precision psychrometer for measuring relative humidity (RH) and air temperature (Tair), and a precision photometer for measuring illuminance.Ideally, these measurements should be conducted at least twice, once during the hot and once during the cold period but this is not always feasible due to practical constraints.
The advantage of point-in-time measurements is that they can be used to map the distribution of examined environmental parameters in space, something that longitudinal measurements cannot do unless a very large number of data loggers is used.These measurements are part of a building envelope and HVAC inspection procedure within the PRIME Energy-Indoor system.
The measurements are converted to distribution maps through spatial interpolation using the Inverse Distance Weighted (IDW) algorithm within the boundaries of each occupied space.The Predicted Mean Vote (PMV) thermal comfort index is calculated using Tair and RH, assuming a homogenous radiant environment (MRT equal to Tair) and by assigning seasonal clothing values (0.5 for summer, 0.8 for winter) and a fixed metabolic activity (desk work, 1.2 Met).Thermal comfort and light level scores are then calculated as an expression of the percentage of occupied floor area that is within acceptable thresholds (Table 2).
Temporal distribution of IEQ: Longitudinal measurements for the two case study buildings are conducted for a six-month period from September 2022 to February 2023 to observe the indoor climate for both hot and cold periods.We utilize an IoT network of wireless data loggers (DeltaOHM HD35) which can radio transmit data to data collection base points placed strategically within the buildings.The data is then sent over the internet to a cloud-based platform at a step of 15'.Fifteen data loggers were used in total, consisting of the following types: • Five Tair, RH, and CO2 level data loggers • Eight Tair, RH, and Illuminance data loggers • Two Tair and RH data loggers suitable for outdoors.
The technical specifications of the data logger sensors are presented in Table 3.In addition, six HOBO human occupancy and light use data loggers are used.These allowed us to verify building occupancy hours, which is essential to correctly filter time-series data.Finally, a class 1 IEC61672 certified sound level meter spectrum analyzer is used to measure equivalent continuous sound pressure level (Leq) outside and inside the examined buildings and for different conditions (with/without visitors, with/without operating HVAC).
An important part of the scoring system is to define acceptable thresholds for the IEQ components (Table 2).For thermal comfort we use the ±0.5PMV threshold specified by ASHRAE 55 [14] and ISO 7730 [15] standards.It's worth noting that ISO standards specify a more laxed ±0.7PMV threshold for existing buildings that might be used as an alternative.For visual comfort we use the 300 -2000 Lux thresholds suggested for the calculation of Useful Daylight Illuminance [16].The lower limit has been an industry standard for offices, while the upper limit is associated with a higher probability of glare [16].
For air quality we use the "1000ppm" rule which was mentioned in older ASHRAE standards.Although the rule has been removed from ASHRAE 62.1 and its usefulness is disputed [20], it is still widely used [24].Furthermore, recent studies have indicated that at around 1000ppm CO2 may begin to have an influence on cognitive performance [18,19,25].For noise we use the threshold of 45dB(A) which corresponds to the upper level of noise in an airconditioned open office [21][22][23].
The collected data is then post-processed to derive a score.Firstly, sub-hourly measurements are down sampled to hourly measurements, averaging observations from different sensors.Next, the timeseries are filtered to keep only the days and hours of building occupancy.PMV is calculated using the same assumptions as above.A basic threshold analysis is conducted for air temperature, relative humidity, CO2, illuminance, and PMV, and the percentage of occupied hours that falls within acceptable thresholds is calculated.Finally, the percentages of occupied hours within thresholds are used to derive a score (Table 2).
A final UX indoor score is then calculated from calculating the weighted average of scores from the survey, longitudinal measurements and point-in-time measurements.The exact weights are determined by important findings of the pilot study which are described in Section 5.

The case study buildings
The proposed methodology was applied to two case study buildings located in Thessaloniki, Greece (Fig. 2).Both buildings are public, housing administrative and social welfare services.A reason for selecting public office buildings was the recent nationwide "Electra" subsidy programme for public building retrofits provided by the Greek government.
Building A was the ex-town hall of the Municipality of Triandria, a four-story building erected in 1977 with a total floorspace of 1018m², of which 826m² are conditioned.Building A's energy certificate categorizes it in the "E" class due to its outdated gas boiler, single-pane windows, no insulation, and old split units for air conditioning.The ground floor is used as a Citizen Service Center, while the other floors are offices.The last floor has a large hall for meetings and ceremonies.The building was chosen for its age and glaring energy performance issues that render it a good candidate for retrofitting.
Building B is the Social Welfare Directorate of Thessaloniki, a six-story building erected in 2006, with a total floorspace of 1998m², of which 1574m² are conditioned.Building B's energy certificate places it in the "C" class, below the current baseline for new buildings.The building houses, among other offices, a social pharmacy and a social clinic, which include electricity-consuming devices such as fridges and medical equipment.It was selected because it belongs to a newer generation of public buildings that were erected before the current building energy code was enacted.Examining Building B could reveal IEQ issues of more recently built public buildings.
A measurement campaign was conducted for six months from 09/2022 to 02/2023 for both buildings.Five IoT data loggers and one outdoor logger were installed on building A, while eight IoT data loggers and one outdoor logger were installed on building B. Device location was determined by on several factors such as radio signal strength, safety, and the need to correctly measure the examined parameters.For example, data loggers with integrated light meters were placed horizontally on desks and away from direct sunlight.Data loggers with CO2 meters were placed on walls at a height of 1.5m and away from locations where human breath or plants could directly affect readings.
The buildings were inspected on different days with handheld instruments.Illuminance measurements were conducted at the height of real or imaginary working planes (0.75m above floor) with both lights on and off, and local shading was left intact as found.Air temperature and relative humidity were measured with the psychrometer.The noise meter mounted on a tripod was used to record indoor noise levels (Leq, Lmax and Lmin) with several 5' measurements at different occupied rooms and during periods without rainfall and non-typical noise sources (e.g.passing aircraft or sirens).
Point-in-time measurements conducted in February found that the second floor is overheated, as occupants adjusted the thermostat to a higher setting.Informal communication revealed that the heating system was slow to heat up during winter morning and occupants preferred leaving the thermostat as is.The building's side entrance is the coldest part by far due to being almost always open to the outside.However, as it is unoccupied it is not accounted during scoring calculations.Point-in-time measurements rated thermal comfort highly (5/5) as the occupied spaces were within acceptable PMV thresholds.Visual comfort (Fig. 4) was rated by survey respondents highly (4/5) although significant glare issues (1/5) and lack of daylight control (2/5) were reported.Survey results seem to be in accordance with point-intime measurements (4/5), although measured daylighting levels are lower than expected as we chose not to retract any shading devices during observations.Instead, longitudinal measurements gave a strikingly poorer rating (1/5) since 67% of occupied hours fell below the 300-lux threshold.We discovered a similar trend in Building B which led us to provide an explanation and change the way scores are calculated, as discussed in Section 5.
Air quality (Fig. 5) was rated highly (4/5) although respondents complained about dust/pollen (2/5) and poor outdoor air quality (1/5).Again, we notice here a striking difference with the poor rating of longitudinal measurements (1/5), as 46% of occupied time is above the maximum CO2 threshold that indicates poor ventilation.
Finally, the acoustic environment (Fig. 6) was poorly rated (2/5) with respondents complaining about both indoor and outdoor sources of noise.Occupant responses are in accordance with findings from noise measurements that revealed noise levels exceeding the maximum threshold of 45db(A) for 100% of occupied time, with a mean building Leq = 58.0db(A).

Building B results
Results for Building B from the Social Welfare Directorate of Thessaloniki are summarized below in text as the scorecard format was presented extensively for Building A. Building B received an overall score of 3/5 in the questionnaire, with longitudinal and point-in-time measurements scoring 4/5 and 2/5, and 3/5, respectively.
Thermal comfort satisfaction was rated highly (4/5), but respondents reported slight dissatisfaction with cold drafts and lack of thermal environment uniformity (3/5).Longitudinal measurements showed that thermal comfort conditions occurred for over 70% of occupied hours (4/5), but summer overheating was a problem, accounting for 22% of occupied hours.Point-in-time thermal comfort measurements scored 4/5, with 75% of occupied floor area within acceptable limits.Noise dissatisfaction was reported (3/5), indicating both indoor and outdoor sources.Longitudinal measurements revealed poor acoustic performance (mean Leq = 50dBA, 2/5).
Air quality satisfaction was high (4/5), verified by CO2 longitudinal measurements that never exceeded 1000ppm (5/5).However, dissatisfaction with air quality specifically from mechanical ventilation (2/5) and slight dissatisfaction with dust and odors (3/5) were reported.Respondents mentioned large operable windows as a plus, but low outdoor air quality and external noise discouraged natural ventilation.
Respondent satisfaction with artificial and natural light levels was equally high (4/5), despite slight dissatisfaction with glare and lack of uniform light levels (3/5).Point-in-time measurements scored 3/5, with 74% and 73% of occupied floor space within acceptable limits.Longitudinal measurements showed 94% of occupied time falling below the 300 Lux threshold, possibly due to obstructed devices and some users being satisfied with lower light levels (personal communication).

Discussion and Final Score
Table 4. shows a summary of the scores for the two buildings.Longitudinal measurements receive the lowest scores and point-in-time measurements the highest.This discrepancy is more apparent for visual comfort where the latter are closer to survey results.The issue here is complex but easily explained: Surveys reveal the subjective user experience and satisfaction.Building users may be satisfied or dissatisfied with different levels of thermal and visual comfort or may under-or overreport air quality and noise levels according to personal assumptions.
Longitudinal measurement results are not infallible either.Data logger positioning is critical for correct light level measurement but even slight re-arrangements of the workplace and changes in seasonal sun trajectory may induce significant errors due to local shade or exposure to sun rays.Even in ideal measurement conditions, building users may simply choose to keep shading devices deployed with no apparent glare problems, switch on only half of the lights available or change the thermostat settings to something different than what is considered optimal.
Hence, longitudinal measurements reveal how a building operates but not how it can perform under ideal conditions.Finally, point-in-time measurements allow us to take temperature and lighting measurements that can reflect ideal conditions more accurately, if we choose to temporarily operate thermostat, lighting and shading controls.In this study we chose to only operate light switches and leave the rest intact.The above findings are important and can help increasing the robustness of the UX indoor workflow, by adapting the measurement protocol and revising scores and thresholds.Here, we suggest the use of a weighted average to calculate a score for each IEQ parameter and finally for each building.Weights are split evenly for thermal comfort, air quality and noise while for visual comfort longitudinal measurements only account for 10% of total score (Table 5).Table 6 shows the final scores, with both buildings receiving 3/5 but for different reasons.During this feasibility study we were able to detect several methodological limitations that need to be overcome before the proposed framework matures.These are concisely presented below: Lack of scientific robustness of thresholds: While we utilized commonly acceptable thresholds, we acknowledge that there is a need for vigorous research that should focus on establishing reliable analysis thresholds.These should ideally change according to building type, occupant and climatic characteristics.In our work we relied on CO2 concentrations to indirectly estimate air quality.Ideally the concentrations of different pollutants should be tested (PM2.5, PM10, VOCs, CO, Radon, NOx etc.) but that would further increase the cost of our methodology (see next paragraph).
Dedication of resources: While we strived for a costefficient methodology, it still requires considerable investment of time, money and effort.Currently there is no readily available solution that is both "low-cost" and accurate enough for IEQ studies, especially when considering noise levels and CO2 concentration.Both the point-in-time measurements and the IoT device network setup required approximately 2 hours per building.These issues pose a significant challenge when considering a wider application of the framework to multiple buildings simultaneously.
IoT limitations: IoT devices add an additional layer of complexity, as signal transmission range limits data logger placement.While longitudinal measurements of illuminance sounds good in theory, in practice it offers very little.There is no way to differentiate between daylighting or artificial lighting.Data loggers need to be placed on horizontal surfaces, usually desks, which can easily become cluttered.Occupants may also simply choose not to use all lighting available, which may be misinterpreted as insufficient lighting.
Quality vs quantity in data collection: Digital questionnaires are the cheapest solution but provide only qualitative data.The small number of occupant population in some buildings may require near-100% return rate for statistically significant results, which not always feasible.Point-in-time measurements offer the greatest control over the measurement process; however, data collection is limited to a specific time window during the building inspection.

Conclusions
The proposed UX indoor scoring framework constitutes a useful tool for assessing IEQ by combining occupant surveys, point-in-time measurements using handheld instruments, and longitudinal measurements with IoT devices.The feasibility study conducted in two public buildings in Thessaloniki, Greece, demonstrated the ability of UX indoor to provide quick insights on IEQ in a standardized way.This allows the comparison of IEQ across different buildings of the same type which can inform strategic decision-making processes related to energy retrofits.
The study also highlighted the limitations of the tested methodological framework, which are: (i) the lack of a more robust IEQ threshold values, (ii) the required capital, time and effort that currently prevents a wider application of the method to multiple buildings simultaneously, (iii) the additional complexities IoT devices bring to the table versus the relative usefulness of certain gathered data (illuminance) and (iv) the balance between quantity and quality of data that needs to be established in order for the framework to mature.The proposed scoring system is a significant component of the ongoing "PRIME Energy-Indoor" project that aims to develop a platform that can prioritize, monitor, and validate energy retrofits of a large building portfolio using real energy consumption and IEQ data.The findings of this feasibility study will be used to improve the next iterations of the UX indoor framework.
Funding: This research was carried out as part of the project "PRIME Energy-INDOOR (PRIoritise, Manage and Evaluate buildings in use -Energy performance and INDOOR environment)" (Project code: ΚΜΡ6-0075423) under the framework of the Action "Investment Plans of Innovation" of the Operational Program "Central Macedonia 2014 2020", that is co-funded by the European Regional Development Fund and Greece

Fig. 3 .
Fig. 3. Survey results presented in the scorecard format (Total score and thermal comfort results).

Fig. 4 .
Fig. 4. Survey results presented in the scorecard format (Visual comfort results).

Fig. 5 .
Fig. 5. Survey results presented in the scorecard format (Air quality results).

Fig. 6 .
Fig. 6.Survey results presented in the scorecard format (Noise results and occupant comments).

Fig. 7 .
Fig. 7. Longitudinal measurements results presented in the scorecard format (Total score and thermal comfort results).

Fig. 8 .
Fig. 8. Longitudinal measurements results presented in the scorecard format (Visual comfort results).

Fig. 9 .
Fig. 9. Longitudinal measurements results presented in the scorecard format (Air quality results).

Fig. 11 .
Fig. 11.Point in time results presented in the scorecard format (Total score and thermal comfort results).

Fig. 12 .
Fig. 12. Point in time measurements results presented in the scorecard format (Visual comfort results).

Table 1 .
The scoring scheme for different types of questions.

Table 2 .
Scoring for point-in-time measurements, longitudinal measurements and specified thresholds (bottom left).

Table 3 .
Resolution and accuracy of data logger sensors.