Assessing the effect of a classroom IEQ on student satisfaction, engagement and performance

. Inappropriate indoor environmental quality (IEQ) conditions are shown to reduce occupants’ satisfaction, wellbeing and performance. Therefore, it is crucial to provide an excellent classroom IEQ in order to minimize learning loss among students. This study determines the effect of different thermal and indoor air quality (IAQ) conditions in a classroom on students’ satisfaction, study engagement and cognitive performance. Three data collection campaigns were performed in a university classroom. Data collection consisted of (1) continuous IEQ monitoring (i.e., temperature, relative humidity, CO 2 , …), (2) frequent assessments of students’ satisfaction with IEQ and study engagement via here-and-now surveys, and (3) evaluations of students’ memory and processing speed. The IEQ conditions were varied by adapting (1) a room temperature setpoint between 18°C to 26°C and (2) a CO 2 -setpoint between 500 ppm and 1000 ppm. A mixed-effects regression analysis on the collected data showed statistically significant relationships between students’ study engagement and the room temperature trend, air enthalpy, relative humidity and TVOC-concentrations in the classroom. The addition of data on students’ satisfaction with IEQ only improved a minority of model fits. A statistically significant difference in students’ memory was found between test conditions. However, a randomized experimental design is needed to determine the relationship between the classroom IEQ and students’ cognitive performance.


Introduction
Students spend a lot of their time in classroom environments. However, the indoor environmental quality (IEQ) in classrooms is often perceived as unsatisfactory by students and teachers [1], [2]. Unsatisfactory IEQ conditions could have negative effects on occupants' productivity, which leads to learning losses in case of a school setting. The literature review of Brink et al. [3] already indicated that the IEQ of a classroom can effect students' short term academic performance. Furthermore, studies already showed an improvement in underaged students' performance by lowering the indoor temperature [4] or CO2concentration [5] in the classroom.
Existing studies also indicate that the IEQ affects occupants' mental wellbeing, such as work engagement. The study of Feige et al. [6] revealed a positive correlation between office employees' comfort and their work engagement. Furthermore, Deng et al. [7] showed that work engagement is affected by the lighting conditions in a space. Increasing work engagement of employees reduces their risk of burnout, which consequently, reduces the risk of job turnover and staff absenteeism in the company [8]. Likewise, in school environments it is important to optimise students' study engagement to avoid burnout among them. * Corresponding author: quinten.carton@kuleuven.be This study aims to gain insights in the effects of a university classrooms IEQ on adult students' satisfaction, study engagement and cognitive performance.

Materials & Methods
In this study, data was collected during three different experimental campaigns in the same university classroom. Experimental campaigns consisted of simultaneously measuring IEQ-conditions and regularly assessing students' satisfaction and study engagement via a here-and-now surveys. Furthermore, the subtests from the cognitive test COVAT-3 [9] were used to measure short-term memory, long-term memory and processing speed

Case study classroom and monitoring equipment
This study was performed in a university classroom located in Ghent, Belgium. The classroom is equipped with a VAV-based all-air HVAC system. The ventilation flow rate and supply air temperature are demand-controlled based on a room temperature and CO2-setpoint. The all-air system has a maximum air flow rate of 2200 m³/h. The air distribution is designed to be a displacement ventilation system. The classroom has a floor area of 140 m² and a volume of 380 m³. The window to wall and window to floor ratios are 27% and 13%, respectively. The classroom is designed to be occupied by maximum 80 persons. More information on the case study classroom can be found in [10].
During the experimental campaigns, the room temperature and CO2 setpoints were varied in order to expose the participants to different conditions. The room temperature setpoint was changed between 18°C to 26°C. Originally, the classroom used the setpoint of 23°C. Furthermore, two different CO2-setpoints were set, i.e., 500 ppm (assuming maximum ventilation flow rate during occupancy) and 1000 ppm.
The classroom was equipped with multiple sensors and a weather station to measure both indoor and outdoor environmental conditions. Moreover, an IAQ sensor was placed in the middle of the classroom. Furthermore, a thermal comfort meter (Testo 480) was placed in the classroom to monitor the properties of the thermal environment in the classroom. The measured parameters and their accuracies are shown in Table 1

Experimental campaign 1
The first experimental campaign was performed from November 30 th 2021 to January 20 th 2022. In this experimental campaign a group of 39 international students participated. A survey completion schedule was proposed to the participants based on their course schedule in the classroom. Students were mostly a whole day in the classroom for five days a week. The subtests of the COVAT-3 were not performed in this measurement campaign, since the participating students were non-Dutch speaking, while the test was only available in Dutch.

Experimental campaign 2
This experimental campaign was performed during a full academic semester from February 14 th to May 19 th 2022. Two groups of bachelor students, who had regular courses in the classroom, were participants in this experimental campaign. The participants were asked to complete the survey upon entering and leaving the classroom. The survey completion frequency depended on how often the students had courses in the classroom, namely, one to four times per week. Furthermore, two completion moments for the subtests of the COVAT-3 were organized for each group at the end of a course. A total of 65 students participated in this experimental campaign.

Experimental campaign 3
The third experimental campaign consisted of six consecutive study sessions on Tuesday afternoons from 13:30 -17:30 from April 19 th to May 25 th 2022. During these study sessions up to 17 participants could freely use the classroom as a study room. The participants were recruited via a flyer and poster campaign on the university campus. During the study sessions, participants were informed to complete the here-andnow survey via a 30-minute timer. At the end of the first and last study session, the online cognitive test was completed by the participants.

Survey design
The students' satisfaction and study engagement were assessed via repeated here-and-now surveys. The students participating in the first or second experimental campaign were asked to complete the here-and-now survey when they entered the classroom and at the end of a lecture. The participants in the third experimental campaign were asked to complete the survey when they entered the classrooms and when a 30-minute timer gave an alarm.

Survey content
The here-and-now survey started with questioning students unique ID, which was used to follow the students in the analysis. After the data collection, the students' ID was pseudonymised. The survey also questioned students' location in the classroom and how long they were already present in the classroom. The content of the survey changed depending on how long the students had been in the classroom. Detailed questions on students' perception of the thermal environment were only shown when students were in the classroom for longer than 15 minutes. Likewise, students' perception of the IAQ was assessed more in detail when students indicated they were in the classroom for less than 15 minutes. In experimental campaign 3, students' perception of the IAQ was always assessed in order to study the difference between adapted and non-adapted occupants. Furthermore, students' satisfaction with IEQ was always assessed independently on how long the students were present in the room. Moreover, students' study engagement was evaluated when the students indicated that they were in the classroom for over 15 minutes. The survey ended with an optional text box in which the participants could give additional comments regarding the IEQ.
The here-and-now surveys were preceded by a starting survey, in which the students were asked to give their informed consent and relevant demographic information (e.g., age, gender, height, weight). The starting survey was only completed once by the students. The design and distribution of the online surveys was done using the software Qualtrics [11].

Satisfaction assessment
The students' satisfaction with IEQ was assessed via a 5-point scale (1= Dissatisfied, 3= Satisfied, 5= Very Satisfied) [2]. Students were able to give score on their satisfaction with the IAQ, thermal, acoustic, visual and overall IEQ conditions in the classroom at the moment of completing the survey. Scores lower than three indicated dissatisfaction and led to a follow-up question, in which the students could specify why they were dissatisfied with a certain IEQ condition.

Study engagement assessment
Study engagement was assessed by asking the participants to what level they agreed or disagreed with nine statements on study engagement. The students' level of agreement was assessed on a 7-point Likert scale (0= Strongly Disagree, 1= Disagree, 2= Somewhat Disagree, 3= Neutral, 4= Somewhat Agree, 5= Agree, 6= Strongly Agree). The study engagement statements were derived from the shortened Dutch standardised questionnaire Utrecht Work Engagement Scale Student version (UWES-S) [12]. The statements were rephrased in order to fit the repeated measures design of the current study. Instead of asking how students generally felt, they were asked to rate how each statement according to how they currently felt. The nine statements could be divided into three subscales, i.e., vigor, dedication and absorption (see Table 2). Each subscale resembled one of the three dimensions of work engagement as defined by Schaufeli et al. [8]. The statements in the first subscale 'vigor' referred to students' high levels of energy and resilience, and their willingness to study. A high score on vigor indicated that a participant had a lot of energy and resilience, while a low score indicated exhaustion. The second subscale 'dedication' referred to students' involvement in their studies and their experience of pride, enthusiasm, inspiration and challenge. A high score on dedication indicated that a participant experienced its studies as meaningful and that he or she was proud of doing these studies. On the other hand, a low score on dedication showed that the participant is cynical about its studies. The last subscale 'absorption' referred to students' concentration and immersion in their studies. When a high score on absorption was given, it indicated that the participant was happily engrossed in his or her studies and that he or she easily forgot track of time when they were involved in their studies. A low score on absorption indicated that the participant did not feel immersed in his or her studies. Table 2 gives an overview of the nine study engagement statements and the subscale to which they belong. Table 2. Overview and division of study engagement statements

Subscales Statements
Vigor "I feel bursting with energy" "I feel strong and vigorous" "I felt happy when I was studying, following my courses or engaging in group work"

Dedication
"I am enthusiastic about my courses" "I am proud of the courses that I'm taking" "My studies inspire me" Absorption "I feel eager to continue studying, following class or engaging in group work" "I get carried away during my courses" "I am immersed in my courses" The study engagement questions were analysed per subscale (i.e., Vigor, Dedication and Absorption). Therefore, the engagement assessments were aggregated per subscale. The engagement assessment aggregation consisted of two steps. First, the responses on the 7-point scale were translated into a numeric score (0: Strongly Disagree -6: Strongly Agree). Secondly, the mean score of the three statements within one subscale was calculated. This was also done for all nine statements together. As a result, a score per subscale (i.e., vigor, dedication and absorption) and for study engagement as a whole was obtained, which could reach between 0 and 6. Afterwards, the scores per subscale were classified into very low, low, average, high and very high. The cut off points for the classification were based on the norm scores defined in the UWES-Smanual [12].

Memory and processing speed assessment
Memory and processing speed of the students was assessed using subtests of the COVAT-3, an online cognitive test developed by Thomas More [9]. The COVAT-3 is based on the Cattell-Horn-Carroll model (CHC) of intelligence [13]. The test assessed the students' short-and long-term visual memory and their processing speed. The test was only available in Dutch. The participants in the first experimental campaign were non-Dutch speaking, therefore, the test was not done in the first experimental campaign. The groups of students that participated in the second and third experimental campaign completed the COVAT-3 two times. At least four weeks were kept between the two tests in order to minimize learning effects. Furthermore, the online test was at the same time of the day each time (i.e., same hour, day of the week and course) in order to control for effects of circadian rhythm. A different temperature and/or CO2-setpoint was used during the two test moments.

Dataset construction
The collected survey data, including assessments of students' satisfaction and study engagement, was combined with IEQ, outdoor climate and HVAC system-related parameters into one dataset. The responses from the here-and-now survey were merged with the nearest objective measurement in time of the IEQ sensors and weather station. After the data was merged, a data cleaning process was performed. Firstly, survey responses, which lacked an assessment of occupant satisfaction, were deleted from the dataset. As a result, 8 datapoints on a total of 907 were discarded from the dataset. Secondly, the time between the survey response and the environmental measurement was checked. In case the time difference was 15 minutes or more, the values of the environmental measurement data were set to NaN. As a result, only representative environmental conditions were considered in the analysis.
After the data cleaning process additional variables were added to the dataset. The air enthalpy (EA) and running mean outdoor temperature (Trm) were added using the relevant functions in the pythermalcomfort package [14] for Python. Furthermore, the trends for room temperature (TTrend), relative humidity (RHTrend) and CO2 (CO2,Trend) were calculated over 60 minutes and added to the dataset.

Statistical analyses
Statistical analyses were performed in order to investigate the effect on the students' study engagement of (1) the classroom IEQ and (2) students' satisfaction with the IEQ. Furthermore, a possible effect of the classroom IEQ on students' memory and processing speed was explored. All analyses were performed using the software R [15] and a statistical significance level of p< 0.05 was applied.

Effect of IEQ and satisfaction on study engagement
The study had a repeated measures design, thus the same participants assessed their satisfaction and engagement multiple times throughout several experimental campaigns. Therefore, a mixed-effects modelling approach, in which the participants' unique ID was set as a random effect, was performed in order to account for these dependencies. Furthermore, mixed-effects models are, in contrast to a repeated measures ANOVA, capable of tackling missing observations. The dependent variables, in this case the engagement assessments, were ordinal variables. Therefore, ordinal logistic regression models were fitted using the package ordinal [16]. The mixed-effects analysis methodology used in this study was inspired by the approach of Schweiker et al. [17] in which the effect of personality traits on occupants' behaviour and thermal perception was investigated.
Before the start of the regression analysis, all continuous variables (e.g., room temperature, relative humidity, and CO2-concentration) were scaled and centred, in order to account for their differences in mean and standard deviation. As a result, the importance of the factors could be easily determined from the obtained regression coefficients.
The mixed-effects regression analysis started with fitting a null model, which consisted of only the random effect and no fixed effects. Afterwards, a backwards model selection procedure was performed using the Akaike Information Criterion (AIC) as an evaluation metric. AIC balances the model complexity and goodness of fit. A lower AIC value indicates a better model fit. Only the IEQ-related variables were considered in the backwards elimination process. The result of the model selection gave the best model fit with a set of the IEQ variables. The function buildclmm from the R-package buildmer [18] was used to perform an automated model selection. After the model selection process, the students' satisfaction was added as an interaction term to the resulting model. The student satisfaction was a binary variable which was true (satisfaction score ≥ 3) or false (satisfaction score ≤ 3). A likelihood ratio test was performed to determine if there was statistically significant difference between the model with and without satisfaction variable. The addition of a satisfaction variable was done five times to test each of the different satisfaction variables, i.e., thermal, IAQ, acoustic, visual and IEQ satisfaction.
This process was performed for each of the four possible dependent variables, i.e., the aggregated engagement assessments for vigor, dedication, absorption and study engagement as a whole.

Analysis of students' memory and processing speed
The students' memory and processing speed were assessed two times per group of students via subtests of the cognitive test COVAT-3. A different temperature and/or CO2-setpoint were used during each of the tests, in order to determine the effect of the classroom IEQ on the students' memory and processing speed. A pairwise two-tailed t-test was performed to determine if the difference in performance scores of the first and second testing moment was statistically significant. The test was performed on each of the three tests, i.e., short term memory, processing speed and long-term memory. The test results of the participants that did not participate in both test moments were discarded from the dataset. The assumption for normality was checked before the analysis using the Shapiro-Wilk normality test.

Descriptive statistics
A total of 899 complete survey responses and 90 cognitive test results were gathered from 121 different participants over the three experimental campaigns. For the analysis a subset of this dataset is used only consisting of 412 survey responses including an engagement assessment. The cleaned engagement dataset consists of 384 datapoints since 28 datapoints are removed because they lacked a representative IEQ measurement. The descriptive statistics of all predictor variables in the dataset before scaling are shown in Table 3.

Internal consistency check
The internal consistency of the subscales used for the assessment of students' study engagement (UWES-S) is determined using a Cronbach's α test [19]. In order to aggregate the assessment per subscale (i.e., vigor, dedication and absorption), a minimum value for α would be needed be 0.7. The internal consistency of the engagement related questions is found to be sufficient for each of the three subscales, i.e., vigor (α = 0.848), dedication (α = 0.883), and absorption (α = 0.784). For study engagement as a whole, combining al nine statements, a high internal consistency (α = 0.918) is obtained. Therefore, aggregation of the engagement scores is acceptable. Figure 1 gives an overview of the aggregated study engagement assessments. The shown results are the percentage of assessments per category (i.e., very low, low, average, high and very high). The percentages are visualised for each of the three dimensions of engagement (i.e., vigor, dedication and absorption) and for study engagement as a whole. The dimension vigor is most often assessed negatively (34.7% represent very low or low) compared to, dedication (12.9%), absorption (11.9%) and engagement (16.3%). The low assessments for vigor shows that students often felt exhausted and lacked energy throughout the experimental campaign. However, students indicated less often that they were feeling unenthusiastic or disinterested about their studies during the experimental campaigns.

Overview of the study engagement responses
The dimension absorption is assessed most positively with 25.2% of the scores that can be categorised as high or very high. This shows that the students were often happily engrossed in their study work. Only a small number of assessments can be categorised of as very high. The percentage of assessments categorised as very high ranges from 0.5% (engagement) to 2.4% (absorption). For all categories, the majority of assessments could be categorised as average.

Relationship with IEQ-related variables
The results of the backwards elimination process are shown in Table 4. In each of the four fitted models, a statistically significant relationship was found with one or two of the IEQ predictors. The 'vigor' and 'absorption' model both show a negative association with the temperature trend in the classroom. Indicating that when the temperature increased with a rate higher than 0.19 K/h, the probability of a lower score on the dimensions vigor and absorption increased. Moreover, the relative humidity indoors and the air enthalpy show a positive relationship with the vigor and dedication score, respectively. The most occurring predictor in the models is the TVOC-concentration. TVOC shows a positive relationship with the scores on dedication, absorption, and engagement, indicating that a TVOCconcentration higher than 130ppb would increase students' study engagement. A positive effect of TVOCconcentration on study engagement is unexpected. The increase of the TVOC could be correlated with other effects which are not present in the current model. Therefore, further analysis is needed to determine the cause of this positive association.
The R² values of the fitted models can be categorised as low, showing that a significant portion of the variance could not be explained by the predictors used in these models. Nonetheless, the obtained R²-values are in the magnitude of those described for models describing occupants' behaviour (R²fixed = 0.04 -0.15) and thermal perception (R²fixed= 0.02 -0.04) [17]. A better model fit could be obtained by investigating non-linear relationships between the IEQ and study engagement. Furthermore, contextual parameters, such as, students' location in the classroom, time of day and clothing level, could be added as predictors to achieve a better model fit. However, the aim of this study was to investigate how the IEQ and students' satisfaction with the IEQ affect their study engagement.

Effect of students' satisfaction with IEQ
For each of the four fitted models, shown in Table 4, five alternative models are fitted in which the students' satisfaction (thermal, IAQ, acoustic, visual or IEQ satisfaction) is added as an interaction term. In total, 20 new models are fitted. Only two of these models showed a statistically significant difference in model fit compared to the original model. Firstly, the addition of acoustic satisfaction slightly improved the model fit for vigor (R²full = 0.17, R²fixed= 0.03). In this model, only acoustic satisfaction has a statistically significant

Analysis of students' memory and processing speed
In total, 90 COVAT-3 test results were collected during six test sessions in three different groups of students. Two test sessions were performed per group. Table 5 gives an overview of the classroom IEQ conditions during the tests. The used temperature and CO2setpoints are shown together with the mean PMV-value and CO2-concentration during the test and the occupation period of the group before the test (e.g., lecture or study session). The PMV-value was calculated using the environmental parameters measured by the comfort meter and by assuming a METvalue of 1.1 (sedentary activity sitting) and a CLO-value of 1. Table 5 indicates that the difference in measured CO2-concentrations by adjusting the CO2-setpoints are not as expected. In case of group 3, the higher CO2setpoint during the second testing moment did not lead to a higher CO2-concentration in the classroom. This was due to a lower occupancy level during the second test compared to the first. The difference in the thermal environment during the two testing conditions is, however, clearer in all three groups. Table 6 shows the results of the paired t-tests. The results are visualised per group and per type of test (i.e., processing speed, short-and long-term visual memory). The statistical significance (two-tailed p-value) and power are given. A statistical power of 0.8 is often used as a minimum threshold to classify the results as a true effect. Due to the limited sample size in group 3 (n=7), only sufficient statistical power was achieved for the ttest performed on the change in long-term memory. Furthermore, the analysis of the processing speed in group 2 did not achieve sufficient statistical power.
In all cases, except for the short-term memory results of group 3, a statistically significant difference was found between the results of tests 1 and 2. However, the results for processing speed show a standard deviation exceeding the mean difference in scores between test 1 and 2. This is due to an unequal change in scores between the participants. Some participants scored a much higher score on processing speed during test 2, while a majority of the participants obtained largely the same score as during the first test. As a result, we cannot reliably conclude that there is a statistically significant difference in processing speed scores.
In all cases the mean score of the test increased in the second testing moment compared to the first. The increase in scores between both testing moments can be due to the learning effect of redoing the test. This effect was minimized by leaving at least four weeks between both test moments. Although, the change in IEQ conditions is different for each group, we could not determine a distinct difference in increase of the scores. As a result, the relationship between IEQ and students' cognitive performance could not be established with adequate certainty.

Conclusions
The analysis using mixed-effects models showed a statistically significant relationship between study engagement and the following IEQ parameters  ). The addition of students' satisfaction with IEQ only slightly improved the model fit statistically significant in two out of 20 tested models. A larger dataset would be needed in order to strengthen the obtained insights.
The fitted models in this study only explain a low proportion of the variance. Further research could focus on achieving a better model fit by exploring the nonlinear relationships between the IEQ parameters and the engagement assessments, and by adding contextual variables as predictors.
The analysis of the COVAT-3 subtests show a statistically significant difference in students' memory between the two different testing moments. We could, however, not relate the difference in memory and processing speed to the change in IEQ conditions. A different experimental design, in which the learning effect can be removed, is necessary to accurately determine the relationship between the classroom IEQ and students' memory and processing speed.