A Transfer Active Learning Framework to Predict Thermal Comfort

*
Annamalai Natarajan1 , Emil Laftchiev 2

1 Philips Research, Cambridge, MA, 02141, USA

[email protected]

2 Mitsubishi Electric Research Labs, Cambridge, MA, 02139, USA

[email protected]

A BSTRACT by 74%. The same chilly office worker had a reduced output
(productivity) of 46%. Another study by IJzerman and Semin
Personal thermal comfort is the feeling that individuals have
(IJzerman & Semin, 2009) showed that warmth in the office
about how hot, cold or comfortable they are. Studies have
environment encouraged closeness and friendliness. Thus,
shown that thermal comfort is a key component of human
improving and maintaining thermal comfort in the office envi-
performance in the work place and that personalized thermal
ronment can yield significant benefits in terms of improving
comfort models can be learned from user labeled data that
worker performance. In fact, the improved work performance
is collected from wearable devices and room sensors. These
was estimated to be as much a 12.5% increase in worker wages
personalized thermal comfort models can then be used to opti-
(Hedge et al., 2005).
mize the thermal comfort of room occupants to maximize their
performance. Unfortunately, personalized thermal comfort
models can only be learned after extensive dataset collection 30
and user labeling. This paper addresses this challenge by 28
Skin Temp.

proposing a transfer active learning framework for thermal 26

comfort prediction that reduces the burdensome task of collect-
24
ing large labeled datasets for each new user. The framework
leverages domain knowledge from prior users and an active 22
learning strategy for new users that reduces the necessary size 20
of the labeled dataset. When tested on a real dataset collected 18 User 1 User 3 User 5
User 2 User 4 User 2, 12 days
from five users, this framework achieves a 70% reduction in 1626 28 30 32 34 36 26 28 30 32 34 36
the required size of the labeled dataset as compared to the Room Temperature
fully supervised learning approach. Specifically, the frame- (a) (b)
work achieves a mean error of 0.82±0.05, while the supervised
learning approach achieves a mean error of 0.85±0.04. Figure 1. Scatter plot of skin temperature and room tempera-
ture (a) at the instant in time when five users reported being
1. I NTRODUCTION AND BACKGROUND thermally comfortable (b) at the instant in time for one user
over multiple days who reported being thermally comfortable
Personal thermal comfort is feeling that individuals have about
how hot, cold or comfortable they are. Importantly, thermal There are two principal difficulties when modeling thermal
comfort is a good predictor of human performance. Studies comfort. First, personal thermal comfort varies from one in-
have shown that making office workers comfortable is critical dividual to the next. Often this variation can be explained
to improving worker productivity and improving the office en- by gender, ethnicity, location, and season. Second, personal
vironment. In one study, Hedge et. al. (Hedge, Wafa, & Anshu, thermal comfort varies within the individual because of their
2005) showed that reducing temperatures such that the average physical state, including conditions such as tiredness and sick-
female office worker felt chilly increased her typing mistakes ness. Because of these difficulties, state of the art methods of
thermal comfort estimation provide only coarse estimation of
* Research performed while at Mitsubishi Electric Research Labs. thermal comfort for large groups of individuals. Additionally,
Annamalai Natarajan et al. This is an open-access article distributed under
the terms of the Creative Commons Attribution 3.0 United States License,
these models rely on a-priori assumptions about the composi-
which permits unrestricted use, distribution, and reproduction in any medium, tion of the group of occupants which increases the model error
provided the original author and source are credited. rates in today’s diverse workforce (Belluck, 2015).

International Journal of Prognostics and Health Management, ISSN 2153-2648, 2019 003 1
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

The variation in personal thermal comfort is illustrated in Fig- short. However, a study in the steam-based setting remains to
ure 1. In subplot (a) five users’ skin temperature and room be performed at a future time. In this work, we evaluate the
temperature are shown, color coded, at instants of feedback. feasibility of using active learning techniques to minimize the
Here, feedback refers to thermal comfort ratings as provided labeling effort in thermal comfort prediction using wearable
by users. For these examples, the users reported being ther- devices. To the best of our knowledge ours is the first work to
mally comfortable. In this figure, one can clearly see the dif- demonstrate this in a user cohort.
ference in preference among the five users. In subplot (b) the
The work presented in this paper is tested on a dataset collected
same plot is repeated for a single user’s data across multiple
from five users who were exposed to a variety of conditions
days. Here one can clearly observe the variation in preference
from high temperature and high humidity to low temperature
of a single user. In both cases, we note that the preferences
and low humidity. The results of this study show that is it
overlap, underlining the fact that some part of the individual
possible to reduce the number of required labeled examples
preferences is shared and necessarily common through the un-
by 70%, on average over all five users. Specifically, our frame-
derlying human thermo-regulation, but preferences vary both
work achieves a mean root mean square error (RMSE) over
between users and between days for the same subject.
five users of 0.818 with standard error of ±0.05 with just 25
Given the importance of thermal comfort in human perfor- labeled examples in comparison to a mean RMSE of 0.845
mance, it is desirable to find an approach to overcome the with standard error of ±0.04 when using 82 labeled exam-
difficulty in modeling so that we can understand and maxi- ples. This indicates that our framework can achieve the similar
mize personal thermal comfort. This requires the development performance albeit with less resources.
of personalized thermal comfort models which rely on the
In summary, this paper makes three contributions:
availability of large quantities of labeled data examples. Such
an approach was recently used by several authors (Laftchiev • A transfer active learning framework for thermal comfort
& Nikovski, 2016; Ranjan & Scott, 2016; Huang, Yang, & prediction which leverages domain knowledge via transfer
Newman, 2015) who all note the difficulty in securing the learning and minimizes the number of labeled examples
cooperation of users during the experiments. Thus obtaining via active learning.
labeled data examples is emerging as a key obstacle.
• Two query-by-committee querying strategies for active
This paper addresses the problem of modeling thermal com- learning in regression settings with a novel disagreement
fort with a minimal number of labeled examples. Our goal is score.
develop a machine learning framework that uses data collected • An empirical study using five participants which shows:
from an IoT system (Laftchiev & Nikovski, 2016) to create ac- (a) The feasibility of accurately modeling thermal comfort
curate models that can be used to predict personalized thermal using machine learning for multiple individuals, (b) The
comfort. To achieve this goal a novel transfer active learning feasibility of reducing the labeling effort of new partici-
framework is presented. The first part of the framework lever- pants using the described framework.
ages knowledge from a few base users (a group of initial users
as part of a controlled experiment) using transfer learning to 2. BACKGROUND AND R ELATED W ORK
learn a general model of thermal comfort. This model is then
personalized using data from a new user which is obtained This section begins by first providing some background on
through queries issued by an active learning algorithm. The thermal comfort prediction. This is followed by a short back-
results in this paper show that while still in the supervised ground on transfer learning and active learning and concludes
machine learning setting, this approach greatly reduces the with a comparison between the method developed in this paper
number of labeled examples that must be provided by a user. and previously published approaches. Because the volume
of work addressed here is large, and our space is limited, we
The active learning research presented in this paper is set invite the reader to study transfer learning and active learn-
in the pool-based setting. In pool-based active learning, the ing in more depth in (Pan & Yang, 2010) and (Settles, 2010)
assumption is that we have access to all unlabeled examples respectively.
but only few examples are chosen to be queried for labels.
This is the standard setting for the active learning problem. 2.1. Thermal Comfort Estimation
However, within this setting and within this paper’s research
careful attention is paid to choose specific techniques (and to In this paper our goal is to build on prior work in (Laftchiev
tune those techniques) such that the transition to future work & Nikovski, 2016; Ranjan & Scott, 2016; Huang et al., 2015)
in the stream-based setting can be facilitated. Specifically, the to model individual personal thermal comfort. The first study
methods chosen consider that in the stream-based setting user to model thermal comfort was performed by Povl Ole Fanger
data is arriving continuously (as possible data examples that (Fanger, 1967; Ergonomics of the thermal environment – An-
can be labeled by users), and the labeling window is inherently alytical determination and interpretation of thermal comfort
using calculation of the PMV and PPD indices and local

2
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

thermal comfort criteria, 2005). Fanger did not focus on the bient measurements of temperature, humidity, and airspeed,
nuance of modeling a single individual and instead modeled and feedback from the user, it is possible to accurately model
the mean thermal comfort vote of a group of people. During individual thermal comfort. The caveat is that the user must
Fanger’s experiment, thermal comfort feedback was given on provide a sufficiently large number of labeled examples in
an integer scale between 1 and 7. This scale is typically called order to develop accurate prediction models.
the Bedford Scale or when offset to be between -3 and 3, the
Another study that employed wearable sensors and room sen-
ASHRAE scale. Fanger’s model is calibrated such that at most
sors was performed by Huang et al. (Huang et al., 2015). In
5% of respondents are dissatisfied when the predicted mean
this study the authors did not use the standard scale of comfort
vote is comfortable. This model, developed in the 1970s, was
measurement (the ASHRAE or Bedford Scale). Instead they
later adopted as an international standard in ISO 7730.
proposed a new five level scale that combined thermal and
Fanger’s model is based on heat balance equations that de- comfort sensation indices. A classifier is then trained on fea-
scribe the transfer of heat from the body to the environment, tures extracted from the sensors data in duration intervals of 5
with model constants learned from a group study. This model and 30 minutes. Notably, these authors also point out that the
requires one input, room temperature, and makes assump- main challenge in developing personalized thermal comfort
tions about other input factors such as metabolic rate, effective prediction models for a user is the lack of user labeling across
mechanical power produced by the body, clothing insulation, a diverse number of conditions.
surface area of the body, mean radiant temperature, relative
A third study of interest was performed by Ranjan et al. (Ranjan
air velocity, humidity, convective heat transfer, and clothing
& Scott, 2016) using thermographic images for thermal com-
surface temperature. These assumptions are also at the core
fort prediction. Here the authors leverage the intuition that
of the critiques of the model since they were made based on a
blood rushes in to parts of the body to keep warm and rushes
very homogeneous group of individuals.
out to keep the body cool. Thus heatmaps of the body can be
Another class of models called adaptive models that explain constructed from the thermographic images to reveal whether
thermal comfort as a function of outdoor and indoor tempera- a person is hot or cold. During the course of this study, partici-
ture have been proposed. Examples of this literature include pants had thermographic images taken twice per day in their
the European Committee for Standartizations CEN method office. At the same time, each participant provided labeled
(Indoor Environmental Input Parameters for Design and As- examples about their comfort. Despite the small labeling effort
sessment of Energy Performance of Buildings - Addressing required, the authors noted the difficulty in obtaining labeled
Indoor Air Quality, Thermal Environment, Lighting and Acous- examples from users - the minimum labeled examples per
tics, 2006), and the American Society of Heating, Refriger- user collected were 8, the median number of labeled examples
ating, and Air-Conditioning Engineers (ASHRAE) method collected was 24, while the maximum number collected was
(Thermal Environmental Conditions for Human Occupancy, 33. The expected number of labeled examples per user was
2013). Building on these approaches, Haldi proposed a prob- 50.
abilistic model for thermal comfort (Haldi, 2010). These
models are typically calibrated by season (Summer, Winter, 2.2. Transfer Learning for Regression
etc.) and address the critique of the physiological models that
Transfer learning is a type of machine learning where knowl-
they are incapable of capturing seasonal variation in individual
edge from one domain is transferred to another with the goal
preferences. Lastly, data driven approaches have been pro-
of facilitating learning. In our problem setup, given N users,
posed by Jiang and Yao (Jiang & Yao, 2016) and Farhan et al.
domains refer to different users. Specifically the source do-
(Farhan, Pattipati, Wang, & Luh, 2015), which these focus on
main pertains to data from N − 1 users and the target domain
a few machine learning models or the prediction of comfort
refers to data from the N th user. Predicting thermal comfort
on a limited scale, respectively.
falls under inductive transfer learning where labeled data are
All modeling efforts suffer from the drawback of necessarily available in both source and target domains but the crucial
requiring assumptions about the individual. However, with difference is we do not assume we have access to all labeled
the advent of wearable sensors and Internet of Things (IoT) data in target domain.
technology, it is now possible to measure many of the vari-
One of approaches to inductive transfer learning is parameter
ables estimated in the prior models. Recent work by Laftchiev
transfer - where the assumption is that parameters for indi-
and Nikovski (Laftchiev & Nikovski, 2016) seeks to capture
vidual prediction models for similar tasks should be sampled
this possibility by designing a new IoT system to explicitly
from the same prior distribution (Pan & Yang, 2010). Parame-
sense as many features as possible from an individual user
ter transfer has been shown to work for classification models
and then use supervised machine learning to identify an in-
like support vector machines (Evgeniou & Pontil, 2004) and
dividual’s model of thermal comfort. This work showed that
conditional random field models (Natarajan et al., 2014).
using biometric measurements from wearable devices and am-

3
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

For regression problems, parameter transfer has been restricted the generalization error (e.g., RMSE, MSE, etc) is decomposed
to Gaussian Processes (GP) (Bonilla, Chai, & Williams, 2008; of three terms – error from model misspecification, error from
Schwaighofer, Tresp, & Yu, 2005). The general idea is to share labeling noise and error from model variance. The first two
a GP prior that captures the dependencies between different terms are fixed by the choice of prediction model and experi-
tasks and/or domains. This approach is ideal when representa- ment design. For this reason, most of the published research
tive data is simultaneously available to jointly learn a GP prior. has focused on minimizing the model variance such that the
This is challenging in the present problem setup because it is total generalization error (made up of all three components)
assumed data in the target domain is scarce. is minimized. Typically model variance reduction techniques
have relied on computing a variant of Fisher information which
Hence our approach to parameter sharing is sequential where
sets a lower bound on the variance of model parameter’s esti-
we first learn the parameters in the source domain and utilize
mates (Settles, 2010).
this information as data becomes available in the target do-
main. Specifically, our approach to parameter sharing is to first The challenge in using variance reduction techniques for re-
learn the source domain parameters and second to penalize the gression is that the statistics such as Fisher information must
deviation of target domain model parameters from source do- be computed on the whole data distribution and is therefore
main model parameters. This has the added advantage that in not feasible to be computed when samples arrive sequentially.
the absence of target domain data the prediction model would The last caveat is important to this work because the final goal
fall back on the source domain model to make predictions. of the framework presented in this paper is to be transplanted
This is clearly better than making predictions using a model into the stream-based setting where knowledge of the complete
initialized with random parameters, or assuming a baseline data distribution is unknown.
comfort score.
A second approach to model-based active learning is query-by-
committee (QBC). The goal of QBC is to minimize the space
2.3. Active Learning for Regression
of possible predictions (also known as hypothesis, version
Active learning is a type of machine learning where a predic- space) given the current labeled dataset (Burbidge, Rowland,
tion model achieves good performance when it is allowed to & King, 2007). To achieve this goal QBC relies on a com-
choose the examples from which to learn (Lewis & Catlett, mittee to vote on available pool of examples and the most
1994). An active learner chooses a sample to be labeled via controversial example is chosen to be labeled. Once this sam-
querying and then requests an oracle to provide a label for the ple is chosen, the committee members who disagreed the most
chosen sample. The majority of published results in active with the provided ground truth label update their prediction
learning focus on classification problems, in contrast, few pa- models to minimize disagreement on similar data points in
pers address the work of developing active learning approaches the future. Classically proposed QBC is unlikely to perform
for regression problems (Cai, Zhang, & Zhou, 2013). well in the case of thermal comfort model learning because the
committee of users have fixed thermal comfort models which
Active learning for regression can be subdivided into model-
cannot simply be retrained when a new user provides a label.
free and model-based approaches. The model-free strategies
are active learning approaches that do not rely on a prediction
2.4. Transfer and Active Learning Combined
model to determine which data samples to label. Instead these
approaches rely only on the statistics of the data distribution The work most closely related to the active transfer learning
(O‘Neill, Delany, & MacNamee, 2017). The most popular framework proposed in this paper is that of Wang et al., (Wang,
model-free approach is a density-based querying approach Huang, & Schneider, 2014). In this paper the authors develop
which seeks labeled examples for data points residing in high a unified framework to perform active transfer learning for
density regions of the dataset. These regions are hypothesized the case where a regression model is to be identified. The
to be representative of the underlying data distribution and proposed approach is a domain adaptation approach that is
thus of labeling interest (Settles, 2010). The difficulty faced based on the differences in both the marginal (P(X)) and con-
in model-free active learning approaches is that successive ditional distributions (P(Y |X)) in source and target domains.
queries do not account for prior knowledge gained and often To account for the difference in the marginal distribution, the
end up issuing redundant queries. Therefore the model-free authors perform a covariate shift and to account for the dif-
active learning approach is not suitable in this paper because ferences in the conditional distribution the authors propose
when the problem setting involves human user labeling there two approaches both subject to the smoothness assumption:
is an extreme constraint with respect to the number of queries the first, is to match the conditional distributions between the
that a user is willing to label. source and target domains; the second, is to use the Gaussian
Processes to model the source, target tasks and the offset in
On the other hand, most model-based active learning for the
between. The authors leverage active learning to choose which
case of building a regression model builds on the early work by
examples in target domain should be labeled.
Geman et al.(Geman, Bienenstock, & Doursat, 2008), where

4
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

A second paper of importance where domain adaptation is each real valued, xi ∈ Rk , and each corresponding to data from
performed and then is augmented by active learning is the wearable and ambient room sensors. The index i denotes the
work of Sugiyama et al., (Sugiyama, 2006). In this paper the sample number while k denotes the length of the vector which
authors assume that the marginal distributions differ but the corresponds to the number of features used in the prediction
conditional distributions remain the same between the source model. The target values yi are drawn from a pre-defined set,
and target domains. The difference in the marginal distribu- yi ∈ {0, ±1, ±2, ±3}. These correspond to thermal comfort
tions is viewed as a covariate shift problem and is addressed rating given as feedback from the users.
by computing importance weights such that the distribution
The goal of this paper and the framework is to learn a predic-
is rebalanced in the source domain to match the distribution
tion model, h, h : xi → yi that for any input vector xi outputs
in the target domain. The authors then perform active learn-
ing, choosing examples to be labeled in the target domain, to a prediction target value ŷi = h(xi ). Because in this paper the
learn the importance weights used in the distribution rebal- prediction model will be learned using a regression method
ancing problem. Concretely, the active learning approach is (more details under framework development), we stipulate that
batch active learning where a subset of available examples is the predicted target value must not deviate more than ε, in the
sampled to be labeled in a single shot. In practice, multiple squared sense, from the actual target value ŷi as (yi − ŷi )2 < ε
subsets need to be chosen in order to identify the subset which ∀i. For convenience the n labeled data samples are all ex-
minimizes generalization error. pressed as matrix, which we call the design matrix, X, with
n rows and k columns. We also represent all the labels as a
There are two crucial differences between our work and these vector y of length n.
two prior work: first, both papers rely on computing impor-
tance weights to handle covariate shift. These importance 3.2. Transfer Learning for Linear Regression
weights are typically computed by estimating the probability
densities of the marginal distribution in the source and target For the development in this work, the target values, y, are
domain. This is generally challenging in datasets which are treated as continuous values that are restricted to the range
high dimensional but have a low sample counts. This is the {-3,+3}. The inherent assumption here is that while users are
case for the dataset in this paper. Second, both approaches op- forced to discretize their state into 7 levels, in practice their
erate in a pool-based settings where active learner can choose thermal comfort is much more nuanced.
examples in small batches (Sugiyama et al., (Sugiyama, 2006)). Furthermore, treating the problem of thermal comfort predic-
Wang et al., (Wang et al., 2014) take advantage of the pool- tion as a regression problem addresses the problem of class
based setting to identify regions of high utility from low utility imbalance. In particular because most users are in an HVAC
and strategically place queries. Such approaches cannot be controlled space, we anticipate that most feedback received
deployed to the stream-based settings and must be necessarily will be in the range {-1,+1} leading to severe class imbal-
re-worked. In contrast, the work in this paper presents an ances for the -3, -2, +2, and +3 classes. Thus using regression
approach that can be adapted the stream-based stetting in a methods is a natural approach when training thermal comfort
future study. predictors.

3. T RANSFER ACTIVE L EARNING F RAMEWORK FOR To provide maximum transparency to the model, this paper
T HERMAL C OMFORT P REDICTION focuses on the problem of linear regression. Linear regression
provides an easier quantification of the effect of each feature
Having reviewed the case for thermal comfort prediction as on the model output. Linear regression is parametrized by
well as the relevant background in transfer learning and active weight vector, W , such that the product of design matrix and
learning, here we return to the problem of developing personal- weight vector results in ŷ as,
ized thermal comfort model with a minimal number of labeled
examples. This section develops the framework of transfer X T W = ŷ. (1)
active learning that was described in the introduction. The
overall goal is to leverage prior knowledge to rapidly learn The standard approach to finding the regressor weight vector
a prediction model of thermal comfort and tune this model is called ordinary least squares (OLS), where the goal of OLS
given only a few labeled examples of data. is to minimize the squared sum of the differences between
the estimated target values and the real target values. These
3.1. Notation differences are called the residuals and the sum of the residuals,
often written as an optimization objective is expressed as,
To begin we present notation for the development of the frame-
work. In this work we assume that we are given a dataset kX T W − yk22 . (2)
D which contains n labeled samples of the form D = (xi , yi )
∀i ∈ {1 . . . n}. Here each xi corresponds to a feature vector,
The OLS estimate of W is prone to high variance in the model

5
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

weights and poor allocation (selection) of the weights among which does not differ from one person to the next. It is simply
the features. Furthermore the classical, analytical solution to the preferences of the individual that differ.
this problem is not well posed, suffering from numerical issues
One convenient prior for transfer learning in the case of ther-
in the event that the design matrix is not easily invertible.
mal comfort modeling is a general thermal comfort over a
To remedy these issues, a penalty is introduced on the regressor group of users. That is, suppose that we have N datasets
weight vector. In this paper we choose this penalty to take the collected from N distinct users. Then we can find a general
form of the `2 -norm, which means that the derivation below linear regressor, using equation (3), that describes the data
follows the Ridge Regression framework. Here the `2 -norm is from N − 1 users. We call this regressor our population model,
chosen because of its more beneficial treatment of correlated Wp . We then use equation (4).
features (Hastie, Tibshirani, & Friedman, 2009). The added
Solving equation (4), will then yield the personalized thermal
penalty parameter reduces the model variance and results in
comfort model for the N th user. This approach to introducing
a solution where some feature weight may be close to zero.
a prior intuitively captures the idea that new user’s coefficients,
This minimizes overfitting and reduces variance in estimates.
W , should be very similar to other users while allowing for
The new objective function to solve is thus,
individual differences.
kX T W − yk22 +λ kW k22 . (3) Setting this up as an optimization problem, the ridge regression
coefficients are learned by minimizing the following objective
In equation (3), λ , is the penalty parameter that determines function,
the weight of the penalty term in the solution. Increasing λ
leads to smaller weight coefficients in W , and decreasing λ Ŵ ridge = arg min(kX T W − yk22 +λ kW −Wp k22 ). (5)
W
leads to larger weight coefficients in W . Because of this, λ is
said to control the shrinkage of the regressor coefficients.
In this formulation, the first term is the loss function, which
Classically, when utilizing Ridge Regression, the shrinkage has the usual format of equation (3), the second term penalizes
parameter is optimized such that the coefficients are driven the deviation of ridge coefficients of the new model W from
towards zero without compromising the model error perfor- the prior model Wp . Taking the derivative of this objective
mance. This approach to Ridge Regression has a Bayesian with respect to the new regressor weight vector W and setting
interpretation where the weight vector coefficients are sam- it equal to zero results in analytical solution, which we term
pled fromr a prior normal distribution with mean zero and modified ridge regression,
1
variance= . Ŵ ridge = (X T X + λ I)−1 (X T Y + λWp ). (6)
λ
An alternate approach to Ridge Regression is to shrink the
coefficients towards a non-zero prior distribution. When this 3.3. Incorporating Active Learning
approach is taken, the non-zero prior distribution represents
The goal of this framework is to create regression models that
some prior knowledge about the problem. In this case, it is
predict personal thermal comfort but do not require large quan-
said that the shrinkage of the coefficients toward the prior
tities of labeled examples per user. So far we have introduced
distribution induces a transfer of domain knowledge because
the transfer learning component of the framework, however,
the weight vector we find should be as close to the prior distri-
in order to personalize the model to the N th user, this user
bution as possible.The modified ridge regression problem has
must provide some feedback. Combining active learning with
the following form,
transfer learning is a logical approach to reducing the labeling
kX T W − yk22 +λ kW −Wp k22 . (4) effort for thermal comfort modeling.
In pool-based active learning, solutions often begin with the
In equation (4), Wp is a vector containing a sample regressor introduction of, A , the pool of all available examples that are
vector. This vector represents the mean of the prior distribution yet to be labeled and, L , the set of labeled examples which are
described above. Note that setting Wp to zeros results in the chosen through some active learning strategy. Importantly, in
classical ridge OLS from equation (3). the pool-based setting all labels exists, but there is some cost
associated to obtain a label that is to be minimized through
Multiple approaches exists to estimate the prior regressor,
sample selection. The overall goal of active learning is to
Wp . In this paper, we posit that when the goal is to derive
choose an optimal subset of m (where m << n) labeled exam-
a personalized thermal comfort model, we can assume that
ples L such that it achieves good generalization performance
there are strong similarities between users, and that the model
on the test set.
must only be slightly modified to fit a new individual. This
assumption is rooted in the physiology of thermoregulation, There are two important components of active learning; the

6
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

Algorithm 1 Transfer Active Learning

1: procedure T HERMAL _C OMFORT(N, Budget) . Predict Thermal comfort for user N using labeling budget Budget
2: Train ← {} . Labeled examples L of the N th user
3: Available ← Data(N) . Available examples A of the N th user
4: Other ← Data(N − 1) . Data from N − 1 users
5: Wp ← Ridge(Other) . Wp is learned on N − 1 users; Equation 3
6: RidgeWp ← random weights . Initial model; Wp is used from line #5 and used as in Equation 5
7: while Budget 6= 0 & Available 6= ∅ do . Budget not empty and sample available to query
8: (x, y) ← query_strategy(RidgeWp , Available) . Pick and return labeled example using querying strategy
9: Train ← Train ∪ (x, y) . Add labeled example to L
10: RidgeWp ← RidgeWp (Train) . Update model; Wp is used from line #5 and used as in Equation 5
11: Available ← Available − (x, y) . Remove labeled example from A
12: Budget ← Budget − 1 . Decrease budget
13: end while
14: return RidgeWp . Return final model
15: end procedure

labeling budget and the querying strategy. The labeling budget In equation (7), C is the number of committee members, ŷic is
is simply the total number of labeled examples that can be the prediction associated to the cth committee member and ŷiL
obtained. In the context of personalized thermal comfort mod- corresponds to the prediction made by the N th user’s prediction
eling this is the number of labeled examples that each user is model which has been trained only using the labeled examples,
allowed to be asked. Because in this problem the user should L , obtained thus far. We compute di for all examples in A
not be disturbed frequently, the labeling budget should be as and pick the example with the maximum disagreement score
small as possible. to be labeled. It is important to note that the disagreement
score defined in this manner prefers labeled examples that
The querying strategy is the approach used to determine which
predict thermal comfort with opposite signs, for example −2
examples in the set A should be labeled. In this paper we pro-
predicted as +2, versus examples that predict thermal com-
pose a modified QBC approach. In a typical QBC approach,
fort with the same sign but differ significantly in magnitude,
the labeled dataset L is used to update the committee mem-
for example −3 predicted as −2. This disagreement score
bers. Here we choose not to update the committee members,
accommodates individual differences in thermo-regulatory be-
but instead we update only the N th user’s current predictive
havior, for example the layering of clothes, while focusing on
model. There are two reasons for choosing to update N th
difference that may arise in dataset collected from different
user’s predictive model: first, a labeled example from the N th
individuals; for example N th user’s model predicts cold when
user could benefit only those committee members that ex-
all other users feel hot under similar conditions.
hibit a significant overlap in thermo-regulatory behavior. The
consequence of using labeled examples to update committee Combining the transfer learning and the active learning, the
members who are significantly different will result in noisy complete transfer active learning framework is presented in
predictions when issuing subsequent queries; second, the goal Algorithm 1. In Algorithm 1, the function ‘Thermal_Comfort’
of this work is to develop personalized prediction models with has two arguments: the user, N, for whom we personalize
as few labeled examples as possible and hence updating the thermal comfort predictive model and labeling budget that
N th user’s predictive model gets us towards that goal quickly. determines the maximum number of active learning queries to
The proposed QBC strategy is thus to choose examples which be issued. The algorithm then creates, A , a pool of available
cause the committee members and the N th user’s predictive data examples from the N th user and combines data from N −1
model to maximally disagree. Intuitively this means that the users into ‘Other’ (lines #2 − #4). Then in line #5, the data
proposed QBC technique prefers examples for which the N th from N − 1 users is used to train a ridge regression model
user’s model is uncertain about but the committee is fairly which is the population level model, Wp . Wp is used then to
certain about. learn the N th user’s initial model via transfer learning in line
#6.
A key point to address here is the notion of disagreement. We
choose to define a disagreement score, di , for the ith example After the initial models are created, the algorithm iterates
in A is computed as, Budget times, discovering a point to label in each iteration. To
! do this, a label is queried from the pool of available examples
1 C i (line #8), the training dataset is updated (line #9) and the
di = ∑ ŷc − yiL
C c=1
(7)
ridge regression model penalized by the population model is
retrained (line #10).

7
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

The loop ends when the training budget is exhausted or there provided via voice recognition with seven possible comfort
are no more samples available to label. While this loop is levels: very cold (-3), cold (-2), chilly (-1), comfortable (0),
running, the algorithm updates the available dataset and adjust warm (1), hot (2), and very hot (3). All data were collected
the budget of remaining labeled examples (lines #11 − #12). via a custom Windows phone application which was written
When the algorithm terminates, an updated ridge regression in-house to provide streaming access to the band data. Each
model is returned. sensor provided a sample approximately every 8 seconds.
In addition to the wearable features, room sensors were also
4. E MPIRICAL P ROTOCOLS
deployed. Room temperature was recorded from three separate
Having developed the active transfer learning framework this room thermometers (placed at different locations in the office).
section introduces the empirical protocols used to verify the A gradient of the room temperature was created by subtract-
approach. In particular, this section first introduces the par- ing instantaneous values of the temperature sensors from one
ticipants which took part in the study. Then the experimental another. This gradient serves as a proxy for the variation in
setup is discussed, including the sensors that were used and temperature across the room and the potential motion of air
the feedback method which the users used. The data prepro- mass in the room. Two of the temperature sensors, DHT11 and
cessing is described along with data partitioning to train and the NEST, also record humidity. In addition to temperature
test the framework. The section concludes by describing the and humidity sensors, a hot-wire anemometer was used detect
experimental methods used for validation. airflow in the room. Specifically, the hot-wire anemometer
was placed near the HVAC output vent to detect room air flow.
4.1. Study Design and Participants All room sensors were sampled using an Arduino connected to
a Raspberry Pi. Sensor samples were collected approximately
The study described in this paper included five participants,
every 2-15 seconds. All feature data (room data and wearable
three male, two female, mean age 30 ± 3, who participated in
data) was aggregated in the Microsoft Azure cloud.
the study for an average of 14 days. This study was carried out
in a corporate research environment in which the participants
4.2. Sensor Features
each had an individual office space which was isolated from
the common environment through a door. Study participants In the preceding section it was noted that sensors in the data
were briefed on the experiment design and participated in the collection system were sampled at different rates. The primary
experiments only after giving informed consent. concern during data collection was to collect a complete train-
ing dataset. For this reason each sensor was sampled as close
To create a uniformly sampled and labeled dataset, the user’s
as possible to their maximal sampling rate as possible. The
environment was varied by changing the thermostat to a lower
variance in sampling rates originates in technical limitations
or higher setting, by adding a space heater, and by adding hu-
of the Microsoft Windows phone and the Raspberry Pi sensors.
midifier to the environment. For each parameter combination
To capture the dynamics surrounding user provided thermal
(e.g., added heat and humidity) the settings remained fixed for
comfort ratings we extract features from time windows im-
batches of four hours. However, subsequent four hour batches
mediately preceding user provided ratings. Specifically we
were alternated between hot and cold temperatures. Each user
aggregate features in five minute windows preceding each ther-
also experienced several days of environmental stability where
mal comfort rating. The length of the window is a parameter,
the office HVAC controlled the room parameters.
but the choice of five minutes is motivated by prior work in
The participants were instructed to wear the Microsoft Band thermoregulatory behavior (Schlader, Stannard, & Mündel,
1 during work hours (9 AM to 5 PM) and to take them off 2010).
only when they leave their offices for extended periods of
We only extract features associated with user provided ratings
time (e.g., lunch, hour-long meetings; short bathroom breaks
that are spaced at least five minutes apart to avoid redundancy
were excluded). Participants were instructed not to manually
in ratings as well as to minimize noise in user supplied ratings.
modulate their comfort level by wearing additional layers of
Table 1 provided details of the dataset including available
clothes when cold or removing layers of clothing when hot.
and usable number of user provided ratings. Here the usable
The features measured by the Microsoft band include heart number of ratings represents the number of feedback reports
rate, skin temperature, calories consumed, metabolic rate, al- given to the user that are spaced at least five minutes apart.
timeter, barometer, steps taken, and elevation. Data provided In addition to the averaged features, within each five minute
by the Microsoft band is labeled with user provided thermal- window we also compute the mean, variance, median, min
comfort ratings. The users were instructed to provide such and max for eight features: heart rate, skin temperature, core
ratings approximately every 20 minutes. The feedback was body temperature, preferred temperature, wind speed, room
temperature 1, room temperature 2 and room humidity. We
1 All users wore Microsoft band II except for one participant who wore Mi- estimate the user’s core body temperature from the user’s heart
crosoft band I.

8
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

rate using a Kalman filter. This filter was shown in prior work We compute the disagreement score as in equation (7), the first
to accurately track the core body temperature (Buller et al., term, ŷCi , we set C to equal K nearest neighbors. Then compute
2013). We also derive a preferred temperature feature which the mean rating over the K nearest neighbors, where neighbors
uses skin and core body temperatures as follows, correspond to labeled examples from N − 1 users and the no-
tion of nearest is defined by Euclidean distance. The number
pre f erredTemp = −0.3 × skinTemp (8) of neighbors used in the estimate of the mean labeled examples
× (coreBodyTemp − 36.3) + 44 was empirically tested for neighbor values K = 5, 10, 15, 20.
Of these, it was observed that 10 neighbors yielded optimum
These features are motivated by prior work in thermal comfort performance. The second term in equation (7), ŷiL , is com-
estimation and prediction (Laftchiev & Nikovski, 2016). Note puted using the N th user’s current prediction model which is
here that it is always possible to add more features to this type trained only using labeled examples L . Specifically at budget,
of model. The current set of features represent the features B, L would hold at most B labeled examples, all from the N th
with the most significant contribution in terms of minimizing user. This strategy is a model-based querying strategy which
the model output error. utilizes the model of the N th user. Therefore, the prediction
The statistics which are computed over five minute windows model is retrained after each labeling point is added to L .
are supplemented with simple moving averages to capture In the second active learning strategy, each of the N − 1 users
trends in sensor data over short time scales. We compute sim- is treated as a committee member who is allowed to make a
ple moving averages between two to nine samples immediately prediction for all available examples in A . That is, for each
preceding the user provided ratings within each five minute committee member a thermal comfort model is learned using
window. This brings the total number of features to 104. only data from that user. A 5-fold cross-validation over each
user’s data is performed to choose hyperparameters. Each
4.3. Data Partitioning and Preprocessing committee member then predicts a thermal comfort rating for
Having collected the data, an important question is how to best all available examples in the pool. Then a weighted mean of
split the complete dataset into training and testing datasets. the committee ratings is computed for each sample. Higher
The optimal choice of this split is a study parameter that needs weights are assigned to users that overlap with the N th user
to be empirically evaluated, however for this work the labeled in feature space. These weights are computed as inverse of
dataset was split into two halves for each day of the experiment AUROC between N th user and N − 1 users in pairs. The
and for each user. The first half is used to train and the second remaining details of the strategy are the same as above in the
half is used to test the comfort prediction model. first strategy. In the results section, the performance of both
strategies is compared to random sampling of the available
Each collected feature is centered by subtracting the mean data samples.
and dividing by the standard deviation to bring all features
to the same scale. This ensures that no single feature will 4.5. Evaluating the Performance of the Approach
dominate the regression model. Both train and test datasets
were transformed using the mean and standard deviation com- To evaluate the effectiveness of the proposed transfer active
puted only on the training partition of the dataset within each learning framework the model that is used is fixed and several
user. Labeled examples were also centered using, again, nor- decisions are made about the error reporting metrics and the
malization coefficients derived from the training data. Here format of error. This section details those decisions. The
only the mean was subtracted from each rating. Normalizing section begins with a discussion of the model choice and
the labeled examples obviates the need to fit an intercept in implementation. This is followed by a description of the error
regression settings (Hastie et al., 2009). and how it is calculated. The section then concludes with four
error metrics calculated throughout the framework to show the
4.4. Active Learning - Querying Strategies effectiveness of the proposed approach.

For this paper we experimented with two strategies. Each 4.5.1. Model Selection: Ridge Regression
strategy is set in the pool-based active learning setting, but has
been optimized for the streaming setting which is the natural For this work, the model chosen was ridge regression. Note
extension of this work. here that this model can be readily kernelized to learn a non-
linear model of the data. This may be desirable in future work
The first active learning strategy leverages a K nearest neigh- that focuses exclusively on determining the lowest possible
bors approach (QBC-K). The main idea of this labeling strat- error rate of the thermal comfort prediction model. In this
egy is to compute the disagreement score for all available work the focus is on demonstrating that the transfer active
examples in the pool, A . Then from this set of disagreement learning framework is capable of reducing the number of
scores, the example chosen is that which had the maximum required labeled examples to achieve an error rate comparable
disagreement score. The label for this example is queried.

9
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

User Days No. user No. usable No. non-overlap. Mean duration Label distribution
ratings labels labels (5m) b/w labels (5m) -3 -2 -1 0 1 2 3
1 7 149 97 82 ~27.47m 0 2 16 40 9 11 4
2 12 320 263 185 ~21.72m 8 13 23 94 22 16 9
3 10 201 138 133 ~30.68m 16 14 23 54 18 7 1
4 13 143 119 113 ~44.69m 4 18 6 52 16 17 0
5 29 514 400 300 ~48.69m 1 10 75 117 67 28 2
Total 71 1337 1017 813 – 29 57 143 357 132 79 16

Table 1. Description of the dataset including number of days and the distribution of user supplied thermal comfort ratings that
are spaced at least five minutes (5m) apart

to the strictly supervised model learning approach. For clarity of the ridge regression model when it has been trained on data
of presentation, the model used is linear. from N − 1 users. When training this model the complete
dataset for the N − 1 users is employed. This means that both
In order to implement the framework a custom implementation
the training and testing partitions of those users’ data is used
of ridge regression was used. This in-house implementation
during training. Testing the model occurs on the N th user’s
was used to modify the OLS objective function in accordance
test data only. The data is preprocessed as discussed above,
with equation (5) to perform transfer learning. The implemen-
and a N − 1-fold cross-validation is performed on the training
tation is written in Python using the scipy function minimize
dataset to choose the ridge penalty parameter. Here the hyper-
(Foundation, 2010) and the gradients with respect to W are
parameters follow the same range as within-user evaluation
manually derived and supplied within the code. Because both
protocol.
the design matrix X and the target values y are centered, the
fitting of ridge regression is performed without fitting an in-
4.5.5. Active Learning (AL) Error Trajectory:
tercept. The method chosen for the optimization solver is the
Newton conjugate gradient method with default settings. To evaluate the performance of active learning strategies, this
paper presents the trajectory of the generalization error of the
4.5.2. Reporting Performance ridge regression model as a function of number of training
examples. To isolate the active learning trajectory from prior
In this paper, error is reported as the mean root mean square
knowledge, the model is first initialized with a random weight
error (RMSE) over five users. That is, an RMSE is determined
vector with small magnitudes (±1e − 4). The ridge penalty
for each user and the mean of the five RMSEs is reported.
parameter is also set to a default value of 1e − 4.
Each error is accompanied by a set of standard error bars. To
minimize the effects of randomly seeding the analysis (e.g., Then using equation (5) with the population level model, Wp ,
cross-validation folds used to pick hyperparameters, randomly set to a vector of zeros to eliminate any effect of transfer
initializing weights) each experiment is run for ten times per learning. The reported error for each set of labeled examples,
user. This means that each user RMSE is averaged over the is the generalization error learned from this objective function.
ten runs before being reported. Active learning is performed for each user by iterating this
objective after each label is received up to a given query budget
4.5.3. Within-users (W): B. For each user the training dataset is used as a pool of
available examples and the model is tested on a held out test
A key metric to evaluate the active transfer learning frame-
partitions.
work is within-user performance. This error represents the
generalization error when a model is learned (and tested) using As previously noted, the two query strategies employed are K
only data from each specific user. To find the within-user error, nearest neighbors and the committee of N − 1 users. In both
the ridge regression model is trained and tested on training strategies, we pick examples with the maximum disagreement
and testing data partitions which are determined as described score. We compare this against a query strategy of randomly
above. For each user, a 5-fold cross-validation is performed issuing queries. For these experiments, the total permissible
on the training dataset to choose the optimal ridge penalty label budget is B = 40 because this is the maximum number
parameter, λ . The parameter range is set from 1e−4 to 1e4 . of available examples (for the user with the smallest pool of
Our observation is that the optimal parameter at no time falls labeled examples) in the training partition given a 50%/50%
on the boundary of this range. train/test split. A 5-fold cross-validation is performed for each
of the following label budgets, 10, 15, 20, 25, 30, 35, and 40.
4.5.4. Between-users (B): This cross-validation determines the optimal ridge penalty
parameter using only the labeled examples for each user.
To determine the starting point before active learning is em-
ployed, the between-users error shows the generalization error

10
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

1.6
Baseline
1.5 Random(TAL)
Random(AL)
1.4 QBC-K(TAL)
1.3 QBC-K(AL)
QBC-U(TAL)
RMSE

1.2
1.1
1.0
0.9
0.8
B 10 15 20 25 30 35 40 W
Budget
Figure 2. The performance of the Active Transfer Learning Framework in personalized thermal comfort prediction. Comparing
within-user, between-user, active learning and transfer active learning evaluation protocols.

4.5.6. Transfer Active Learning (TAL) Error Trajectory: method which predicts that the user is always comfortable.
Lastly, to evaluate the effectiveness of the transfer active learn- The figure gives the reader two important conclusions. First,
ing framework, we show the trajectory of the generalization transfer active learning is always better than active learning.
error when the between-users model is further tuned using Recall here that in active learning the model is initialized
labeled data that is actively queried from the N th user. Here with a random weight vector. Second, the QBC-K strategy
we use between-user model to learn the weights which is set proposed in this paper outperforms other querying strategies.
to Wp in the objective function found in equation (5). The first Notably, QBC-K outperforms the random query strategy. This
query is then issued using this model. Our querying strategy means that QBC-K meaningfully adds selectivity with respect
is same as the one outlined for active learning above. Then to data points which are important.
for each subsequently acquired labeled example, the model is
These results verify the underlying assumption of this work
re-trained using equation (5). All other details are same as the
that there is an overlap in the thermo-regulatory behavior
active learning error trajectory.
of users and leveraging this information leads to better per-
formance. Furthermore, the out-performance of the QBC-K
5. R ESULTS AND D ISCUSSION
strategy as compared to random sampling, confirms the sec-
The results of the active transfer learning framework using ond hypothesis of this work which is that neighboring labeled
ridge regression and the modified QBC query strategies are examples from other users carry relevant information that can
shown in Figure 2. This figure serves to summarize the results guide the personalized thermal comfort prediction model to
and to display the effectiveness of this approach. Note that all pick and choose examples with high utility. The better perfor-
error metrics described in the previous section are included in mance of QBC-K can be attributed to its choice of committee
the figure. members which are chosen from one or more users spanning
multiple days whereas for QBC-U each of the N − 1 users
Figure 2 is constructed by placing the between-users (B) error
are committee members and not all of them may be useful in
rate at the extreme left of the x-axis. Here, no labeled examples
computing disagreement scores.
have been queried from the N th user. On the right extreme of
the x-axis, we place the within-users (W) error rate. This error With regard to model performance, we observe two favorable
rate represents the generalization error of the model if it were outcomes. First, the within and between subject model er-
to only be trained with data from the N th user. In between ror evaluation shows the expected trend of a mean RMSE at
these two evaluation protocols are plotted the results from 0.845 ± 0.04 and 1.288 ± 0.44 respectively. This confirms that
active learning and transfer active learning. On the y-axis we the between subject model is more coarse than the within sub-
show the RMSE for each of the corresponding methods. At ject model with room for improvement. Second, all prediction
the top of the graph, with a dotted line, we show a baseline models perform significantly better than the baseline model

11
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

which assumes the users are always at thermal comfort with a user labeled examples under all possible conditions within a
zero rating. single day to facilitate exhaustive supervised model learning.
The advent of this framework and specifically utilizing the
The trajectory observed for transfer active learning algorithms
warm start approach described in this paper, it is now possible
is mostly as expected, lower than between-subjects RMSE,
to choose the most important data points to label within a day
with some exceptions which we will explain below. This fol-
thereby improving the performance of the within-user model.
lows intuitively from the framework development because the
framework penalizes the deviation of weights from the be-
6. C ONCLUSION
tween subject’s weights. The intermittent peak in the transfer
active learning trajectory is a result of hyperparameter learn- This paper demonstrated a new approach to learning personal-
ing that is performed in cross-validation when 15 and then 20 ized thermal comfort models for new user in a personalized
labeled examples are acquired respectively. The reason for this thermal comfort prediction system. This new approach was
intermittent hyperparameter tuning is that the tuning proved presented as a framework that combined the machine learning
to be very computationally expensive while RMSE suffered if fields of active and transfer learning to reduce the labeling
tuning was not performed. effort needed to obtain an accurate model of thermal comfort.
Importantly the approach shown here was able to reduce the
Acceptable RMSE values for untuned hyperparameters did not
labeled examples needed from new users by 70% as compared
occur until the label budgets exceeded 50 labeled examples.
to a fully supervised approach. Specifically, the framework
For extremely low label budgets, for example < 10, we refer
achieves a mean error of 0.82±0.05 while the fully supervised
to these as burn-in periods where transfer active learning used
learning approach achieves a mean error of 0.85±0.04. These
a hyperparameter tuned on N − 1 users and active learning
results indicate a significant improvement in thermal comfort
used a default penalty parameter of 1e − 4. A significant result
prediction at reduced quantities of user supplied labels.
of the transfer active learning with the QBC-K query strategy
is that the resulting model is able to achieve a performance
(RMSE) that is very close (0.35% difference) to the RMSE R EFERENCES
observed in the classically trained within-subject model with
a small budget of 10 to 25 labeled examples. On average, Belluck, P. (2015). Chilly at work? office formula was devised
these results point to a 70% reduction in labeling effort over for men. New York Times.
five users. In comparison, the within-user performance was Bonilla, E. V., Chai, K. M., & Williams, C. (2008). Multi-
achieved with an average of 82.6 labeled examples. To further task gaussian process prediction. In Advances in neural
illustrate the results, the mean error across users with models information processing systems (nips) (pp. 153–160).
trained on their respective data using the fully supervised Buller, M. J., Tharion, W. J., Cheuvront, S. N., Montain, S.,
approach (within-user) is 0.845 ± 0.04. The transfer active Kenefick, R., Castellani, J., . . . Hoyt, R. W. (2013).
learning approach proposed in this work achieves a mean error Estimation of human core temperature from sequential
of 0.818 ± 0.05. All errors are noted on the ASHRAE comfort heart rate observations. Physiological measurement, 34
scale. Here it should be noted that the model error is due to 7, 781-98.
several factors including uncertainty in sensor measurements Burbidge, R., Rowland, J. J., & King, R. D. (2007). Active
and incomplete sampling of our data space. An important topic learning for regression based on query by committee. In
to study in a future study is the contribution of uncertainty International conference on intelligent data engineering
in sensor measurements. This is because understanding this and automated learning (pp. 209–218).
contribution can lead us to practical suggestions about the Cai, W., Zhang, Y., & Zhou, J. (2013). Maximizing expected
type and grade of sensor that should be used in real systems model change for active learning in regression. In Inter-
performing online thermal comfort estimation. national conference on data mining (icdm) (pp. 51–60).
We hypothesize that the improvement in performance, with Ergonomics of the thermal environment – analytical deter-
respect to the number of labeled examples needed to achieve a mination and interpretation of thermal comfort using
certain RMSE value, comes from the observation that a single calculation of the pmv and ppd indices and local ther-
model trained over all data from a single user (within-user) mal comfort criteria (Tech. Rep.). (2005). ISO 7730.
would necessarily have a larger modeling error, especially Evgeniou, T., & Pontil, M. (2004). Regularized multi–task
when including any outlier labeled examples provided by the learning. In Acm sigkdd international conference on
user, than a model trained on a small but relevant set of exam- knowledge discovery and data mining (pp. 109–117).
ples. However, until the development of this transfer active Fanger, P. O. (1967). Calculation of thermal comfort: Intro-
learning framework it was not possible to determine which duction of a basic comfort equation. ASHRAE Transac-
examples are most important to a user on any given day. This tions, 73.
is because one user does not provide a sufficient quantity of Farhan, A. A., Pattipati, K., Wang, B., & Luh, P. (2015).
Predicting individual thermal comfort using machine

12
 I NTERNATIONAL J OURNAL OF P ROGNOSTICS AND H EALTH M ANAGEMENT

learning algorithms. In International conference on IEEE Transactions on Knowledge and Data Engineer-
automation science and engineering (case) (pp. 708– ing, 22(10), 1345–1359.
713). Ranjan, J., & Scott, J. (2016). Thermalsense: Determin-
Foundation, P. S. (2010). Python language reference, ing dynamic thermal comfort preferences using thermo-
version 2.7. https://docs.scipy.org/doc/ graphic imaging. In Proceedings of the international
scipy/reference/generated/scipy.optimize joint conference on pervasive and ubiquitous computing
.minimize.html. (ubicomp) (pp. 1212–1222).
Geman, S., Bienenstock, E., & Doursat, R. (2008). Neural net- Schlader, Z. J., Stannard, S. R., & Mündel, T. (2010). Human
works and the bias/variance dilemma. Neural Networks, thermoregulatory behavior during rest and exercise -
4(1). a prospective review. Physiology & behavior, 99(3),
Haldi, F. (2010). Towards a unified model of occupants’ 269–275.
behaviour and comfort for building energy simulation Schwaighofer, A., Tresp, V., & Yu, K. (2005). Learning
(Unpublished doctoral dissertation). EPFL. gaussian process kernels via hierarchical bayes. In Ad-
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements vances in neural information processing systems (nips)
of statistical learning: Data mining, inference, and (pp. 1209–1216).
prediction. Springer. Settles, B. (2010). Active learning literature survey (Tech.
Hedge, A., Wafa, S., & Anshu, A. (2005). Thermal effects on Rep.).
office productivity. In Proceedings of the human factors Sugiyama, M. (2006). Active learning in approximately linear
and ergonomics society annual meeting (Vol. 49, pp. regression based on conditional expectation of general-
823–827). ization error. Journal of Machine Learning Research, 7,
Huang, C.-C. J., Yang, R., & Newman, M. W. (2015). The 141–166.
potential and challenges of inferring thermal comfort at
Thermal environmental conditions for human occupancy
home using commodity sensors. In Proceedings of the
(Tech. Rep.). (2013). ASHRAE.
international joint conference on pervasive and ubiqui-
Wang, X., Huang, T.-K., & Schneider, J. (2014). Active trans-
tous computing (ubicomp) (pp. 1089–1100).
fer learning under model shift. In International confer-
IJzerman, H., & Semin, G. R. (2009). The thermometer of so-
ence on machine learning (icml) (pp. 1305–1313).
cial relations mapping social proximity on temperature.
Psychological Science, 20(10), 1214-1220.
Indoor environmental input parameters for design and assess- B IOGRAPHIES
ment of energy performance of buildings - addressing
Annamalai Natarajan Annamalai Natara-
indoor air quality, thermal environment, lighting and
jan is a research scientist with Philips Re-
acoustics (Tech. Rep.). (2006). European Standards
search. His research interests are in ma-
Commission.
chine learning for healthcare with emphasis
Jiang, L., & Yao, R. (2016). Modelling personal thermal
on physiological time series. He holds a PhD
sensations using c-support vector classification (c-svc)
in computer science from UMass Amherst,
algorithm. Building and Environment, 99, 98–106.
a M.S. in computer science from Colorado
Laftchiev, E., & Nikovski, D. (2016). An iot system to estimate
State University, Fort Collins and a B. Tech.
personal thermal comfort. In World forum on internet
in Information Technology from University of Madras, India.
of things (wf-iot) (p. 672-677).
Lewis, D. D., & Catlett, J. (1994). Heterogeneous uncertainty Emil Laftchiev Emil I. Laftchiev received
sampling for supervised learning. In Machine learning the B.S. and Ph.D. degrees in Electrical En-
proceedings (p. 148 - 156). San Francisco, CA. gineering from the Pennsylvania State Uni-
Natarajan, A., Gaiser, E., Angarita, G., Malison, R., Ganesan, versity in 2007 and 2015, respectively. In
D., & Marlin, B. (2014). Conditional random fields for 2015 Emil joined MERL in Cambridge, MA,
morphological analysis of wireless ecg signals. In Acm USA as a research scientist. He has a wide
conference on bioinformatics, computational biology, range of research interests including machine
and health informatics (pp. 370–379). learning, the Internet of Things, time series
O‘Neill, J., Delany, S. J., & MacNamee, B. (2017). Model-free analysis, and signal processing. Within these fields Emil is par-
and model-based active learning for regression. In Ad- ticularly interested in developing subfields of machine learning
vances in computational intelligence systems (pp. 375– such as transfer learning, active learning, and generative mod-
386). Springer. eling. Emil is a member of IEEE and SIGKDD. His work has
Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. appeared in IEEE Conferences, Journals and Workshops.

13