Combining Exploration and Exploitation in Active Learning

Abstract

This thesis investigates the active learning in the presence of model bias. State of the art approaches advocate combining exploration and exploitation in active learning. However, they suffer from premature exploitation or unnecessary exploration in the later stages of learning. We propose to combine exploration and exploitation in active learning by discarding instances outside a sampling window that is centered around the estimated decision boundary and uniformly draw sample from this window. Initially the window spans the entire feature space and is gradually constricted, where the rate of constriction models the exploration-exploitation tradeoff. The desired effect of this approach (CExp) is that we get an increasing sampling density in informative regions as active learning progresses, resulting in a continuous and natural transition from exploration to exploitation, limiting both premature exploitation and unnecessary exploration. We show that our approach outperforms state of the art on real world multiclass datasets. We also extend our approach to spatial mapping problems where the standard active learning assumption of uniform costs is violated. We show that we can take advantage of \emph{spatial continuity} in the data by geographically partitioning the instances in the sampling window and choosing a single partition (region) for sampling, as opposed to taking a random sample from the entire window, resulting in a novel spatial active learning algorithm that combines exploration and exploitation. We demonstrate that our approach (CExp-Spatial) can generate cost-effective sampling trajectories over baseline sampling methods. Finally, we present the real world problem of mapping benthic habitats where bathymetry derived features are typically not strong enough to discriminate the fine details between classes identified from high-resolution imagery, increasing the possiblity of model bias in active learning. We demonstrate, under such conditions, that CExp outperforms state of the art and that CExp-Spatial can generate more cost-effective sampling trajectories for an Autonomous Underwater Vehicle in contrast to baseline sampling strategies.