iv
Abstract
This thesis is in the eld of machine learning: the use of data to automatically learn
a hypothesis to predict the future behavior of a system. It summarizes three of my
research projects.
We rst investigate the role of margins in the phenomenal success of the Boosting
Algorithms. AdaBoost (Adaptive Boosting) is an algorithm for generating an ensem-
ble of hypotheses for classication. The superior out-of-sample performance of Ad-
aBoost has been attributed to the fact that it can generate a classier which classies
the points with a large margin of condence. This led to the development of many
new algorithms focusing on optimizing the margin of condence. It was observed
that directly optimizing the margins leads to a poor performance. This apparent
contradiction has been the topic of a long unresolved debate in the machine-learning
community. We introduce new algorithms which are expressly designed to test the
margin hypothesis and provide concrete evidence which refutes the margin argument.
We then propose a novel algorithm for Adaptive sampling under Monotonicity
constraint. The typical learning problem takes examples of the target function as
input information and produces a hypothesis that approximates the target as an
output. We consider a generalization of this paradigm by taking dierent types of
information as input, and producing only specic properties of the target as output.
This is a very common setup which occurs in many dierent real-life settings where
the samples are expensive to obtain. We show experimentally that our algorithm
achieves better performance than the existing methods, such as Staircase procedure
and PEST.
One of the major pitfalls in machine learning research is that of selection bias.
评论1