CHAPTER 1. INTRODUCTION
patterns from raw data. This capability is known as
machine learning
. The
introduction of machine learning enabled computers to tackle problems involving
knowledge of the real world and make decisions that appear subjective. A simple
machine learning algorithm called
logistic regression
can determine whether to
recommend cesarean delivery (Mor-Yosef 1990et al., ). A simple machine learning
algorithm called naive Bayes can separate legitimate e-mail from spam e-mail.
The performance of these simple machine learning algorithms depends heavily
on the
representation
of the data they are given. For example, when logistic
regression is used to recommend cesarean delivery, the AI system does not examine
the patient directly. Instead, the doctor tells the system several pieces of relevant
information, such as the presence or absence of a uterine scar. Each piece of
information included in the representation of the patient is known as a
feature
.
Logistic regression learns how each of these features of the patient correlates with
various outcomes. However, it cannot influence how features are defined in any
way. If logistic regression were given an MRI scan of the patient, rather than
the doctor’s formalized report, it would not be able to make useful predictions.
Individual pixels in an MRI scan have negligible correlation with any complications
that might occur during delivery.
This dependence on representations is a general phenomenon that appears
throughout computer science and even daily life. In computer science, operations
such as searching a collection of data can proceed exponentially faster if the collec-
tion is structured and indexed intelligently. People can easily perform arithmetic
on Arabic numerals but find arithmetic on Roman numerals much more time
consuming. It is not surprising that the choice of representation has an enormous
effect on the performance of machine learning algorithms. For a simple visual
example, see figure .1.1
Many artificial intelligence tasks can be solved by designing the right set of
features to extract for that task, then providing these features to a simple machine
learning algorithm. For example, a useful feature for speaker identification from
sound is an estimate of the size of the speaker’s vocal tract. This feature gives a
strong clue as to whether the speaker is a man, woman, or child.
For many tasks, however, it is difficult to know what features should be
extracted. For example, suppose that we would like to write a program to detect
cars in photographs. We know that cars have wheels, so we might like to use the
presence of a wheel as a feature. Unfortunately, it is difficult to describe exactly
what a wheel looks like in terms of pixel values. A wheel has a simple geometric
shape, but its image may be complicated by shadows falling on the wheel, the sun
glaring off the metal parts of the wheel, the fender of the car or an object in the
3