ML in NLP June 02
NASSLI 5
ML in NLP
Chomsky’s Challenge to Empiricism
(1) Colorless green ideas sleep furiously.
(2) Furiously sleep ideas green colorless.
… It is fair to assume that neither sentence (1) nor
(2) (nor indeed any part of these sentences) has
ever occurred in an English discourse. Hence, in
any statistical model for grammaticalness, these
sentences will be ruled out on identical grounds as
equally ‘remote’ from English. Yet (1), though
nonsensical, is grammatical, while (2) is not.
Chomsky 57
ML in NLP
The Return of Empiricism
n Empiricist methods
work
:
n Markov models can capture a surprising fraction of
the unpredictability in language
n Statistical information retrieval methods beat
alternatives
n Statistical parsers are more accurate than
competitors based on rationalist methods
n Machine-learning, statistical techniques close to
human performance in part-of-speech tagging, sense
disambiguation
n
Just engineering tricks?
ML in NLP
Unseen Events
n Chomsky’s implicit assumption:
any model must
assign zero probability to unseen events
n naïve estimation of Markov model probabilities from
frequencies
n no
latent (hidden)
events
n Any such model
overfits
data: many events are
likely to be missing in any finite sample
\ The learned model
cannot generalize
to unseen
data
\ Support for
poverty of the stimulus
arguments
ML in NLP
The Science of Modeling
n Probability estimates can be
smoothed
to
accommodate unseen events
n
Redundancy
in language supports
effective statistical inference procedures
\
the stimulus is richer than it might seem
n
Statistical learning theory
:
generalization
ability
of a model class can be measured
independently of
model representation
n
Beyond Markov models:
effects of
latent
conditioning variables
can be estimated
from data
评论2