Python自然语言处理入门经典教程

需积分: 5 61 浏览量更新于2024-07-18 收藏 4.49MB PDF 举报

"Natural Language Processing with Python" 《Natural Language Processing with Python》是一本专注于自然语言处理（NLP）的书籍，由Steven Bird、Ewan Klein和Edward Loper共同编写。这本书是2009年出版的英文原版，针对那些希望理解和应用NLP技术的读者。在中文版翻译存在疑问的情况下，读者可以通过此英文原版获取更准确的理解。 NLP是计算机科学的一个领域，它研究如何让计算机理解、解析、生成和操作人类自然语言。这本书涵盖了NLP的基础知识和实践技巧，使用Python编程语言作为工具。Python因其简洁易用和丰富的库支持而成为NLP领域广泛应用的编程语言，尤其是与NLTK（自然语言工具包）结合时，能提供强大的NLP功能。 NLTK是Python中一个广泛使用的NLP库，包含了大量的语料库、分词器、词性标注器、命名实体识别器等工具。通过这本书，读者可以学习到如何使用NLTK进行文本预处理（如去除停用词、标点符号和数字）、词干提取、词性标注、句法分析、情感分析等任务。此外，书中可能还涵盖了机器学习方法在NLP中的应用，如朴素贝叶斯分类器、最大熵模型等，用于实现更复杂的任务，如文本分类、情感分析和机器翻译。作者Steven Bird是Charles Darwin University的研究员，拥有164篇出版物和5017次引用记录；Ewan Klein是The University of Edinburgh的学者，有129篇出版物和5045次引用记录。他们两位以及其他作者在NLP领域的深入研究和实践经验确保了这本书的权威性和实用性。这本书的内容包括但不限于：语言数据的处理、词汇和语法分析、句法树的构造、信息抽取、情感分析、语义解析等。对于初学者，它可以提供一个全面的入门指南，而对于有一定基础的读者，它也能提供进阶的实践案例和理论讲解。《Natural Language Processing with Python》是一本详尽的指南，旨在帮助读者掌握使用Python进行自然语言处理的方法和技术，适合学术研究者、软件开发者以及对NLP感兴趣的任何人。书中丰富的示例代码和实际应用将使读者能够快速上手并深入理解NLP的核心概念。

we can use to break a line into its words. To apply a method to an object, we write the

object name, followed by a period, followed by the method name, i.e., line.split().

Third, methods have arguments expressed inside parentheses. For instance, in the ex-

ample, word.endswith('ing') had the argument 'ing' to indicate that we wanted words

ending with ing and not something else. Finally—and most importantly—Python is

highly readable, so much so that it is fairly easy to guess what this program does even

if you have never written a program before.

We chose Python because it has a shallow learning curve, its syntax and semantics are

transparent, and it has good string-handling functionality. As an interpreted language,

Python facilitates interactive exploration. As an object-oriented language, Python per-

mits data and methods to be encapsulated and re-used easily. As a dynamic language,

Python permits attributes to be added to objects on the fly, and permits variables to be

typed dynamically, facilitating rapid development. Python comes with an extensive

standard library, including components for graphical programming, numerical pro-

cessing, and web connectivity.

Python is heavily used in industry, scientific research, and education around the world.

Python is often praised for the way it facilitates productivity, quality, and main-

tainability of software. A collection of Python success stories is posted at http://www

.python.org/about/success/.

NLTK defines an infrastructure that can be used to build NLP programs in Python. It

provides basic classes for representing data relevant to natural language processing;

standard interfaces for performing tasks such as part-of-speech tagging, syntactic pars-

ing, and text classification; and standard implementations for each task that can be

combined to solve complex problems.

NLTK comes with extensive documentation. In addition to this book, the website at

http://www.nltk.org/ provides API documentation that covers every module, class, and

function in the toolkit, specifying parameters and giving examples of usage. The website

also provides many HOWTOs with extensive examples and test cases, intended for

users, developers, and instructors.

Software Requirements

To get the most out of this book, you should install several free software packages.

Current download pointers and instructions are available at http://www.nltk.org/.

Python

The material presented in this book assumes that you are using Python version 2.4

or 2.5. We are committed to porting NLTK to Python 3.0 once the libraries that

NLTK depends on have been ported.

NLTK

The code examples in this book use NLTK version 2.0. Subsequent releases of

NLTK will be backward-compatible.

Preface | xiii

NLTK-Data

This contains the linguistic corpora that are analyzed and processed in the book.

NumPy (recommended)

This is a scientific computing library with support for multidimensional arrays and

linear algebra, required for certain probability, tagging, clustering, and classifica-

tion tasks.

Matplotlib (recommended)

This is a 2D plotting library for data visualization, and is used in some of the book’s

code samples that produce line graphs and bar charts.

NetworkX (optional)

This is a library for storing and manipulating network structures consisting of

nodes and edges. For visualizing semantic networks, also install the Graphviz

library.

Prover9 (optional)

This is an automated theorem prover for first-order and equational logic, used to

support inference in language processing.

Natural Language Toolkit (NLTK)

NLTK was originally created in 2001 as part of a computational linguistics course in

the Department of Computer and Information Science at the University of Pennsylva-

nia. Since then it has been developed and expanded with the help of dozens of con-

tributors. It has now been adopted in courses in dozens of universities, and serves as

the basis of many research projects. Table P-2 lists the most important NLTK modules.

Table P-2. Language processing tasks and corresponding NLTK modules with examples of

functionality

Language processing task NLTK modules Functionality

Accessing corpora nltk.corpus Standardized interfaces to corpora and lexicons

String processing nltk.tokenize, nltk.stem Tokenizers, sentence tokenizers, stemmers

Collocation discovery nltk.collocations t-test, chi-squared, point-wise mutual information

Part-of-speech tagging nltk.tag n-gram, backoff, Brill, HMM, TnT

Classification nltk.classify, nltk.cluster Decision tree, maximum entropy, naive Bayes, EM, k-means

Chunking nltk.chunk Regular expression, n-gram, named entity

Parsing nltk.parse Chart, feature-based, unification, probabilistic, dependency

Semantic interpretation nltk.sem, nltk.inference Lambda calculus, first-order logic, model checking

Evaluation metrics nltk.metrics Precision, recall, agreement coefficients

Probability and estimation nltk.probability Frequency distributions, smoothed probability distributions

Applications nltk.app, nltk.chat Graphical concordancer, parsers, WordNet browser, chatbots

xiv | Preface

Language processing task NLTK modules Functionality

Linguistic fieldwork nltk.toolbox Manipulate data in SIL Toolbox format

NLTK was designed with four primary goals in mind:

Simplicity

To provide an intuitive framework along with substantial building blocks, giving

users a practical knowledge of NLP without getting bogged down in the tedious

house-keeping usually associated with processing annotated language data

Consistency

To provide a uniform framework with consistent interfaces and data structures,

and easily guessable method names

Extensibility

To provide a structure into which new software modules can be easily accommo-

dated, including alternative implementations and competing approaches to the

same task

Modularity

To provide components that can be used independently without needing to un-

derstand the rest of the toolkit

Contrasting with these goals are three non-requirements—potentially useful qualities

that we have deliberately avoided. First, while the toolkit provides a wide range of

functions, it is not encyclopedic; it is a toolkit, not a system, and it will continue to

evolve with the field of NLP. Second, while the toolkit is efficient enough to support

meaningful tasks, it is not highly optimized for runtime performance; such optimiza-

tions often involve more complex algorithms, or implementations in lower-level pro-

gramming languages such as C or C++. This would make the software less readable

and more difficult to install. Third, we have tried to avoid clever programming tricks,

since we believe that clear implementations are preferable to ingenious yet indecipher-

able ones.

For Instructors

Natural Language Processing is often taught within the confines of a single-semester

course at the advanced undergraduate level or postgraduate level. Many instructors

have found that it is difficult to cover both the theoretical and practical sides of the

subject in such a short span of time. Some courses focus on theory to the exclusion of

practical exercises, and deprive students of the challenge and excitement of writing

programs to automatically process language. Other courses are simply designed to

teach programming for linguists, and do not manage to cover any significant NLP con-

tent. NLTK was originally developed to address this problem, making it feasible to

cover a substantial amount of theory and practice within a single-semester course, even

if students have no prior programming experience.

Preface | xv

A significant fraction of any NLP syllabus deals with algorithms and data structures.

On their own these can be rather dry, but NLTK brings them to life with the help of

interactive graphical user interfaces that make it possible to view algorithms step-by-

step. Most NLTK components include a demonstration that performs an interesting

task without requiring any special input from the user. An effective way to deliver the

materials is through interactive presentation of the examples in this book, entering

them in a Python session, observing what they do, and modifying them to explore some

empirical or theoretical issue.

This book contains hundreds of exercises that can be used as the basis for student

assignments. The simplest exercises involve modifying a supplied program fragment in

a specified way in order to answer a concrete question. At the other end of the spectrum,

NLTK provides a flexible framework for graduate-level research projects, with standard

implementations of all the basic data structures and algorithms, interfaces to dozens

of widely used datasets (corpora), and a flexible and extensible architecture. Additional

support for teaching using NLTK is available on the NLTK website.

We believe this book is unique in providing a comprehensive framework for students

to learn about NLP in the context of learning to program. What sets these materials

apart is the tight coupling of the chapters and exercises with NLTK, giving students—

even those with no prior programming experience—a practical introduction to NLP.

After completing these materials, students will be ready to attempt one of the more

advanced textbooks, such as Speech and Language Processing, by Jurafsky and Martin

(Prentice Hall, 2008).

This book presents programming concepts in an unusual order, beginning with a non-

trivial data type—lists of strings—then introducing non-trivial control structures such

as comprehensions and conditionals. These idioms permit us to do useful language

processing from the start. Once this motivation is in place, we return to a systematic

presentation of fundamental concepts such as strings, loops, files, and so forth. In this

way, we cover the same ground as more conventional approaches, without expecting

readers to be interested in the programming language for its own sake.

Two possible course plans are illustrated in Table P-3. The first one presumes an arts/

humanities audience, whereas the second one presumes a science/engineering audi-

ence. Other course plans could cover the first five chapters, then devote the remaining

time to a single area, such as text classification (Chapters 6 and 7), syntax (Chapters

8 and 9), semantics (Chapter 10), or linguistic data management (Chapter 11).

Table P-3. Suggested course plans; approximate number of lectures per chapter

Chapter Arts and Humanities Science and Engineering

Chapter 1, Language Processing and Python 2–4 2

Chapter 2, Accessing Text Corpora and Lexical Resources 2–4 2

Chapter 3, Processing Raw Text 2–4 2

Chapter 4, Writing Structured Programs 2–4 1–2

xvi | Preface

剩余504页未读，继续阅读

greecehalf

粉丝: 0
资源: 1

Python自然语言处理入门经典教程

Python自然语言处理聊天机器人框架

Python自然语言处理入门

掌握Python进行自然语言处理的英文原版教程

Hands－On_Natural_Language_Processing_with_Python

Getting_Started_on_Natural_Language_Processing_with_Python.pdf.pdf

Natural_Language_Processing_with_Machine_Learning

Natural_language_processing:python中的NLP进程

Natural_Language_Processing

Natural_Language_processing-Structured_Text_Generation：生成乐曲

natural_language_processing:自然语言处理相关代码的存储库

最新资源