使用 Python 进行自然语言处理技术指南

需积分: 50 24 浏览量更新于2024-07-24 1 收藏 5.18MB PDF 举报

Python自然语言处理概述 Python 是一种流行的编程语言，广泛应用于自然语言处理（Natural Language Processing，NLP）领域。自然语言处理是人工智能和语言学的交叉领域，旨在开发算法和模型来处理、分析和理解人类语言。Python 语言的 simplicity、flexibility 和 extensive libraries 使其成为 NLP 领域的理想选择。本书《Natural Language Processing with Python》由 Steven Bird、Ewan Klein 和 Edward Loper 合著，旨在为读者提供 Python 语言在 NLP 领域的实践经验。该书涵盖了 NLP 的基本概念、文本预处理、词汇分析、句法分析、语义角色标注、机器翻译等方面的知识。 Python 在 NLP 领域的应用非常广泛，包括文本分类、情感分析、命名实体识别、 dependency parsing 等。Python 提供了多种 NLP 库和工具，如 NLTK、spaCy、Stanford CoreNLP 等，使开发者能够快速构建 NLP 应用程序。本书的主要内容包括： 1. 文本预处理：文本 Tokenization、词汇 normalization、停用词 removal 等。 2. 词汇分析：词汇 frequency analysis、词汇 co-occurrence analysis 等。 3. 句法分析：句法树构建、句法角色标注等。 4. 语义角色标注：语义角色标注、事件抽取等。 5. 机器翻译：基于规则的机器翻译、基于统计的机器翻译等。本书还涵盖了 Python 在 NLP 领域的其他应用，例如文本分类、情感分析、命名实体识别等。此外，本书还提供了许多实践经验和案例，帮助读者更好地理解 Python 在 NLP 领域的应用。本书《Natural Language Processing with Python》是一本非常实用的书籍，对于想学习 Python 在 NLP 领域的应用的读者非常有帮助。

NLTK-Data

This contains the linguistic corpora that are analyzed and processed in the book.

NumPy (recommended)

This is a scientific computing library with support for multidimensional arrays and

linear algebra, required for certain probability, tagging, clustering, and classifica-

tion tasks.

Matplotlib (recommended)

This is a 2D plotting library for data visualization, and is used in some of the book’s

code samples that produce line graphs and bar charts.

NetworkX (optional)

This is a library for storing and manipulating network structures consisting of

nodes and edges. For visualizing semantic networks, also install the Graphviz

library.

Prover9 (optional)

This is an automated theorem prover for first-order and equational logic, used to

support inference in language processing.

Natural Language Toolkit (NLTK)

NLTK was originally created in 2001 as part of a computational linguistics course in

the Department of Computer and Information Science at the University of Pennsylva-

nia. Since then it has been developed and expanded with the help of dozens of con-

tributors. It has now been adopted in courses in dozens of universities, and serves as

the basis of many research projects. Table P-2 lists the most important NLTK modules.

Table P-2. Language processing tasks and corresponding NLTK modules with examples of

functionality

Language processing task NLTK modules Functionality

Accessing corpora nltk.corpus Standardized interfaces to corpora and lexicons

String processing nltk.tokenize, nltk.stem Tokenizers, sentence tokenizers, stemmers

Collocation discovery nltk.collocations t-test, chi-squared, point-wise mutual information

Part-of-speech tagging nltk.tag n-gram, backoff, Brill, HMM, TnT

Classification nltk.classify, nltk.cluster Decision tree, maximum entropy, naive Bayes, EM, k-means

Chunking nltk.chunk Regular expression, n-gram, named entity

Parsing nltk.parse Chart, feature-based, unification, probabilistic, dependency

Semantic interpretation nltk.sem, nltk.inference Lambda calculus, first-order logic, model checking

Evaluation metrics nltk.metrics Precision, recall, agreement coefficients

Probability and estimation nltk.probability Frequency distributions, smoothed probability distributions

Applications nltk.app, nltk.chat Graphical concordancer, parsers, WordNet browser, chatbots

xiv | Preface

Language processing task NLTK modules Functionality

Linguistic fieldwork nltk.toolbox Manipulate data in SIL Toolbox format

NLTK was designed with four primary goals in mind:

Simplicity

To provide an intuitive framework along with substantial building blocks, giving

users a practical knowledge of NLP without getting bogged down in the tedious

house-keeping usually associated with processing annotated language data

Consistency

To provide a uniform framework with consistent interfaces and data structures,

and easily guessable method names

Extensibility

To provide a structure into which new software modules can be easily accommo-

dated, including alternative implementations and competing approaches to the

same task

Modularity

To provide components that can be used independently without needing to un-

derstand the rest of the toolkit

Contrasting with these goals are three non-requirements—potentially useful qualities

that we have deliberately avoided. First, while the toolkit provides a wide range of

functions, it is not encyclopedic; it is a toolkit, not a system, and it will continue to

evolve with the field of NLP. Second, while the toolkit is efficient enough to support

meaningful tasks, it is not highly optimized for runtime performance; such optimiza-

tions often involve more complex algorithms, or implementations in lower-level pro-

gramming languages such as C or C++. This would make the software less readable

and more difficult to install. Third, we have tried to avoid clever programming tricks,

since we believe that clear implementations are preferable to ingenious yet indecipher-

able ones.

For Instructors

Natural Language Processing is often taught within the confines of a single-semester

course at the advanced undergraduate level or postgraduate level. Many instructors

have found that it is difficult to cover both the theoretical and practical sides of the

subject in such a short span of time. Some courses focus on theory to the exclusion of

practical exercises, and deprive students of the challenge and excitement of writing

programs to automatically process language. Other courses are simply designed to

teach programming for linguists, and do not manage to cover any significant NLP con-

tent. NLTK was originally developed to address this problem, making it feasible to

cover a substantial amount of theory and practice within a single-semester course, even

if students have no prior programming experience.

Preface | xv

A significant fraction of any NLP syllabus deals with algorithms and data structures.

On their own these can be rather dry, but NLTK brings them to life with the help of

interactive graphical user interfaces that make it possible to view algorithms step-by-

step. Most NLTK components include a demonstration that performs an interesting

task without requiring any special input from the user. An effective way to deliver the

materials is through interactive presentation of the examples in this book, entering

them in a Python session, observing what they do, and modifying them to explore some

empirical or theoretical issue.

This book contains hundreds of exercises that can be used as the basis for student

assignments. The simplest exercises involve modifying a supplied program fragment in

a specified way in order to answer a concrete question. At the other end of the spectrum,

NLTK provides a flexible framework for graduate-level research projects, with standard

implementations of all the basic data structures and algorithms, interfaces to dozens

of widely used datasets (corpora), and a flexible and extensible architecture. Additional

support for teaching using NLTK is available on the NLTK website.

We believe this book is unique in providing a comprehensive framework for students

to learn about NLP in the context of learning to program. What sets these materials

apart is the tight coupling of the chapters and exercises with NLTK, giving students—

even those with no prior programming experience—a practical introduction to NLP.

After completing these materials, students will be ready to attempt one of the more

advanced textbooks, such as Speech and Language Processing, by Jurafsky and Martin

(Prentice Hall, 2008).

This book presents programming concepts in an unusual order, beginning with a non-

trivial data type—lists of strings—then introducing non-trivial control structures such

as comprehensions and conditionals. These idioms permit us to do useful language

processing from the start. Once this motivation is in place, we return to a systematic

presentation of fundamental concepts such as strings, loops, files, and so forth. In this

way, we cover the same ground as more conventional approaches, without expecting

readers to be interested in the programming language for its own sake.

Two possible course plans are illustrated in Table P-3. The first one presumes an arts/

humanities audience, whereas the second one presumes a science/engineering audi-

ence. Other course plans could cover the first five chapters, then devote the remaining

time to a single area, such as text classification (Chapters 6 and 7), syntax (Chapters

8 and 9), semantics (Chapter 10), or linguistic data management (Chapter 11).

Table P-3. Suggested course plans; approximate number of lectures per chapter

Chapter Arts and Humanities Science and Engineering

Chapter 1, Language Processing and Python 2–4 2

Chapter 2, Accessing Text Corpora and Lexical Resources 2–4 2

Chapter 3, Processing Raw Text 2–4 2

Chapter 4, Writing Structured Programs 2–4 1–2

xvi | Preface

剩余503页未读，继续阅读

绝缘胶囊

粉丝: 0
资源: 1

使用 Python 进行自然语言处理技术指南

使用Python进行自然语言处理：使用自然语言工具包分析文本Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit

使用python进行自然语言处理

使用python进行自然语言处理.pdf

图书：使用Python进行自然语言处理

nltk_book_exercises:我对“使用Python进行自然语言处理”练习的解决方案

Natural-Language-Understanding:使用python进行自然语言处理的练习，重点是NLU。 2021年在特伦托大学的自然语言理解课程中开发的作业和项目

Natural-Language-Processing-with-Python-Quick-Start-Guide:Packt发行的《使用Python进行自然语言处理快速入门指南》

Hands-On-Natural-Language-Processing-with-Python:Packt发行的《使用Python进行自然语言处理》（Hands On Natural Language Processing with Python），由Packt发布

用Python进行自然语言处理.pdf_afternoonz8z_用python进行自然语言处理_

使用Python的自然语言处理库进行股票舆情分析

最新资源