Java 6文本处理实战

需积分: 10 35 浏览量更新于2024-07-20 收藏 825KB PDF 举报

"《Text Processing with Java 6》是由Bob Carpenter、Mitzi Morris和Breck Baldwin合著的一本关于使用Java进行文本处理的书籍，由LingPipe Publishing于2011年在纽约出版。该书主要面向的是对自然语言处理（NLP）感兴趣的Java开发者，特别是那些使用Lingpipe库的读者。书中详细介绍了如何利用Java进行文本处理，并涵盖了相关的编程基础知识和技术。\n\n在本书中，作者首先引导读者了解进行文本处理所需的工具，包括一个简单的‘Hello World’示例，以及如何使用Ant构建和管理项目。接着，作者深入讲解了Java的基本概念，如数值类型、对象、数组和同步机制。对于字符和字符串处理，书中详细阐述了字符编码的重要性，如何在Java程序中处理编码问题，char类型的特性，以及Character类和CharSequence接口的功能。此外，作者还详细讨论了String类的使用，以及在需要频繁修改字符串时更高效的StringBuilder类。\n\n3.1章节中，作者强调了字符编码对于正确处理文本至关重要，因为不同的编码标准（如ASCII、ISO-8859-1和Unicode）可能会导致乱码问题。3.2章节解释了如何确保Java源代码在不同的操作系统和环境中能正确编译和运行，这涉及到文件的编码设置。3.4章节介绍了Java中的char类型，它用于存储单个Unicode字符。3.6章节则专门探讨了String类，它是Java中不可变的字符序列，经常用于文本处理。\n\n通过本书，读者不仅可以掌握Java编程基础，还能深入了解如何利用Java进行高效、准确的文本处理，这对于开发涉及NLP的应用程序，如信息检索、机器翻译或情感分析等，具有极大的帮助。同时，由于Lingpipe库的配合，使得这本书成为Java开发者学习NLP技术的理想参考资料。"

4 CHAPTER 1. GETTING STARTED

(defun java-mode-untabify ()

(save-excursion

(goto-char (point-min))

(if (search-forward "t" nil t)

(untabify (1- (point)) (point-max))))

nil)

(add-hook ’java-mode-hook

’(lambda ()

(make-local-variable ’write-contents-hooks)

(add-hook ’write-contents-hooks ’java-mode-untabify)))

(setq indent-tabs-mode nil)

1.1.4 Java Standard Edition 6

We chose Java as the basis for LingPipe because we felt it provided the best

tradeoﬀ among eﬃciency, usability, portability, and library availability.

The presentation here assumes the reader has a basic working knowledge of

the Java programming language. We will focus on a few aspects of Java that

are particularly crucial for processing textual language data, such as character

and string representations, input and output streams and character encodings,

regular expressions, parsing HTML and XML markup, etc. In explaining LingPipe’s

design, we will also delve into greater detail on general features of Java such as

concurrency, generics, ﬂoating point representations, and the collection package.

This book is based on the latest currently supported standard edition of the

Java platform (Java SE), which is version 6. You will need the Java development

kit (JDK) in order to compile Java programs. A java virtual machine (JVM) is

required to execute compiled Java programs. A Java runtime environment (JRE)

contains platform-speciﬁc support and integration for a JVM and often interfaces

to web browsers for applet support.

Java is available in 32-bit and 64-bit versions. The 64-bit version is required

to allocate JVMs with heaps larger than 1.5 or 2 gigabytes (the exact maximum

for 32-bit Java depends on the platform).

Licensing terms for the JRE are at

http://www.java.com/en/download/license.jsp

and for the JDK at

http://java.sun.com/javase/6/jdk-6u20-license.txt

Java is available for the Windows, Linux, and Solaris operating systems in

both 32-bit and 64-bit versions from

http://java.sun.com/javase/downloads/index.jsp

The Java JDK and JRE are included as part of Mac OS X. Updates are available

through

1.1. TOOLS OF THE TRADE 5

http://developer.apple.com/java/download/

Java is updated regularly and it’s worth having the latest version. Updates in-

clude bug ﬁxes and often include performance enhancements, some of which

can be quite substantial.

Java must be added to the operating system’s executables path so that the

shell commands and other packages like Ant can ﬁnd it.

We have to manage multiple versions of Java, so typically we will deﬁne

an environment variable JAVA_HOME, and add ${JAVA_HOME}/bin (Unix) or

%JAVA_HOME%\bin (Windows) to the PATH environment variable (or its equiva-

lent for your operating system). We then set JAVA_HOME to either JAVA_1_5,

JAVA_1_6, or JAVA_1_7 depending on use case. Note that JAVA_HOME is one

level above Java’s bin directory containing the executable Java commands.

You can test whether you can run Java with the following command, which

should produce similar results.

> java -version

java version "1.6.0_18"

Java(TM) SE Runtime Environment (build 1.6.0_18-b07)

Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode)

Similarly, you can test for the Java compiler version, using

> javac -version

javac 1.6.0_20

Java Source

You can download the Java source ﬁles from the same location as the JDK. The

source is subject to yet anoher license, available at

http://java.sun.com/javase/6/scsl_6-license.txt

The source provides a fantastic set of examples of how to design, code, and doc-

ument Java programs. We especially recommend studying the source for strings,

I/O and collections. Further, like all source, it provides a deﬁnitive answer to the

questions of what a particular piece of code does and how it does it. This can be

useful for understanding deeper issues like thread safety, equality and hashing,

eﬃciency and memory usage.

1.1.5 Ant

We present examples for compilation and for running programs using the

Apache Ant build tool. Ant has three key features that make it well suited for

expository purposes. First, it’s portable across all the platforms that support

Java. Second, it provides clear XML representations for core Java concepts such

as classpaths and command-line arguments. Third, invoking a target in Ant di-

rectly executes the dependent targets and then all of the commands in the target.

Thus we ﬁnd Ant builds easier to follow than those using the classic Unix build

6 CHAPTER 1. GETTING STARTED

tool Make or its modern incarnation Apache Maven, both of which attempt to re-

solve dependencies among targets and determine if targets are up to date before

executing them.

Although we’re not attempting to teach Ant, we’ll walk through a basic Ant

build ﬁle later in this chapter to give you at least a reading knowledge of Ant.

If you’re using an IDE like Eclipse or NetBeans, you can import Ant build ﬁles

directly to create a project.

Ant is is an Apache project, and as such, is subject to the Apache license,

http://ant.apache.org/license.html

Ant is available from

http://ant.apache.org/

You only need one of the binary distributions, which will look like

apache-ant-version-bin.tar.gz.

First, you need to unpack the distribution. We like directory structures with

release names, which is how ant unpacks, using top-level directory names like

apache-ant-1.8.1. Then, you need to put the bin subdirectory of the top-level

directory into the PATH environment variable so that Ant may be executed from

the command line.

Ant requires the JAVA_HOME environment variable to be set to the path above

the bin directory containing the Java executables. Ant’s installation instructions

suggest setting the ANT_HOME directory in the same way, and then adding but it’s

not necessary unless you will be scripting calls to Ant.

Ant build ﬁles may be imported directly into either the Eclipse or NetBeans

IDEs (see below for a description of these).

You can test whether Ant is installed with

> ant -version

Apache Ant version 1.8.1 compiled on April 30 2010

1.1.6 Integrated Development Environment

Many pepeople prefer to write (and compile and debug) code in an integrated de-

velopment environment (IDE). IDEs oﬀer advantages such as automatic method,

class and constant completion, easily conﬁgurable text formatting, stepwise de-

buggers, and automated tools for code generation and refactoring.

LingPipe development may be carried out through an IDE. The two most pop-

ular IDEs for Java are Eclipse and NetBeans.

Eclipse IDE

Eclipse provides a full range of code checking, auto-completion and code gen-

eration, and debugging facilities. Eclipse is an open-source project with a wide

range of additional tools available as plugins. It also has modules for languages

other than Java, such as C++ and PHP.

The full set of Eclipse downloads is listed on the following page,

1.1. TOOLS OF THE TRADE 7

http://download.eclipse.org/eclipse/downloads/

You’ll want to make sure you choose the one compatible with the JDK you are

using. Though originally a Windows-based system, Eclipse has been ported to

Mac OS X (though Carbon) and Linux.

Eclipse is released under the Eclipse Public License (EPL), a slightly modiﬁed

version of the Common Public License (CPL) from IBM, the full text of which is

available from

http://www.eclipse.org/legal/epl-v10.html

NetBeans IDE

Unlike Eclipse, the NetBeans IDE is written entirely in Java. Thus it’s possible to

run it under Windows, Linux, Solaris Unix, and Mac OS X. There are also a wide

range of plug-ins available for NetBeans.

Netbeans is free, and may be downloaded from its home page,

http://netbeans.org/

Its licensing is rather complex, being released under a dual license consisting of

the Common Development and Distribution License (CDDL) and version 2 of the

GNU Public License version 2 (GPLv2). Full details are at

http://netbeans.org/about/legal/license.html

1.1.7 Statistical Computing Environment

Although not strictly necessary, if you want to draw nice plots of your data or

results, or compute p values for some of the statistics LingPipe reports, it helps

to have a statistical computing language on hand. Such languages are typically

interpreted, supported interactive data analysis and plotting.

In some cases, we will provide R code for operations not supported in Ling-

Pipe. All of the graphs drawn in the book were rendered as PDFs in R and in-

cluded as vector graphics.

R Project for Statistical Computing

The most popular open-source statistical computing language is still the R

Project for Statistical Computing (R). R runs under all the major operating sys-

tems, though the Windows installers and compatibility with libraries seem to be

a bit more advanced than those for Linux or Mac OS X.

R implements a dialect of the S programming language for matrices and

statistics. It has good built-in functionality for standard distributions and ma-

trix operations. It also allows linking to C or Fortran back-end implementations.

There are also plug ins (of varying quality) for just about everything else you can

imagine. The oﬃcially sanctioned plug ins are available from the Comprehensive

R Archive Network (CRAN).

R is available from its home page,

8 CHAPTER 1. GETTING STARTED

http://www.r-project.org/

The CRAN mirrors are also linked from the page and usually the ﬁrst web search

results for statistics-related queries in R.

R is distributed under the GNU Public License version 2 (GPLv2).

SciPy and NumPy

The Python programming/scripting language is becoming more and more pop-

ular for both preprocessing data and analyzing statistics. The library NumPy

provides a matrix library on top of Python and SciPy a scientiﬁc library including

statistics and graphing. The place to get started is the home page,

http://numpy.scipy.org/

NumPy and SciPy and Python itself are distributed under the much more ﬂexible

BSD License.

Like R, Python is interpreted and not very fast at operations like looping.

NumPy helps by allowing some operations to be vectorized (as does R). Like R,

Python also allows linking C and Fortran back-end packages for eﬃciency.

1.1.8 Full LingPipe Distribution

LingPipe is distributed under both commercial licenses and under a royalty-free

license. A copy of the royalty free license is available at:

http://alias-i.com/lingpipe/licenses/lingpipe-license-1.txt

LingPipe may be downloaded with full source from its web site:

http://alias-i.com/lingpipe/

Other than unpacking the gzipped tarball, there is nothing required for installa-

tion. Downloading LingPipe is not technically necessary for running the examples

in this book because the LingPipe library jar is included with this book’s source

download.

1.1.9 Book Source and Libraries

The code samples from this book are available via anonymous subversion check-

out from the LingPipe sandbox. Speciﬁcally, the entire content of the book, in-

cluding code and text, may be checked out anonymously using,

> svn co https://aliasi.devguard.com/svn/sandbox/lpbook

The distribution contains all of the code and Ant build ﬁles for running it. It also

contains the L

X source for the book itself as well as the library we created to

extract text from the source for automatic inclusion in the book. This last feature

is actually critical to ensure the code in the distribution and in the book match.

剩余200页未读，继续阅读

huang_owen

粉丝: 4
资源: 2

Java 6文本处理实战

Natural Language Processing with Java

Natural.Language.Processing.with.Java.178439179

Natural Language Processing with Java(PACKT,2015)

Packt.Natural.Language.Processing.with.Java.Cookbook..rar

Developing Language Processing Components with GATE

XML Processing with Perl, Python, and PHP (2002).pdf

Deep Learning: Practical Neural Networks with Java 完整高清英文azw3版

java和java虚拟机发展史思维导图

Java SOAP协议

Notepad++ Text Processing Script: Automating Text Tasks to Enhance Work Efficiency

最新资源