精通GNU Awk：脚本编程指南

需积分: 11 139 浏览量更新于2024-07-25 收藏 2.2MB PDF 举报

"《GAWK: Effective AWK Programming》是GNU Awk用户指南的第四版，由Arnold D. Robbins编写，旨在帮助用户熟练掌握GNU Awk的各种编程技巧。这本书详细介绍了awk语言的用法，是学习和精通awk脚本编程的重要资源。" 在信息技术领域，awk是一种强大的文本分析工具，尤其在处理结构化数据时，它的效率和灵活性深受程序员喜爱。GAWK是GNU项目提供的awk实现，遵循POSIX标准，并提供了更多的功能和扩展。本书《GAWK: Effective AWK Programming》是学习和理解awk语言的权威指南，特别针对GNU Awk版本4.0.0及以上。本书的核心内容可能包括以下几个方面： 1. **基本语法和概念**：讲解awk的基本语法，如模式匹配、动作、字段分隔符以及变量的使用，使读者能够快速上手编写简单的awk脚本。 2. **内置函数和变量**：详细介绍GAWK中的内置函数，如字符串操作函数、数学函数、时间函数等，以及预定义的变量，如字段变量$0和$NF等，帮助用户深入理解awk的功能。 3. **控制结构**：涵盖条件语句（if-else）、循环（for、while）以及函数定义，让读者能够编写复杂的逻辑流程。 4. **模式和动作**：深入解释awk中的模式匹配机制，包括正则表达式以及BEGIN和END规则，使得数据处理更加灵活和精确。 5. **输入和输出**：讨论如何处理输入文件、管道和标准输入，以及如何格式化和打印输出，包括自定义输出格式和记录分离符。 6. **高级特性**：介绍数组、自定义函数、错误处理、多文件处理、Awk程序设计模式等进阶主题，提升awk编程能力。 7. **实际应用示例**：提供丰富的实际问题解决方案，涵盖日志分析、数据转换、报告生成等多种场景，帮助读者将理论知识应用于实践。 8. **GNU Awk特定功能**：详细解析GAWK特有的扩展功能，如数组排序、三元运算符、自定义函数库等，这些特性使GAWK比其他awk实现更强大。 9. **调试和性能优化**：指导如何调试awk脚本，以及如何优化代码以提高性能，确保程序的稳定性和效率。通过阅读和实践《GAWK: Effective AWK Programming》，读者不仅可以掌握awk语言的基础，还能深入理解其高级特性和实际应用，从而在日常工作中更有效地处理文本数据，提升工作效率。同时，由于awk与sed、bash等工具常被一起用于shell脚本编程，熟悉awk也对提升整体系统管理技能大有裨益。

2 GAWK: Eﬀective AWK Programming

to prototype an algorithm or design in AWK to get it running quickly and expose problems

early. Often, the interpreted performance is adequate and the AWK prototype becomes the

product.

The new pgawk (proﬁling gawk), produces program execution counts. I recently exper-

imented with an algorithm that for n lines of input, exhibited ∼ Cn

performance, while

theory predicted ∼ C n log n behavior. A few minutes poring over the ‘awkprof.out’ pro-

ﬁle pinpointed the problem to a single line of code. pgawk is a welcome addition to my

programmer’s toolbox.

Arnold has distilled over a decade of experience writing and using AWK programs, and

developing gawk, into this book. If you use AWK or want to learn how, then read this book.

Michael Brennan

Author of mawk

March, 2001

Preface 3

Preface

Several kinds of tasks occur repeatedly when working with text ﬁles. You might want to

extract certain lines and discard the rest. Or you may need to make changes wherever

certain patterns appear, but leave the rest of the ﬁle alone. Writing single-use programs for

these tasks in languages such as C, C++, or Java is time-consuming and inconvenient. Such

jobs are often easier with awk. The awk utility interprets a special-purpose programming

language that makes it easy to handle simple data-reformatting jobs.

The GNU implementation of awk is called gawk; it is fully compatible with the POSIX

speciﬁcation of the awk language and with the Unix version of awk maintained by Brian

Kernighan. This means that all properly written awk programs should work with gawk.

Thus, we usually don’t distinguish between gawk and other awk implementations.

Using awk allows you to:

• Manage small, personal databases

• Generate reports

• Validate data

• Produce indexes and perform other document preparation tasks

• Experiment with algorithms that you can adapt later to other computer languages

In addition, gawk provides facilities that make it easy to:

• Extract bits and pieces of data for processing

• Sort data

• Perform simple network communications

This book teaches you about the awk language and how you can use it eﬀectively. You

should already be familiar with basic system commands, such as cat and ls,

as well as

basic shell facilities, such as input/output (I/O) redirection and pipes.

Implementations of the awk language are available for many diﬀerent computing en-

vironments. This book, while describing the awk language in general, also describes the

particular implementation of awk called gawk (which stands for “GNU awk”). gawk runs

on a broad range of Unix systems, ranging from Intel



-architecture PC-based computers

up through large-scale systems, such as Crays. gawk has also been ported to Mac OS X,

Microsoft Windows (all versions) and OS/2 PCs, and VMS. (Some other, obsolete systems

to which gawk was once ported are no longer supported and the code for those systems has

been removed.)

History of awk and gawk

Recipe For A Programming Language

1 part egrep 1 part snobol

2 parts ed 3 parts C

The 2008 POSIX standard can be found online at http://www.opengroup.org/onlinepubs/9699919799/.

These commands are available on POSIX-compliant systems, as well as on traditional Unix-based sys-

tems. If you are using some other operating system, you still need to be familiar with the ideas of I/O

redirection and pipes.

4 GAWK: Eﬀective AWK Programming

Blend all parts well using lex and yacc. Document minimally and release.

After eight years, add another part egrep and two more parts C. Document

very well and release.

The name awk comes from the initials of its designers: Alfred V. Aho, Peter J. Wein-

berger and Brian W. Kernighan. The original version of awk was written in 1977 at AT&T

Bell Laboratories. In 1985, a new version made the programming language more powerful,

introducing user-deﬁned functions, multiple input streams, and computed regular expres-

sions. This new version became widely available with Unix System V Release 3.1 (1987).

The version in System V Release 4 (1989) added some new features and cleaned up the

behavior in some of the “dark corners” of the language. The speciﬁcation for awk in the

POSIX Command Language and Utilities standard further clariﬁed the language. Both the

gawk designers and the original Bell Laboratories awk designers provided feedback for the

POSIX speciﬁcation.

Paul Rubin wrote the GNU implementation, gawk, in 1986. Jay Fenlason completed

it, with advice from Richard Stallman. John Woods contributed parts of the code as

well. In 1988 and 1989, David Trueman, with help from me, thoroughly reworked gawk for

compatibility with the newer awk. Circa 1994, I became the primary maintainer. Current

development focuses on bug ﬁxes, performance improvements, standards compliance, and

occasionally, new features.

In May of 1997, J¨urgen Kahrs felt the need for network access from awk, and with a

little help from me, set about adding features to do this for gawk. At that time, he also

wrote the bulk of TCP/IP Internetworking with gawk (a separate document, available as

part of the gawk distribution). His code ﬁnally became part of the main gawk distribution

with gawk version 3.1.

John Haque rewrote the gawk internals, in the process providing an awk-level debugger.

This version became available as gawk version 4.0, in 2011.

See Section A.8 [Major Contributors to gawk], page 307, for a complete list of those who

made important contributions to gawk.

A Rose by Any Other Name

The awk language has evolved over the years. Full details are provided in Appendix A [The

Evolution of the awk Language], page 301. The language described in this book is often

referred to as “new awk” (nawk).

Because of this, there are systems with multiple versions of awk. Some systems have an

awk utility that implements the original version of the awk language and a nawk utility for

the new version. Others have an oawk version for the “old awk” language and plain awk for

the new one. Still others only have one version, which is usually the new one.

All in all, this makes it diﬃcult for you to know which version of awk you should run

when writing your programs. The best advice we can give here is to check your local

documentation. Look for awk, oawk, and nawk, as well as for gawk. It is likely that you

already have some version of new awk on your system, which is what you should use when

running your programs. (Of course, if you’re reading this book, chances are good that you

have gawk!)

Often, these systems use gawk for their awk implementation!

Preface 5

Throughout this book, whenever we refer to a language feature that should be available

in any complete implementation of POSIX awk, we simply use the term awk. When referring

to a feature that is speciﬁc to the GNU implementation, we use the term gawk.

Using This Book

The term awk refers to a particular program as well as to the language you use to tell this

program what to do. When we need to be careful, we call the language “the awk language,”

and the program “the awk utility.” This book explains both how to write programs in the

awk language and how to run the awk utility. The term awk program refers to a program

written by you in the awk programming language.

Primarily, this book explains the features of awk as deﬁned in the POSIX standard. It

does so in the context of the gawk implementation. While doing so, it also attempts to

describe important diﬀerences between gawk and other awk implementations.

Finally, any

gawk features that are not in the POSIX standard for awk are noted.

This book has the diﬃcult task of being both a tutorial and a reference. If you are a

novice, feel free to skip over details that seem too complex. You should also ignore the many

cross-references; they are for the expert user and for the online Info and HTML versions of

the document.

There are subsections labeled as Advanced Notes scattered throughout the book. They

add a more complete explanation of points that are relevant, but not likely to be of interest

on ﬁrst reading. All appear in the index, under the heading “advanced features.”

Most of the time, the examples use complete awk programs. Some of the more advanced

sections show only the part of the awk program that illustrates the concept currently being

described.

While this book is aimed principally at people who have not been exposed to awk, there

is a lot of information here that even the awk expert should ﬁnd useful. In particular,

the description of POSIX awk and the example programs in Chapter 12 [A Library of awk

Functions], page 211, and in Chapter 13 [Practical awk Programs], page 241, should be of

interest.

Chapter 1 [Getting Started with awk], page 11, provides the essentials you need to know

to begin using awk.

Chapter 2 [Running awk and gawk], page 25, describes how to run gawk, the meaning of

its command-line options, and how it ﬁnds awk program source ﬁles.

Chapter 3 [Regular Expressions], page 37, introduces regular expressions in general, and

in particular the ﬂavors supported by POSIX awk and gawk.

Chapter 4 [Reading Input Files], page 49, describes how awk reads your data. It intro-

duces the concepts of records and ﬁelds, as well as the getline command. I/O redirection

is ﬁrst described here. Network I/O is also brieﬂy introduced here.

Chapter 5 [Printing Output], page 73, describes how awk programs can produce output

with print and printf.

Chapter 6 [Expressions], page 89, describes expressions, which are the basic building

blocks for getting most things done in a program.

All such diﬀerences appear in the index under the entry “diﬀerences in awk and gawk.”

6 GAWK: Eﬀective AWK Programming

Chapter 7 [Patterns, Actions, and Variables], page 111, describes how to write patterns

for matching records, actions for doing something when a record is matched, and the built-in

variables awk and gawk use.

Chapter 8 [Arrays in awk], page 135, covers awk’s one-and-only data structure: associa-

tive arrays. Deleting array elements and whole arrays is also described, as well as sorting

arrays in gawk. It also describes how gawk provides arrays of arrays.

Chapter 9 [Functions], page 147, describes the built-in functions awk and gawk provide,

as well as how to deﬁne your own functions.

Chapter 10 [Internationalization with gawk], page 185, describes special features in gawk

for translating program messages into diﬀerent languages at runtime.

Chapter 11 [Advanced Features of gawk], page 195, describes a number of gawk-speciﬁc

advanced features. Of particular note are the abilities to have two-way communications

with another process, perform TCP/IP networking, and proﬁle your awk programs.

Chapter 12 [A Library of awk Functions], page 211, and Chapter 13 [Practical awk

Programs], page 241, provide many sample awk programs. Reading them allows you to see

awk solving real problems.

Chapter 14 [dgawk: The awk Debugger], page 285, describes the awk debugger, dgawk.

Appendix A [The Evolution of the awk Language], page 301, describes how the awk

language has evolved since its ﬁrst release to present. It also describes how gawk has

acquired features over time.

Appendix B [Installing gawk], page 309, describes how to get gawk, how to compile it on

POSIX-compatible systems, and how to compile and use it on diﬀerent non-POSIX systems.

It also describes how to report bugs in gawk and where to get other freely available awk

implementations.

Appendix C [Implementation Notes], page 325, describes how to disable gawk’s exten-

sions, as well as how to contribute new code to gawk, how to write extension libraries, and

some possible future directions for gawk development.

Appendix D [Basic Programming Concepts], page 341, provides some very cursory back-

ground material for those who are completely unfamiliar with computer programming. Also

centralized there is a discussion of some of the issues surrounding ﬂoating-point numbers.

The [Glossary], page 347, deﬁnes most, if not all, the signiﬁcant terms used throughout

the book. If you ﬁnd terms that you aren’t familiar with, try looking them up here.

[GNU General Public License], page 357, and [GNU Free Documentation License],

page 369, present the licenses that cover the gawk source code and this book, respectively.

Typographical Conventions

This book is written in Texinfo, the GNU documentation formatting language. A single

Texinfo source ﬁle is used to produce both the printed and online versions of the documen-

tation. Because of this, the typographical conventions are slightly diﬀerent than in other

books you may have read.

Examples you would type at the command-line are preceded by the common shell pri-

mary and secondary prompts, ‘$’ and ‘>’. Input that you type is shown like this. Output

from the command is preceded by the glyph “

”. This typically represents the command’s

剩余409页未读，继续阅读

skysmiler

粉丝: 1
资源: 3

精通GNU Awk：脚本编程指南

Gawk Effective awk Programming

effective awk programming

高效awk编程

GAWK: Effective AWK Programming；AWK说明书

GAWK-Effective AWK Programming - GNU Awk编程经典

GAWK：Effective AWK Programming Edition 4.2

GAWK- Effective AWK Programming4.2.pdf

GNU Awk编程指南：GAWK-Effective AWK Programming

Effective AWK Programming.pdf

GAWK：Effective AWK 编程指南

最新资源