精通GAWK：GNU Awk编程指南

需积分: 11 44 浏览量更新于2024-07-29 收藏 2.2MB PDF 举报

GAWK: Effective AWK Programming 是一本专为GNU Awk编写的用户指南，由Arnold D. Robbins编写，于2011年6月发布，适用于GNU Awk 4.0.0及更高版本。这本书是该领域的权威指南，旨在帮助读者掌握AWK语言的有效编程技巧。 AWK是一种强大的文本处理工具，它结合了正则表达式、数据流处理和编程功能，广泛应用于数据分析、脚本编写和文件转换等场景。本书的核心内容包括但不限于以下知识点： 1. **AWK语言基础**：介绍了AWK语言的基本语法，如变量、常量、控制结构（如if-else语句、循环）、算术运算和字符串操作等，帮助读者快速上手AWK。 2. **正则表达式**：书中深入讲解了AWK中的正则表达式规则，这对于处理文本中的模式匹配至关重要，包括特殊字符、重复符、分组和引用等。 3. **流程控制**：讨论了AWK的流程控制语句，如BEGIN、ACTION和END块，以及条件控制（NR、NF等变量）的应用，如何根据输入灵活地控制程序执行。 4. **函数与模块**：介绍了如何定义和使用内置函数，以及如何创建自定义函数，增强程序的复用性和可维护性。 5. **文件处理**：讲述了如何读取、写入和操作文件，以及处理多行和字段数据的方法，这对于数据清洗和转换非常有用。 6. **数组和记录处理**：讲解了数组在AWK中的使用，包括索引数组和关联数组，以及如何处理记录（即行）的分割和合并。 7. **性能优化**：针对大规模数据处理，书中分享了提高AWK性能的策略，如减少内存使用、优化循环结构和利用内建函数。 8. **示例与实践**：书中包含大量的实战案例，通过具体的任务演示如何运用所学知识，帮助读者理解和应用AWK。 9. **版权与许可**：强调了GNU Free Documentation License (FDL)的使用，确保读者可以自由地复制、分发和修改文档，符合开源软件的精神。这本手册是GAWK程序员的宝典，无论是初学者还是经验丰富的用户，都能从中找到提升技能和解决问题的资源。对于从事数据处理、自动化脚本编写或系统管理的IT专业人士来说，熟练掌握GAWK能极大地提高工作效率。

2 GAWK: Eﬀective AWK Programming

to prototype an algorithm or design in AWK to get it running quickly and expose problems

early. Often, the interpreted performance is adequate and the AWK prototype becomes the

product.

The new pgawk (proﬁling gawk), produces program execution counts. I recently exper-

imented with an algorithm that for n lines of input, exhibited ∼ Cn

performance, while

theory predicted ∼ C n log n behavior. A few minutes poring over the ‘awkprof.out’ pro-

ﬁle pinpointed the problem to a single line of code. pgawk is a welcome addition to my

programmer’s toolbox.

Arnold has distilled over a decade of experience writing and using AWK programs, and

developing gawk, into this book. If you use AWK or want to learn how, then read this book.

Michael Brennan

Author of mawk

March, 2001

Preface 3

Preface

Several kinds of tasks occur repeatedly when working with text ﬁles. You might want to

extract certain lines and discard the rest. Or you may need to make changes wherever

certain patterns appear, but leave the rest of the ﬁle alone. Writing single-use programs for

these tasks in languages such as C, C++, or Java is time-consuming and inconvenient. Such

jobs are often easier with awk. The awk utility interprets a special-purpose programming

language that makes it easy to handle simple data-reformatting jobs.

The GNU implementation of awk is called gawk; it is fully compatible with the POSIX

speciﬁcation of the awk language and with the Unix version of awk maintained by Brian

Kernighan. This means that all properly written awk programs should work with gawk.

Thus, we usually don’t distinguish between gawk and other awk implementations.

Using awk allows you to:

• Manage small, personal databases

• Generate reports

• Validate data

• Produce indexes and perform other document preparation tasks

• Experiment with algorithms that you can adapt later to other computer languages

In addition, gawk provides facilities that make it easy to:

• Extract bits and pieces of data for processing

• Sort data

• Perform simple network communications

This book teaches you about the awk language and how you can use it eﬀectively. You

should already be familiar with basic system commands, such as cat and ls,

as well as

basic shell facilities, such as input/output (I/O) redirection and pipes.

Implementations of the awk language are available for many diﬀerent computing en-

vironments. This book, while describing the awk language in general, also describes the

particular implementation of awk called gawk (which stands for “GNU awk”). gawk runs

on a broad range of Unix systems, ranging from Intel



-architecture PC-based computers

up through large-scale systems, such as Crays. gawk has also been ported to Mac OS X,

Microsoft Windows (all versions) and OS/2 PCs, and VMS. (Some other, obsolete systems

to which gawk was once ported are no longer supported and the code for those systems has

been removed.)

History of awk and gawk

Recipe For A Programming Language

1 part egrep 1 part snobol

2 parts ed 3 parts C

The 2008 POSIX standard can be found online at http://www.opengroup.org/onlinepubs/9699919799/.

These commands are available on POSIX-compliant systems, as well as on traditional Unix-based sys-

tems. If you are using some other operating system, you still need to be familiar with the ideas of I/O

redirection and pipes.

4 GAWK: Eﬀective AWK Programming

Blend all parts well using lex and yacc. Document minimally and release.

After eight years, add another part egrep and two more parts C. Document

very well and release.

The name awk comes from the initials of its designers: Alfred V. Aho, Peter J. Wein-

berger and Brian W. Kernighan. The original version of awk was written in 1977 at AT&T

Bell Laboratories. In 1985, a new version made the programming language more powerful,

introducing user-deﬁned functions, multiple input streams, and computed regular expres-

sions. This new version became widely available with Unix System V Release 3.1 (1987).

The version in System V Release 4 (1989) added some new features and cleaned up the

behavior in some of the “dark corners” of the language. The speciﬁcation for awk in the

POSIX Command Language and Utilities standard further clariﬁed the language. Both the

gawk designers and the original Bell Laboratories awk designers provided feedback for the

POSIX speciﬁcation.

Paul Rubin wrote the GNU implementation, gawk, in 1986. Jay Fenlason completed

it, with advice from Richard Stallman. John Woods contributed parts of the code as

well. In 1988 and 1989, David Trueman, with help from me, thoroughly reworked gawk for

compatibility with the newer awk. Circa 1994, I became the primary maintainer. Current

development focuses on bug ﬁxes, performance improvements, standards compliance, and

occasionally, new features.

In May of 1997, J¨urgen Kahrs felt the need for network access from awk, and with a

little help from me, set about adding features to do this for gawk. At that time, he also

wrote the bulk of TCP/IP Internetworking with gawk (a separate document, available as

part of the gawk distribution). His code ﬁnally became part of the main gawk distribution

with gawk version 3.1.

John Haque rewrote the gawk internals, in the process providing an awk-level debugger.

This version became available as gawk version 4.0, in 2011.

See Section A.8 [Major Contributors to gawk], page 307, for a complete list of those who

made important contributions to gawk.

A Rose by Any Other Name

The awk language has evolved over the years. Full details are provided in Appendix A [The

Evolution of the awk Language], page 301. The language described in this book is often

referred to as “new awk” (nawk).

Because of this, there are systems with multiple versions of awk. Some systems have an

awk utility that implements the original version of the awk language and a nawk utility for

the new version. Others have an oawk version for the “old awk” language and plain awk for

the new one. Still others only have one version, which is usually the new one.

All in all, this makes it diﬃcult for you to know which version of awk you should run

when writing your programs. The best advice we can give here is to check your local

documentation. Look for awk, oawk, and nawk, as well as for gawk. It is likely that you

already have some version of new awk on your system, which is what you should use when

running your programs. (Of course, if you’re reading this book, chances are good that you

have gawk!)

Often, these systems use gawk for their awk implementation!

Preface 5

Throughout this book, whenever we refer to a language feature that should be available

in any complete implementation of POSIX awk, we simply use the term awk. When referring

to a feature that is speciﬁc to the GNU implementation, we use the term gawk.

Using This Book

The term awk refers to a particular program as well as to the language you use to tell this

program what to do. When we need to be careful, we call the language “the awk language,”

and the program “the awk utility.” This book explains both how to write programs in the

awk language and how to run the awk utility. The term awk program refers to a program

written by you in the awk programming language.

Primarily, this book explains the features of awk as deﬁned in the POSIX standard. It

does so in the context of the gawk implementation. While doing so, it also attempts to

describe important diﬀerences between gawk and other awk implementations.

Finally, any

gawk features that are not in the POSIX standard for awk are noted.

This book has the diﬃcult task of being both a tutorial and a reference. If you are a

novice, feel free to skip over details that seem too complex. You should also ignore the many

cross-references; they are for the expert user and for the online Info and HTML versions of

the document.

There are subsections labeled as Advanced Notes scattered throughout the book. They

add a more complete explanation of points that are relevant, but not likely to be of interest

on ﬁrst reading. All appear in the index, under the heading “advanced features.”

Most of the time, the examples use complete awk programs. Some of the more advanced

sections show only the part of the awk program that illustrates the concept currently being

described.

While this book is aimed principally at people who have not been exposed to awk, there

is a lot of information here that even the awk expert should ﬁnd useful. In particular,

the description of POSIX awk and the example programs in Chapter 12 [A Library of awk

Functions], page 211, and in Chapter 13 [Practical awk Programs], page 241, should be of

interest.

Chapter 1 [Getting Started with awk], page 11, provides the essentials you need to know

to begin using awk.

Chapter 2 [Running awk and gawk], page 25, describes how to run gawk, the meaning of

its command-line options, and how it ﬁnds awk program source ﬁles.

Chapter 3 [Regular Expressions], page 37, introduces regular expressions in general, and

in particular the ﬂavors supported by POSIX awk and gawk.

Chapter 4 [Reading Input Files], page 49, describes how awk reads your data. It intro-

duces the concepts of records and ﬁelds, as well as the getline command. I/O redirection

is ﬁrst described here. Network I/O is also brieﬂy introduced here.

Chapter 5 [Printing Output], page 73, describes how awk programs can produce output

with print and printf.

Chapter 6 [Expressions], page 89, describes expressions, which are the basic building

blocks for getting most things done in a program.

All such diﬀerences appear in the index under the entry “diﬀerences in awk and gawk.”

6 GAWK: Eﬀective AWK Programming

Chapter 7 [Patterns, Actions, and Variables], page 111, describes how to write patterns

for matching records, actions for doing something when a record is matched, and the built-in

variables awk and gawk use.

Chapter 8 [Arrays in awk], page 135, covers awk’s one-and-only data structure: associa-

tive arrays. Deleting array elements and whole arrays is also described, as well as sorting

arrays in gawk. It also describes how gawk provides arrays of arrays.

Chapter 9 [Functions], page 147, describes the built-in functions awk and gawk provide,

as well as how to deﬁne your own functions.

Chapter 10 [Internationalization with gawk], page 185, describes special features in gawk

for translating program messages into diﬀerent languages at runtime.

Chapter 11 [Advanced Features of gawk], page 195, describes a number of gawk-speciﬁc

advanced features. Of particular note are the abilities to have two-way communications

with another process, perform TCP/IP networking, and proﬁle your awk programs.

Chapter 12 [A Library of awk Functions], page 211, and Chapter 13 [Practical awk

Programs], page 241, provide many sample awk programs. Reading them allows you to see

awk solving real problems.

Chapter 14 [dgawk: The awk Debugger], page 285, describes the awk debugger, dgawk.

Appendix A [The Evolution of the awk Language], page 301, describes how the awk

language has evolved since its ﬁrst release to present. It also describes how gawk has

acquired features over time.

Appendix B [Installing gawk], page 309, describes how to get gawk, how to compile it on

POSIX-compatible systems, and how to compile and use it on diﬀerent non-POSIX systems.

It also describes how to report bugs in gawk and where to get other freely available awk

implementations.

Appendix C [Implementation Notes], page 325, describes how to disable gawk’s exten-

sions, as well as how to contribute new code to gawk, how to write extension libraries, and

some possible future directions for gawk development.

Appendix D [Basic Programming Concepts], page 341, provides some very cursory back-

ground material for those who are completely unfamiliar with computer programming. Also

centralized there is a discussion of some of the issues surrounding ﬂoating-point numbers.

The [Glossary], page 347, deﬁnes most, if not all, the signiﬁcant terms used throughout

the book. If you ﬁnd terms that you aren’t familiar with, try looking them up here.

[GNU General Public License], page 357, and [GNU Free Documentation License],

page 369, present the licenses that cover the gawk source code and this book, respectively.

Typographical Conventions

This book is written in Texinfo, the GNU documentation formatting language. A single

Texinfo source ﬁle is used to produce both the printed and online versions of the documen-

tation. Because of this, the typographical conventions are slightly diﬀerent than in other

books you may have read.

Examples you would type at the command-line are preceded by the common shell pri-

mary and secondary prompts, ‘$’ and ‘>’. Input that you type is shown like this. Output

from the command is preceded by the glyph “

”. This typically represents the command’s

剩余409页未读，继续阅读

profove

粉丝: 1
资源: 17

精通GAWK：GNU Awk编程指南

GAWK: Effective AWK Programming；AWK说明书

GAWK：Effective AWK Programming Edition 4.2

GAWK-Effective AWK Programming - GNU Awk编程经典

如何使用GAWK（GNU Awk）进行基本的文本数据处理和模式匹配？

如何利用GAWK进行文本文件中的关键字搜索并输出匹配行？请提供具体的命令行示例。

如何通过GAWK编程实现对文本文件中的数据进行统计计数以及条件筛选？

gawk: error while loading shared libraries: libmpfr.so.6: cannot open shared object file: No such file or directory

configure: error: *** These critical programs are missing or too old: gawk *** Check the INSTALL file for required versions.

使用gawk能否接受外界的变量

awk在shell脚本中运行报错出现awk未找到

最新资源

configure: error: * These critical programs are missing or too old: gawk * Check the INSTALL file for required versions.