NumPy用户指南：矩阵操作与性能优化

1星需积分: 50 188 浏览量更新于2024-07-20 收藏 399KB PDF 举报

"numpy官方用户指南手册1.8.1.pdf" NumPy是Python中的一个核心库，专为处理多维数据而设计。它提供了一个高效的数据结构，即ndarray（n-dimensional array），用于存储和操作大型数组。NumPy库不仅支持基本的数学运算，还包含各种高级数学功能，如线性代数、傅立叶变换和随机数生成等。这个官方用户指南手册1.8.1版本详细介绍了NumPy的各个方面。 1. **介绍** - **什么是NumPy？** NumPy是Python科学计算的基础，它提供了高性能的多维数组对象以及一系列用于操作这些数组的工具。 - **构建与安装NumPy** 指南中涵盖了如何在不同的操作系统上安装NumPy，包括从源代码编译安装的方法。 - **查找文档** 提供了如何访问和使用NumPy的官方文档，以便于用户学习和解决问题。 2. **NumPy基础** - **数据类型（Datatypes）** NumPy支持多种数据类型，如整型、浮点型、复数、布尔型等，每种类型都有不同的字节大小和精度。 - **数组创建（Array creation）** 通过各种方法创建数组，如`numpy.array()`，`numpy.zeros()`，`numpy.ones()`等。 - **输入/输出（I/O with Numpy）** 学习如何使用NumPy读取和写入数据到文件，如使用`numpy.save()`和`numpy.load()`。 - **索引（Indexing）** 包括切片、整数索引、布尔索引以及轴向索引，以及高级索引的概念。 - **广播（Broadcasting）** 当操作的数组形状不匹配时，NumPy会自动进行广播，使得操作能够进行。 - **字节交换（Byte-swapping）** 当遇到字节顺序问题时，NumPy提供了函数来改变数组元素的字节顺序。 - **结构化数组（Structured arrays, aka “Record arrays”）** 这种数组允许在一个数组中存储多种类型的数据，类似于数据库的记录。 - **子类化ndarray** NumPy允许创建ndarray的子类，以实现自定义行为。 3. **性能** - 本部分可能涵盖优化NumPy代码的方法，如使用内存对齐、减少复制和优化计算效率。 4. **其他（Miscellaneous）** - **IEEE 754浮点特殊值**：讨论浮点数的无穷大、非数字(NaN)和其他特殊值的处理。 - **NumPy如何处理数值异常**：解释了如何控制浮点数运算中的异常，如除以零或溢出。 - **示例**：提供各种使用NumPy的代码示例，帮助理解概念。 - **与C、Fortran和C++接口**：介绍如何在NumPy中使用C、Fortran和C++扩展功能。 - **方法与函数**：解释了NumPy中方法和函数的区别，以及何时使用它们。 5. **使用NumPy C-API** - **如何扩展NumPy**：详细介绍了如何使用NumPy的C API来创建自定义的数组操作函数。 - **使用Python作为胶水**：讲解如何将Python代码与C扩展结合，利用NumPy的优势。 - **编写自己的ufunc**：ufunc（通用函数）是NumPy中的一个关键特性，可以处理数组的元素级运算，这里讲述了如何创建自定义ufunc。 - **超越基础**：这部分深入到更复杂的NumPy用法和技巧。此用户指南是NumPy初学者和进阶用户的宝贵资源，涵盖了从安装到高级编程的全部内容，确保用户能充分利用NumPy的强大功能。

NumPy User Guide, Release 1.8.1

and types

>>> x = np.array([[ 1.+0.j, 2.+0.j], [ 0.+0.j, 0.+0.j], [ 1.+1.j, 3.+0.j]])

2.2.3 Intrinsic Numpy Array Creation

Numpy has built-in functions for creating arrays from scratch:

zeros(shape) will create an array ﬁlled with 0 values with the speciﬁed shape. The default dtype is ﬂoat64.

>>> np.zeros((2, 3)) array([[ 0., 0., 0.], [ 0., 0., 0.]])

ones(shape) will create an array ﬁlled with 1 values. It is identical to zeros in all other respects.

arange() will create arrays with regularly incrementing values. Check the docstring for complete information on the

various ways it can be used. A few examples will be given here:

>>> np.arange(10)

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> np.arange(2, 10, dtype=np.float)

array([ 2., 3., 4., 5., 6., 7., 8., 9.])

>>> np.arange(2, 3, 0.1)

array([ 2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9])

Note that there are some subtleties regarding the last usage that the user should be aware of that are described in the

arange docstring.

linspace() will create arrays with a speciﬁed number of elements, and spaced equally between the speciﬁed beginning

and end values. For example:

>>> np.linspace(1., 4., 6)

array([ 1. , 1.6, 2.2, 2.8, 3.4, 4. ])

The advantage of this creation function is that one can guarantee the number of elements and the starting and end

point, which arange() generally will not do for arbitrary start, stop, and step values.

indices() will create a set of arrays (stacked as a one-higher dimensioned array), one per dimension with each repre-

senting variation in that dimension. An example illustrates much better than a verbal description:

>>> np.indices((3,3))

array([[[0, 0, 0], [1, 1, 1], [2, 2, 2]], [[0, 1, 2], [0, 1, 2], [0, 1, 2]]])

This is particularly useful for evaluating functions of multiple dimensions on a regular grid.

2.2.4 Reading Arrays From Disk

This is presumably the most common case of large array creation. The details, of course, depend greatly on the format

of data on disk and so this section can only give general pointers on how to handle various formats.

Standard Binary Formats

Various ﬁelds have standard formats for array data. The following lists the ones with known python libraries to read

them and return numpy arrays (there may be others for which it is possible to read and convert to numpy arrays so

check the last section as well)

HDF5: PyTables

FITS: PyFITS

12 Chapter 2. Numpy basics

NumPy User Guide, Release 1.8.1

Examples of formats that cannot be read directly but for which it is not hard to convert are those formats supported by

libraries like PIL (able to read and write many image formats such as jpg, png, etc).

Common ASCII Formats

Comma Separated Value ﬁles (CSV) are widely used (and an export and import option for programs like Excel). There

are a number of ways of reading these ﬁles in Python. There are CSV functions in Python and functions in pylab (part

of matplotlib).

More generic ascii ﬁles can be read using the io package in scipy.

Custom Binary Formats

There are a variety of approaches one can use. If the ﬁle has a relatively simple format then one can write a simple

I/O library and use the numpy fromﬁle() function and .toﬁle() method to read and write numpy arrays directly (mind

your byteorder though!) If a good C or C++ library exists that read the data, one can wrap that library with a variety of

techniques though that certainly is much more work and requires signiﬁcantly more advanced knowledge to interface

with C or C++.

Use of Special Libraries

There are libraries that can be used to generate arrays for special purposes and it isn’t possible to enumerate all of

them. The most common uses are use of the many array generation functions in random that can generate arrays of

random values, and some utility functions to generate special matrices (e.g. diagonal).

2.3 I/O with Numpy

2.3.1 Importing data with genfromtxt

Numpy provides several functions to create arrays from tabular data. We focus here on the genfromtxt function.

In a nutshell, genfromtxt runs two main loops. The ﬁrst loop converts each line of the ﬁle in a sequence of strings.

The second loop converts each string to the appropriate data type. This mechanism is slower than a single loop, but

gives more ﬂexibility. In particular, genfromtxt is able to take missing data into account, when other faster and

simpler functions like loadtxt cannot.

Note: When giving examples, we will use the following conventions:

>>> import numpy as np

>>> from StringIO import StringIO

Deﬁning the input

The only mandatory argument of genfromtxt is the source of the data. It can be a string corresponding to the name

of a local or remote ﬁle, or a ﬁle-like object with a read method (such as an actual ﬁle or a StringIO.StringIO

object). If the argument is the URL of a remote ﬁle, this latter is automatically downloaded in the current directory.

The input ﬁle can be a text ﬁle or an archive. Currently, the function recognizes gzip and bz2 (bzip2) archives. The

type of the archive is determined by examining the extension of the ﬁle: if the ﬁlename ends with ’.gz’, a gzip

archive is expected; if it ends with ’bz2’, a bzip2 archive is assumed.

2.3. I/O with Numpy 13

NumPy User Guide, Release 1.8.1

Splitting the lines into columns

The delimiter argument

Once the ﬁle is deﬁned and open for reading, genfromtxt splits each non-empty line into a sequence of strings.

Empty or commented lines are just skipped. The delimiter keyword is used to deﬁne how the splitting should take

place.

Quite often, a single character marks the separation between columns. For example, comma-separated ﬁles (CSV) use

a comma (,) or a semicolon (;) as delimiter:

>>> data = "1, 2, 3\n4, 5, 6"

>>> np.genfromtxt(StringIO(data), delimiter=",")

array([[ 1., 2., 3.],

[ 4., 5., 6.]])

Another common separator is "\t", the tabulation character. However, we are not limited to a single character, any

string will do. By default, genfromtxt assumes delimiter=None, meaning that the line is split along white

spaces (including tabs) and that consecutive white spaces are considered as a single white space.

Alternatively, we may be dealing with a ﬁxed-width ﬁle, where columns are deﬁned as a given number of characters.

In that case, we need to set delimiter to a single integer (if all the columns have the same size) or to a sequence of

integers (if columns can have different sizes):

>>> data = " 1 2 3\n 4 5 67\n890123 4"

>>> np.genfromtxt(StringIO(data), delimiter=3)

array([[ 1., 2., 3.],

[ 4., 5., 67.],

[ 890., 123., 4.]])

>>> data = "123456789\n 4 7 9\n 4567 9"

>>> np.genfromtxt(StringIO(data), delimiter=(4, 3, 2))

array([[ 1234., 567., 89.],

[ 4., 7., 9.],

[ 4., 567., 9.]])

The autostrip argument

By default, when a line is decomposed into a series of strings, the individual entries are not stripped of leading nor

trailing white spaces. This behavior can be overwritten by setting the optional argument autostrip to a value of

True:

>>> data = "1, abc , 2\n 3, xxx, 4"

>>> # Without autostrip

>>> np.genfromtxt(StringIO(data), dtype="|S5")

array([[’1’, ’ abc ’, ’ 2’],

[’3’, ’ xxx’, ’ 4’]],

dtype=’|S5’)

>>> # With autostrip

>>> np.genfromtxt(StringIO(data), dtype="|S5", autostrip=True)

array([[’1’, ’abc’, ’2’],

[’3’, ’xxx’, ’4’]],

dtype=’|S5’)

The comments argument

The optional argument comments is used to deﬁne a character string that marks the beginning of a comment. By

default, genfromtxt assumes comments=’#’. The comment marker may occur anywhere on the line. Any

character present after the comment marker(s) is simply ignored:

14 Chapter 2. Numpy basics

NumPy User Guide, Release 1.8.1

>>> data = """#

... # Skip me !

... # Skip me too !

... 1, 2

... 3, 4

... 5, 6 #This is the third line of the data

... 7, 8

... # And here comes the last line

... 9, 0

... """

>>> np.genfromtxt(StringIO(data), comments="#", delimiter=",")

[[ 1. 2.]

[ 3. 4.]

[ 5. 6.]

[ 7. 8.]

[ 9. 0.]]

Note: There is one notable exception to this behavior: if the optional argument names=True, the ﬁrst commented

line will be examined for names.

Skipping lines and choosing columns

The skip_header and skip_footer arguments

The presence of a header in the ﬁle can hinder data processing. In that case, we need to use the skip_header

optional argument. The values of this argument must be an integer which corresponds to the number of lines to skip

at the beginning of the ﬁle, before any other action is performed. Similarly, we can skip the last n lines of the ﬁle by

using the skip_footer attribute and giving it a value of n:

>>> data = "\n".join(str(i) for i in range(10))

>>> np.genfromtxt(StringIO(data),)

array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.])

>>> np.genfromtxt(StringIO(data),

... skip_header=3, skip_footer=5)

array([ 3., 4.])

By default, skip_header=0 and skip_footer=0, meaning that no lines are skipped.

The usecols argument

In some cases, we are not interested in all the columns of the data but only a few of them. We can select which

columns to import with the usecols argument. This argument accepts a single integer or a sequence of integers

corresponding to the indices of the columns to import. Remember that by convention, the ﬁrst column has an index of

0. Negative integers behave the same as regular Python negative indexes.

For example, if we want to import only the ﬁrst and the last columns, we can use usecols=(0, -1):

>>> data = "1 2 3\n4 5 6"

>>> np.genfromtxt(StringIO(data), usecols=(0, -1))

array([[ 1., 3.],

[ 4., 6.]])

If the columns have names, we can also select which columns to import by giving their name to the usecols

argument, either as a sequence of strings or a comma-separated string:

2.3. I/O with Numpy 15

NumPy User Guide, Release 1.8.1

>>> data = "1 2 3\n4 5 6"

>>> np.genfromtxt(StringIO(data),

... names="a, b, c", usecols=("a", "c"))

array([(1.0, 3.0), (4.0, 6.0)],

dtype=[(’a’, ’<f8’), (’c’, ’<f8’)])

>>> np.genfromtxt(StringIO(data),

... names="a, b, c", usecols=("a, c"))

array([(1.0, 3.0), (4.0, 6.0)],

dtype=[(’a’, ’<f8’), (’c’, ’<f8’)])

Choosing the data type

The main way to control how the sequences of strings we have read from the ﬁle are converted to other types is to set

the dtype argument. Acceptable values for this argument are:

• a single type, such as dtype=float. The output will be 2D with the given dtype, unless a name has been

associated with each column with the use of the names argument (see below). Note that dtype=float is the

default for genfromtxt.

• a sequence of types, such as dtype=(int, float, float).

• a comma-separated string, such as dtype="i4,f8,|S3".

• a dictionary with two keys ’names’ and ’formats’.

• a sequence of tuples (name, type), such as dtype=[(’A’, int), (’B’, float)].

• an existing numpy.dtype object.

• the special value None. In that case, the type of the columns will be determined from the data itself (see below).

In all the cases but the ﬁrst one, the output will be a 1D array with a structured dtype. This dtype has as many ﬁelds as

items in the sequence. The ﬁeld names are deﬁned with the names keyword.

When dtype=None, the type of each column is determined iteratively from its data. We start by checking whether a

string can be converted to a boolean (that is, if the string matches true or false in lower cases); then whether it can

be converted to an integer, then to a ﬂoat, then to a complex and eventually to a string. This behavior may be changed

by modifying the default mapper of the StringConverter class.

The option dtype=None is provided for convenience. However, it is signiﬁcantly slower than setting the dtype

explicitly.

Setting the names

The names argument

A natural approach when dealing with tabular data is to allocate a name to each column. A ﬁrst possibility is to use an

explicit structured dtype, as mentioned previously:

>>> data = StringIO("1 2 3\n 4 5 6")

>>> np.genfromtxt(data, dtype=[(_, int) for _ in "abc"])

array([(1, 2, 3), (4, 5, 6)],

dtype=[(’a’, ’<i8’), (’b’, ’<i8’), (’c’, ’<i8’)])

Another simpler possibility is to use the names keyword with a sequence of strings or a comma-separated string:

>>> data = StringIO("1 2 3\n 4 5 6")

>>> np.genfromtxt(data, names="A, B, C")

16 Chapter 2. Numpy basics

剩余108页未读，继续阅读

宁静以致墨

粉丝: 787

NumPy用户指南：矩阵操作与性能优化

numpy-1.10.1.api参考手册.chm

《Numpy 中文用户指南》PDF

numpy中文文档.zip

numpy源码 numpy-1.8.1.tar.gz

numpy-MKL-1.8.1.win32-py3.4.exe

numpy-1.8.1.zip

Numpy：科学计算的基石.pdf

numpy-user-v1.7.0.pdf

numpy完全详解--jalen.pdf

NumPy中ndarray索引的使用.pdf

最新资源