Python可视化入门：数据处理与图形制作指南

需积分: 10 31 浏览量更新于2024-07-17 收藏 7.02MB PDF 举报

《Python可视化入门：打造视觉数据转换脚本》是一本由Apress出版的综合文档，旨在引导读者探索和掌握Python在数据可视化领域的应用。本书以作者个人对数学和计算机的热爱为背景，特别关注了这两个领域的交叉——数值分析，该领域将数学理论与计算机技术紧密结合。本书共分为十个章节，以实用和理论相结合的方式展开教学： 1. **第一章：探索数据可视化的世界** - 通过这个章节，读者将了解数据可视化的基本概念和重要性，以及如何在实际问题中选择合适的可视化工具。 2. **第二章：环境设置** - 讲解Python环境的配置，包括必要的软件安装、库的导入以及如何创建一个适合数据可视化的开发环境。 3. **第三章：Python编程基础** - 针对程序员，介绍了Python语言的基础知识，如变量、数据类型、控制结构等，以便后续章节进行更深入的可视化操作。 4. **第四章：数据组织** - 学习如何有效地组织和处理数据，包括数据清洗、数据结构的选择，为可视化做好数据准备。 5. **第五章：文本文件处理** - 介绍如何使用Python处理文本文件，这对于数据预处理是至关重要的步骤。 6. **第六章：图形与图表** - 进一步讲解各种图表类型（如折线图、柱状图、散点图等）的创建和定制，帮助读者理解和展示数据的模式。 7. **第七章：数学游戏与教育应用** - 通过有趣的例子展示数学在可视化中的应用，提升学习兴趣，同时教授基本的数学概念。 8. **第八章：科学与可视化** - 探讨科学实验中的可视化技术，如何将复杂的数据和理论结果以直观的方式呈现出来。 9. **第九章：图像处理** - 学习如何使用Python进行图像分析和编辑，这对于需要处理图像数据的可视化场景至关重要。 10. **第十章：高级文件处理** - 对于进阶用户，章节涵盖了更复杂的文件操作和数据流处理技术，提升数据处理的效率。附录中提供了额外的代码源代码清单，供读者参考和实践。全书最后还有一份详细的索引，方便查找特定主题。《Python可视化入门》是一本结合作者个人经验，从基础知识到实际应用的全面指南，不仅适合初学者入门，也适合有一定编程基础的读者深化理解并提升数据可视化技能。

CHAPTER 1 ■ NAVIGATING THE WORLD OF DATA VISUALIZATION

len(x) tells us the size of the array of lists. It’s also a crude way for us to ensure that data was actually read into

the array.

The second line in the function is called a docstring, and it is characterized by three quotes (""") surrounding the

text in the following manner: """dosctring""". In this case, a docstring is used to document the function; that is, it

enables us to explain what it does. Issuing the command help(funcname) yields its docstring:

>>> help(read_csv_file)

Help on function read_csv_file in module __main__:

read_csv_file(filename)

Reads a CSV file and return it as a list of rows.

You should use help() whenever you need a reminder of what a function does. help() can be invoked with

functions as well as modules. For example, the following invokes help on module csv:

>>> help(csv)

Help on module csv:

NAME

csv - CSV parsing and writing.

DESCRIPTION

This module provides classes that assist in the reading and writing

of Comma Separated Value (CSV) files, and implements the interface

described by PEP 305. Although many CSV files are simple to parse,

the format is not formally defined by a stable specification and

is subtle enough that parsing lines of a CSV file with something

like line.split(",") is bound to fail. The module supports three

basic APIs: reading, writing, and registration of dialects.

The line data = [] declares a variable named data and initializes it as an empty list. We will use data to store the

values from the CSV file.

The csv module helps us read CSV files by automating a lot of the tasks associated with reading them. I will

discuss CSV files and the csv module in more detail in Chapters 4 and 5; this chapter will only provide an overview.

Here are the steps for reading CSV files with the csv module:

1. Open the file for reading.

2. Create a csv.reader object. The csv.reader object has functions that help us read CSV files.

a. Using the csv.reader object, read the data from the file, a row at a time.

b. Append every row to the variable data.

3. Close the file.

CHAPTER 1 ■ NAVIGATING THE WORLD OF DATA VISUALIZATION

Let’s try this, a step at a time:

>>> import csv

>>> f = open('../data/GPS-2008-06-04-09-03-45.csv')

>>> cr = csv.reader(f)

>>> for row in cr:

... print(row)

['$GPGSA', 'A', '3', '21', '18', '15', '24', '', '22', '', '', '', '', '', '',

'03.5', '02.2', '02.7*09']

['$GPGSV', '3', '1', '12', '21', '86', '267', '39', '18', '66', '286', '44',

'15', '51', '060', '43', '24', '45', '125', '30*7A']

['$GPGSV', '3', '2', '12', '06', '28', '300', '33', '22', '27', '265', '31',

'03', '18', '312', '27', '29', '15', '185', '31*7C']

['$GPGSV', '3', '3', '12', '09', '15', '138', '31', '16', '00', '301', '',

'19', '00', '332', '', '00', '00', '000', '*70']

['$GPRMC', '140706.24', 'A', '4455.6241', 'N', '09328.0519', 'W', '011.4',

'152.7', '040608', '001.2', 'E', 'A*25']

['$GPGGA', '140706.24', '4455.6241', 'N', '09328.0519', 'W', '1', '04', '03.0',

'00295.1', 'M', '-030.7', 'M', '', '*51']

['$GPGSA', 'A', '3', '21', '18', '15', '24', '', '', '', '', '', '', '', '',

'08.9', '03.0', '08.4*04']

>>> f.close()

First, we open the data file and assign it to variable f. The opened file can now be referred to by the variable

f. Next, we create a csv.reader object, cr. We associate the csv.reader object, cr, with the file f. We then iterate

through every row of the csv.reader object and print that row. Lastly, we close the file by calling f.close(). It is

considered good practice to close the file once you’re done with it; but if you neglect to do so, Python will close the file

automatically once the variable f is no longer in use.

Note ■ You may, after issuing the commands, receive an error similar to this: UnicodeEncodeError: 'charmap'

codec can't encode character '\uABCD'

. If this happens, open the GPS file in a text editor and make sure the file

contains proper alpha-numeric characters. Be sure to delete lines with non-alpha-numeric characters.

Python also lets you implement cascade functions, where you can call new functions based on the results of other

functions. This process can be repeated several times. Cascading (usually) adds clarity and produces more elegant

scripts. In our case, the variable f isn’t really important to us, so we discard it after we attach it to a csv.reader object.

Instead of the preceding code, we can write the following, by cascading the functions:

>>> cr = csv.reader(open('../data/GPS-2008-06-04-09-03-45.csv')):

>>> for row in cr:

... print(row)

CHAPTER 1 ■ NAVIGATING THE WORLD OF DATA VISUALIZATION

The same holds true for variable cr, for which we can cascade several functions and generate a more compact

line of code:

>>> for row in csv.reader(open('../data/GPS-2008-06-04-09-03-45.csv')):

... print(row)

While the script might be shorter, there’s no performance gain. It is therefore suggested that you cascade

functions only if it adds clarity; there’s a good chance you’ll be editing this code later on, and it’s important to be able

to understand what’s going on. In fact, not cascading functions might be useful at times because you might need

access to intermediate variables (such as f and cr in our case).

The csv.reader object converts each row we read into a row of fields, in the form of a list. That row is then

appended to a list of rows and stored in the variable data.

Note ■ By now, you’ve seen the dot symbol (.) used several times. Its use might be a bit confusing, so an explanation

is in order. The dot symbol is used to access function members of modules, as well as function members of objects

(classes). You’ve seen it in member functions of modules, such as csv.reader(), but also for objects, such as f.read().

In the latter, it means that the file object has a member function read() and that the function is called to operate on

variable f. To access these functions, we use the dot operator. We’ll touch on this again in Chapter 3. Lastly, we use the

ellipsis symbol (...) to denote line continuation when interactively entering commands in Python.

Analyzing GPS Data

Also known as NMEA 0183, the GPS format contains many header stamps, some of which hold useful information for

our task. Let’s take a closer look at the GPS data:

Each row seems to start with a text header stamp, beginning with the characters • $GP.

There are several header stamps (e.g., • $GPGSA and $GPRMC).

Additional values follow the header, most of which are numeric.•

Note ■ NMEA stands for the National Marine Electronics Association; see http://www.nmea.org for more information.

The NMEA 0183 data format is described at http://www.gpsinformation.org/dale/nmea.htm.

As mentioned earlier, several $GP header stamps appear in our data files, but we need to determine which ones

exactly are relevant to us. First, it would be nice to know which header stamps from the NMEA standard are even

present in our data files. One option would be to open the files, look for the headers, and jot down every new header

once we see it. Another, of course, would be to use Python to do that for us.

Python is a very high-level programming language. As such, it has built-in support for dictionaries (also known

as associative arrays in Perl), which are data structures that have a one-to-one relationship between a key and a value,

very much like real dictionaries. Traditional dictionaries, however, often have several values for a key; that is, they

have several interpretations (values) for one word (key). You can easily implement this in Python using the dictionary

object, dict, as well by assigning a list value to a key. That way, you can have several entries per one key, because the

key is associated with a list that can hold several values. In reality, it’s still a one-to-one relationship, but enough about

that for now. I’ll cover dictionaries in more detail in future chapters. What we want to do here is use a dictionary object

to hold the number of times a header is encountered. Our key will be the GPS header stamp, and our value will be a

number that indicates occurrence. We’ll increment the value whenever a key is encountered, as shown in Listing 1-5.

CHAPTER 1 ■ NAVIGATING THE WORLD OF DATA VISUALIZATION

Listing 1-5. Function list_gps_commands( )

def list_gps_commands(data):

"""Counts the number of times a GPS command is observed.

Returns a dictionary object."""

gps_cmds = dict()

for row in data:

try:

gps_cmds[row[0]] += 1

except KeyError:

gps_cmds[row[0]] = 1

return gps_cmds

There are few things to keep in mind about this function. First, the docstring spans multiple lines, which is one

of the key benefits of docstrings. Docstrings will display all the spaces and line breaks as shown in the function itself.

Second, we initialize a variable, gps_cmds, to be our dictionary. We then process every list in the GPS data: we only

care about the first element of every row, as that’s the value that holds the GPS header stamps. We then increment the

value associated with the key: gps_cmds[row[0]] += 1. We use the += operation to increment the value by 1, similar

to how it’s done in C (Python, however, does not use the ++ operator). If the key does not exist, which will happen

whenever we encounter a new header stamp, an exception will be raised. We catch the exception with our except

KeyError statement. In the case of an exception, we set the dictionary value associated with the key to 1.

We can write the function list_gps_commands() even more compactly using the dictionary method get();

see Chapter 3 for details.

Let’s analyze some GPS data:

>>> x = read_csv_file('../data/GPS-2008-05-30-09-00-50.csv')

>>> list_gps_commands(x)

{'$GPGSA': 282, '$GPGSV': 846, '$GPGGA': 282, '$GPRMC': 283}

It turns out there are four distinct GPS headers being generated by my GPS. Of those, only two interest us: $GPGSV,

which holds the number of satellites in view (Hey! It’s really important!); and $GPRMC, which holds location and

velocity information.

What we’d like to do is code a function that takes the GPS data and, whenever the header field is $GPGSV or

$GPRMC, extracts the information and stores it in numerical arrays that will be easier to manipulate later on. Numerical

arrays are introduced with the NumPy module, so we have to import numpy. Since we’ll be using a lot of the

functionality of NumPy, SciPy, and matplotlib, an easier approach would be to import pylab, which imports all these

modules, as follows:

>>> from pylab import *

Note ■ The name PyLab comes from Python and MATLAB. The PyLab module provides MATLAB-like functionality

in Python.

CHAPTER 1 ■ NAVIGATING THE WORLD OF DATA VISUALIZATION

Extracting GPS Data

In the case of a $GPGSV header, the number of satellites is the fourth entry. In the case of a $GPRMC header, we have a

bit more interesting information. The second field is the timestamp, the fourth field is the latitude, the sixth field is

the longitude, and the eighth field is the velocity. Again, refer to the NMEA 0183 format for more details. Table 1-1

summarizes the fields and their values in a $GPRMC line.

Table 1-1. $GPRMC Information (Excerpt)

Field Name Index Format

Header 0 $GPRMC (fixed)

Timestamp 1 hhmmss.ss

Latitude 3 DDMM.MMM

Longitude 5 DDDMM.MMM

Velocity 7 VVV.V

We need to keep in mind some caveats regarding the information in $GPRMC. For example, let’s look at the

timestamp of an arbitrary line:

>>> x[12]

['$GPRMC', '140055.00', 'A', '4454.1740', 'N', '09325.0143', 'W', '000.0',

'128.7', '300508', '001.1', 'E', 'A*28']

In this output, the timestamp appears as '140055.00'. This follows the format hhmmss.ss where hh are two digits

representing the hour (it will always consist of two digits—if the hour is one digit, say 7 in the morning, a 0 will be

added before it), mm are two digits representing the minute (again, always two digits), and ss.ss are five characters

(four digits plus the dot) representing seconds and fractions of seconds. There’s also a North/South field, as well as

an East/West field. Here, for simplicity, we assume northern hemisphere, but you can easily change these values by

reading the entire $GPRMC structure.

Note ■ In the ISO time format, we’ve used HHMMSS to denote hours minutes and seconds. In this case, we follow

the convention in NMEA, which uses hhmmss.ss for hours, minutes, and seconds, and then sets DD and MM to angular

degrees and minutes.

The timestamp string is a bit hard to work with, especially when plotting data. The first reason is that it’s a string,

not a number. But even if you translate it to a number, the system does not lend itself nicely to plotting because there

are 60 seconds in a minute, not 100. So what we want to do is “linearize” the timestamp. To achieve this, we translate

the timestamp as seconds elapsed since midnight, as follows: T = hh * 3600 + mm * 60 + ss.ss.

剩余404页未读，继续阅读

weixin_38743968

粉丝: 404

Python可视化入门：数据处理与图形制作指南

Beginning Python Visualization - Crafting Visual Transformation Scripts.pdf

Mastering Python Data Visualization.pdf

学习教程：Python _Scripting_ arcgis 2011：2022.pdf

英文原版-Beginning Python Visualization 1st Edition

3d Linear Transformation Visualization:可视化3d线性变换。-开源

Python可视化入门：《Crafting Visual Transformation Scripts》

Python可视化入门：构建视觉转换脚本

Python可视化入门：塑造数据转换脚本

Python可视化入门：塑造视觉转换脚本实战

Python视觉入门：打造数据可视化转译脚本

最新资源