Python数据分析第二版：Pandas、NumPy与IPython实用教程

需积分: 7 173 浏览量更新于2024-07-19 收藏 9.19MB PDF 举报

《Python for Data Analysis》第二版是一本专门针对数据分析领域使用的Python编程指南，由Wes McKinney撰写。这本书是英文版，适用于Python 3环境，旨在帮助读者掌握如何利用Pandas、NumPy和IPython等库进行数据清洗、整理和分析。作者Wes McKinney在书中分享了他的专业知识，强调了Python在数据科学中的核心地位。该书的核心内容包括但不限于： 1. **Pandas介绍**：Pandas是Python中的一个强大数据处理库，本书深入讲解了DataFrame对象的创建、操作和查询，以及Series数据结构的使用，这对于数据分析师来说是必备的基础技能。 2. **NumPy支持**：NumPy提供了高性能的多维数组和矩阵运算，是科学计算的基础。书中会涉及如何结合Pandas与NumPy进行数值计算和数据转换。 3. **IPython扩展**：IPython是一个交互式计算环境，书中展示了如何利用IPython Notebook（现称Jupyter Notebook）进行数据探索和可视化，以及其在交互式数据分析中的重要作用。 4. **数据清洗与预处理**：书中详细介绍了如何处理缺失值、异常值、重复值，以及如何进行数据类型转换和数据格式规范化，这些都是数据清洗过程中的关键步骤。 5. **数据合并与重塑**：通过学习如何合并多个数据源，以及如何重塑数据格式以适应分析需求，读者可以更好地理解数据的内在结构。 6. **数据分组与聚合**：通过学习如何对数据进行分组和计算汇总统计，读者能够深入了解数据的分布和趋势。 7. **数据可视化**：书中不仅提供使用Matplotlib等工具进行基础图表绘制的方法，还探讨了如何利用Seaborn等高级库创建更美观且具有洞察力的图形。然而，值得注意的是，提到的“代码部分有问题，格式不对”，这可能意味着某些示例代码可能存在错误或不适用于当前版本的Python，因此在阅读时需要仔细核对，并根据最新文档或在线资源进行修正。此外，如果你打算下载电子版，建议选择官方出版或已更新的版本以确保正确性和有效性。《Python for Data Analysis》第二版于2017年9月首次发布修订版，适合那些希望通过Python进行实际数据处理和分析工作的专业人士，无论是初学者还是有经验的数据分析师，都能从中受益匪浅。通过学习这本书，读者将能掌握Python工具链在数据科学项目中的应用，并提升自己的数据分析能力。

Python can be a challenging language for building highly concurrent, multithreaded applications,

particularly applications with many CPU-bound threads. The reason for this is that it has what is

known as the global interpreter lock (GIL), a mechanism that prevents the interpreter from

executing more than one Python instruction at a time. The technical reasons for why the GIL

exists are beyond the scope of this book. While it is true that in many big data processing

applications, a cluster of computers may be required to process a dataset in a reasonable amount

of time, there are still situations where a single-process, multithreaded system is desirable.

This is not to say that Python cannot execute truly multithreaded, parallel code. Python C

extensions that use native multithreading (in C or C++) can run code in parallel without being

impacted by the GIL, so long as they do not need to regularly interact with Python objects.

1.3 Essential Python Libraries

For those who are less familiar with the Python data ecosystem and the libraries used throughout

the book, I will give a brief overview of some of them.

NumPy

NumPy , short for Numerical Python, has long been a cornerstone of numerical computing in

Python. It provides the data structures, algorithms, and library glue needed for most scientific

applications involving numerical data in Python. NumPy contains, among other things:

A fast and efficient multidimensional array object ndarray

Functions for performing element-wise computations with arrays or mathematical

operations between arrays

Tools for reading and writing array-based datasets to disk

Linear algebra operations, Fourier transform, and random number generation

A mature C API to enable Python extensions and native C or C++ code to access NumPy’s

data structures and computational facilities

Beyond the fast array-processing capabilities that NumPy adds to Python, one of its primary uses

in data analysis is as a container for data to be passed between algorithms and libraries. For

numerical data, NumPy arrays are more efficient for storing and manipulating data than the other

built-in Python data structures. Also, libraries written in a lower-level language, such as C or

Fortran, can operate on the data stored in a NumPy array without copying data into some other

memory representation. Thus, many numerical computing tools for Python either assume NumPy

arrays as a primary data structure or else target seamless interoperability with NumPy.

pandas

pandas provides high-level data structures and functions designed to make working with

structured or tabular data fast, easy, and expressive. Since its emergence in 2010, it has helped

enable Python to be a powerful and productive data analysis environment. The primary objects in

pandas that will be used in this book are the DataFrame , a tabular, column-oriented data

structure with both row and column labels, and the Series , a one-dimensional labeled array

object.

pandas blends the high-performance, array-computing ideas of NumPy with the flexible data

manipulation capabilities of spreadsheets and relational databases (such as SQL). It provides

sophisticated indexing functionality to make it easy to reshape, slice and dice, perform

aggregations, and select subsets of data. Since data manipulation, preparation, and cleaning is

such an important skill in data analysis, pandas is one of the primary focuses of this book.

As a bit of background, I started building pandas in early 2008 during my tenure at AQR Capital

Management, a quantitative investment management firm. At the time, I had a distinct set of

requirements that were not well addressed by any single tool at my disposal:

Data structures with labeled axes supporting automatic or explicit data alignment — this

prevents common errors resulting from misaligned data and working with differently

indexed data coming from different sources

Integrated time series functionality

The same data structures handle both time series data and non–time series data

Arithmetic operations and reductions that preserve metadata

Flexible handling of missing data

Merge and other relational operations found in popular databases (SQL-based, for example)

I wanted to be able to do all of these things in one place, preferably in a language well suited to

general-purpose software development. Python was a good candidate language for this, but at

that time there was not an integrated set of data structures and tools providing this functionality.

As a result of having been built initially to solve finance and business analytics problems, pandas

features especially deep time series functionality and tools well suited for working with time-

indexed data generated by business processes.

For users of the R language for statistical computing, the DataFrame name will be familiar, as

the object was named after the similar R data.frame object. Unlike Python, data frames are built

into the R programming language and its standard library. As a result, many features found in

pandas are typically either part of the R core implementation or provided by add-on packages.

The pandas name itself is derived from panel data , an econometrics term for multidimensional

structured datasets, and a play on the phrase Python data analysis itself.

matplotlib

matplotlib is the most popular Python library for producing plots and other two-dimensional data

visualizations. It was originally created by John D. Hunter and is now maintained by a large team

of developers. It is designed for creating plots suitable for publication. While there are other

visualization libraries available to Python programmers, matplotlib is the most widely used and

as such has generally good integration with the rest of the ecosystem. I think it is a safe choice as

a default visualization tool.

IPython and Jupyter

The IPython project began in 2001 as Fernando Pérez’s side project to make a better interactive

Python interpreter. In the subsequent 16 years it has become one of the most important tools in

the modern Python data stack. While it does not provide any computational or data analytical

tools by itself, IPython is designed from the ground up to maximize your productivity in both

interactive computing and software development. It encourages an execute-explore workflow

instead of the typical edit-compile-run workflow of many other programming languages. It also

provides easy access to your operating system’s shell and filesystem. Since much of data

analysis coding involves exploration, trial and error, and iteration, IPython can help you get the

job done faster.

In 2014, Fernando and the IPython team announced the Jupyter project , a broader initiative to

design language-agnostic interactive computing tools. The IPython web notebook became the

Jupyter notebook, with support now for over 40 programming languages. The IPython system

can now be used as a kernel (a programming language mode) for using Python with Jupyter.

IPython itself has become a component of the much broader Jupyter open source project, which

provides a productive environment for interactive and exploratory computing. Its oldest and

simplest “mode” is as an enhanced Python shell designed to accelerate the writing, testing, and

debugging of Python code. You can also use the IPython system through the Jupyter Notebook,

an interactive web-based code “notebook” offering support for dozens of programming

languages. The IPython shell and Jupyter notebooks are especially useful for data exploration

and visualization.

The Jupyter notebook system also allows you to author content in Markdown and HTML,

providing you a means to create rich documents with code and text. Other programming

languages have also implemented kernels for Jupyter to enable you to use languages other than

Python in Jupyter.

For me personally, IPython is usually involved with the majority of my Python work, including

running, debugging, and testing code.

In the accompanying book materials , you will find Jupyter notebooks containing all the code

examples from each chapter.

SciPy

SciPy is a collection of packages addressing a number of different standard problem domains in

scientific computing. Here is a sampling of the packages included:

scipy.integrate

Numerical integration routines and differential equation solvers

scipy.linalg

statistics professor Jonathan Taylor, who implemented a number of regression analysis models

popular in the R programming language. Skipper Seabold and Josef Perktold formally created

the new statsmodels project in 2010 and since then have grown the project to a critical mass of

engaged users and contributors. Nathaniel Smith developed the Patsy project, which provides a

formula or model specification framework for statsmodels inspired by R’s formula system.

Compared with scikit-learn, statsmodels contains algorithms for classical (primarily frequentist)

statistics and econometrics. This includes such submodules as:

Regression models: Linear regression, generalized linear models, robust linear models,

linear mixed effects models, etc.

Analysis of variance (ANOVA)

Time series analysis: AR, ARMA, ARIMA, VAR, and other models

Nonparametric methods: Kernel density estimation, kernel regression

Visualization of statistical model results

statsmodels is more focused on statistical inference, providing uncertainty estimates and p -

values for parameters. scikit-learn, by contrast, is more prediction-focused.

As with scikit-learn, I will give a brief introduction to statsmodels and how to use it with NumPy

and pandas.

1.4 Installation and Setup

Since everyone uses Python for different applications, there is no single solution for setting up

Python and required add-on packages. Many readers will not have a complete Python

development environment suitable for following along with this book, so here I will give detailed

instructions to get set up on each operating system. I recommend using the free Anaconda

distribution. At the time of this writing, Anaconda is offered in both Python 2.7 and 3.6 forms,

though this might change at some point in the future. This book uses Python 3.6, and I encourage

you to use Python 3.6 or higher.

Windows

To get started on Windows, download the Anaconda installer . I recommend following the

installation instructions for Windows available on the Anaconda download page, which may

have changed between the time this book was published and when you are reading this.

Now, let’s verify that things are configured correctly. To open the Command Prompt application

(also known as cmd.exe ), right-click the Start menu and select Command Prompt. Try starting

the Python interpreter by typing python . You should see a message that matches the version of

Anaconda you installed:

C:\Users\wesm>python

Python 3.5.2 |Anaconda 4.1.1 (64-bit)| (default, Jul 5 2016, 11:41:13)

剩余2915页未读，继续阅读

xiaojiqiren_1

粉丝: 0

Python数据分析第二版：Pandas、NumPy与IPython实用教程

掌握数据分析：Python for Data Analysis第二版

《Python for Data Analysis》PDF电子书内容概览

Python数据分析实战：Python for Data Analysis

Python for Data Analysis_Data(2nd)_dataanalysis_python_英文原版_数据分析

Python for Data Analysis_dataanalysispython_dataanalysis_python_

Python for Data Analysis_python/digital/KNN_dataanalysis_

Python_for_Data_Analysis_Chinese_version.pdf.zip_data analysis_e

Python_DataAnalysis_Workshop

Python_for_Data_Analysis：Python_for_Data_Analysis

pandas powerful Python data analysis_pandas_dataanalysis_

最新资源