R编程与统计计算：数据分析软件精选

需积分: 9 3 浏览量更新于2024-07-23 收藏 4.36MB PDF 举报

"Software_for_data_analysis_Programme" 是一个专注于数据分析软件和编程的资源，特别是使用R语言和统计学方法。这个资源适合已经具备一定R语言基础的读者，旨在帮助他们提升编写稳定R代码的能力，从而进行更高效、更深入的数据分析。在R语言方面，"Chambers: Software for Data Analysis: Programming with R" 是一本经典著作，由R语言的主要开发者之一John Chambers撰写，深入讲解如何利用R进行数据分析编程。这本书涵盖了R语言的基础、高级编程技巧以及统计分析的应用，是R语言学习者的必备参考。 "Dalgaard: Introductory Statistics with R" 由Knut-David Dalgaard撰写，是R语言初学者学习统计学概念和方法的好教材。它将统计理论与R的实际应用相结合，帮助读者快速上手R语言并理解统计分析的基本原理。 "Venables/Ripley: Modern Applied Statistics with S, 4th ed." 是另一本经典书籍，介绍了使用S（R的前身）进行现代应用统计的实践方法，第四版更新了R语言的内容，对于理解统计学和R的结合使用非常有帮助。在统计分析和计算方面，"Gentle: ElementsofComputationalStatistics" 和 "Lange: NumericalAnalysisforStatisticians" 分别探讨了计算统计学的基本原理和数值分析在统计中的应用，对于进行复杂统计计算和模拟的读者来说十分有价值。此外，"Gentle: Random Number Generation and Monte Carlo Methods, 2nd ed." 讨论了随机数生成和蒙特卡洛方法，这是进行模拟研究和统计推断的重要工具。 "Venables/Ripley: ModernAppliedStatisticswithS,4thed." 和 "Pinheiro/Bates: Mixed-Effects Models in S-PLUS" 提及了混合效应模型，这对于处理具有层次结构或相关数据的研究者来说是必不可少的。资源中还提到了其他一些书籍，如"XploRe: An Interactive Statistical Computing Environment" 和 " GraphicsofLargeDatasets: VisualizingaMillion"，分别涉及交互式统计计算环境和大规模数据集的可视化，这些都是现代数据科学领域的重要技能。 "Software_for_data_analysis_Programme" 资源提供了全面的学习路径，涵盖了从R编程到统计分析，再到数值计算和数据可视化的多个关键领域，对于想要提升数据科学能力的专业人士来说是一份宝贵的资料。通过学习这些资源，读者可以增强在实际数据分析项目中的能力，并掌握处理复杂问题的技巧。

2 CHAPTER 1. INTRODUCTION: PRINCIPLES AND CONCEPTS

How can these techniques help people to carry out this speciﬁc kind of

exploration?

Ensuring that software for data analysis exists for such purposes is an

important, exciting, and challenging activity. Later chapters examine how

we can select and develop software using

R and other systems.

The importance, excitement, and challenge all come from the central

role that data and computing have come to play in modern society. Science,

business and many other areas of soc iety continually rely on understanding

data, and that understanding frequently involves large and complicated data

processes.

A few examples current as the book is written can suggest the ﬂavor:

• Many ambitious projects are underway or propose d to deploy sensor

networks, that is, coordinated networks of devices to record a variety

of meas urements in an ongoing program. The data resulting is essen-

tial to understand environmental quality, the mechanisms of weather

and climate, and the future of biodiversity in the earth’s ecosystems.

In both scale and diversity, the challenge is unprecedented, and will

require merging techniques from many disciplines.

• Astronomy and cosmology are undergoing profound changes as a result

of large-scale digital mappings enabled by both sate llite and ground

recording of huge quantities of data. The scale of data collected allows

questions to be addressed in an overall sense that before could only be

examined in a few, local regions.

• Much business activity is now carried out largely through distributed,

computerized processes that both generate large and complex streams

of data and also oﬀer through such data an unprecedented opportu-

nity to understand one’s business quantitatively. Telecommunications

in North America, for example, generates databases with conceptually

billions of records. To explore and understand such data has great

attraction for the business (and for society), but is enormously chal-

lenging.

These and many other possible examples illustrate the importance of what

John Tukey long ago characterized as “the peaceful collision of computing

and data analysis”. Progress on any of these examples will require the ability

to explore the data, ﬂexibly and in a reasonable time frame.

1.2. TRUSTWORTHY SOFTWARE: THE PRIME DIRECTIVE 3

1.2 Trustworthy Software: The Prime Directive

Exploration is our mission; we and those who use our software want to

ﬁnd new paths to understand the data and the underlying processes. The

mission is, indeed, to boldly go where no one has gone before. But, we need

boldness to be balanced by our responsibility. We have a responsibility for

the results of data analysis that provides a key compensating principle.

The complexity of the data processes and of the computations applied

to them mean that those who receive the results of modern data analysis

have limited opportunity to verify the results by direct observation. Users of

the analysis have no option but to trust the analysis, and by extension the

software that produced it. Both the data analyst and the software provider

therefore have a strong responsibility to produce a result that is trustworthy,

and, if possible, one that can be shown to be trustworthy.

This is the second principle: the computations and the software for data

analysis should be trustworthy: they should do what they claim, and be seen

to do so. Neither those who view the results of data analysis nor, in many

cases, the statisticians performing the analysis can directly validate exten-

sive computations on large and complicated data processes. Ironically, the

steadily increasing computer power applied to data analysis often distances

the results further from direct checking by the recipient. The many com-

putational steps between original data source and displayed results must all

be truthful, or the eﬀect of the analysis may be worthless, if not pernicious.

This places an obligation on all creators of software to program in such a

way that the computations can be understood and trusted. This obligation

I label the Prime Directive.

Note that the directive in no sense discourages exploratory or approx-

imate methods. As John Tukey often remarked, better an approximate

answer to the right question than an exact answer to the wrong question.

We should seek answers boldly, but always explaining the nature of the

method applied, in an open and understandable format, supported by as

much e vidence of its quality as can be produced. As we will see, a number

of more technically speciﬁc choices can help us satisfy this obligation.

Readers who have seen the Star Trek



television series

may rec ognize

the term “prime directive”. Captains Kirk, Picard, and Janeway and their

crews were bound by a directive which (slightly paraphrased) was: Do noth-

ing to interfere with the natural course of a new civilization. Do not distort

Actually, at least ﬁve series, from “The Original” in 1966 through “Enterprise”, not

counting the animated version, plus many ﬁlms. See startrek.com and the many reruns

if this is a gap in your cultural background.

4 CHAPTER 1. INTRODUCTION: PRINCIPLES AND CONCEPTS

the development. Our directive is not to distort the message of the data,

and to provide computations whose content can be trusted and understood.

The prime directive of the space explorers, notice, was not their mission

but rather an important safeguard to apply in pursuing that mission. Their

mission was to explore, to “boldly go where no one has gone before”, and

all that. That’s really our mission too: to explore how software can add

new abilities for data analysis. And our own prime directive, likewise, is an

important caution and guiding principle as we create the software to support

our mission.

Here, then, are two motivating principles: the mission, which is bold

exploration; and the prime directive, trustworthy software. We will examine

in the rest of the book how to selec t and program software for data analysis,

with these principles as guides. A few aspects of

R will prove to be es pecially

relevant; let’s examine those next.

1.3 Concepts for Programming with R

The software and the programming techniques to be discussed in later chap-

ters tend to share some concepts that make them helpful for data analysis.

Exploiting these concepts will often beneﬁt both the eﬀectiveness of pro-

gramming and the quality of the results. Each of the concepts arises nat-

urally in later chapters, but it’s worth outlining them together here for an

overall picture of our strategy in programming for data analysis.

Functional Programming

Software in R is written in a functional style that helps both to understand

the intent and to ensure that the implementation corresponds to that intent.

Computations are organized around functions, which can encapsulate spe-

ciﬁc, meaningful computational results, with implementations that can be

examined for their correctness. The style derives from a more formal theory

of functional programming that restricts the computations to obtain well-

deﬁned or eve n formally veriﬁable results. Clearly, programming in a fully

functional manner would contribute to trustworthy software. The

S lan-

guage does not enforce a strict functional programming approach, but does

carry over some of the ﬂavor, particularly when you make some eﬀort to

emphasize simple functional deﬁnitions with minimal use of non-functional

computations.

As the scope of the software expands, much of the beneﬁt from functional

style can be retained by using functional methods to deal with varied types

1.3. CONCEPTS FOR PROGRAMMING WITH R 5

of data, within the general goal deﬁned by the generic function.

Classes and Methods

The natural complement to functional style in programming is the deﬁnition

of classes of objects. Where functions should clearly encapsulate the actions

in our analysis, classes should encapsulate the nature of the objects used

and returned by calls to functions. The duality between function calls and

objects is a recurrent theme of programming with

R. In the design of new

classes, we seek to capture an underlying concept of what the objects mean.

The relevant techniques combine directly specifying the contents (the slots),

relating the new class to existing classes (the inheritance), and expressing

how objects should be created and validated (methods for initializing and

validating).

Method deﬁnitions knit together functions and classes. Well-designed

methods extend the generic deﬁnition of what a function does to provide a

speciﬁc computational method when the argument or arguments come from

speciﬁed classes, or inherit from those classes. In contrast to methods that

are s olely class-base d, as in common object-oriented programming languages

such as

C++ or Java, methods in R are part of a rich but complex network

of functional and object-based computation.

The ability to deﬁne classes and methods in fact is itself a major advan-

tage in adhering to the Prime Directive. It gives us a way to isolate and

deﬁne formally what information certain objects should contain and how

those objects should behave when functions are applied to them.

Data Frames

Trustworthy data analysis depends ﬁrst on trust in the data being analyzed.

Not so much that the data must be perfect, which is impossible in nearly

any application and in any case beyond our control, but rather that trust

in the analysis depends on trust in the relation between the data as we use

it and the data as it has entered the process and then has been recorded,

organized and transformed.

In serious modern applications, the data usually comes from a process

external to the analysis, whether generated by scientiﬁc observations, com-

mercial transactions or any of many other human activities. To access the

data for analysis by well-deﬁned and trustworthy computations, we will ben-

eﬁt from having a description, or model, for the data that corresponds to

its natural home (often in DBMS or spreadsheet software), but can also be

6 CHAPTER 1. INTRODUCTION: PRINCIPLES AND CONCEPTS

a meaningful basis for data as used in the analysis. Transformations and

restructuring will often be needed, but these should be understandable and

defensible.

The model we will emphasize is the data frame, essentially a formulation

of the traditional view of observations and variables. The data frame has a

long history in the

S language but modern techniques for classes and meth-

ods allow us to extend the use of the concept. Particularly useful techniques

arise from using the data frame concept both within

R, for model-ﬁtting,

data visualization, and other computations, and also for eﬀective commu-

nication with other systems. Spreadsheets and relational database software

both relate naturally to this model; by using it along with unambiguous

mechanisms for interfacing with such software, the meaning and structure

of the data can be preserved. Not all applications suit this approach by

any means, but the general data frame model provides a valuable basis for

trustworthy organization and treatment of many sources of data.

Open Source Software

Turning to the general characteristics of the languages and systems available,

note that many of those discussed in this book are open-source software

systems; for example,

R, Perl, Python, many of the database sys tems , and

the

Linux op e rating system. These systems all provide access to source code

suﬃcient to generate a working version of the software. The arrangement is

not equivalent to “public-domain” software, by which people usually mean

essentially unrestricted use and copying. I nstead, most open-source systems

come with a copyright, usually held by a related group or foundation, and

with a license restricting the use and modiﬁcation of the software. There

are several versions of license, the best know n being the Gnu Public License

and its variants (see gnu.org/copyleft/gpl.html), the famous GPL.

R is

distributed under a version of this license (see the

"COPYING" ﬁle in the home

directory of

R). A variety of other licenses exists; those accepted by the Open

Source Initiative are described at opensource.org/licenses.

Distinctions among open-source licenses generate a good deal of he at

in some discussions, often centered on what eﬀect the license has on the

usability of the software for commercial purposes. For our focus, particularly

for the conce rn with trustworthy software for data analysis, these issues are

not directly relevant. The popularity of open-source systems certainly owes

a lot to their being thought of as “free”, but for our goal of trustworthy

software, this is also not the essential property. Two other characteristics

contribute more. First, the simple openness itself allows any suﬃciently

剩余513页未读，继续阅读

alexwar

粉丝: 0

R编程与统计计算：数据分析软件精选

free_and_open_source_software_for_development.pdf

FT_Write_String_Data.zip_LabVIEW FTDI_ftdi labview

IJ1_macro_for_image_analysis:用于生物学家的ImageJ Macro Programme量化IHCICC图片

csc-um_phd_programme_application_form.docx

DLPC350_Programme_Guide.pdf

java_chat_programme.zip_java

Programme_of_static_LIB.zip_驱动编程_Unix_Linux_

DCT程序实现文档.rar_DCT dsp_dct_dct programme_dct 变换

TestCodeCamp_Programme

programme_site

最新资源