入门数据科学：来自哥伦比亚大学课程的实践指南

5星 · 超过95%的资源需积分: 9 131 浏览量更新于2024-07-19 收藏 28MB PDF 举报

"Doing Data Science" 是一本由 Rachel Schutt 和 Cathy O’Neil 合著的书籍，内容来源于哥伦比亚大学的“数据科学入门”课程。这本书面向已经掌握线性代数、概率统计和有一定编程经验的读者，旨在帮助他们进入数据科学领域。书中通过各行业领先公司的数据科学家进行客座讲座，分享实际案例和算法代码，涵盖机器学习、统计模型、预测与描述、探索性数据分析、通信与可视化、数据处理、大数据、编程伦理以及如何提出好的问题等多个主题。在当前数据驱动的时代，无论是初创公司还是财富500强企业，都在利用数据驱动的洞察力来抓住机遇。四个强大的趋势——新型数据收集和管理方法、云 computing 提供的低成本存储和弹性计算能力、复杂数据的可视化技术，以及使任何人都能使用数据工具的普及，共同推动了这一变革。通过 O'Reilly 的 Strata 产品系列，读者可以掌握大数据并将其转化为洞察力，创造新产品或改进现有产品，理解消费者行为，并获取数据优势。书中，读者将了解到数据科学家的工作内容，学习如何运用各种算法和方法，如机器学习和数据挖掘，以及统计模型和方法。同时，书中探讨了预测与描述之间的区别，强调了探索性数据分析的重要性，因为这有助于发现数据中的模式和关系。此外，有效的沟通和可视化技巧是数据科学家的重要技能，它们可以帮助将复杂的数据转化为有说服力的故事。书中还讨论了数据处理的挑战，特别是在大数据环境下，以及编程在数据科学中的应用。在大数据领域，如何处理和分析海量数据成为关键。本书将介绍如何利用云技术和工具来应对这一挑战。编程部分将指导读者如何利用代码来实现数据处理和分析。同时，书中还会涉及伦理问题，提醒数据科学家在进行工作时要考虑数据隐私和道德规范。最后，书中强调了提出好问题的能力，这是数据科学项目成功的关键。只有明确问题，才能有效地收集、分析数据并得出有价值的结论。通过本书，读者不仅可以掌握数据科学的基础知识，还能了解这个领域的最新发展和实践应用。

Being an Ethical Data Scientist 354

Career Advice 359

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361

xii | Table of Contents

www.it-ebooks.info

might find? After I got my PhD, I worked at Google for a few years

around the same time that “data science” and “data scientist” were be‐

coming terms in Silicon Valley.

The world is opening up with possibilities for people who are quan‐

titatively minded and interested in putting their brains to work to solve

the world’s problems. I consider it my goal to help these students to

become critical thinkers, creative solvers of problems (even those that

have not yet been identified), and curious question askers. While I

myself may never build a mathematical model that is a piece of the

cure for cancer, or identifies the underlying mystery of autism, or that

prevents terrorist attacks, I like to think that I’m doing my part by

teaching students who might one day do these things. And by writing

this book, I’m expanding my reach to an even wider audience of data

scientists who I hope will be inspired by this book, or learn tools in it,

to make the world better and not worse.

Building models and working with data is not value-neutral. You

choose the problems you will work on, you make assumptions in those

models, you choose metrics, and you design the algorithms.

The solutions to all the world’s problems may not lie in data and tech‐

nology—and in fact, the mark of a good data scientist is someone who

can identify problems that can be solved with data and is well-versed

in the tools of modeling and code. But I do believe that interdiscipli‐

nary teams of people that include a data-savvy, quantitatively minded,

coding-literate problem-solver (let’s call that person a “data scientist”)

could go a long way.

Origins of the Class

I proposed the class in March 2012. At the time, there were three pri‐

mary reasons. The first will take the longest to explain.

Reason 1: I wanted to give students an education in what it’s like to be

a data scientist in industry and give them some of the skills data sci‐

entists have.

I was working on the Google+ data science team with an interdisci‐

plinary team of PhDs. There was me (a statistician), a social scientist,

an engineer, a physicist, and a computer scientist. We were part of a

larger team that included talented data engineers who built the data

pipelines, infrastructure, and dashboards, as well as built the experi‐

mental infrastructure (A/B testing). Our team had a flat structure.

xiv | Preface

www.it-ebooks.info

Together our skills were powerful, and we were able to do amazing

things with massive datasets, including predictive modeling, proto‐

typing algorithms, and unearthing patterns in the data that had huge

impact on the product.

We provided leadership with insights for making data-driven deci‐

sions, while also developing new methodologies and novel ways to

understand causality. Our ability to do this was dependent on top-

notch engineering and infrastructure. We each brought a solid mix of

skills to the team, which together included coding, software engineer‐

ing, statistics, mathematics, machine learning, communication, visu‐

alization, exploratory data analysis (EDA), data sense, and intuition,

as well as expertise in social networks and the social space.

To be clear, no one of us excelled at all those things, but together we

did; we recognized the value of all those skills, and that’s why we

thrived. What we had in common was integrity and a genuine interest

in solving interesting problems, always with a healthy blend of skep‐

ticism as well as a sense of excitement over scientific discovery. We

cared about what we were doing and loved unearthing patterns in the

data.

I live in New York and wanted to bring my experience at Google back

to students at Columbia University because I believe this is stuff they

need to know, and because I enjoy teaching. I wanted to teach them

what I had learned on the job. And I recognized that there was an

emerging data scientist community in the New York tech scene, and I

wanted students to hear from them as well.

One aspect of the class was that we had guest lectures by data scientists

currently working in industry and academia, each of whom had a dif‐

ferent mix of skills. We heard a diversity of perspectives, which con‐

tributed to a holistic understanding of data science.

Reason 2: Data science has the potential to be a deep and profound

research discipline impacting all aspects of our lives. Columbia Uni‐

versity and Mayor Bloomberg announced the Institute for Data

Sciences and Engineering in July 2012. This course created an oppor‐

tunity to develop the theory of data science and to formalize it as a

legitimate science.

Reason 3: I kept hearing from data scientists in industry that you can’t

teach data science in a classroom or university setting, and I took that

on as a challenge. I thought of my classroom as an incubator of data

Preface | xv

www.it-ebooks.info

science teams. The students I had were very impressive and are turning

into top-notch data scientists. They’ve contributed a chapter to this

book, in fact.

Origins of the Book

The class would not have become a book if I hadn’t met Cathy O’Neil,

a mathematician-turned-data scientist and prominent and out-

spoken blogger on mathbabe.org, where her “About” section states that

she hopes to someday have a better answer to the question, “What can

a nonacademic mathematician do that makes the world a better

place?” Cathy and I met around the time I proposed the course and

she was working as a data scientist at a startup. She was encouraging

and supportive of my efforts to create the class, and offered to come

and blog it. Given that I’m a fairly private person, I initially did not

feel comfortable with this idea. But Cathy convinced me by pointing

out that this was an opportunity to put ideas about data science into

the public realm as a voice running counter to the marketing and hype

that is going on around data science.

Cathy attended every class and sat in the front row asking questions,

and was also a guest lecturer (see Chapter 6). As well as documenting

the class on her blog, she made valuable intellectual contributions to

the course content, including reminding us of the ethical components

of modeling. She encouraged me to blog as well, and so in parallel to

her documenting the class, I maintained a blog to communicate with

my students directly, as well as capture the experience of teaching data

science in the hopes it would be useful to other professors. All Cathy’s

blog entries for the course, and some of mine, became the raw material

for this book. We’ve added additional material and revised and edited

and made it much more robust than the blogs, so now it’s a full-fledged

book.

What to Expect from This Book

In this book, we want to both describe and prescribe. We want to

describe the current state of data science by observing a set of top-notch

thinkers describe their jobs and what it’s like to “do data science.” We

also want to prescribe what data science could be as an academic

discipline.

xvi | Preface

www.it-ebooks.info

剩余404页未读，继续阅读

真的CK

粉丝: 0
资源: 6

入门数据科学：来自哥伦比亚大学课程的实践指南

[奥莱理] Doing Data Science

the art of doing science enhineering

pca = PCA(n_components=5) pca.fit(Data) Std_pca=pca.transform(Data)

from .data import sql_data ImportError: attempted relative import with no known parent package

_data = GetYamlData(ensure_path_sep("\\common\\config.yaml")).get_yaml_data() config = Config(**_data)

Java 正则表达式匹配链接/doing/seminar/45645.html /doing/market/4561.html /doing/special/456235.html

tx_data[2] = READ_ANG_Z & 0xFFFFF;解释

Data truncation: Incorrect date value: '2017' for column 'birth' at row 1

请帮我提取关于这篇文献Cleaning GeoNames Data: A Case Study for Natural Language Processing的The Case Study部分的内容

person 类中定义一个私有属性 name 和一个私有方法 doing ； 在 doing 方法中打印 name 。

最新资源

person 类中定义一个私有属性 name 和一个私有方法 doing ；在 doing 方法中打印 name 。