BigQuery Analytics实战指南：掌握复杂查询与API开发

5星 · 超过95%的资源需积分: 10 78 浏览量更新于2024-07-22 收藏 9.33MB PDF 举报

《Google BigQuery Analytics》是由John Wiley & Sons, Inc.出版于2014年的专业书籍，专为商业分析人员和数据分析师设计，旨在提供最新的关于如何运行复杂查询和与BigQuery API交互的实用技巧。本书以实际案例为基础，深入讲解了如何在Google Compute Engine中利用Hadoop进行数据流处理、AppEngine数据存储集成，以及如何结合GViz和Tableau生成查询结果的图表，使得读者能够有效地掌握BigQuery的实际操作。该书的核心内容围绕着Google BigQuery，不仅介绍了其底层Dremel查询引擎的架构，帮助读者理解查询背后的逻辑，从而优化查询性能。通过阅读这本书，读者可以学习到如何有效地利用BigQuery的强大功能，如分布式处理、海量数据处理能力，以及如何将其与其他Google云服务整合，形成一个完整的数据分析生态系统。作者们，Jordan Tigani和Siddartha Naidu，以其丰富的经验和深入的专业知识，确保了书中的理论知识与实践经验相结合，使读者能够迅速上手并提升技能。此外，书中还包含了版权信息，提醒读者在合法范围内使用和复制内容，遵守版权法规。《Google BigQuery Analytics》是一本实用的指南，适合希望在大数据分析领域进一步发展的专业人士，无论是初次接触BigQuery还是已有一定经验的用户，都能从中获益匪浅。通过阅读和实践书中的内容，读者不仅能提升自己的数据处理能力，还能紧跟Google BigQuery技术的发展动态，适应不断变化的数据分析需求。

xiv Introduction

 ast.indd 01:51:1:PM 05/08/2014 Page xiv

annual developer conference. Two years later, also at Google I/O, Jordan was

on hand to announce public availability.

In the intervening time, we have both been active contributors to the design

and implementation of the BigQuery service. The system has grown consider-

ably, both in the number of users as well as what it can do. We’ve also helped

support the product via mailing lists and the BigQuery area on StackOver ow.

We’ve paid close attention to how the product was used, the kinds of problems

that people had, and the interesting applications they found for the technology.

Along the way we built up a collection of cool things you can do using BigQuery:

different techniques, ways of querying, and methods of working around limita-

tions. These tricks and best practices are in this book.

This book doesn’t attempt to replace the online documentation. Instead, it is

a good format to dive deeper into the service and address different audiences,

particularly those familiar with traditional data warehousing products. In addi-

tion, this book gives you a sense of how BigQuery works and answers some of

the “whys” behind the design.

Although BigQuery, as a product, moves quickly, we’ve written the book

with changes (expected and otherwise) in mind. A new version of BigQuery is

released every week or so, and each one  xes bugs, adds features, or improves

scalability and performance. By the time you read this book, BigQuery may

have undergone signi cant evolution.

The BigQuery team works hard at making sure all changes to BigQuery will

be backward compatible. When new functionality is introduced that con icts

with existing behavior, the policy is to ensure the old version continues to work

for at least 12 months. The code and examples in this book should continue to

work for a considerable amount of time. Both code and SQL queries that were

written to interact with BigQuery at the public launch in 2012 still work today.

Even though the BigQuery service maintains backward compatibility, the best

way to achieve a particular result sometimes changes as new features become

available. We have chosen to document in this book core concepts that are useful

independent of details that might change over time. When in doubt, consult the

of cial documentation at https://developers.google.com/bigquery.

Moreover, this book describes integration with other technologies, such as

R, Microsoft Excel, and Tableau. Although each of these technologies is also

evolving, we’ve tried to include things that will likely continue to work for the

foreseeable future. For instance, Chapter 13, “Using BigQuery from Third-Party

Tools,” describes using BigQuery via an ODBC connection. Although each

version of Microsoft Windows may change the user interface slightly around

setting up an ODBC connection, ODBC is a stable technology that will still

work years down the road.

www.it-ebooks.info

Introduction xv

 ast.indd 01:51:1:PM 05/08/2014 Page xv

How This Book Is Organized

This book is divided into four sections:

■

BigQuery Fundamentals (Chapters 1 through 4): Walks you through

how to start with BigQuery and describes the basic abstractions used by

the service. If you’re familiar with BigQuery, you might want to just skim

this section; although, it might be helpful to read it to make sure you have

a  rm grounding in the primary concepts.

■

Basic BigQuery (Chapters 5 through 8): Shows how to use the API at a

raw HTTP level and via the higher-level clients, as well as how to write

SQL queries. This section culminates with an AppEngine app that ties

the various pieces of the API together and shows how they are useful in

a real-world scenario.

■

Advanced BigQuery (Chapters 9 through 11): Goes into detail about

how BigQuery works, gives advanced query tricks and recipes, and gives

advice on data management strategies. This section can help you under-

stand why a query that seems like it should work might return an error,

and can help you create queries that can be dif cult to express in SQL.

■

BigQuery Applications (Chapters 12 through 14): Shows how to tie

BigQuery in with other systems. For instance, this section shows how

to visualize your data with Tableau, how to query BigQuery tables from

Microsoft Excel, and how to query your Google Analytics data.

How to Read This Book

You can, of course, read this book straight through from cover to cover. However,

depending on your goals, you may want to skip some sections without losing

much context. For example, if you are a data analyst trying to learn how to write

better BigQuery queries, you may not be as interested in the HTTP API as a

developer who wants to mirror a production database to BigQuery. Following

are a couple of suggested paths though the book for different user pro les.

Data Scientist

A data scientist is a user who doesn’t care about the details of the HTTP API but

wants to get the most out of queries. Their primary interaction with BigQuery

www.it-ebooks.info

xvi Introduction

 ast.indd 01:51:1:PM 05/08/2014 Page xvi

will be through SQL and the various BigQuery tools provided by Google. The

data scientist can follow this chapter progression:

■

Chapters 1–4 (fundamentals): For anyone not familiar with BigQuery,

these chapters describe what BigQuery is and how to start using it. Chapter

4, “Understanding the BigQuery Object Model,” is important because the

fundamental abstractions in BigQuery differ slightly from other relational

database systems.

■

Chapter 7, “Running Queries”: You may want to skim the API portion,

but you probably shouldn’t skip it completely because it describes what

is possible via the API. (And all the API features should be available in

the web UI or in the bq command-line client.) Some table management

operations that you may expect to be in SQL (such as creating a tempo-

rary table) are done via the API in BigQuery. This chapter also discusses

how BigQuery SQL is different from standard SQL and walks through a

number of BigQuery queries.

■

Chapter 9, “Understanding Query Execution”: This chapter describes

the architecture of the systems that underlie BigQuery. If you want to

write good queries and understand why query X is faster than query Y,

this chapter is important. Most users of relational databases develop an

intuition about how to write ef cient queries, and because BigQuery uses

a different fundamental architecture, some of these previous intuitions

could get you in trouble. This chapter can help you develop similar intu-

ition about the types of queries that can run well in BigQuery.

■

Chapter 10, “Advanced Queries”: This chapter shows some queries that

you might not think of writing and provides advanced query recipes. You

may want to refer back to this chapter when you run into data modeling

or query problems.

■

Chapter 11, “Managing Data Stored in BigQuery”: You might want to

skip or skim this chapter, but the portions on how to partition your data

or how to make use of the query cache may be useful.

■

Chapter 12, “External Data Processing”: The second half of this chap-

ter, which describes running queries from Microsoft Excel and Google

Spreadsheets, will likely be interesting if your organization uses a lot of

spreadsheets.

■

Chapter 13, “Using BigQuery from Third-Party Tools”: You should read

this chapter if you’re interested in data visualization, client-side encryp-

tion, R, or using BigQuery via ODBC.

■

Chapter 14, “Querying Google Data Sources”: If you have data from a

Google project (AdSense, Google Analytics, or DoubleClick) that you want

to query, this is the chapter for you.

www.it-ebooks.info

Introduction xvii

 ast.indd 01:51:1:PM 05/08/2014 Page xvii

Software Developer

If you’re a developer who wants to integrate BigQuery with your data pipelines

or create a dashboard using BigQuery, you might be most interested in the fol-

lowing chapters:

■

Chapters 1–4 (fundamentals): If you’re going to use the API, carefully

read Chapter 4, “Understanding the BigQuery Object Model.” You need to

understand the BigQuery object model, the difference between a Dataset

and a Table, and what kinds of things you can do with Jobs.

■

Chapter 5, “Talking to the BigQuery API”: This chapter gives an overview

of the HTTP API that you’ll use if you write code to talk to BigQuery.

■

Chapter 6, “Loading Data”: If you want to get your data into BigQuery,

read this chapter.

■

Chapter 7, “Running Queries”: This chapter discusses the BigQuery query

language and the query API. You’ll likely want to be familiar with how

to run queries via the API as well as the various query options.

■

Chapter 8, “Putting It Together”: This chapter walks you through an

end-to-end AppEngine application that uses BigQuery for logging, dash-

boarding, and ad-hoc querying. If you write code that uses BigQuery, the

online resources for this chapter will be particularly interesting because

you may cut and paste a lot of the code that is provided.

■

Chapter 11, “Managing Data Stored in BigQuery”: This may be interesting

because it gives best practices for how to partition your data effectively,

and gives tips and tricks for reducing the cost of using BigQuery.

■

Chapter 12, “External Data Processing”: If you want to process your data

outside of BigQuery, this chapter will be useful. Maybe you want to run

Hadoop over your BigQuery data, or you want to download your tables

locally to process them on-premise.

■

Chapter 13, “Using BigQuery from Third-Party Tools”: There are a

number of third-party tools that can make it easier for you to integrate

with BigQuery. For example, if you already use ODBC to connect to a

database source, the Simba ODBC driver for BigQuery may allow you to

run queries without having to write any additional code.

Technology Evaluator

Maybe you’re considering using BigQuery and would like to compare it against

other options such as Amazon.com’s Redshift or Cloudera Impala. Or maybe

you’re just curious about the architecture. If you’re reading this book because

www.it-ebooks.info

xviii Introduction

 ast.indd 01:51:1:PM 05/08/2014 Page xviii

you’re interested in what the technology can do and how it works, consider the

following chapters:

■

Chapter 2, “Big Query Fundamentals”: This chapter is a good introduc-

tion to BigQuery and what it can do. It also describes what BigQuery does

not do and gives some comparisons to other technologies.

■

Chapter 4, “Understanding the BigQuery Object Model”: This chapter

will be worth skimming, even if you’re interested only in how BigQuery

works. Other chapters reference it heavily and assume that you know the

difference between a Table and a Dataset.

■

Chapters 5–8 (BigQuery API): These chapters may be worth skimming,

if to see only what BigQuery can do and what the code to use for various

features would look like.

■

Chapter 9, “Understanding Query Execution”: This chapter has archi-

tectural comparisons to other Big Data frameworks such as Hadoop.

It describes how BigQuery works, including the Dremel architecture,

Colossus, and the other building blocks that BigQuery uses to provide a

comprehensive service.

■

Chapter 12, “External Data Processing”: This chapter describes a number

of ways to interact with BigQuery; it will be interesting if you want to

 gure out how to integrate with your existing systems.

■

Chapter 13, “Using BigQuery from Third-Party Tools”: This chapter

gives a survey of visualization, connection, and other types of tools that

are built on top of BigQuery.

■

Chapter 14, “Querying Google Data Sources”: If you use other Google

services, such as AdSense or Double Click, this chapter is worth skim-

ming to see how you can access the data you already have within Google

via BigQuery.

Tools You Need

If you’re an analyst or someone who just wants to use BigQuery directly

out-of-the-box (that is, without writing code to interact with it), the only thing

you need is a web browser (and perhaps a credit card if your queries exceed

the free monthly quota).

If you’re a developer who wants to integrate BigQuery into your processes,

either by streaming logs into BigQuery, writing dashboards on top of the ser-

vice, or writing custom Hadoop pipelines that use BigQuery data, you need

www.it-ebooks.info

剩余529页未读，继续阅读

vanridin

粉丝: 108
资源: 1187

BigQuery Analytics实战指南：掌握复杂查询与API开发

Clojure语言的Google BigQuery API客户端库

Logstash输出插件发布：Google BigQuery集成详解

掌握Pandas与Google BigQuery的接口：pandas-gbq包使用教程

SQL脚本：100多个SQL脚本-PostgreSQL，MySQL，Google BigQuery，MariaDB，AWS Athena。 DevOps DBA Analytics性能工程。 Google BigQuery ML机器学习分类

google-analytics-premium-bigquery-statistics

neo4j-bigquery:将Neo4j数据导出到Google BigQuery

bigquery-grafana：适用于Grafana的Google BigQuery数据源插件

pandas-gbq:熊猫Google BigQuery

bigquery-geo-viz：使用Google Maps Platform API可视化Google BigQuery地理空间数据

bigrquery：R到Google BigQuery的接口

最新资源