使用Kafka Streams进行实时流处理

Kafka

需积分: 14 48 浏览量更新于2024-07-18 收藏 15.6MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"Kafka Streams in Action 是一本由 William P. Bejeck Jr. 编著，Neha Narkhede 前言的书籍，专注于介绍如何使用 Kafka Streams 进行实时流处理。本书旨在帮助读者掌握在 Kafka 集群中处理数据流的方法，以便快速获取数据洞察，同时简化复杂性和减少额外工具的需求。通过阅读本书，读者将能够将 Kafka Streams 应用于他们的项目中，轻松高效地利用数据。 Kafka Streams 是一个强大的库，专为处理流入 Kafka 集群的数据而设计。随着实时数据分析的重要性日益增强，但现有的解决方案可能过于复杂，需要额外的工具来执行查找和聚合操作，Kafka Streams 提供了一种更为简洁的解决方案。书中详细阐述了如何利用 Kafka Streams 实现微服务和实时应用的构建，涵盖了各种处理模式，如源、过滤、选择键和处理器等。书中的例子包括了不同的处理器模式，如 Masking、Filtering、Select-key，以及针对不同场景的应用，如 Electronics、Cafe 和 Rewards 的源和接收器（sink）。这些模式和实例展示了如何在实际业务场景中应用 Kafka Streams，例如处理电子产品的购买记录、咖啡店的消费数据，以及奖励系统的积分计算。 Kafka Streams 的核心优势在于它能够将复杂的流处理任务集成到 Kafka 平台内部，从而降低了系统复杂性，提高了数据处理速度。通过学习本书，读者可以了解到如何设计和实现这些处理管道，以及如何处理数据流中的事件时间，确保数据的一致性和准确性。此外，书中可能还涵盖了容错机制、状态管理、窗口和会话的概念，这些都是进行高效流处理的关键。 Kafka Streams in Action 是一本全面的指南，适合那些希望在 Kafka 环境中实现高效实时数据处理的开发者和数据工程师。通过深入理解并实践书中的概念和示例，读者将能够构建出能够快速响应数据变化的微服务和应用程序，充分利用数据的潜力。"

资源详情

资源推荐

xiv

acknowledgments

First and foremost, I’d like to thank my wife Beth and acknowledge all the support I

received from her during this process. Writing a book is a time-consuming task, and

without her encouragement, this book never would have happened. Beth, you are fan-

tastic, and I’m very grateful to have you as my wife. I’d also like to acknowledge my

children, who put up with Dad sitting in his office all day on most weekends and

accepted the vague answer “Soon” when they asked when I’d be finished writing.

Next, I thank Guozhang Wang, Matthias Sax, Damian Guy, and Eno Thereska, the

core developers of Kafka Streams. Without their brilliant insights and hard work,

there would be no Kafka Streams, and I wouldn’t have had the chance to write about

this game-changing tool.

I thank my editor at Manning, Frances Lefkowitz, whose expert guidance and

infinite patience made writing a book almost fun. I also thank John Hyaduck for his

spot-on technical feedback, and Valentin Crettaz, the technical proofer, for his excel-

lent work reviewing the code. Additionally, I thank the reviewers for their hard work

and invaluable feedback in making the quality of this book better for all readers:

Alexander Koutmos, Bojan Djurkovic, Dylan Scott, Hamish Dickson, James Frohnhofer,

Jim Manthely, Jose San Leandro, Kerry Koitzsch, László Hegedüs, Matt Belanger,

Michele Adduci, Nicholas Whitehead, Ricardo Jorge Pereira Mano, Robin Coe, Sumant

Tambe, and Venkata Marrapu.

Finally, I’d like to acknowledge all the Kafka developers for building such high-

quality software, especially Jay Kreps, Neha Narkhede, and Jun Rao—not just for start-

ing Kafka in the first place, but also for founding Confluent, a great and inspiring

place to work.

ABOUT THIS BOOK

xvi

How this book is organized: a roadmap

This book has four parts spread over nine chapters. Part 1 introduces a mental model

of Kafka Streams to show you the big-picture view of how it works. These chapters also

provide the basics of Kafka, for those who need them or want a review:

■

Chapter 1 provides some history of how and why stream processing became

necessary for handling real-time data at scale. It also presents the mental model

of Kafka Streams. I don’t go over any code but rather describe how Kafka

Streams works.

■

Chapter 2 is a primer for developers who are new to Kafka. Those with more

experience with Kafka can skip this chapter and get right into Kafka Streams.

Part 2 moves on to Kafka Streams, starting with the basics of the API and continuing

to the more complex features:

■

Chapter 3 presents a Hello World application and then presents a more realistic

example: developing an application for a fictional retailer, including advanced

features.

■

Chapter 4 discusses state and explains how it’s sometimes required for stream-

ing applications. You’ll learn about state store implementations and how to per-

form joins in Kafka Streams.

■

Chapter 5 explores the duality of tables and streams, and introduces a new con-

cept: the

KTable

. Whereas a

KStream

is a stream of events, a

KTable

is a stream

of related events or an update stream.

■

Chapter 6 goes into the low-level Processor API. Up to this point, you’ve been

working with the high-level DSL, but here you’ll learn how to use the Processor

API when you need to write customized parts of an application.

Part 3 moves on from developing Kafka Streams applications to managing Kafka

Streams:

■

Chapter 7 explains how to test a Kafka Streams application. You’ll learn how to

test an entire topology, unit-test a single processor, and use an embedded Kafka

broker for integration tests.

■

Chapter 8 covers how to monitor your Kafka Streams application, both to see

how long it takes to process records and to locate potential processing bottle-

necks.

Part 4 is the capstone of the book, where you’ll delve into advanced application devel-

opment with Kafka Streams:

■

Chapter 9 covers integrating existing data sources into Kafka Streams using

Kafka Connect. You’ll learn to include database tables in a streaming applica-

tion. Then, you’ll see how to use interactive queries to provide visualization

and dashboard applications while data is flowing through Kafka Streams, with-

out the need for relational databases. The chapter also introduces KSQL,

ABOUT THIS BOOK

xvii

which you can use to run continuous queries over Kafka without writing any

code, by using SQL.

About the code

This book contains many examples of source code both in numbered listings and

inline with normal text. In both cases, source code is formatted in a

fixed-width

font

this

to separate it from ordinary text.

In many cases, the original source code has been reformatted; we’ve added line

breaks and reworked indentation to accommodate the available page space in the

book. In rare cases, even this was not enough, and listings include line-continuation

markers (

➥

). Additionally, comments in the source code have often been removed

from the listings when the code is described in the text. Code annotations accompany

many of the listings, highlighting important concepts.

Finally, it’s important to note that many of the code examples aren’t meant to

stand on their own: they’re excerpts containing only the most relevant parts of what is

currently under discussion. You’ll find all the examples from the book in the accom-

panying source code in their complete form. Source code for the book’s examples is

available from GitHub at https://github.com/bbejeck/kafka-streams-in-action and

the publisher’s website at www.manning.com/books/kafka-streams-in-action.

The source code for the book is an all-encompassing project using the build tool

Gradle (https://gradle.org). You can import the project into either IntelliJ or Eclipse

using the appropriate commands. Full instructions for using and navigating the source

code can be found in the accompanying README.md file.

Book forum

Purchase of Kafka Streams in Action includes free access to a private web forum run by

Manning Publications where you can make comments about the book, ask technical

questions, and receive help from the author and from other users. To access the

forum, go to https://forums.manning.com/forums/kafka-streams-in-action. You can

also learn more about Manning’s forums and the rules of conduct at https://forums

.manning.com/forums/about.

Manning’s commitment to our readers is to provide a venue where a meaningful

dialogue between individual readers and between readers and the author can take

place. It is not a commitment to any specific amount of participation on the part of

the author, whose contribution to the forum remains voluntary (and unpaid). We sug-

gest you try asking him some challenging questions lest his interest stray! The forum

and the archives of previous discussions will be accessible from the publisher’s website

as long as the book is in print.

剩余281页未读，继续阅读

WindStand

粉丝: 35
资源: 367

使用Kafka Streams进行实时流处理

Kafka Stream In Action(MEAP) 前六章

高清彩版 Kafka_Streams_in_Action

kafka实战pdf

分享一下kafka streams的学习资料

生成一个kafka streams 调用kafka java代码

Kafka Streams或Kafka Connect等组件来实现延迟发送。

springboot怎么集成kafka streams实现消息按时间戳消费

kafka stream和kafka区别

kafkastream in action

kafka的dwd层

sparkStream和kafkaStream的区别

spring boot kafka 集成

kafka中的delayservice

kafka-stream

scala代码连接kafka

kafka最好视频教程

stream 根据某个字段分类

这个数据是存储在kafka中的，统计的功能尽量用kafka自带的功能实现

Java Kafka常用API

stream筛选字段

最新资源