使用Kafka Streams构建实时应用与微服务

需积分: 13 127 浏览量更新于2024-07-18 收藏 16.69MB PDF 举报

"Kafka Streams in Action 是一本由 William P. Bejeck Jr. 编写的，由 Neha Narkhede 前言的书籍，由 Manning 出版社出版。这本书专注于使用 Kafka Streams API 开发实时应用程序和微服务。书中通过一系列模式和处理器示例，展示了如何构建复杂的数据流处理系统。" 本书详细介绍了 Kafka Streams，这是一个用于构建实时数据处理应用程序的 Java 库，它允许开发者直接在 Kafka 集群上运行计算任务，而无需单独的流处理平台。Kafka Streams 提供了一种轻量级的方式，使得在分布式环境中处理数据流变得简单且高效。书中的关键知识点包括： 1. **实时应用程序与微服务**：Kafka Streams 为构建实时处理系统提供了一个强大的工具，适合现代微服务架构。它支持快速响应、低延迟的数据处理，这对于实时业务决策和监控至关重要。 2. **Kafka Streams API**：书中详细解释了 API 的使用，包括如何创建流处理作业，设置源和接收器（source and sink），以及中间处理器（processor）的配置和操作。 3. **数据流处理模式**：书中列举并讨论了多个处理模式，如 Masking（掩码）、Select-key（选择键）等，这些模式有助于解决实际场景中的数据转换和过滤问题。 4. **处理器拓扑**：书中展示了如何构建复杂的处理器拓扑，例如 Branch（分支）、Filtering（过滤）和 Cafe、Electronics、Rewards 等特定场景的处理器，帮助读者理解如何根据业务需求设计和实现数据处理流程。 5. **数据转换与过滤**：通过 Cafe、Electronics 和 Rewards 等处理器示例，作者深入讲解了如何使用 Kafka Streams 进行数据筛选和转换，这对于数据清洗、分析和业务规则应用至关重要。 6. **集成与部署**：除了理论知识，书中也涵盖了 Kafka Streams 与其他系统的集成，以及在生产环境中的部署策略，帮助开发者了解如何在实际项目中实施。 7. **最佳实践与性能优化**：作者可能还讨论了使用 Kafka Streams 的最佳实践，包括如何进行性能调优，以确保系统在高并发和大数据量下的稳定性和效率。 8. **案例研究**：书中可能包含实际案例，以帮助读者将理论知识应用于实际业务场景，增强对 Kafka Streams 的理解和应用能力。通过这本书，读者不仅可以学习到 Kafka Streams 的核心概念和技术，还能掌握如何利用这些技术来构建和优化实时数据处理系统，以适应不断变化的业务需求。对于想要深入了解和使用 Kafka Streams 的开发者来说，这是一本不可多得的参考资料。

xiv

acknowledgments

First and foremost, I’d like to thank my wife Beth and acknowledge all the support I

received from her during this process. Writing a book is a time-consuming task, and

without her encouragement, this book never would have happened. Beth, you are fan-

tastic, and I’m very grateful to have you as my wife. I’d also like to acknowledge my

children, who put up with Dad sitting in his office all day on most weekends and

accepted the vague answer “Soon” when they asked when I’d be finished writing.

Next, I thank Guozhang Wang, Matthias Sax, Damian Guy, and Eno Thereska, the

core developers of Kafka Streams. Without their brilliant insights and hard work,

there would be no Kafka Streams, and I wouldn’t have had the chance to write about

this game-changing tool.

I thank my editor at Manning, Frances Lefkowitz, whose expert guidance and

infinite patience made writing a book almost fun. I also thank John Hyaduck for his

spot-on technical feedback, and Valentin Crettaz, the technical proofer, for his excel-

lent work reviewing the code. Additionally, I thank the reviewers for their hard work

and invaluable feedback in making the quality of this book better for all readers:

Alexander Koutmos, Bojan Djurkovic, Dylan Scott, Hamish Dickson, James Frohnhofer,

Jim Manthely, Jose San Leandro, Kerry Koitzsch, László Hegedüs, Matt Belanger,

Michele Adduci, Nicholas Whitehead, Ricardo Jorge Pereira Mano, Robin Coe, Sumant

Tambe, and Venkata Marrapu.

Finally, I’d like to acknowledge all the Kafka developers for building such high-

quality software, especially Jay Kreps, Neha Narkhede, and Jun Rao—not just for start-

ing Kafka in the first place, but also for founding Confluent, a great and inspiring

place to work.

ABOUT THIS BOOK

xvi

How this book is organized: a roadmap

This book has four parts spread over nine chapters. Part 1 introduces a mental model

of Kafka Streams to show you the big-picture view of how it works. These chapters also

provide the basics of Kafka, for those who need them or want a review:

■

Chapter 1 provides some history of how and why stream processing became

necessary for handling real-time data at scale. It also presents the mental model

of Kafka Streams. I don’t go over any code but rather describe how Kafka

Streams works.

■

Chapter 2 is a primer for developers who are new to Kafka. Those with more

experience with Kafka can skip this chapter and get right into Kafka Streams.

Part 2 moves on to Kafka Streams, starting with the basics of the API and continuing

to the more complex features:

■

Chapter 3 presents a Hello World application and then presents a more realistic

example: developing an application for a fictional retailer, including advanced

features.

■

Chapter 4 discusses state and explains how it’s sometimes required for stream-

ing applications. You’ll learn about state store implementations and how to per-

form joins in Kafka Streams.

■

Chapter 5 explores the duality of tables and streams, and introduces a new con-

cept: the

KTable

. Whereas a

KStream

is a stream of events, a

KTable

is a stream

of related events or an update stream.

■

Chapter 6 goes into the low-level Processor API. Up to this point, you’ve been

working with the high-level DSL, but here you’ll learn how to use the Processor

API when you need to write customized parts of an application.

Part 3 moves on from developing Kafka Streams applications to managing Kafka

Streams:

■

Chapter 7 explains how to test a Kafka Streams application. You’ll learn how to

test an entire topology, unit-test a single processor, and use an embedded Kafka

broker for integration tests.

■

Chapter 8 covers how to monitor your Kafka Streams application, both to see

how long it takes to process records and to locate potential processing bottle-

necks.

Part 4 is the capstone of the book, where you’ll delve into advanced application devel-

opment with Kafka Streams:

■

Chapter 9 covers integrating existing data sources into Kafka Streams using

Kafka Connect. You’ll learn to include database tables in a streaming applica-

tion. Then, you’ll see how to use interactive queries to provide visualization

and dashboard applications while data is flowing through Kafka Streams, with-

out the need for relational databases. The chapter also introduces KSQL,

构思模型

虚构的零售商

二元性;

压顶石;

ABOUT THIS BOOK

xvii

which you can use to run continuous queries over Kafka without writing any

code, by using SQL.

About the code

This book contains many examples of source code both in numbered listings and

inline with normal text. In both cases, source code is formatted in a

fixed-width

font

this

to separate it from ordinary text.

In many cases, the original source code has been reformatted; we’ve added line

breaks and reworked indentation to accommodate the available page space in the

book. In rare cases, even this was not enough, and listings include line-continuation

markers (

➥

). Additionally, comments in the source code have often been removed

from the listings when the code is described in the text. Code annotations accompany

many of the listings, highlighting important concepts.

Finally, it’s important to note that many of the code examples aren’t meant to

stand on their own: they’re excerpts containing only the most relevant parts of what is

currently under discussion. You’ll find all the examples from the book in the accom-

panying source code in their complete form. Source code for the book’s examples is

available from GitHub at https://github.com/bbejeck/kafka-streams-in-action and

the publisher’s website at www.manning.com/books/kafka-streams-in-action.

The source code for the book is an all-encompassing project using the build tool

Gradle (https://gradle.org). You can import the project into either IntelliJ or Eclipse

using the appropriate commands. Full instructions for using and navigating the source

code can be found in the accompanying README.md file.

Book forum

Purchase of Kafka Streams in Action includes free access to a private web forum run by

Manning Publications where you can make comments about the book, ask technical

questions, and receive help from the author and from other users. To access the

forum, go to https://forums.manning.com/forums/kafka-streams-in-action. You can

also learn more about Manning’s forums and the rules of conduct at https://forums

.manning.com/forums/about.

Manning’s commitment to our readers is to provide a venue where a meaningful

dialogue between individual readers and between readers and the author can take

place. It is not a commitment to any specific amount of participation on the part of

the author, whose contribution to the forum remains voluntary (and unpaid). We sug-

gest you try asking him some challenging questions lest his interest stray! The forum

and the archives of previous discussions will be accessible from the publisher’s website

as long as the book is in print.

剩余281页未读，继续阅读

daniudaniudaniu

粉丝: 0
资源: 10

使用Kafka Streams构建实时应用与微服务

kafka stream in action

高清彩版 Kafka_Streams_in_Action

Kafka Streams in Action (MEAP V9)

分享一下kafka streams的学习资料

生成一个kafka streams 调用kafka java代码

Kafka Streams或Kafka Connect等组件来实现延迟发送。

springboot怎么集成kafka streams实现消息按时间戳消费

kafka stream和kafka区别

kafkastream in action

kafka的dwd层

最新资源