27.1.1.
概念 ...................................................................
269
27.1.1.
集群架构 ...............................................................
269
27.1.1.1.
Nimbus(master-代码分发给 Supervisor) .........................................269
27.1.1.2.
Supervisor(slave-管理 Worker 进程的启动和终止) ................................269
27.1.1.3.
Worker(具体处理组件逻辑的进程) ..............................................269
27.1.1.4.
Task.......................................................................................................................................270
27.1.1.5.
ZooKeeper.............................................................................................................................270
27.1.2.
编程模型(spout->tuple->bolt) ............................................
270
27.1.2.1.
Topology................................................................................................................................270
27.1.2.2.
Spout .....................................................................................................................................270
27.1.2.3.
Bolt ........................................................................................................................................270
27.1.2.4.
Tuple .....................................................................................................................................270
27.1.2.5.
Stream...................................................................................................................................271
27.1.3.
Topology
运行 ...........................................................
271
(1). Worker(进程) (2). Executor(线程) (3). Task ...........................................................................271
27.1.3.1.
Worker(1 个 worker 进程执行的是 1 个 topology 的子集)...................................................271
27.1.3.2.
Executor(executor 是 1 个被 worker 进程启动的单独线程) .................................................271
27.1.3.3.
Task(最终运行 spout 或 bolt 中代码的单元) ........................................................................272
27.1.4.
Storm Streaming Grouping ............................................................................................272
27.1.4.1.
huffle Grouping......................................................................................................................273
27.1.4.2.
Fields Grouping .....................................................................................................................273
27.1.4.3.
All grouping :广播 ..............................................................273
27.1.4.4.
Global grouping .....................................................................................................................274
27.1.4.5.
None grouping :不分组 .........................................................274
27.1.4.6.
Direct grouping :直接分组 指定分组 ..............................................274
28.
YARN .......................................................................................................................................275
28.1.1.
概念 ...................................................................
275
28.1.2.
ResourceManager .........................................................................................................275
28.1.3.
NodeManager................................................................................................................275
28.1.4.
ApplicationMaster................................................................................................................
276
28.1.5.
YARN 运行流程 ...................................................................................................................
277
29.
机器学习 ........................................................................278
29.1.1.
决 策树 .................................................................
278
29.1.2.
随机森林算法 ...........................................................
278
29.1.3.
逻辑回归 ...............................................................
278
29.1.4. SVM ...............................................................................................................................278
29.1.5.
朴素贝叶斯 .............................................................
278
29.1.6.
K
最近邻算法............................................................
278
29.1.7.
K
均值算法 .............................................................
278
29.1.8.
Adaboost
算法...........................................................
278
29.1.9.
神经网络 ...............................................................
278
29.1.10.
马尔可夫 .............................................................
278
30. 云计算 ..........................................................................
279
30.1.1.
SaaS
279
30.1.2.
PaaS
279
30.1.3.
IaaS
279
30.1.4.
Docker
279
30.1.4.1. 概念 ...........................................................................279
30.1.4.2.
Namespaces .........................................................................................................................280
30.1.4.3.
进程(CLONE_NEWPID 实现的进程隔离) ..............................................................................281
30.1.4.4.
Libnetwork 与网络隔离...........................................................281
30.1.4.5.
资源隔离与 CGroups .............................................................................................................282
30.1.4.6.
镜像与 UnionFS.....................................................................................................................282
30.1.4.7.
存储驱动 .......................................................................282