MySQL 5.7 NDB Cluster 7.5技术概览与新特性

需积分: 10 73 浏览量更新于2024-07-15 收藏 5.33MB PDF 举报

MySQL Cluster NDB 7.5是MySQL 5.7参考手册中的一个详细章节，它介绍了MySQL Cluster这个分布式数据库系统在7.5版本中的关键特性、架构和新功能。该文档不仅提供了技术细节，还包含了法律通知和用户支持指南。首先，文档的前言部分概述了MySQL NDB Cluster 7.5和其后续版本7.6.3的主要内容，以及文档的生成日期（2020年5月22日，修订版本为66080）。读者可以从中了解到关于MySQL Cluster的核心概念，包括集群节点、节点组、副本和分区等基本组件，这些都是构建高可用性和可扩展性数据库的基础。硬件、软件和网络要求在3.3节详细阐述，确保读者了解在实际部署时需要满足的基础设施条件。随着技术的更新，新版本的NDB Cluster引入了许多改进和新特性。在3.4节中，分别列出了NDB Cluster 7.5和7.6.3的新功能亮点，这有助于开发者跟踪技术发展，并确定哪些功能可能对他们的应用有直接影响。 3.5节则关注选项、变量和参数的变化，无论是新增、弃用还是已移除的，这对于开发者理解和配置NDB Cluster至关重要。通过这些信息，他们可以确保他们的数据库配置是最新的，同时避免潜在的兼容性问题。接下来的3.6节讨论了NDB Cluster与InnoDB存储引擎之间的区别。虽然InnoDB是MySQL的标准内存表存储引擎，但NDB Cluster提供了不同的设计哲学，适用于处理大规模并发和高吞吐量的应用场景。这里会对比两者在存储模型、工作负载适应性和性能上的异同。 MySQL Cluster NDB 7.5的这份提取文档是MySQL 5.7用户和管理员的重要参考资料，提供了关于集群设计、性能优化、新功能和最佳实践的深入洞察。对于希望利用NDB Cluster进行分布式数据库管理的用户，阅读并理解这部分内容将极大地提升他们的技术水平和工作效率。

NDB Cluster Core Concepts

Figure 3.1 NDB Cluster Components

All these programs work together to form an NDB Cluster (see Chapter 6, NDB Cluster Programs.

When data is stored by the NDB storage engine, the tables (and table data) are stored in the data

nodes. Such tables are directly accessible from all other MySQL servers (SQL nodes) in the cluster.

Thus, in a payroll application storing data in a cluster, if one application updates the salary of an

employee, all other MySQL servers that query this data can see this change immediately.

Although an NDB Cluster SQL node uses the mysqld server daemon, it differs in a number of critical

respects from the mysqld binary supplied with the MySQL 5.7 distributions, and the two versions of

mysqld are not interchangeable.

In addition, a MySQL server that is not connected to an NDB Cluster cannot use the NDB storage

engine and cannot access any NDB Cluster data.

The data stored in the data nodes for NDB Cluster can be mirrored; the cluster can handle failures of

individual data nodes with no other impact than that a small number of transactions are aborted due

to losing the transaction state. Because transactional applications are expected to handle transaction

failure, this should not be a source of problems.

Individual nodes can be stopped and restarted, and can then rejoin the system (cluster). Rolling

restarts (in which all nodes are restarted in turn) are used in making configuration changes and

software upgrades (see Section 7.5, “Performing a Rolling Restart of an NDB Cluster”). Rolling restarts

are also used as part of the process of adding new data nodes online (see Section 7.15, “Adding NDB

Cluster Data Nodes Online”). For more information about data nodes, how they are organized in an

NDB Cluster, and how they handle and store NDB Cluster data, see Section 3.2, “NDB Cluster Nodes,

Node Groups, Replicas, and Partitions”.

Backing up and restoring NDB Cluster databases can be done using the NDB-native functionality

found in the NDB Cluster management client and the ndb_restore program included in the NDB

Cluster distribution. For more information, see Section 7.3, “Online Backup of NDB Cluster”, and

Section 6.24, “ndb_restore — Restore an NDB Cluster Backup”. You can also use the standard

MySQL functionality provided for this purpose in mysqldump and the MySQL server. See mysqldump

— A Database Backup Program, for more information.

NDB Cluster nodes can employ different transport mechanisms for inter-node communications; TCP/IP

over standard 100 Mbps or faster Ethernet hardware is used in most real-world deployments.

3.1 NDB Cluster Core Concepts

NDB Cluster Core Concepts

NDBCLUSTER (also known as NDB) is an in-memory storage engine offering high-availability and data-

persistence features.

The NDBCLUSTER storage engine can be configured with a range of failover and load-balancing

options, but it is easiest to start with the storage engine at the cluster level. NDB Cluster's NDB storage

engine contains a complete set of data, dependent only on other data within the cluster itself.

The “Cluster” portion of NDB Cluster is configured independently of the MySQL servers. In an NDB

Cluster, each part of the cluster is considered to be a node.

Note

In many contexts, the term “node” is used to indicate a computer, but when

discussing NDB Cluster it means a process. It is possible to run multiple nodes

on a single computer; for a computer on which one or more cluster nodes are

being run we use the term cluster host.

There are three types of cluster nodes, and in a minimal NDB Cluster configuration, there will be at

least three nodes, one of each of these types:

• Management node: The role of this type of node is to manage the other nodes within the NDB

Cluster, performing such functions as providing configuration data, starting and stopping nodes, and

running backups. Because this node type manages the configuration of the other nodes, a node of

this type should be started first, before any other node. An MGM node is started with the command

ndb_mgmd.

• Data node: This type of node stores cluster data. There are as many data nodes as there are

replicas, times the number of fragments (see Section 3.2, “NDB Cluster Nodes, Node Groups,

Replicas, and Partitions”). For example, with two replicas, each having two fragments, you need four

data nodes. One replica is sufficient for data storage, but provides no redundancy; therefore, it is

recommended to have 2 (or more) replicas to provide redundancy, and thus high availability. A data

node is started with the command ndbd (see Section 6.1, “ndbd — The NDB Cluster Data Node

Daemon”) or ndbmtd (see Section 6.3, “ndbmtd — The NDB Cluster Data Node Daemon (Multi-

Threaded)”).

NDB Cluster tables are normally stored completely in memory rather than on disk (this is why we

refer to NDB Cluster as an in-memory database). However, some NDB Cluster data can be stored

on disk; see Section 7.13, “NDB Cluster Disk Data Tables”, for more information.

• SQL node: This is a node that accesses the cluster data. In the case of NDB Cluster, an SQL node

is a traditional MySQL server that uses the NDBCLUSTER storage engine. An SQL node is a mysqld

process started with the --ndbcluster and --ndb-connectstring options, which are explained

elsewhere in this chapter, possibly with additional MySQL server options as well.

An SQL node is actually just a specialized type of API node, which designates any application which

accesses NDB Cluster data. Another example of an API node is the ndb_restore utility that is

used to restore a cluster backup. It is possible to write such applications using the NDB API. For

basic information about the NDB API, see Getting Started with the NDB API.

Important

It is not realistic to expect to employ a three-node setup in a production

environment. Such a configuration provides no redundancy; to benefit from NDB

Cluster's high-availability features, you must use multiple data and SQL nodes.

The use of multiple management nodes is also highly recommended.

For a brief introduction to the relationships between nodes, node groups, replicas, and partitions in

NDB Cluster, see Section 3.2, “NDB Cluster Nodes, Node Groups, Replicas, and Partitions”.

Configuration of a cluster involves configuring each individual node in the cluster and setting up

individual communication links between nodes. NDB Cluster is currently designed with the intention

NDB Cluster Core Concepts

that data nodes are homogeneous in terms of processor power, memory space, and bandwidth. In

addition, to provide a single point of configuration, all configuration data for the cluster as a whole is

located in one configuration file.

The management server manages the cluster configuration file and the cluster log. Each node in

the cluster retrieves the configuration data from the management server, and so requires a way to

determine where the management server resides. When interesting events occur in the data nodes,

the nodes transfer information about these events to the management server, which then writes the

information to the cluster log.

In addition, there can be any number of cluster client processes or applications. These include

standard MySQL clients, NDB-specific API programs, and management clients. These are described in

the next few paragraphs.

Standard MySQL clients. NDB Cluster can be used with existing MySQL applications written in

PHP, Perl, C, C++, Java, Python, Ruby, and so on. Such client applications send SQL statements to

and receive responses from MySQL servers acting as NDB Cluster SQL nodes in much the same way

that they interact with standalone MySQL servers.

MySQL clients using an NDB Cluster as a data source can be modified to take advantage of the ability

to connect with multiple MySQL servers to achieve load balancing and failover. For example, Java

clients using Connector/J 5.0.6 and later can use jdbc:mysql:loadbalance:// URLs (improved

in Connector/J 5.1.7) to achieve load balancing transparently; for more information about using

Connector/J with NDB Cluster, see Using Connector/J with NDB Cluster.

NDB client programs. Client programs can be written that access NDB Cluster data directly from

the NDBCLUSTER storage engine, bypassing any MySQL Servers that may be connected to the cluster,

using the NDB API, a high-level C++ API. Such applications may be useful for specialized purposes

where an SQL interface to the data is not needed. For more information, see The NDB API.

NDB-specific Java applications can also be written for NDB Cluster using the NDB Cluster Connector

for Java. This NDB Cluster Connector includes ClusterJ, a high-level database API similar to object-

relational mapping persistence frameworks such as Hibernate and JPA that connect directly to

NDBCLUSTER, and so does not require access to a MySQL Server. Support is also provided in NDB

Cluster for ClusterJPA, an OpenJPA implementation for NDB Cluster that leverages the strengths of

ClusterJ and JDBC; ID lookups and other fast operations are performed using ClusterJ (bypassing the

MySQL Server), while more complex queries that can benefit from MySQL's query optimizer are sent

through the MySQL Server, using JDBC. See Java and NDB Cluster, and The ClusterJ API and Data

Object Model, for more information.

NDB Cluster also supports applications written in JavaScript using Node.js. The MySQL Connector

for JavaScript includes adapters for direct access to the NDB storage engine and as well as for the

MySQL Server. Applications using this Connector are typically event-driven and use a domain object

model similar in many ways to that employed by ClusterJ. For more information, see MySQL NoSQL

Connector for JavaScript.

The Memcache API for NDB Cluster, implemented as the loadable ndbmemcache storage engine

for memcached version 1.6 and later, can be used to provide a persistent NDB Cluster data store,

accessed using the memcache protocol.

The standard memcached caching engine is included in the NDB Cluster 7.5 distribution. Each

memcached server has direct access to data stored in NDB Cluster, but is also able to cache data

locally and to serve (some) requests from this local cache.

For more information, see ndbmemcache—Memcache API for NDB Cluster.

Management clients. These clients connect to the management server and provide commands for

starting and stopping nodes gracefully, starting and stopping message tracing (debug versions only),

showing node versions and status, starting and stopping backups, and so on. An example of this type

of program is the ndb_mgm management client supplied with NDB Cluster (see Section 6.5, “ndb_mgm

NDB Cluster Nodes, Node Groups, Replicas, and Partitions

— The NDB Cluster Management Client”). Such applications can be written using the MGM API, a C-

language API that communicates directly with one or more NDB Cluster management servers. For

more information, see The MGM API.

Oracle also makes available MySQL Cluster Manager, which provides an advanced command-line

interface simplifying many complex NDB Cluster management tasks, such restarting an NDB Cluster

with a large number of nodes. The MySQL Cluster Manager client also supports commands for getting

and setting the values of most node configuration parameters as well as mysqld server options

and variables relating to NDB Cluster. See MySQL™ Cluster Manager 1.4.8 User Manual, for more

information.

Event logs. NDB Cluster logs events by category (startup, shutdown, errors, checkpoints, and

so on), priority, and severity. A complete listing of all reportable events may be found in Section 7.6,

“Event Reports Generated in NDB Cluster”. Event logs are of the two types listed here:

• Cluster log: Keeps a record of all desired reportable events for the cluster as a whole.

• Node log: A separate log which is also kept for each individual node.

Note

Under normal circumstances, it is necessary and sufficient to keep and examine

only the cluster log. The node logs need be consulted only for application

development and debugging purposes.

Checkpoint. Generally speaking, when data is saved to disk, it is said that a checkpoint has been

reached. More specific to NDB Cluster, a checkpoint is a point in time where all committed transactions

are stored on disk. With regard to the NDB storage engine, there are two types of checkpoints which

work together to ensure that a consistent view of the cluster's data is maintained. These are shown in

the following list:

• Local Checkpoint (LCP): This is a checkpoint that is specific to a single node; however, LCPs

take place for all nodes in the cluster more or less concurrently. An LCP usually occurs every few

minutes; the precise interval varies, and depends upon the amount of data stored by the node, the

level of cluster activity, and other factors.

Prior to NDB 7.6.4, an LCP involved saving all of a node's data to disk. NDB 7.6.4 introduces

support for partial LCPs, which can significantly improve recovery time under some conditions. See

Section 3.4.2, “What is New in NDB Cluster 7.6”, for more information, as well as the descriptions of

the EnablePartialLcp and RecoveryWork configuration parameters which enable partial LCPs

and control the amount of storage they use.

• Global Checkpoint (GCP): A GCP occurs every few seconds, when transactions for all nodes are

synchronized and the redo-log is flushed to disk.

For more information about the files and directories created by local checkpoints and global

checkpoints, see NDB Cluster Data Node File System Directory.

3.2 NDB Cluster Nodes, Node Groups, Replicas, and Partitions

This section discusses the manner in which NDB Cluster divides and duplicates data for storage.

A number of concepts central to an understanding of this topic are discussed in the next few

paragraphs.

Data node. An ndbd or ndbmtd process, which stores one or more replicas—that is, copies of the

partitions (discussed later in this section) assigned to the node group of which the node is a member.

Each data node should be located on a separate computer. While it is also possible to host multiple

data node processes on a single computer, such a configuration is not usually recommended.

NDB Cluster Nodes, Node Groups, Replicas, and Partitions

It is common for the terms “node” and “data node” to be used interchangeably when referring to an

ndbd or ndbmtd process; where mentioned, management nodes (ndb_mgmd processes) and SQL

nodes (mysqld processes) are specified as such in this discussion.

Node group. A node group consists of one or more nodes, and stores partitions, or sets of replicas

(see next item).

The number of node groups in an NDB Cluster is not directly configurable; it is a function of the number

of data nodes and of the number of replicas (NoOfReplicas configuration parameter), as shown here:

[# of node groups] = [# of data nodes] / NoOfReplicas

Thus, an NDB Cluster with 4 data nodes has 4 node groups if NoOfReplicas is set to 1 in the

config.ini file, 2 node groups if NoOfReplicas is set to 2, and 1 node group if NoOfReplicas is

set to 4. Replicas are discussed later in this section; for more information about NoOfReplicas, see

Section 5.3.6, “Defining NDB Cluster Data Nodes”.

Note

All node groups in an NDB Cluster must have the same number of data nodes.

You can add new node groups (and thus new data nodes) online, to a running NDB Cluster; see

Section 7.15, “Adding NDB Cluster Data Nodes Online”, for more information.

Partition. This is a portion of the data stored by the cluster. Each node is responsible for keeping at

least one copy of any partitions assigned to it (that is, at least one replica) available to the cluster.

The number of partitions used by default by NDB Cluster depends on the number of data nodes and

the number of LDM threads in use by the data nodes, as shown here:

[# of partitions] = [# of data nodes] * [# of LDM threads]

When using data nodes running ndbmtd, the number of LDM threads is controlled by the setting

for MaxNoOfExecutionThreads. When using ndbd there is a single LDM thread, which means

that there are as many cluster partitions as nodes participating in the cluster. This is also the

case when using ndbmtd with MaxNoOfExecutionThreads set to 3 or less. (You should be

aware that the number of LDM threads increases with the value of this parameter, but not in a

strictly linear fashion, and that there are additional constraints on setting it; see the description of

MaxNoOfExecutionThreads for more information.)

NDB and user-defined partitioning. NDB Cluster normally partitions NDBCLUSTER tables

automatically. However, it is also possible to employ user-defined partitioning with NDBCLUSTER tables.

This is subject to the following limitations:

1. Only the KEY and LINEAR KEY partitioning schemes are supported in production with NDB tables.

2. The maximum number of partitions that may be defined explicitly for any NDB table is 8 *

[number of LDM threads] * [number of node groups], the number of node groups

in an NDB Cluster being determined as discussed previously in this section. When running ndbd

for data node processes, setting the number of LDM threads has no effect (since ThreadConfig

applies only to ndbmtd); in such cases, this value can be treated as though it were equal to 1 for

purposes of performing this calculation.

See Section 6.3, “ndbmtd — The NDB Cluster Data Node Daemon (Multi-Threaded)”, for more

information.

For more information relating to NDB Cluster and user-defined partitioning, see Section 3.7, “Known

Limitations of NDB Cluster”, and Partitioning Limitations Relating to Storage Engines.

Replica. This is a copy of a cluster partition. Each node in a node group stores a replica. Also

sometimes known as a partition replica. The number of replicas is equal to the number of nodes per

node group.

剩余637页未读，继续阅读

Liantao007

粉丝: 0
资源: 4

MySQL 5.7 NDB Cluster 7.5技术概览与新特性

Linux 6.x 下mysql 5.7 galera 集群实战

mysql-cluster-excerpt-5.1-en.a4.pdf

mysql-tutorial-excerpt-8.0-en.a4.pdf

如何使用jQuery进行个人博客网页的设计与实现？

python知乎热榜数据爬取

用scrapy编写一个知乎热榜的爬虫

IBMMQ jmsadmin

redis bind 失败

爬取网站为https://cq.58.com/ershoufang/的重庆挂牌出售的全部二手房信息信息。爬取信息包括卖点、楼盘、楼盘地址、房屋户型、楼层、建筑年代、每平单价、房屋总价。

要动态采集两个栏目的数据，如果这一页中最新发布的数据和当前时间相比相差3天及以上，需要给我微信推送一条超期信息； 1、实现数据写入mysql数据库的思路 2、实现微信推送代码开发

最新资源