Bloomberg的Comdb2：高可用分布式关系数据库系统

138 浏览量更新于2024-08-25 收藏 1.21MB PDF 举报

"Comdb2是Bloomberg公司开发的一个分布式关系型数据库系统，设计目标是实现地理复制和高可用性。与当前的数据库趋势不同，Comdb2支持完全的事务处理，采用标准的关系模型，并提供了SQL语言的强大表达能力。此外，系统还允许使用一种Lua方言编写丰富的存储过程。Comdb2实现了一个可串行化的系统，在任何节点上进行读取操作总是返回最新值。通过内置的服务发现和标准API中的复杂重试逻辑，它提供透明的高可用性。Comdb2不仅有关系数据模型，还实现了发布者到订阅者的消息传递队列，这些队列可以与表触发器结合，实现时间一致的日志分布。" Comdb2是一个由Bloomberg开发的分布式数据库系统，它的核心特性在于地理复制和高可用性，这使得它在分布式环境中特别适合处理跨地域的数据同步和容灾。Comdb2坚持传统的事务处理特性，这意味着它支持ACID（原子性、一致性、隔离性和持久性）属性，保证了数据的一致性和完整性。其使用标准的关系模型和SQL语言，易于理解和使用，同时也具备了高度的灵活性和适应性。系统中的存储过程支持使用一种类似于Lua的方言，这为用户提供了在数据库内部执行复杂逻辑的能力，增强了数据库的功能。这种内置的编程能力使得Comdb2能够处理更复杂的业务场景，而无需额外依赖外部应用。 Comdb2的可串行化特性确保了在分布式环境下的数据一致性，任何节点上的读操作都能获取到最新的数据状态，避免了脏读和不可重复读的问题。此外，它通过内置的服务发现机制和API中的智能重试逻辑，实现了数据库服务的高可用性，即使在部分节点故障的情况下，也能保证服务的连续性和数据的完整性。 Comdb2不仅仅局限于传统的表格存储，还引入了消息队列功能，支持发布者到订阅者的消息传递。这种设计使得数据的实时更新和异步处理成为可能，尤其适用于事件驱动的应用场景。通过与表触发器的结合，可以实现时间一致的日志分布，这对于金融交易等对时间敏感的业务至关重要。 Comdb2是一个集成了传统关系数据库优势和现代分布式系统特性的强大数据库解决方案，它在保证数据一致性的同时，提供了高可用性和扩展性，是应对复杂业务需求的理想选择。

reconnect, allowing for tran s p a rent failure masking of IN-

SERT, UPDATE, and DELETE statements. Transactions

are supported as well. T h e OC C sys tem does not perform

any actual upda tes until a transaction commits, so retrying

is possible before commit. Global sequencing is used from

the cl ie nt to server all owing for retries a n d detections of re-

plays. The system does not return errors to applications

when a no d e in a cluster fails. Instead all forms of errors

resulting from node failures a re masked by transparent re-

tries.

3.2 Isolation Levels

Comdb2 allows applications to chose from a weaker ver-

sion of ANSI read committe d to an ANSI-compliant se-

rializable implementation depending on their correctness

and performance requirements.

Block is the weakest isolation level in Comdb 2. This is the

only isolation level available to non-SQL clients. In this

isolation level, only committed data can be seen. Comdb2

never o↵ers the ability to see uncommitted data. This iso-

lation level makes no e↵ort to mask the underlying OCC

nature of Comdb2, and as such, read s wit h in a tra n sa c t io n

are unable to see uncommitted writes that have occ u rred

within the same transac t io n . Many applications are ab le

to function properly with the phenomena present in this

level.

Read Committed behaves like block,butadditionallyal-

lows clients to read rows which have been written within

the current transaction. Reads merge committ ed data with

the current transac ti o n ’s uncommitted changes. Changes

local to the transaction are stored in data structures de-

scribed in Section 4.

Snapshot Isolation imp lements Snapsh ot Isolation as de-

ﬁned i n [2]. Private copies of rows are synthesized as needed

when pages are read which have been modiﬁed after the

start LSN of the transaction.

Serializable implem ents a fully serializable system. As an

OCC system, any transaction which would result in a non

serializable history is aborted a t commit time and returns a

non-serializable error. Transactions do not block or dead-

lock. Serializable isolation adds additiona l validation to

Snapshot Isolation in the form of read-w rit e conﬂict detec-

tion.

3.3 Optimistic Concurrency Control

Concurrency control models fall into two categories: op-

timistic and pessimistic. An optimistic model anticipates

a workload where resource contention will not often occur,

whereas a pessimistic model anticipates a workload ﬁlled

with contention.

Most commercialized database eng i n es adopt a pessimistic

approach whereby rows are manipulated under a safe lock,

speciﬁcally: (1) a read operation will block a write, (2) a

write will block a read, and (3) multiple reads will hold a

“shared” lock tha t blocks any write to the same row. In a

classical two-phase locking (2 P L) scheme every acquired lock

is held until the tra n sac t io n is committed or aborted, hence

blocking every transaction that tries to work on the data

under a lock. Even MVCC based systems acquire tran s a c-

tion duration write locks on rows being modiﬁed while an

OCC system never obt a i n s long term write locks.

In an OCC system, transactions are executed concurrently

without having to wait for each other to access the rows.

Read operations, in p a rt ic u l ar, will have no restrictions as

they cannot compromise the integrity of the data. Write

operations will operate on temporary copies of rows. Since

persisting the transactions as they are execute d would likely

violate the ACID pro pert ie s, the execution of each transac-

tion has to be validated against the others.

Comdb2 uses a form of Backwards Optimistic Concur-

rency Control (BOCC) [14] with concurrent validat io n . Two

distinct validation phases prevent anoma lies such as over-

writing uncommitted data, unrepeatable reads and write skew.

This is a hybrid system using locking for some functions,

while adhering to a more traditional OCC approach for oth-

ers.

In order to detect Write-Write conﬂicts, Comdb2 uses a

form of deferred 2PL to allow for concurrent validation with-

out a critical section [33]. This is based on the notion of a

genid - GENeration IDentiﬁer - associated with each row.

Every modiﬁcation to a row changes its genid,and

genids can never be reused. Genids are latched -i.e.

remembered - during the execution of a transaction when

rows are modiﬁed, and later validated at commit time using

2PL on the rows. The struct u re used to record such modi-

ﬁcations is referred to as the Block Processor Log (bplog).

As the genid forms a key into the data internally, the

existence of a genid in the system is suﬃcient to assert the

existence and stability of a row before committing. Comdb2

incurs no extra overh ea d in recording all overlapping write

sets for validation, as a standard Write-Ahead Log (WAL)

protocol demands that write sets be logged al rea d y.

Read-Write conﬂicts are add ress ed by n o n - d u ra b ly record-

ing the read set of a transaction as degenerate predicates

consisting of rows, ra n g es and tables. During the validati o n

phase the overlapping write sets fro m the WAL are checked

for conﬂicts against the transa c ti o n ’s read set. Validation

runs backwards in several phases.

The ultimate commit operation oc c urs in a critical sec-

tion but pre-validation ensures that duration will be brief.

Replicants running a transaction are able to begin validation

concurrently up to the LSN pres ent on that node. The vali-

dation burden then moves to t h e master in a repeated cycle

of validations outsid e the critical section. The critical sec-

tion is entered for ﬁnal validation onc e pre validation is near

enough to the current LSN as determined by a tunable.

3.4 Replication

A transaction in Comdb2 goes through several distinct

phases on various nodes o f the cluster, as shown in Fig. 2. In

the initial ph a se, the client connects to a geographi c al ly close

replicant (Fig. 2a), typically in th e same data center. The

interactive phase o f the transaction (SELECT, INSERT,

UPDATE, DELETE operations (Fig. 2b)) o c c u rs entirely

on that replicant. We will refer to this as the OCC phase

of the transaction lifecycle as no locks are acquired. During

the exec u ti o n of this phase write operations are recorded for

purposes of later execution and valid a tio n . This reco rd in g

occurs o n the bplog which is continually shipped to the mas-

ter and bu↵ered. When the client application ﬁnally COM-

MITs, the master begins the second pha se of the tra n s a ct i o n

lifecycle (Fig. 2c). This is a 2PL phase in which ope ra ti o n s

are both written and validated to detect O CC read-write or

write-write conﬂicts (Fig. 2d ). The master generates phys-

1379

剩余11页未读，继续阅读

weixin_38632247

粉丝: 8
资源: 1000

Bloomberg的Comdb2：高可用分布式关系数据库系统

cpp-Comdb2基于乐观并发控制技术的群集RDBMS

CNACE - Com Name Arbiter ComDB Editor-开源

comdb2:彭博的分布式 RDBMS

comdb:一个PouchDB插件，透明地加密和解密其数据

dotenv-schema:为 dotenv 定义架构

彭博的分布式RDBMS-C/C++开发

解决win7COM2~COM7被占用

吴怀玉2017年SimpleDB批量存储系统述职报告

weixin151云匹面粉直供微信小程序+springboot后端毕业源码案例设计.zip

1_基于CEEMDAN-EDO的行波波头标定算法研究_李英春.pdf

最新资源