融合关系型与NoSQL数据库：应对大数据挑战

需积分: 3 164 浏览量更新于2024-07-19 收藏 5.76MB PDF 举报

"这篇文档是关于融合关系型数据库与NoSQL数据库的探讨，旨在解决不同数据库系统间的集成问题。" 在IT行业中，数据库系统扮演着至关重要的角色，尤其是在大数据时代。传统的关系型数据库（Relational Database）如Oracle、MySQL等，因其严格的数据结构和ACID（原子性、一致性、隔离性、持久性）特性，长期主导了数据存储领域。然而，随着互联网和移动应用的爆发式增长，传统数据库在处理海量、高并发、非结构化数据时暴露出局限性，这催生了NoSQL（Not Only SQL）数据库的兴起。标题"《Bridging Relational and NoSQL Databases》"揭示了本书的主题，即研究如何将关系型数据库与NoSQL数据库的优势相结合，以应对现代数据管理的需求。描述虽简洁，但暗示了作者将深入探讨两种数据库之间的差异、互补性和集成策略。标签"NoSQL"表明内容将重点围绕NoSQL数据库的特点和发展展开。NoSQL数据库以键值对、文档型、列族和图形数据库等多种数据模型，提供了高扩展性、灵活性和水平扩展的能力，适应了大数据时代的挑战。书中内容涵盖了以下几个方面： 1. **章节1：《The End of Relational Databases Domination》** - 这部分讨论了数据库发展的三个阶段，强调了关系型数据库的基石，如规范化理论和ACID属性。同时，介绍了BASE（基本可用、软状态、最终一致性）原则和CAP定理，这些原则为NoSQL设计提供了理论基础。此外，还讨论了分区（Sharding）现象，以及关系型数据库在处理大规模数据时面临的限制。 2. **章节2：《NoSQL Database Phenomenon》** - 本章深入到NoSQL的起源，分析了大数据带来的挑战，特别是Hadoop分布式文件系统的出现。NoSQL数据库的产生是为了应对大数据的处理需求，其特点是无固定模式（Schema-free/Schemaless），并按照键值存储、文档型、列族和图形四种类型进行分类。 3. **章节3：《NoSQL Data Modeling》** - 数据建模在关系型数据库中的重要性不言而喻，但在NoSQL中，数据建模呈现出不同的方式。本章介绍了超越实体关系模型的数据建模方法，如概念建模和对象建模，这些都是在NoSQL环境中进行有效数据组织的关键。 4. **章节4：《How NoSQL Databases Work》** - 该章讲解了NoSQL数据库的工作原理，包括分布式存储架构、一致性模型以及查询执行机制。这些内容对于理解NoSQL如何处理高并发和分布式环境下的数据至关重要。 5. **Section2：《Bringing Together Relational and NoSQL Databases》** - 这部分可能是全书的核心，将讨论如何在实际应用中结合关系型数据库和NoSQL数据库，实现数据的无缝交互。可能包括数据迁移策略、混合架构的设计、数据一致性和事务处理的解决方案。通过以上内容，读者不仅可以深入了解NoSQL数据库的特性和运作机制，还能掌握如何在实际项目中结合两种数据库，以实现更高效、更灵活的数据管理和处理。这种融合策略在当今的云计算、物联网和大数据分析等领域具有广泛的实践价值。

schemaless, that is they enable developers to start storing data right away, without losing time on

schema creation. However, the time savings at the beginning of the project are usually the cause

of many problems later, both in the data storage itself and in the efficiency of their maintenance

and query performance. This is usually manifested when handling large amounts of data, which

requires data reorganization to speed up the access and update times and find the appropriate

balance between consistency, availability, and access speed. However, if the amount of data is

large, their reorganization can be expensive, both in terms of time and the space necessary for

temporary storage, all of which can cause reduced availability of data for users. If the database is

implemented without previously creating a model (schemaless), the only way to understand the

data after the implementation is to speak with the developer or to analyze the code. The user’s

dependence on developers to understand the code significantly reduces his/her possibility of

proposing changes and expansions of data and may create additional pressure on developers.

The integration of relational and NoSQL databases is one of the possible ways of bridging them.

This approach is also generating specific challenges, including the following:

• Increased responsibility of developers in the optimization of database processes, which

means that additional programming and time are necessary for completing this task.

• Difficulties in finding developers with good knowledge of NoSQL databases, while

finding developers with knowledge about both worlds, relational and NoSQL, is still equal

to a jackpot.

• Finding an appropriate driver or tool for a particular problem is a time-consuming and

exhaustive process because the market is full of many different drivers and tools offered

by third vendors.

As a result, big database vendors (Oracle, Microsoft, and IBM) are oriented toward NoSQL

databases, with the main aim of offering the best from both worlds in one database management

system to their users.

SEARCHING FOR A SOLUTION

The very first motif behind the development of NoSQL databases is to find solutions for the

abovementioned Big Data challenges. The solutions have been searched in innovative NoSQL

features, including a distributed storage architecture that comprises: distributed, cluster-oriented,

and horizontally scalable features; a consistency model that refers to CAP (consistency,

availability, partitioning) and BASE (basically available, soft-state, eventually consistent)

features; a query execution that refers to the schemaless feature.

Data distribution and use of clusters are natural features of NoSQL databases. The ability of

NoSQL databases to run on a large cluster is the feature that has drawn attention to this type of

databases.

One of the pillars of relational databases is the ACID transaction approach. The implementation

of ACID transaction properties ensures efficient concurrency control and recovery process in

case of limited scale-up and shared-something environments, which are natural environments for

relational databases. Nevertheless, with the emergence of the Big Data phenomenon and the

urgent need for storing an enormous quantity of data across multiple database servers, the ACID

approach has become a serious limitation. The result has been the development of the BASE

approach, which has become popular, especially with the development of NoSQL databases.

These databases, which are distributed naturally, have been designed to avoid the restrictions of

strict ACID consistency and to be promoted as databases that do not support the ACID

transaction approach. This often leads to the misconception that NoSQL databases provide only a

weak or, at best, eventual consistency, and that their consistency implementation mechanisms are

very simple. However, NoSQL databases offer different consistency guarantees, including strict

consistency, even if only at the level of an individual object. Also, owing to their departure from

strict and predictable rules of ACID models, most NoSQL databases have developed complex

mechanisms for establishing acceptable levels of consistency.

The increasing number of NoSQL database applications has brought about a growing awareness

of the need for data modeling in these systems, too. Hence, efforts are made to find a solution

that will enable modeling both relational and NoSQL databases. Wang (2016) proposes the use

of unified data modeling (UML) techniques that support features like the document schema of

NoSQL databases, reverse engineering of data from an existing database, and visual refactoring

of existing databases.

Also, Hills (2016) proceeds from the idea that a single model can be used for both relational and

NoSQL databases at the logical level, whereas on the physical side multiple models can be

considered. This is why Hills’s (2016) concept and object modeling notation (COMN) was

designed to enable data modeling in both relational and NoSQL databases. The basic idea of

COMN notation is to integrate real objects, data, and implementation to allow all operations and

functions, from user requests to logical models, to functional databases, to be represented in a

single model, regardless of whether it is executed in a relational or NoSQL database management

system.

The integration of relational and NoSQL databases is one of the possible solutions of bridging

them. Three approaches to integration are discussed: native, hybrid, and reduction to one

solution, either relational or NoSQL.

A native solution involves the use of standard drivers and ways in which the business layer

communicates with a particular database. Since a part of the data is in NoSQL and the other part

in the relational database, it is necessary to collate data. This is implemented in the business

layer. When data is extracted from a database, they are linked and converted into a format which

is suitable for use in the user layer. When data is stored in a database, the business layer is

programmed to prepare data for storage and pass them on to the particular database.

A hybrid solution introduces an additional layer, which provides SQL communication between

the business layer and the data layer. Developers continue to use familiar SQL programming

patterns in the business layer, and employ a new layer to translate these patterns into the NoSQL

programming interface for communication with the NoSQL database.

Reducing to one option, either relational or NoSQL, refers to vendors' efforts in adding the

functionalities of the “counterpart” option, to convince the developer community that their

solution is sufficient.

The development of databases showed that NoSQL was moved away from relational databases

because of the structural relational constraints it imposed, especially related to the ACID

transactions which became a serious limitation to scaling and dealing with large data sets.

Although the umbrella name for the new generation of databases is NoSQL, their recent

development is showing that they are not necessarily opposed to SQL, i.e. that NoSQL means

not just SQL. Today, a generally accepted view is that the aim of NoSQL databases is not to

abandon SQL, but to find solutions to overcome the technical limitations of relational databases,

especially when it comes to working with Big Data. One of the reasons for the shift to SQL

probably lies in the fact that, when it comes to data manipulation and search, the requirements of

NoSQL databases are almost identical to those of relational databases. Namely, both the

databases must allow entering, updating, deleting, and searching data, except that all this has

already been solved long ago in SQL, the query language that has been associated mainly with

relational databases until now. After the initial enthusiasm regarding various NoSQL

programming languages and APIs, SQL turned out to have important advantages. It is a language

with a high-level abstraction that simplifies access and manipulation of data. Besides, it is a

language in which literally millions of database users are conversant, and hundreds of popular BI

and analytic tools use it under the hood as the means for getting at data (Harrison, 2015).

While vendors of NoSQL databases are oriented toward SQL, the vendors of relational

databases, at least the leading three (i.e., Oracle, Microsoft, and IBM) are working on extending

their databases in order to include NoSQL capabilities. They realized that NoSQL databases

bring some innovative and good solutions to the problems they had been facing for years (e.g.,

Big Data, data distribution, and high availability). The three largest DBMS vendors have

innovated and expanded their relational databases and, thus, they brought them closer to NoSQL

databases. These expansions primarily involve manipulating unstructured data (e.g., JSON

documents), providing sharding, improved possibilities of SQL, and high availability.

Database vendors, at least the biggest ones, are fully aware of their responsibilities, not just in

ensuring storage of vast amounts of quality data, but also in providing solid data foundation,

which is crucial for the development and successful implementation of many other technologies

(e.g., IoT [Internet of Things], AI [Artificial Intelligence], predictive analytics, cloud, DBaaS

[Database as a Service], and blockchain).

Database evolution could take place by bridging the gap between relational and NoSQL

databases. Indeed, this would meet different and often opposite users’ requirements, by enabling

combinations of both approach and tunable and configurable capabilities.

ORGANIZATION OF THE BOOK

The book is organized into eight chapters and divided into two main sections:

• Section 1: The Birth of NoSQL Databases

• Section 2: Bringing Together Relational and NoSQL Databases

The first section of the book describes the reasons behind the appearance of NoSQL databases,

their main characteristics, and their classification.

Chapter 1 is an introductory chapter that gives an overview of the three main stages of database

development: Hierarchical and network database models, relational database model, and NoSQL

databases. It gives an overview of the pillars of relational databases: Relational data model,

ACID transaction, and SQL language. Also, the concepts that make the base for NoSQL database

development are explained, including the CAP theorem, the BASE approach, and the sharding

phenomenon. At last, the limitations of relational databases which led to the development of

NoSQL databases are discussed.

Chapter 2 sustains that NoSQL databases emerged as an attempt for resolving the limitations of

relational databases in coping with Big Data. The issue of Big Data is related to extensive

requests for the storage and management of complex, dynamic, evolving, distributed, and

heterogeneous data from different sources and platforms. The chapter provides an overview of

the technologies, including Google File System (GFS), MapReduce, Hadoop, and Hadoop

Distributed File System (HDFS), which were the first responses to Big Data challenges and main

driving forces for the development of NoSQL databases. Also, the chapter asserts that NoSQL is

an umbrella term related to numerous databases with different architectures and purposes, which

can be classified in four basic categories: Key-value, column-family, document, and graph

stores. This chapter discusses the general features of NoSQL databases, as well as the specific

features of each of the four basic categories of NoSQL databases.

Chapter 3 discusses the necessity for data modeling in NoSQL world. Namely, relational

databases are generally based on the entity-relationship (ER) logical model, opposite to NoSQL

databases. Indeed, one of the main features of NoSQL databases is that they are schema-free, that

is they allow data manipulation without the need for the previous modeling or developing an ER

or similar model. Although the absence of a schema can be an advantage in some situations, with

the increase in the number of NoSQL database implementations, it appears that the absence of a

conceptual model can be a source of certain problems. This is why this chapter emphasizes the

need for data modeling with NoSQL databases as well. In order to better understand the need for

data modeling in NoSQL databases, first the basic structure of an ER model and an analysis of its

limitations are summarized, especially in terms of application in NoSQL databases. Hill’s

COMN model is then presented as one of the possible solutions for data modeling in NoSQL

databases. The COMN model is designed to allow to present the real world, or its objects and

concepts—data about the real world—but also objects in a computer’s memory, or to present

everything from the naming of requirements to a functional database running by either a NoSQL

or a SQL database management system.

Chapter 4 explains how NoSQL databases work. Since different NoSQL databases are classified

into four categories (key-value, column-family, document, and graph stores), three main features

of NoSQL databases are chosen, and their practical implementation is explained using examples

of one or two typical NoSQL databases from each NoSQL database category. The three chosen

features are: Distributed storage architecture that comprises the distributed, cluster-oriented, and

horizontally scalable features; consistency model that refers to the CAP and BASE features;

query execution that refers to the schemaless feature. These features are chosen because, through

them, it is possible to describe most of the new and innovative approaches that NoSQL databases

bring to the database world.

The second section of the book shows the necessity and possible ways of integration the two, at

the beginning very opposite worlds: Relational and NoSQL databases. Also, the convergence

processes from relational to NoSQL and from NoSQL to relational databases are analyzed, as

well as the features of next database generation.

Chapter 5 discusses the fact that the development and use of NoSQL databases showed that

neither everything was good in NoSQL nor everything was so bad in relational databases.

Namely, when operating with data, NoSQL databases have identical requirements for entering,

updating, deleting or searching data, or for the data manipulation that SQL already resolved long

ago. Therefore, it is not surprising that further development of many NoSQL databases shifted

towards supporting SQL, which is one of the topics of this chapter. First, NoSQL databases were

promoted as databases which were developed for coders. One of their primary goals was to avoid

the eternal impedance mismatch between the object-oriented approach to programming and

relational databases which lead to SQL avoidance and the focus on the development of specific

APIs and programming languages. However, such an approach soon turned out to be a limiting

factor of wider implementation of NoSQL databases, because it lacked the critical mass of well-

trained developers. On the other hand, database users are generally not concerned with details

about how data is stored. Rather, they want to have the possibility to view and analyze data

together, regardless of whether the data is stored in relational or NoSQL databases. Therefore,

producers of relational databases were forced, partly for the sake of solving customers'

requirements and partly for the sake of keeping market share, to look for solutions within

relational databases that would allow them to work with data stored in NoSQL databases as well.

This chapter describes how this was accomplished by three largest database producers: Oracle,

Microsoft, and IBM.

Chapter 6 proposes three ways of integration of the two different worlds of relational and

NoSQL databases: Native, hybrid, and reducing to one option, either relational or NoSQL. The

native solution includes using vendors’ standard APIs and integration on the business layer. In a

relational environment, APIs are based on SQL standards, while the NoSQL world has its own,

unstandardized solutions. The native solution for integrating these two environments is to use the

APIs of the individual systems that need to be connected, leaving to the business-layer coding

the task of linking and separating data in extraction and storage operations. A hybrid solution

introduces an additional layer that provides SQL communication between the business layer and

the data layer. Developers continue to use familiar SQL programming patterns in the business

layer, and employ a new layer to translate these patterns into the NoSQL programming interface

for communication with the NoSQL database. The third integration solution includes vendors’

effort to foresee functionalities of “opposite” side, thus convincing developers’ community that

their solution is sufficient.

Chapter 7 presents a real case study of the integration of relational and NoSQL databases. The

example of a real project related to vehicle registration, particularly to testing vehicles for

compliance with environmental standards, explains how this two worlds can be integrated.

Oracle database is used as a relational database, while MongoDB is used as NoSQL database.

The chapter sustains that the COMN notation can be successfully used in the process of

modeling both relational and nonrelational data. All three ways of integration of relational and

NoSQL databases, which are explained in chapter 6, are tested. The native solution was tested by

using of native drivers for communication with Oracle and MongoDB databases. The hybrid

solution (i.e., development of an additional level of communication between the business and the

data layer) used a Unity product. The reducing-to-one option, in this case SQL, was tested on

Oracle database. The capabilities of Oracle 12c database to work both with relational and

nonrelational data by using SQL were tested.

Chapter 8 concludes the story about bridging relational and NoSQL worlds. It presents how

relational databases answer to typical NoSQL features, and, vice versa, how NoSQL databases

answer to typical relational features. Open issues related to the integration of relational and

NoSQL databases, as well as next database generation features are discussed. The big relational

database vendors (Oracle, Microsoft, and IBM) have continuously worked to incorporate NoSQL

features into their databases, as well as NoSQL vendors are trying to make their products more

like relational databases. The convergence of these two groups of databases has been a driving

force in the evolution of database market, in establishing a new level of focus to resolving big

data requirements, and in enabling users to fully use data potential, wherever data is stored, in

relational or NoSQL databases. In turn, the database of choice in the future will likely be one that

provides the best of both worlds: a flexible data model, high availability, and enterprise

reliability.

REFERENCES

Codd, E. F. (1970). A relational model of data for large shared data banks. Communications of

the ACM , 13(6), 377–387. doi:10.1145/362384.362685

Harrison, G. (2015). Next generation databases: NoSQL, NewSQL, and big data . Apress.

doi:10.1007/978-1-4842-1329-2

剩余283页未读，继续阅读

sygwin_net

粉丝: 40
资源: 43

融合关系型与NoSQL数据库：应对大数据挑战

consensus bridging theory and practice

Consensus Bridging Theory and Practice

Wireless Transceiver Architecture Bridging RF and Digital Communication

Bridging Vision and Language from the Video-to-Text Perspective

「安全威胁」Bridging Emulation and the Real World with the Nintendo

PEGASUS_Bridging_Polynomial_and_Non-polycrypt.pdf

Bridging_Emulation_and_the_Real_World_with_the_Nintendo

Metal Triangles versus Metal Chains and Terminal versus Bridging Hydrogen Atoms in Trinuclear Osmium Carbonyl Hydride Chemistry

Synthesis and Crystal Structure of a TetravalentCerium Complex with a Bridging Triphenolate Ligand (2014年)

Bridging The Gap

最新资源