Oracle GoldenGate 10.4管理手册：异构数据复制关键

需积分: 10 4 浏览量更新于2024-07-23 收藏 3.15MB PDF 举报

Oracle GoldenGate 是一款专为数据库间异构复制而设计的高级数据复制解决方案，由Oracle公司开发。该软件允许实时或近实时地在不同的数据库系统之间传输和同步数据，广泛应用于企业级的数据集成、灾备恢复和数据仓库场景中。《Oracle GoldenGate Administration Guide》版本10.4发布于2009年10月，是针对管理员的专业手册，包含了详细的安装、配置、管理和维护GoldenGate的指导。此文档的核心知识点包括： 1. **安装与配置**： - 安装过程涉及选择合适的组件、网络设置和授权管理，以及环境准备。 - 配置参数设置，如数据源、目标端、通道（Connection Pools）以及数据转换规则（Data Pump Templates）。 2. **数据复制流程**： - 描述了GoldenGate的复制机制，包括捕获阶段（Capturing）、传输阶段（Transmission）和应用阶段（Application），确保数据的一致性和完整性。 - 实时复制（Real-time Replication）与延迟复制（Batch Replication）的区别和应用场景。 3. **监控与管理**： - 提供了GoldenGate的性能监控工具，如 GG Monitor 和 GGSCI（GoldenGate Server Control Interface），用于跟踪和诊断数据传输状况。 - 定期检查日志文件，识别潜在问题，并进行故障排查。 4. **安全与审计**： - 系统权限管理，确保只有授权用户能够访问和操作GoldenGate。 - 数据传输的加密和审计功能，保护敏感数据的安全。 5. **许可证和法律注意事项**： - Oracle GoldenGate受到版权保护，用户必须遵守许可协议，限制了诸如修改、分发等行为。 - 如果软件被提供给美国政府或其代表，文档中可能包含特定的政府权利声明，强调了对软件使用的法律限制。 6. **更新与支持**： - 文档中可能包含了关于版本更新、技术支持和兼容性信息，帮助管理员保持系统的最新状态。《Oracle GoldenGate Administration Guide》对于那些负责维护和优化Oracle数据库间数据同步的IT专业人员来说是一本不可或缺的参考资料，提供了全面且深入的指导，以确保高效、稳定和合规的数据复制环境。

Introduction to GoldenGate

Overview of the GoldenGate architecture

Oracle® GoldenGate Administration Guide

..............................................................................

Overview of Extract

The Extract process runs on the source system and is the capture mechanism of

GoldenGate. You can configure Extract in one of the following ways:

● Initial loads: For initial data loads, Extract extracts a current set of data directly from

their source objects.

● Change synchronization: To keep source data synchronized with another set of data,

Extract extracts transactional changes made to data (inserts, updates, and deletes)

after the initial synchronization has taken place. DDL changes and sequences are also

extracted, if supported for the type of database being used.

When processing transactional data changes, Extract obtains the data from a data source

that can be one of the following.

● The database transaction logs (such as the Oracle redo logs or SQL/MX audit trails).

This method is known as log-based extraction. When Extract is installed as a log-based

implementation, it can read the transaction logs directly.

● A GoldenGate Vendor Access Module (VAM). The VAM is a communication layer that

passes data changes and transaction metadata to the Extract process. The database

vendor provides the components that extract the data changes and pass it to Extract.

This method is known as VAM-based extraction.

Extract captures all of the changes that are made to objects that you configure for

synchronization, but it only sends the data from committed transactions to the trail for

propagation to the target system. When Extract captures the commit record of a

transaction, all of the log records for that transaction are written to the trail as a

sequentially organized transaction unit. This design ensures both speed and data

integrity.

Multiple Extract processes can operate on different objects at the same time. For example,

one process could continuously extract transactional data changes and stream them to a

decision-support database, while another process performs batch extracts for periodic

reporting. Or, two Extract processes could extract and transmit in parallel to two Replicat

processes (with two trails) to minimize target latency when the databases are large. To

differentiate among different processes, you assign each one a group name (see “Overview

of groups” on page 20).

Figure 2 GoldenGate logical architecture

Introduction to GoldenGate

Overview of the GoldenGate architecture

Oracle® GoldenGate Administration Guide

..............................................................................

For the SQL/MX database, the Extract module also includes a program named VAMSERV.

Extract starts

VAMSERV, and together they retrieve and process database changes from the

audit trails created by TMF-enabled applications on a NonStop system.

Overview of data pumps

A data pump is a secondary Extract group within the source GoldenGate configuration. If

a data pump is not used, Extract must send data to a remote trail on the target. In a typical

configuration that includes a data pump, however, the primary Extract group writes to a

trail on the source system. The data pump reads this trail and sends the data across the

network to a remote trail on the target. The data pump adds storage flexibility and also

serves to isolate the primary Extract process from TCP/IP activity.

Like a primary Extract group, a data pump can be configured for either online or batch

processing. It can perform data filtering, mapping, and conversion, or it can be configured

in pass-through mode, where data is passively transferred as-is, without manipulation.

Pass-through mode increases the throughput of the data pump, because all of the

functionality that looks up object definitions is bypassed.

In most business cases, it is best practice to use a data pump. Some reasons for using a data

pump include the following:

● Protection against network and target failures: In a basic GoldenGate configuration,

with only a trail on the target system, there is nowhere on the source system to store

data that Extract continuously extracts into memory. If the network or the target

system becomes unavailable, the primary Extract could run out of memory and abend.

However, with a trail and data pump on the source system, captured data can be moved

to disk, preventing the abend. When connectivity is restored, the data pump extracts

the data from the source trail and sends it to the target system(s).

● You are implementing several phases of data filtering or transformation. When using

complex filtering or data transformation configurations, you can configure a data pump

to perform the first transformation either on the source system or on the target system,

and then use another data pump or the Replicat group to perform the second

transformation.

● Consolidating data from many sources to a central target. When synchronizing multiple

source databases with a central target database, you can store extracted data on each

source system and use data pumps on each of those systems to send the data to a trail

on the target system. Dividing the storage load between the source and target systems

reduces the need for massive amounts of space on the target system to accommodate

data arriving from multiple sources.

● Synchronizing one source with multiple targets. When sending data to multiple target

systems, you can configure data pumps on the source system for each target. If network

connectivity to any of the targets fails, data can still be sent to the other targets.

Overview of Replicat

The Replicat process runs on the target system. Replicat reads extracted data changes and

DDL changes (if supported) that are specified in the Replicat configuration, and then it

replicates them to the target database. You can configure Replicat in one of the following

ways:

● Initial loads: For initial data loads, Replicat can apply data to target objects or route

them to a high-speed bulk-load utility.

Introduction to GoldenGate

Overview of the GoldenGate architecture

Oracle® GoldenGate Administration Guide

..............................................................................

● Change synchronization: To maintain synchronization, Replicat applies extracted

transactional changes to target objects using native database calls, statement caches,

and local database access. Replicated DDL and sequences are also applied, if supported

for the type of database that is being used. To preserve data integrity, Replicat applies

the replicated changes in the same order as those changes were committed to the

source database.

You can use multiple Replicat processes with multiple Extract processes in parallel to

increase throughput. Each set of processes handles different objects. To differentiate

among processes, you assign each one a group name (see “Overview of groups” on page 20).

You can delay Replicat so that it waits a specific amount of time before applying data to

the target database. A delay may be desirable for such purposes as to prevent the

propagation of errant SQL, to control data arrival across different time zones, and to allow

time for other planned events to occur. The length of the delay is controlled by the

DEFERAPPLYINTERVAL parameter.

Overview of trails

To support the continuous extraction and replication of supported database changes,

GoldenGate stores those changes temporarily on disk in a series of files called a trail. A

trail can exist on the source or target system, or on an intermediary system, depending on

how you configure GoldenGate. On the local system it is known as an extract trail (or local

trail). On a remote system it is known as a remote trail.

By using a trail for storage, GoldenGate supports data accuracy and fault tolerance (see

“Overview of checkpoints” on page 18). The use of a trail also allows extraction and

replication activities to occur independently of each other. With these processes separated,

you have more choices for how data is delivered. For example, instead of extracting and

replicating changes continuously, you could extract changes continuously but store them

in the trail for replication to the target later, whenever the target application needs them.

Processes that write to, and read, a trail

The primary Extract process writes to a trail. Only one Extract process can write to a trail.

Processes that read the trail are:

● Data-pump Extract: Extracts data from a local trail for further processing, if needed,

and transfers it to the target system or to the next GoldenGate process downstream in

the GoldenGate configuration.

● Replicat: Reads a trail to apply change data to the target database.

Trail maintenance

Trail files are created as needed during processing, and they are aged automatically to

allow processing to continue without interruption for file maintenance. By default, each file

in a trail is 10 MB in size. All file names in a trail begin with the same two characters,

which you assign when you create the trail. As files are created, each name is appended

with a unique, six-digit serial (sequence) number from 000000 through 999999, for

example c:

\ggs\dirdat\tr000001. By default, trails are stored in the dirdat sub-directory of the

GoldenGate directory.

You can create more than one trail to separate the data of different objects or applications.

You link the objects that are specified in a

TABLE or SEQUENCE parameter to a trail that is

Introduction to GoldenGate

Overview of the GoldenGate architecture

Oracle® GoldenGate Administration Guide

..............................................................................

specified with an EXTTRAIL or RMTTRAIL parameter in the Extract parameter file. Aged trail

files can be purged by using the Manager parameter

PURGEOLDEXTRACTS.

How processes write to a trail

To maximize throughput, and to minimize I/O load on the system, extracted data is sent

into and out of a trail in large blocks. Transactional order is preserved. By default,

GoldenGate writes data to the trail in universal data format, a proprietary format which

allows it to be exchanged rapidly and accurately among heterogeneous databases.

However, data can be written in other formats that are compatible with different

applications.

By default, Extract operates in append mode, where if there is a process failure, a recovery

marker is written to the trail and Extract appends recovery data to the file so that a history

of all prior data is retained for recovery purposes.

In append mode, the Extract initialization determines the identity of the last complete

transaction that was written to the trail at startup time. With that information, Extract

ends recovery when the commit record for that transaction is encountered in the data

source; then it begins new data capture with the next committed transaction that qualifies

for extraction and begins appending the new data to the trail. A data pump or Replicat

starts reading again from that recovery point.

Overwrite mode is another version of Extract recovery that was used in versions of

GoldenGate prior to version 10.0. In these versions, Extract overwrites the existing

transaction data in the trail after the last write-checkpoint position, instead of appending

the new data. The first transaction that is written is the first one that qualifies for

extraction after the last read checkpoint position in the data source.

If the version of GoldenGate on the target is older than version 10, Extract will

automatically revert to overwrite mode to support backward compatibility. This behavior

can be controlled manually with the

RECOVERYOPTIONS parameter.

Trail format

As of GoldenGate version 10.0, each file of a trail contains a file header record that is stored

at the beginning of the file. The file header contains information about the trail file itself.

Previous versions of GoldenGate do not contain this header.

Each data record in a GoldenGate trail file contains a header area and a data area. The

header contains information about the transaction environment, and the data area

contains the actual data values that were extracted. For more information about the

GoldenGate trail record format, see Appendix 1.

File header and versioning

Because all of the GoldenGate processes are decoupled and thus can be of different

GoldenGate versions, each trail file or extract file has a version that is stored in the file

header. By default, the version of a trail is the current version of the process that created

the file. To set the version of a trail, use the

FORMAT option of the EXTTRAIL, EXTFILE, RMTTRAIL,

or RMTFILE

parameter.

To ensure forward and backward compatibility of files among different GoldenGate process

versions, the file header fields are written in a standardized token format. New tokens that

are created by new versions of a process can be ignored by older versions, so that backward

Introduction to GoldenGate

Overview of the GoldenGate architecture

Oracle® GoldenGate Administration Guide

..............................................................................

compatibility is maintained. Likewise, newer GoldenGate versions support older tokens.

Additionally, if a token is deprecated by a new process version, a default value is assigned

to the token so that older versions can still function properly. The token that specifies the

file version is

COMPATIBILITY and can be viewed in the Logdump utility and also by retrieving

it with the

GGFILEHEADER option of the @GETENV function.

A trail or extract file must have a version that is equal to, or lower than, that of the process

that reads it. Otherwise the process will abend. Additionally, GoldenGate forces the output

trail or file of a data pump to be the same version as that of its input trail or file. Upon

restart, Extract rolls a trail to a new file to ensure that each file is of only one version

(unless the file is empty).

Overview of extract files

When processing a one-time run, such as an initial load or a batch run that synchronizes

transactional changes (see page 19), GoldenGate stores the extracted changes in an extract

file instead of a trail. The extract file typically is a single file but can be configured to roll

over into multiple files in anticipation of limitations on the size of a file that are imposed

by the operating system. In this sense, it is similar to a trail, except that checkpoints are

not recorded. The file or files are created automatically during the run. The same

versioning features that apply to trails also apply to extract files.

Overview of checkpoints

Checkpoints store the current read and write positions of a process to disk for recovery

purposes. Checkpoints ensure that database changes marked for synchronization are

extracted by Extract and replicated by Replicat, and they prevent redundant processing.

They provide fault tolerance by preventing the loss of data should the system, the network,

or a GoldenGate process need to be restarted. For advanced synchronization

configurations, checkpoints enable multiple Extract or Replicat processes to read from the

same set of trails.

The read checkpoint of a process is always synchronized with the write checkpoint. Thus,

if GoldenGate needs to re-read something that it already sent to the target system (for

example, after a process failure) checkpoints enable accurate recovery to the point where

a new transaction starts, and GoldenGate can resume processing.

Checkpoints work with inter-process acknowledgments to prevent messages from being

lost in the network. GoldenGate has a proprietary guaranteed-message delivery

technology.

Extract creates checkpoints for its positions in the data source and in the trail. Replicat

creates checkpoints for its position in the trail.

A checkpoint system is used for Extract and Replicat processes that operate continuously,

but it is not required for Extract and Replicat processes that run in batch mode (see page

19). A batch process can be re-run from its start point, whereas continuous processing

requires the support for planned or unplanned interruptions that is provided by

checkpoints.

Checkpoint information is maintained in checkpoint files within the

dirchk sub-directory of

the GoldenGate directory. Optionally, Replicat’s checkpoints can be maintained in a

checkpoint table within the target database, in addition to a standard checkpoint file.

剩余367页未读，继续阅读

apollo_gao

粉丝: 0

Oracle GoldenGate 10.4管理手册：异构数据复制关键

Oracle_GoldenGate_11.2.1.0.3 for Oracle_11g_windows_x64

oracle goldengate

oracle goldengate oracle to oracle

oracle GOLDENGATE配置

oracle GoldenGate实施

oracle goldengate ddl

Oracle Goldengate 安装

Mastering Oracle GoldenGate

oracle goldengategit 和 oracle goldengate版本

Oracle GoldenGate Case Study

最新资源