RAC之clusterware

1. Oracle Clusterware简介

Oracle Clusterware, formerly known as Cluster Ready Services (CRS) is an integrated management solution that enables you to link multiple servers so that they function as a single system or cluster. The Oracle Clusterware simplifies the infrastructure required for Oracle RAC because it is integrated with the Oracle Database. In addition, Oracle Clusterware is also available for use with single-instance databases and applications that you deploy on clusters.

cluster

A set of interconnected instances that cooperates to perform the same task.

A comprises multiple interconnected computers or servers that appear as if they are one server to end users and applications. Oracle Database 10g Real Application Clusters (Oracle RAC) enables the clustering of the Oracle Database. Oracle RAC uses Oracle Clusterware for the infrastructure to bind multiple servers so that they operate as a single system.

In Oracle RAC environments, the cluster database instances access one database. The combined processing power of the multiple servers can provide greater throughput and scalability than is available from a single server. Oracle RAC is the Oracle Database option that provides a single system image for multiple servers to access one Oracle database. In Oracle RAC, each Oracle instance usually runs on a separate server.

Having multiple instances access a single database prevents the server from being a single point of failure. Oracle RAC enables you to combine smaller commodity servers into a cluster to create scalable environments that support mission critical business applications.

2. clusterware的组成

clusterware物理组成：voting disk, OCR

The Oracle Clusterware requires two clusterware components: a voting disk to record node membership information and the Oracle Cluster Registry (OCR) to record cluster configuration information. The voting disk and the OCR must reside on shared storage. The Oracle Clusterware requires that each node be connected to a private network by way of a private interconnect.

Oracle recommends that you configure a redundant interconnect to prevent the interconnect from being a single point of failure. Oracle also recommends that you use User Datagram Protocol (UDP) on a Gigabit Ethernet for your cluster interconnect. Crossover cables are not supported for use with Oracle Clusterware or Oracle RAC databases.

Voting Disk—Manages cluster membership by way of a health check and arbitrates(健康检查和仲裁) cluster ownership among the instances in case of network failures. Oracle RAC uses the voting disk to determine which instances are members of a cluster. The voting disk must reside on shared disk. For high availability, Oracle recommends that you have multiple voting disks. The Oracle Clusterware enables multiple voting disks but you must have an odd number of voting disks, such as three, five, and so on. If you define a single voting disk, then you should use external mirroring to provide redundancy.

Oracle Cluster Registry (OCR)—Maintains cluster configuration information as well as configuration information about any cluster database within the cluster. The OCR also manages information about processes that Oracle Clusterware controls. The OCR stores configuration information in a series of key-value pairs within a directory tree structure. The OCR must reside on shared disk that is accessible by all of the nodes in your cluster. The Oracle Clusterware can multiplex the OCR and Oracle recommends that you use this feature to ensure cluster high availability. You can replace a failed OCR online, and you can update the OCR through supported APIs such as Enterprise Manager, the Server Control Utility (SRVCTL), or the Database Configuration Assistant (DBCA).

clusterware进程组成：CRS, CSS, EVM, ONS, OPROCD(hangcheck timer)

The Oracle Clusterware comprises several background processes that facilitate cluster operations.

Cluster Synchronization Services (CSS)—Manages the cluster configuration by controlling which nodes are members of the cluster and by notifying members when a node joins or leaves the cluster. If you are using third-party clusterware, then the css process interfaces with your clusterware to manage node membership information.

Cluster Synchronization Services (CSS)

An Oracle Clusterware component that discovers and tracks the membership state of each node by providing a common view of membership across the cluster. CSS also monitors process health, specifically the health of the database instance. The Global Enqueue Service Monitor (LMON), a background process that monitors the health of the cluster database environment and registers and de-registers from CSS. See also, OCSSD.

ocssd—Manages cluster node membership and runs as the oracle user; failure of this process results in cluster restart.

Cluster Ready Services (CRS)—The primary program for managing high availability operations within a cluster. Anything that the crs process manages is known as a cluster resource which could be a database, an instance, a service, a Listener, a virtual IP (VIP) address, an application process, and so on. The crs process manages cluster resources based on the resource's configuration information that is stored in the OCR. This includes start, stop, monitor and failover operations. The crs process generates events when a resource status changes. When you have installed Oracle RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs. By default, the crs process makes five attempts to restart a resource and then does not make further restart attempts if the resource does not restart.

总结：

1) CSS 利用了 voting disk 来 Manages cluster membership 信息。集群成员信息写入了voting disk。并利用 voting disk 来进行 health check and arbitrates(健康检查和仲裁) 。仲裁哪些节点是集群成员。

2) CRS 利用了 OCR 来中的节点配置信息(即cluster resouces, 也即crs_stat -t的各种资源)来 managing high availability，includes start, stop, monitor and failover operations.

Event Management (EVM)—A background process that publishes events that crs creates.

Oracle Notification Service (ONS)—A publish and subscribe service for communicating Fast Application Notification (FAN) events.

EVM 和 ONS 一起实现了 FAN， 而 FAN 被用来实现： 快速故障转移， 快速连接故障转移等。

RACG—Extends clusterware to support Oracle-specific requirements and complex resources. Runs server callout scripts when FAN events occur. (RACGmain, RACgevt .... support集群用于Oracle数据库时，才具有的特定的一些要求。功能是当FAN事件发生时运行服务器上的脚步)

Process Monitor Daemon (OPROCD)—This process is locked in memory to monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.

总结：

为了好理解，我们可有将clusterware按照实现的功能分成下面四个部分：

1) CSS/voting disk——解决集群由哪些节点组成的问题(脑裂问题)；

2) CRS/OCR——解决集群上的各种resouces的高可用的问题(健忘问题)；

3) FAN event 相关的 EVM, ONS, RACG——辅助功能；

4) IO fencing(hangcheck timer)——解决IO 隔离功能，防止数据库被破坏(IO隔离问题)。