HP rx7640四节点MC/ServiceGuard OPS方式配置过程（完）

10，创建并编写包控制脚本：

[hsedb1:/]#cmmakepkg –s /etc/cmcluster/pkg1/pkg1.cntl

[hsedb1:/]#vi /etc/cmcluster/pkg1/pkg1.cntl

# @(#) A.11.18.00 Date: 02/04/08 PHSS_37243 $

# **********************************************************************

# *                                                                    * 

# *        HIGH AVAILABILITY PACKAGE CONTROL SCRIPT (template)         * 

# *                                                                    *

# *       Note: This file MUST be edited before it can be used.        *

# *                                                                    *

# **********************************************************************

# The environment variables PACKAGE, NODE, SG_PACKAGE, 

# SG_NODE and SG_SCRIPT_LOG_FILE are set by Serviceguard 

# at the time the control script is executed.

# Do not set these environment variables yourself!

# The package may fail to start or halt if the values for

# these environment variables are altered.

# NOTE: Starting from 11.17, all environment variables set by 

# Serviceguard implicitly at the time the control script is

# executed will contain the prefix "SG_". Do not set any variable

# with the defined prefix, or the control script may not

# function as it should.

. ${SGCONFFILE:=/etc/cmcluster.conf}

# UNCOMMENT the variables as you set them.

# Set PATH to reference the appropriate directories.

PATH=$SGSBIN:/usr/bin:/usr/sbin:/etc:/bin

# VOLUME GROUP ACTIVATION: 

# Specify the method of activation for volume groups.

# Leave the default (VGCHANGE="vgchange -a e") if you want volume

# groups activated in exclusive mode. This assumes the volume groups have

# been initialized with 'vgchange -c y' at the time of creation.

#

# Uncomment the first line (VGCHANGE="vgchange -a e -q n"), and comment 

# out the default, if you want to activate volume groups in exclusive mode

# and ignore the disk quorum requirement. Since the disk quorum ensures 

# the integrity of the LVM configuration, it is normally not advisable

# to override the quorum.

#

# Uncomment the second line (VGCHANGE="vgchange -a e -q n -s"), and comment 

# out the default, if you want to activate volume groups in exclusive mode,

# ignore the disk quorum requirement, and disable the mirror 

# resynchronization. Note it is normally not advisable to override the 

# quorum.

#

# Uncomment the third line (VGCHANGE="vgchange -a s"), and comment

# out the default, if you want volume groups activated in shared mode. 

# This assumes the volume groups have already been marked as sharable

# and a part of a Serviceguard cluster with 'vgchange -c y -S y'.

#

# Uncomment the fourth line (VGCHANGE="vgchange -a s -q n"), and comment 

# out the default, if you want to activate volume groups in shared mode

# and ignore the disk quorum requirement. Note it is normally not 

# advisable to override the quorum.

#

# Uncomment the fifth line (VGCHANGE="vgchange -a y") if you wish to 

# use non-exclusive activation mode. Single node cluster configurations

# must use non-exclusive activation.

#

# VGCHANGE="vgchange -a e -q n"

# VGCHANGE="vgchange -a e -q n -s"

# VGCHANGE="vgchange -a s"

# VGCHANGE="vgchange -a s -q n"

VGCHANGE="vgchange -S y -c y"

VGCHANGE="vgchange -a s"                # Default

# CVM DISK GROUP ACTIVATION: 

# Specify the method of activation for CVM disk groups.

# Leave the default

# (CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=exclusivewrite")

# if you want disk groups activated in the exclusive write mode.

#

# Uncomment the first line

# (CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=readonly"),

# and comment out the default, if you want disk groups activated in

# the readonly mode.

#

# Uncomment the second line

# (CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=sharedread"),

# and comment out the default, if you want disk groups activated in the

# shared read mode.

#

# Uncomment the third line

# (CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=sharedwrite"),

# and comment out the default, if you want disk groups activated in the

# shared write mode.

#

# CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=readonly"

# CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=sharedread"

# CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=sharedwrite"

CVM_ACTIVATION_CMD="vxdg -g \$DiskGroup set activation=exclusivewrite" 

# VOLUME GROUPS

# Specify which volume groups are used by this package. Uncomment VG[0]="" 

# and fill in the name of your first volume group. You must begin with 

# VG[0], and increment the list in sequence.

#

# For example, if this package uses your volume groups vg01 and vg02, enter:

#         VG[0]=vg01

#         VG[1]=vg02

#

# The volume group activation method is defined above. The filesystems

# associated with these volume groups are specified below.

#

VG[0]=vg_ops

# CVM DISK GROUPS

# Specify which cvm disk groups are used by this package. Uncomment

# CVM_DG[0]="" and fill in the name of your first disk group. You must 

# begin with CVM_DG[0], and increment the list in sequence.

#

# For example, if this package uses your disk groups dg01 and dg02, enter:

#         CVM_DG[0]=dg01

#         CVM_DG[1]=dg02

#

# The cvm disk group activation method is defined above. The filesystems

# associated with these volume groups are specified below in the CVM_*

# variables.

#

#CVM_DG[0]=""

# NOTE: Do not use CVM and VxVM disk group parameters to reference

# devices used by CFS (cluster file system).  CFS resources are

# controlled by the Disk Group and Mount Multi-node packages.

# 

# VxVM DISK GROUPS

# Specify which VxVM disk groups are used by this package. Uncomment

# VXVM_DG[0]="" and fill in the name of your first disk group. You must 

# begin with VXVM_DG[0], and increment the list in sequence.

#

# For example, if this package uses your disk groups dg01 and dg02, enter:

#         VXVM_DG[0]=dg01

#         VXVM_DG[1]=dg02

#

# The cvm disk group activation method is defined above.

#

#VXVM_DG[0]=""

#

# NOTE: A package could have LVM volume groups, CVM disk groups and VxVM 

#       disk groups.

#

# NOTE: When VxVM is initialized it will store the hostname of the

#       local node in its volboot file in a variable called 'hostid'.

#       The Serviceguard package control scripts use both the values of

#       the hostname(1m) command and the VxVM hostid. As a result

#       the VxVM hostid should always match the value of the

#       hostname(1m) command.

#

#       If you modify the local host name after VxVM has been

#       initialized and such that hostname(1m) does not equal uname -n,

#       you need to use the vxdctl(1m) command to set the VxVM hostid

#       field to the value of hostname(1m). Failure to do so will

#       result in the package failing to start.

# VXVM DISK GROUP IMPORT RETRY

# For packages using VXVM disk groups, if the import of a VXVM

# disk group fails then this parameter allows you to specify if you want 

# to retry the import of disk group. Setting this parameter to "YES" will 

# run the following command "vxdisk scandisks" on the disks which belong to 

# the failed disk group. This command can take a long time on a system which 

# has a large IO subsystem. The use of this parameter is recommended in a

# Metrocluster with EMC SRDF environment.

# The legal values are "YES" and "NO". The default value is "NO"

VXVM_DG_RETRY="NO"

# VOLUME GROUP AND DISK GROUP DEACTIVATION RETRY COUNT

# Specify the number of deactivation retries for each disk group and volume

# group at package shutdown. The default is 2.

DEACTIVATION_RETRY_COUNT=2

# RAW DEVICES

# If you are using raw devices for your application, this parameter allows

# you to specify if you want to kill the processes that are accessing the

# raw devices at package halt time. If raw devices are still being accessed

# at package halt time, volume group or disk group deactivation can fail,

# causing the package halt to also fail. This problem usually happens when

# the application does not shut down properly.

# Note that if you are using Oracle's Cluster Ready Service, killing this

# service could cause the node to reboot.

# The legal values are "YES" and "NO". The default value is "NO".

# The value that is set for this parameter affects all raw devices associated

# with the LVM volume groups and CVM disk groups defined in the package.

KILL_PROCESSES_ACCESSING_RAW_DEVICES="NO"

# FILESYSTEMS

# Filesystems are defined as entries specifying the logical volume, the

# mount point, the mount, umount and fsck options and type of the file system.

# Each filesystem will be fsck'd prior to being mounted. The filesystems 

# will be mounted in the order specified during package startup and will 

# be unmounted in reverse order during package shutdown. Ensure that 

# volume groups referenced by the logical volume definitions below are 

# included in volume group definitions above.

#

# Specify the filesystems which are used by this package. Uncomment 

# LV[0]=""; FS[0]=""; FS_MOUNT_OPT[0]=""; FS_UMOUNT_OPT[0]=""; FS_FSCK_OPT[0]=""

# FS_TYPE[0]="" and fill in the name of your first logical volume, 

# filesystem, mount, umount and fsck options and filesystem type 

# for the file system. You must begin with LV[0], FS[0],

# FS_MOUNT_OPT[0], FS_UMOUNT_OPT[0], FS_FSCK_OPT[0], FS_TYPE[0]

# and increment the list in sequence.

#

# Note: The FS_TYPE parameter lets you specify the type of filesystem to be 

# mounted. Specifying a particular FS_TYPE will improve package failover time.  

# The FSCK_OPT and FS_UMOUNT_OPT parameters can be used to include the 

# -s option with the fsck and umount commands to improve performance for 

# environments that use a large number of filesystems. (An example of a 

# large environment is given below following the decription of the 

# CONCURRENT_MOUNT_AND_UMOUNT_OPERATIONS parameter.)

#

# Example: If a package uses two JFS filesystems, pkg01a and pkg01b, 

# which are mounted on LVM logical volumes lvol1 and lvol2 for read and 

# write operation, you would enter the following:

#      LV[0]=/dev/vg01/lvol1; FS[0]=/pkg01a; FS_MOUNT_OPT[0]="-o rw";

#      FS_UMOUNT_OPT[0]=""; FS_FSCK_OPT[0]=""; FS_TYPE[0]="vxfs"

#

#      LV[1]=/dev/vg01/lvol2; FS[1]=/pkg01b; FS_MOUNT_OPT[1]="-o rw"

#      FS_UMOUNT_OPT[1]=""; FS_FSCK_OPT[1]=""; FS_TYPE[1]="vxfs"

#

#LV[0]=""; FS[0]=""; FS_MOUNT_OPT[0]=""; FS_UMOUNT_OPT[0]=""; FS_FSCK_OPT[0]=""

#FS_TYPE[0]=""

#

# VOLUME RECOVERY

#

# When mirrored VxVM volumes are started during the package control

# bring up, if recovery is required the default behavior is for

# the package control script to wait until recovery has been

# completed.

#

# To allow mirror resynchronization to ocurr in parallel with

# the package startup, uncomment the line

# VXVOL="vxvol -g \$DiskGroup -o bg startall" and comment out the default.

#

# VXVOL="vxvol -g \$DiskGroup -o bg startall"

VXVOL="vxvol -g \$DiskGroup startall"      # Default

# FILESYSTEM UNMOUNT COUNT

# Specify the number of unmount attempts for each filesystem during package

# shutdown.  The default is set to 1.

FS_UMOUNT_COUNT=1

# FILESYSTEM MOUNT RETRY COUNT.

# Specify the number of mount retrys for each filesystem. 

# The default is 0. During startup, if a mount point is busy 

# and FS_MOUNT_RETRY_COUNT is 0, package startup will fail and 

# the script will exit with 1.  If a mount point is busy and

# FS_MOUNT_RETRY_COUNT is greater than 0, the script will attempt 

# to kill the user responsible for the busy mount point 

# and then mount the file system.  It will attempt to kill user and

# retry mount, for the number of times specified in FS_MOUNT_RETRY_COUNT.

# If the mount still fails after this number of attempts, the script

# will exit with 1.

# NOTE: If the FS_MOUNT_RETRY_COUNT > 0, the script will execute

# "fuser -ku" to freeup busy mount point.

FS_MOUNT_RETRY_COUNT=0

#

# Configuring the concurrent operations below can be used to improve the 

# performance for starting up or halting a package.  The maximum value for 

# each concurrent operation parameter is 1024.  Set these values carefully.  

# The performance could actually decrease if the values are set too high 

# for the system resources available on your cluster nodes.  Some examples 

# of system resources that can affect the optimum number of concurrent 

# operations are: number of CPUs, amount of available memory, the kernel 

# configuration for nfile and nproc. In some cases, if you set the number 

# of concurrent operations too high, the package may not be able to start

# or to halt.  For example, if you set CONCURRENT_VGCHANGE_OPERATIONS=5 

# and the node where the package is started has only one processor, then 

# running concurrent volume group activations will not be beneficial. 

# It is suggested that the number of concurrent operations be tuned 

# carefully, increasing the values a little at a time and observing the 

# effect on the performance, and the values should never be set to a value 

# where the performance levels off or declines.  Additionally, the values 

# used should take into account the node with the least resources in the 

# cluster, and how many other packages may be running on the node.  

# For instance, if you tune the concurrent operations for a package so 

# that it provides optimum performance for the package on a node while 

# no other packages are running on that node, the package performance 

# may be significantly reduced, or may even fail when other packages are 

# already running on that node.

#

# CONCURRENT VGCHANGE OPERATIONS

# Specify the number of concurrent volume group activations or

# deactivations to allow during package startup or shutdown. 

# Setting this value to an appropriate number may improve the performance

# while activating or deactivating a large number of volume groups in the

# package. If the specified value is less than 1, the script defaults it 

# to 1 and proceeds with a warning message in the package control script 

# logfile. 

CONCURRENT_VGCHANGE_OPERATIONS=1

#

# USE MULTI-THREADED VGCHANGE

# Specify whether multi-threaded vgchange is to be used if available.

# 0 means that the multi-threaded option is not to be used and 1 means

# that the multi-threaded option is to be used. The default is set to 0.

# Multi-threaded vgchange has potential performance benefits.

# If the activation order of the paths defined in lvmtab is important then

# multi-threaded vgchange should not be used. If mirrored volume groups

# are synced during activation then using multi-threaded vgchange may

# worsen performance.

# Using the multi-threaded vgchange option can improve the activation

# performance of volume groups with multiple disks.

# CONCURRENT_VGCHANGE_OPERATIONS option is beneficial when mutiple

# volume groups need to be activated. To get the best performance for

# volume group activation, use the multi-threaded vgchange option in

# combination with the CONCURRENT_VGCHANGE_OPERATIONS option.

ENABLE_THREADED_VGCHANGE=0

# CONCURRENT FSCK OPERATIONS

# Specify the number of concurrent fsck to allow during package startup. 

# Setting this value to an appropriate number may improve the performance

# while checking a large number of file systems in the package. If the

# specified value is less than 1, the script defaults it to 1 and proceeds 

# with a warning message in the package control script logfile. 

CONCURRENT_FSCK_OPERATIONS=1

# CONCURRENT MOUNT AND UMOUNT OPERATIONS

# Specify the number of concurrent mounts and umounts to allow during 

# package startup or shutdown. 

# Setting this value to an appropriate number may improve the performance

# while mounting or un-mounting a large number of file systems in the package.

# If the specified value is less than 1, the script defaults it to 1 and

# proceeds with a warning message in the package control script logfile. 

CONCURRENT_MOUNT_AND_UMOUNT_OPERATIONS=1

# Example:  If a package uses 50 JFS filesystems, pkg01aa through pkg01bx,

# which are mounted on the 50 logical volumes lvol1..lvol50 for read and write 

# operation, you may enter the following:

#

#      CONCURRENT_FSCK_OPERATIONS=50

#      CONCURRENT_MOUNT_AND_UMOUNT_OPERATIONS=50

#

#      LV[0]=/dev/vg01/lvol1; FS[0]=/pkg01aa; FS_MOUNT_OPT[0]="-o rw";

#      FS_UMOUNT_OPT[0]="-s"; FS_FSCK_OPT[0]="-s"; FS_TYPE[0]="vxfs"

#

#      LV[1]=/dev/vg01/lvol2; FS[1]=/pkg01ab; FS_MOUNT_OPT[1]="-o rw"

#      FS_UMOUNT_OPT[1]="-s"; FS_FSCK_OPT[1]="-s"; FS_TYPE[0]="vxfs"

#         :           :          :

#         :           :          :

#         :           :          :

#      LV[49]=/dev/vg01/lvol50; FS[49]=/pkg01bx; FS_MOUNT_OPT[49]="-o rw"

#      FS_UMOUNT_OPT[49]="-s"; FS_FSCK_OPT[49]="-s"; FS_TYPE[0]="vxfs"

#

# IP ADDRESSES

# Specify the IP and Subnet address pairs which are used by this package.

# You could specify IPv4 or IPv6 IP and subnet address pairs.

# Uncomment IP[0]="" and SUBNET[0]="" and fill in the name of your first

# IP and subnet address. You must begin with IP[0] and SUBNET[0] and 

# increment the list in sequence.

#

# For example, if this package uses an IP of 192.10.25.12 and a subnet of 

# 192.10.25.0 enter:

#          IP[0]=192.10.25.12 

#          SUBNET[0]=192.10.25.0

#          (netmask=255.255.255.0)

#

# Hint: Run "netstat -i" to see the available subnets in the Network field.

#

# For example, if this package uses an IPv6 IP of 2001::1/64 

# The address prefix identifies the subnet as 2001::/64 which is an available

# subnet.

# enter:

#          IP[0]=2001::1

#          SUBNET[0]=2001::/64

#          (netmask=ffff:ffff:ffff:ffff::)

# Alternatively the IPv6 IP/Subnet pair can be specified without the prefix 

# for the IPv6 subnet.

#          IP[0]=2001::1

#          SUBNET[0]=2001::

#          (netmask=ffff:ffff:ffff:ffff::)

#

# Hint: Run "netstat -i" to see the available IPv6 subnets by looking

# at the address prefixes

# IP/Subnet address pairs for each IP address you want to add to a subnet 

# interface card.  Must be set in pairs, even for IP addresses on the same

# subnet.

#

#IP[0]=" "

#IP[1]=" "

#IP[2]= " "

#IP[3]= " "

#SUBNET[0]= " "

# SERVICE NAMES AND COMMANDS.

# Specify the service name, command, and restart parameters which are 

# used by this package. Uncomment SERVICE_NAME[0]="", SERVICE_CMD[0]="",

# SERVICE_RESTART[0]="" and fill in the name of the first service, command,

# and restart parameters. You must begin with SERVICE_NAME[0], SERVICE_CMD[0],

# and SERVICE_RESTART[0] and increment the list in sequence.

#

# For example:

#          SERVICE_NAME[0]=pkg1a 

#          SERVICE_CMD[0]="/usr/bin/X11/xclock -display 192.10.25.54:0"

#          SERVICE_RESTART[0]=""  # Will not restart the service.

#

#          SERVICE_NAME[1]=pkg1b

#          SERVICE_CMD[1]="/usr/bin/X11/xload -display 192.10.25.54:0"

#          SERVICE_RESTART[1]="-r 2"   # Will restart the service twice.

#

#          SERVICE_NAME[2]=pkg1c

#          SERVICE_CMD[2]="/usr/sbin/ping"

#          SERVICE_RESTART[2]="-R" # Will restart the service an infinite 

#                                    number of times.

#

# Note: No environmental variables will be passed to the command, this 

# includes the PATH variable. Absolute path names are required for the

# service command definition.  Default shell is /usr/bin/sh.

#

#SERVICE_NAME[0]=""

#SERVICE_CMD[0]=""

#SERVICE_RESTART[0]=""

# DEFERRED_RESOURCE NAME

# Specify the full path name of the 'DEFERRED' resources configured for

# this package.  Uncomment DEFERRED_RESOURCE_NAME[0]="" and fill in the

# full path name of the resource.

#

#DEFERRED_RESOURCE_NAME[0]=""

# DTC manager information for each DTC.

# Example: DTC[0]=dtc_20

#DTC_NAME[0]=

# HA_NFS_SCRIPT_EXTENSION

# If the package uses HA NFS, this variable can be used to alter the

# name of the HA NFS script.  If not set, the name of this script is

# assumed to be "hanfs.sh".  If set, the "sh" portion of the default

# script name is replaced by the value of this variable.  So if

# HA_NFS_SCRIPT_EXTENSION is set to "package1.sh", for example, the name

# of the HA NFS script becomes "hanfs.package1.sh".  In any case,

# the HA NFS script must be placed in the same directory as the package

# control script.  This allows multiple packages to be run out of the

# same directory, as needed by SGeSAP.

#HA_NFS_SCRIPT_EXTENSION=""

# Setting the log file

log_file=${SG_SCRIPT_LOG_FILE:-$0.log}

# START OF CUSTOMER DEFINED FUNCTIONS

# This function is a place holder for customer define functions.

# You should define all actions you want to happen here, before the service is

# started.  You can create as many functions as you need.  

function customer_defined_run_cmds

{

# ADD customer defined run commands.

: # do nothing instruction, because a function must contain some command.

    /oracle/product/10.2.0/db1/bin/crsctl start crs

        test_return 51

}

# This function is a place holder for customer define functions.

# You should define all actions you want to happen here, after the service is

# halted.

function customer_defined_halt_cmds

{

# ADD customer defined halt commands.

: # do nothing instruction, because a function must contain some command.

    /oracle/product/10.2.0/db1/bin/crsctl stop crs

        test_return 52

}

# END OF CUSTOMER DEFINED FUNCTIONS

11，验证包的运行状况：

[hsedb1:/]#cmviewcl

CLUSTER        STATUS       

cluster1       up           

  NODE           STATUS       STATE        

  hsedb1         up           running      

  hsedb2         up           running      

  hsedb3         up           running      

  hsedb4         up           running      

MULTI_NODE_PACKAGES

  PACKAGE        STATUS        STATE         AUTO_RUN     SYSTEM      

  pkg1           up            running       enabled      no

12，查看crs进程：

[hsedb1:/]#/oracle/product/10.2.0/db1/bin/crs_stat –t                                                                                                                  

Name           Type           Target    State     Host        

------------------------------------------------------------

ora....C1.inst application    ONLINE    ONLINE    hsedb1      

ora....C2.inst application    ONLINE    ONLINE    hsedb2      

ora....C3.inst application    ONLINE    ONLINE    hsedb3      

ora....C4.inst application    ONLINE    ONLINE    hsedb4      

ora.HSERAC.db  application    ONLINE    ONLINE    hsedb1      

ora....B1.lsnr application    ONLINE    ONLINE    hsedb1      

ora.hsedb1.gsd application    ONLINE    ONLINE    hsedb1      

ora.hsedb1.ons application    ONLINE    ONLINE    hsedb1      

ora.hsedb1.vip application    ONLINE    ONLINE    hsedb1      

ora....B2.lsnr application    ONLINE    ONLINE    hsedb2      

ora.hsedb2.gsd application    ONLINE    ONLINE    hsedb2      

ora.hsedb2.ons application    ONLINE    ONLINE    hsedb2      

ora.hsedb2.vip application    ONLINE    ONLINE    hsedb2      

ora....B3.lsnr application    ONLINE    ONLINE    hsedb3      

ora.hsedb3.gsd application    ONLINE    ONLINE    hsedb3      

ora.hsedb3.ons application    ONLINE    ONLINE    hsedb3      

ora.hsedb3.vip application    ONLINE    ONLINE    hsedb3      

ora....B4.lsnr application    ONLINE    ONLINE    hsedb4      

ora.hsedb4.gsd application    ONLINE    ONLINE    hsedb4      

ora.hsedb4.ons application    ONLINE    ONLINE    hsedb4      

ora.hsedb4.vip application    ONLINE    ONLINE    hsedb4