在Rac系统中偶尔会出现一个节点维护,需要关闭相应的服务,
以下就是一个操作过程:
注:在我们正常关闭该节点服务的时候vip并不会自动切换另外一台机器
1、关闭节点1的所有应用
oracle@CHNFSCC1[billdb1]:/home/oracle$ srvctl stop nodeapps -n CHNFSCC1
查看状态节点1的服务已经关闭
oracle@CHNFSCC1[billdb1]:/home/oracle$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....b1.inst application ONLINE ONLINE chnfscc1
ora....b2.inst application ONLINE ONLINE chnfscc2
ora.billdb.db application ONLINE ONLINE chnfscc2
ora....SM1.asm application ONLINE ONLINE chnfscc1
ora....C1.lsnr application OFFLINE OFFLINE
ora....cc1.gsd application OFFLINE OFFLINE
ora....cc1.ons application OFFLINE OFFLINE
ora....cc1.vip application OFFLINE OFFLINE
ora....SM2.asm application ONLINE ONLINE chnfscc2
ora....C2.lsnr application ONLINE ONLINE chnfscc2
ora....cc2.gsd application ONLINE ONLINE chnfscc2
ora....cc2.ons application ONLINE ONLINE chnfscc2
ora....cc2.vip application ONLINE ONLINE chnfscc2
oracle@CHNFSCC1[billdb1]:/home/oracle$
2、关闭节点1的crs
[root@CHNFSCC1:/]#/sbin/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
[root@CHNFSCC1:/]#ps -ef|grep crs
root 28764 1 0 10:47:04 ? 0:00 /bin/sh /sbin/init.d/init.crsd run
root 28812 24594 0 10:47:10 pts/0 0:00 grep crs
关闭节点1 crs后,节点2的crs状态
oracle@CHNFSCC2[billdb2]:/home/oracle$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....b1.inst application ONLINE OFFLINE
ora....b2.inst application ONLINE ONLINE chnfscc2
ora.billdb.db application ONLINE ONLINE chnfscc2
ora....SM1.asm application ONLINE OFFLINE
ora....C1.lsnr application OFFLINE OFFLINE
ora....cc1.gsd application OFFLINE OFFLINE
ora....cc1.ons application OFFLINE OFFLINE
ora....cc1.vip application OFFLINE OFFLINE
ora....SM2.asm application ONLINE ONLINE chnfscc2
ora....C2.lsnr application ONLINE ONLINE chnfscc2
ora....cc2.gsd application ONLINE ONLINE chnfscc2
ora....cc2.ons application ONLINE ONLINE chnfscc2
ora....cc2.vip application ONLINE ONLINE chnfscc2
oracle@CHNFSCC2[billdb2]:/home/oracle$
3、在接到2启动vip,否则客户端连接到节点1vip后需等到超时后才会切换到节点的vip
oracle@CHNFSCC2[billdb2]:/home/oracle$ crs_start ora.chnfscc1.vip -c CHNFSCC2
4、关闭节点1的SERVICE GUARD
[root@CHNFSCC1:/]#cmviewcl -v
CLUSTER STATUS
fscluster up
NODE STATUS STATE
CHNFSCC1 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c4t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d1 up running enabled CHNFSCC1
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Subnet up 192.168.0.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC1 (current)
NODE STATUS STATE
CHNFSCC2 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c8t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d2 up running enabled CHNFSCC2
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC2 (current)
[root@CHNFSCC1:/]#cmhaltnode -f -v CHNFSCC1
Disabling all packages from starting on nodes to be halted.
Disabling all packages from running on CHNFSCC1.
Warning: Do not modify or enable packages until the halt operation is completed.
Disabling automatic failover for failover packages to be halted.
Halting package pkg_fs_d1
Successfully halted package pkg_fs_d1
Enabling switching for Package pkg_fs_d1.
Waiting for nodes to halt ..... done
Successfully halted all nodes specified.
Halt operation complete.
[root@CHNFSCC1:/]#cmviewcl -v
CLUSTER STATUS
fscluster up
NODE STATUS STATE
CHNFSCC1 down halted
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c4t0d1 unknown
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY unknown 0/2/1/0 lan0
PRIMARY unknown 0/5/1/0 lan3
NODE STATUS STATE
CHNFSCC2 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c8t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d2 up running enabled CHNFSCC2
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC2 (current)
UNOWNED_PACKAGES
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d1 down halted enabled unowned
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS NODE_NAME NAME
Subnet unknown CHNFSCC1 192.168.0.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary down CHNFSCC1
此时查看节点crs状态正常
oracle@CHNFSCC2[billdb2]:/home/oracle$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....b1.inst application ONLINE OFFLINE
ora....b2.inst application ONLINE ONLINE chnfscc2
ora.billdb.db application ONLINE ONLINE chnfscc2
ora....SM1.asm application ONLINE OFFLINE
ora....C1.lsnr application OFFLINE OFFLINE
ora....cc1.gsd application OFFLINE OFFLINE
ora....cc1.ons application OFFLINE OFFLINE
ora....cc1.vip application OFFLINE OFFLINE
ora....SM2.asm application ONLINE ONLINE chnfscc2
ora....C2.lsnr application ONLINE ONLINE chnfscc2
ora....cc2.gsd application ONLINE ONLINE chnfscc2
ora....cc2.ons application ONLINE ONLINE chnfscc2
ora....cc2.vip application ONLINE ONLINE chnfscc2
oracle@CHNFSCC2[billdb2]:/home/oracle$
5、关闭节点1机器
[root@CHNFSCC1:/]#shutdown -y -h 0
6、关闭机器
Hp unix关机命令
shutdown -y -h 0 (关闭系统,然后关闭电源)
shutdown -y -r (关闭系统后重新启动,类于widowns里的热起)
shutdown -y 0 (关闭系统,并进到单用户模式)
Shutdown 不加任何参数只是切到单用户模式(系统维护模式)
相当于
#init 0 (先到0状态)
#init S|s (再到单用户状态)
[root@CHNFSCC1:/]#shutdown -y -h 0
SHUTDOWN PROGRAM
03/29/09 10:51:20 EAT
Broadcast Message from root (pts/0) Sun Mar 29 10:51:20...
PLEASE LOG OFF NOW ! ! !
System maintenance about to begin.
All processes will be terminated in 0 seconds.
Broadcast Message from root (pts/0) Sun Mar 29 10:51:20...
SYSTEM BEING BROUGHT DOWN NOW ! ! !
/sbin/auto_parms: DHCP access is disabled (see /etc/auto_parms.log)
i
System shutdown in progress
___________________________
Stop Oracle Cluster Ready Services .............................. OK
Stop Highly Available cluster ................................... N/A
Stop CDE login server ........................................... OK
Stop kwdbd ...................................................... N/A
Stopping HP-UX Apache-based Web Server .......................... OK
Stopping HP-UX Tomcat-based Servlet Engine. ..................... N/A
Stopping HP-UX Webmin-based Admin ............................... OK
Stopping the HPUX Webproxy subsystem ............................ OK
Stopping HP-UX XML Web Server Tools ............................. OK
Stop X print server(s) .......................................... N/A
Stop NFS server subsystem ....................................... OK
Shutting down CIFS Client ....................................... N/A
Stopping Event Monitoring Service ............................... OK
Shutting down the Winbind Daemon ................................ OK
Unload/Uninstall XF86 DLKM Helper Modules ....................... OK
Stopping the gWLM Agent ......................................... OK
Shutting down the System Management HomePage server ............. OK
Stopping USB hub daemon ......................................... OK
Stopping audio server daemon .................................... OK
Stop auditing subsystem ......................................... N/A
Stop environment monitoring daemon .............................. OK
Stop diagnostic subsystem ....................................... OK
Stop clock daemon ............................................... OK
Stop print spooler .............................................. OK
Stopping HP Disk Array Manager daemons. ......................... OK
Stop accounting ................................................. N/A
Stopping time synchronization ................................... OK
Stop vt daemon .................................................. OK
Terminating X Font Server ....................................... OK
Stopping HP-UX Secure Shell ..................................... OK
Stopping Instant Capacity ....................................... N/A
Stop CIM cimserver subsystem .................................... OK
Stop RPC daemon ................................................. OK
Stopping SNMP PCI FDDI Network Management subAgent .............. OK
Stop DCE daemons ................................................ OK
Stop HA cluster SNMP subagent ................................... OK
Stopping OSPF MIB Network Management subAgent ................... N/A
Stopping SNMP HP-UNIX Network Management subAgent ............... OK
Stopping SNMP IPv6 Network Management subAgent .................. OK
Stopping SNMP MIB-2 Network Management subAgent ................. OK
Stopping Native Adapter Agent ................................... OK
Stopping SNMP Trap Dest Network Management subAgent ............. OK
Stopping SNMP Master Network Management daemon .................. OK
Stopping outbound connection daemons for DDFA software .......... N/A
Stopping syslog-ng daemons ...................................... N/A
Stopping cfengine's cfservd daemon .............................. N/A
Stopping mail daemon ............................................ OK
Stopping IPv6 router advertisement daemon ....................... N/A
Stopping remote system status daemon ............................ N/A
Stopping RARP protocol daemon ................................... N/A
Stop PPPoE configuration ........................................ N/A
Stop PPP configuration .......................................... OK
Stopping router discover protocol daemon ........................ N/A
Stopping dynamic routing daemon ................................. N/A
Stopping Internet services daemon ............................... OK
Stopping multicast routing daemon ............................... N/A
Stop the Trusted Mode with NIS+ subsystem ....................... OK
Stop NFS client subsystem ....................................... OK
Stop NIS client subsystem ....................................... OK
Stop ldap client daemon ......................................... OK
Stop NIS/LDAP server subsystem .................................. N/A
Stop NIS server subsystem ....................................... OK
Stop NIS+ client subsystem ...................................... OK
Stop NIS+ server subsystem ...................................... OK
Stop NFS core subsystem ......................................... OK
Stop Runner ..................................................... OK
Stopping ISEE hpservices mad daemon. ............................ OK
Stopping name server daemon ..................................... N/A
VxVM VVR INFO V-5-4-2419 Stopping VRAS daemon ................... OK
VxVM VVR INFO V-5-4-2419 Stopping VxRSync daemon ................ OK
Unconfiguring DHCPv6 Interfaces ................................. OK
VxVM VVR INFO V-5-2-3620 Stopping Communication Daemon .......... OK
Stopping DHCPv6 Server daemon ................................... N/A
Stopping sfmdb PostgreSQL daemons ............................... OK
Unconfigure LAN interfaces ...................................... OK
Unconfigure LAN interfaces for IPv6 ............................. OK
Unconfigure HP gelan Gigabit Ethernet interfaces ................ OK
Unconfigure HP iether 100BT/Gigabit Ethernet interfaces ......... OK
Unconfigure HP igelan Gigabit Ethernet interfaces ............... OK
Stop network tracing and logging daemon ......................... OK
Stop pty allocator daemon ....................................... OK
Stop system message logging daemon .............................. OK
Stop Software Distributor agent daemon .......................... OK
Unconfigure Loopback interfaces (lo0) ........................... OK
Stopping VERITAS Enterprise Administrator Service (vxsvc) ....... OK
Stop Utmp Daemon ................................................ OK
Stop syncer daemon .............................................. OK
Stop Kernel Logging facility .................................... OK
Killing user processes .......................................... OK
Move nextboot kernel configuration into place ................... N/A
Unmount file systems ............................................ OK
VxVM INFO V-5-2-3254 Stopping all Volume Manager I/O Daemons .... OK
Transition to run-level 0 is complete.
Executing "/sbin/reboot -h ".
Shutdown at 10:53 (in 0 minutes)
*** FINAL System shutdown message from root@CHNFSCC1 ***
System going down IMMEDIATELY
System shutdown time has arrived
reboot: redirecting error messages to /dev/console
硬件故障修复完后
1、检查SG状态
[root@CHNFSCC1:/var/opt/resmon/log]#cmviewcl -v
CLUSTER STATUS
fscluster up
NODE STATUS STATE
CHNFSCC1 down halted
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c4t0d1 unknown
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY unknown 0/2/1/0 lan0
PRIMARY unknown 0/5/1/0 lan3
NODE STATUS STATE
CHNFSCC2 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c8t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d2 up running enabled CHNFSCC2
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC2 (current)
UNOWNED_PACKAGES
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d1 down halted enabled unowned
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS NODE_NAME NAME
Subnet unknown CHNFSCC1 192.168.0.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary down CHNFSCC1
2、启动节点1的SG
[root@CHNFSCC1:/var/opt/resmon/log]# cmrunnode -v CHNFSCC1
cmrunnode: Validating network configuration...
Gathering network information
Beginning network probing (this may take a while)
Completed network probing
cmrunnode: Network validation complete
cmrunnode: Validating cluster lock disk .... Done
Waiting for nodes to join ...... done
Cluster successfully formed.
Check the syslog files on all nodes in the cluster to verify that no warnings occurred during startup.
查看启动是否正常
[root@CHNFSCC1:/var/opt/resmon/log]#cmviewcl -v
CLUSTER STATUS
fscluster up
NODE STATUS STATE
CHNFSCC1 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c4t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d1 up running enabled CHNFSCC1
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Subnet up 192.168.0.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC1 (current)
NODE STATUS STATE
CHNFSCC2 up running
Cluster_Lock_LVM:
VOLUME_GROUP PHYSICAL_VOLUME STATUS
/dev/vglock /dev/dsk/c8t0d1 up
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 0/2/1/0 lan0
PRIMARY up 0/5/1/0 lan3
PACKAGE STATUS STATE AUTO_RUN NODE
pkg_fs_d2 up running enabled CHNFSCC2
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled CHNFSCC2 (current)
3、启动crs
查看crs状态
[root@CHNFSCC1:/var/opt/resmon/log]#ps -ef|grep crs
root 2119 1 0 12:10:47 ? 0:00 /bin/sh /sbin/init.d/init.crsd run
root 5574 5108 0 12:26:32 pts/1 0:00 grep crs
启动
[root@CHNFSCC1:/var/opt/resmon/log]#/sbin/init.d/init.crs start
Startup will be queued to init within 30 seconds.
查看进程
[root@CHNFSCC1:/var/opt/resmon/log]#ps -ef|grep crs
oracle 5617 5616 0 12:26:48 ? 0:00 /u01/app/oracle/product/crs/bin/crsctl.bin check boot
root 2119 1 0 12:10:47 ? 0:00 /bin/sh /sbin/init.d/init.crsd run
oracle 5616 2161 0 12:26:48 ? 0:00 sh -c /u01/app/oracle/product/crs/bin/crsctl check boot > /tmp/crsctl.2161
root 5634 5108 0 12:26:51 pts/1 0:00 grep crs
oracle 5622 2134 0 12:26:49 ? 0:00 sh -c /u01/app/oracle/product/crs/bin/crsctl check boot > /tmp/crsctl.2134
oracle 5623 5622 1 12:26:49 ? 0:00 /u01/app/oracle/product/crs/bin/crsctl.bin check boot
oracle 5629 5628 1 12:26:49 ? 0:00 /u01/app/oracle/product/crs/bin/crsctl.bin check boot
oracle 5628 2151 0 12:26:49 ? 0:00 sh -c /u01/app/oracle/product/crs/bin/crsctl check boot > /tmp/crsctl.2151
[root@CHNFSCC1:/var/opt/resmon/log]#ps -ef|grep crs
oracle 5728 5727 0 12:26:59 ? 0:00 /u01/app/oracle/product/crs/bin/oclsmon.bin
oracle 2117 1 0 12:10:47 ? 0:00 sh -c sh -c 'ulimit -c unlimited; cd /u01/app/oracle/product/crs/log/chnfscc1/evmd; exec /u01/app/oracle/product/crs/bin/evmd '
root 2119 1 0 12:10:47 ? 0:00 /u01/app/oracle/product/crs/bin/crsd.bin reboot
oracle 5719 5659 0 12:26:59 ? 0:00 sh -c /bin/sh -c 'cd /u01/app/oracle/product/crs/log/chnfscc1/cssd/oclsmon; ulimit -c unlimited; /u01/app/oracle/product/crs/bi
oracle 5816 5661 0 12:27:01 ? 0:00 sh -c /bin/sh -c 'ulimit -c unlimited; cd /u01/app/oracle/product/crs/log/chnfscc1/cssd; /u01/app/oracle/product/crs/bin/ocssd
oracle 5721 2117 0 12:26:59 ? 0:00 /u01/app/oracle/product/crs/bin/evmd.bin
oracle 5817 5816 0 12:27:01 ? 0:00 /bin/sh -c ulimit -c unlimited; cd /u01/app/oracle/product/crs/log/chnfscc1/cssd; /u01/app/oracle/product/crs/bin/ocssd || ex
oracle 5727 5719 0 12:26:59 ? 0:00 /bin/sh -c cd /u01/app/oracle/product/crs/log/chnfscc1/cssd/oclsmon; ulimit -c unlimited; /u01/app/oracle/product/crs/bin/oclsm
oracle 5818 5817 0 12:27:01 ? 0:00 /u01/app/oracle/product/crs/bin/ocssd.bin
oracle 6234 5721 0 12:27:17 ? 0:00 /u01/app/oracle/product/crs/bin/evmlogger.bin -o /u01/app/oracle/product/crs/evm/log/evmlogger.info -l /u01/app/oracle/product/
oracle 6728 1 0 12:27:36 ? 0:00 /u01/app/oracle/product/crs/opmn/bin/ons -d
oracle 6729 6728 0 12:27:36 ? 0:00 /u01/app/oracle/product/crs/opmn/bin/ons -d
root 7465 5108 0 12:27:55 pts/1 0:00 grep crs
oracle 7004 1 0 12:27:45 ? 0:00 /u01/app/oracle/product/crs/jdk/jre/bin/IA64N/java -DPROGRAM=gsd -classpath /u01/app/oracle/product/crs/jlib/netcfg.jar:/u01/ap
查看节点1的数据库,节点1的instance启动时间稍长
成功启动
oracle@CHNFSCC1[billdb1]:/home/oracle$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....b1.inst application ONLINE ONLINE chnfscc1
ora....b2.inst application ONLINE ONLINE chnfscc2
ora.billdb.db application ONLINE ONLINE chnfscc2
ora....SM1.asm application ONLINE ONLINE chnfscc1
ora....C1.lsnr application ONLINE ONLINE chnfscc1
ora....cc1.gsd application ONLINE ONLINE chnfscc1
ora....cc1.ons application ONLINE ONLINE chnfscc1
ora....cc1.vip application ONLINE ONLINE chnfscc1
ora....SM2.asm application ONLINE ONLINE chnfscc2
ora....C2.lsnr application ONLINE ONLINE chnfscc2
ora....cc2.gsd application ONLINE ONLINE chnfscc2
ora....cc2.ons application ONLINE ONLINE chnfscc2
ora....cc2.vip application ONLINE ONLINE chnfscc2
oracle@CHNFSCC1[billdb1]:/home/oracle$
至此成功启动
gsd offline解决方法:
1)、采用srvctl启动应用
2)、采用gsdctl启动
oracle@CHNFSCC2[billdb2]:/u01/app/oracle/admin/billdb/bdump$ /u01/app/oracle/product/rac10g/bin/gsdctl start
oracle@CHNFSCC2[billdb2]:/u01/app/oracle/admin/billdb/bdump$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....b1.inst application ONLINE ONLINE chnfscc1
ora....b2.inst application ONLINE ONLINE chnfscc2
ora.billdb.db application ONLINE ONLINE chnfscc2
ora....SM1.asm application ONLINE ONLINE chnfscc1
ora....C1.lsnr application ONLINE ONLINE chnfscc1
ora....cc1.gsd application ONLINE ONLINE chnfscc1
ora....cc1.ons application ONLINE ONLINE chnfscc1
ora....cc1.vip application ONLINE ONLINE chnfscc1
ora....SM2.asm application ONLINE ONLINE chnfscc2
ora....C2.lsnr application ONLINE ONLINE chnfscc2
ora....cc2.gsd application ONLINE OFFLINE
ora....cc2.ons application ONLINE ONLINE chnfscc2
ora....cc2.vip application ONLINE ONLINE chnfscc2
crs案例分析
http://www.itpub.net/viewthread.php?tid=1088801&extra=&page=2