发生一下crs的服务出现offline 或 unknown 状态的时候。我们可能看到的状态如下:
ecsrac01$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE UNKNOWN ecsrac02
ora....01.lsnr application ONLINE UNKNOWN ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application ONLINE UNKNOWN ecsrac02
ora....c02.gsd application ONLINE UNKNOWN ecsrac02
ora....c02.ons application ONLINE UNKNOWN ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application OFFLINE OFFLINE
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application OFFLINE OFFLINE
ora....01.lsnr application OFFLINE OFFLINE
ora....c01.gsd application OFFLINE OFFLINE
ora....c01.ons application OFFLINE OFFLINE
ora....c01.vip application OFFLINE OFFLINE
ora....02.lsnr application OFFLINE OFFLINE
ora....c02.gsd application OFFLINE OFFLINE
ora....c02.ons application OFFLINE OFFLINE
ora....c02.vip application OFFLINE OFFLINE
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application OFFLINE OFFLINE
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application OFFLINE OFFLINE
ora....c01.gsd application OFFLINE OFFLINE
ora....c01.ons application OFFLINE OFFLINE
ora....c01.vip application OFFLINE OFFLINE
ora....02.lsnr application ONLINE ONLINE ecsrac02
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
解决方法:
例子1:
ecsrac01$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application OFFLINE UNKNOWN ecsrac01
ora....c01.gsd application OFFLINE UNKNOWN ecsrac01
ora....c01.ons application OFFLINE UNKNOWN ecsrac01
ora....c01.vip application OFFLINE UNKNOWN ecsrac01
ora....02.lsnr application ONLINE ONLINE ecsrac02
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
ecsrac01$srvctl stop nodeapps -n ecsrac01
ecsrac01$srvctl start nodeapps -n ecsrac01
ecsrac01$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application ONLINE ONLINE ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application ONLINE ONLINE ecsrac02
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
ecsrac01$
例子2: 单独启动crs服务
ecsrac01$crs_stop -f ora.ecsrac01.ons
ecsrac01$crs_stop -f ora.ecsrac01.gsd
ecsrac01$crs_stop -f ora.ecsrac01.vip
ecsrac01$crs_stop -f ora.ecsrac01.LISTENER_ECSRAC01.lsnr
ecsrac01$crs_stop -f ora.ecsdb.ecsdb1.inst
ecsrac01$crs_start ora.ecsrac01.ons
ecsrac01$crs_start ora.ecsrac01.gsd
ecsrac01$crs_start ora.ecsrac01.vip
ecsrac01$crs_start ora.ecsrac01.LISTENER_ECSRAC01.lsnr
ecsrac01$crs_start ora.ecsdb.ecsdb1.inst
节点2 因为网络断开一会,导致了vip漂移到节点1 。 监听服务呈现offline 状态 。
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application ONLINE ONLINE ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application OFFLINE OFFLINE
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac01
查看服务名称:
ecsrac02$crs_stat
NAME=ora.ecsdb.db
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsdb.ecsdb1.inst
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsdb.ecsdb2.inst
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac02
NAME=ora.ecsrac01.LISTENER_ECSRAC01.lsnr
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsrac01.gsd
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsrac01.ons
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsrac01.vip
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
NAME=ora.ecsrac02.LISTENER_ECSRAC02.lsnr
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.ecsrac02.gsd
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac02
NAME=ora.ecsrac02.ons
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac02
NAME=ora.ecsrac02.vip
TYPE=application
TARGET=ONLINE
STATE=ONLINE on ecsrac01
试图开启单独监听服务 (因为vip的漂移,一般不会成功) :
ecsrac02$crs_start ora.ecsrac02.LISTENER_ECSRAC02.lsnr
ecsrac01 : CRS-1019: Resource ora.ecsrac02.LISTENER_ECSRAC02.lsnr (application) cannot run on ecsrac01
ecsrac02 : CRS-1018: Resource ora.ecsrac02.vip (application) is already running on ecsrac01
CRS-0223: Resource 'ora.ecsrac02.LISTENER_ECSRAC02.lsnr' has placement error.
我们可以看到报错中有一个 ecsrac02 : CRS-1018: Resource ora.ecsrac02.vip (application) is already running on ecsrac01 错误, 表示节点2的VIP已经漂移到了节点1 ( 这里节点1机器名称为 ecsrac01 ) 。因为VIP的漂移导致节点2的监听不能在节点1上运行 (节点2本身的监听是正常的,但是只显示一个实例,可以通过lsnrctl stat 查看 ) 。 所以需要监听正常, 首先需要节点2的VIP 重新运行到节点2上。 从 ora....c02.vip application ONLINE ONLINE ecsrac01 上可以看到节点2 VIP 是运行在 ecsrac01 节点1机器上 。
我们在处理监听之前,先处理VIP . 查看节点2 VIP 对应的服务名称 (通过 crs_stat -v 查看 )
ecsrac02$crs_stop ora.ecsrac02.vip
Attempting to stop `ora.ecsrac02.vip` on member `ecsrac01`
Stop of `ora.ecsrac02.vip` on member `ecsrac01` succeeded.
ecsrac02$crs_start ora.ecsrac02.vip
Attempting to start `ora.ecsrac02.vip` on member `ecsrac02` --- 重新运行到节点2上,前提是网络恢复正常 。
Start of `ora.ecsrac02.vip` on member `ecsrac02` succeeded.
ecsrac02$crs_start ora.ecsrac02.LISTENER_ECSRAC02.lsnr
Attempting to start `ora.ecsrac02.LISTENER_ECSRAC02.lsnr` on member `ecsrac02`
Start of `ora.ecsrac02.LISTENER_ECSRAC02.lsnr` on member `ecsrac02` succeeded.
ecsrac02$
查看状态:
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application ONLINE ONLINE ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application ONLINE ONLINE ecsrac02
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE UNKNOWN ecsrac02
ora....01.lsnr application ONLINE UNKNOWN ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application ONLINE UNKNOWN ecsrac02
ora....c02.gsd application ONLINE UNKNOWN ecsrac02
ora....c02.ons application ONLINE UNKNOWN ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
[root@ecsrac02 ~]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
/etc/init.d/init.crsd: line 81: /tmp/oratz.3839: No such file or directory
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
[root@ecsrac02 ~]# /etc/init.d/init.crs start
Startup will be queued to init within 90 seconds.
ecsrac02$crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.ecsdb.db application ONLINE ONLINE ecsrac01
ora....b1.inst application ONLINE ONLINE ecsrac01
ora....b2.inst application ONLINE ONLINE ecsrac02
ora....01.lsnr application ONLINE ONLINE ecsrac01
ora....c01.gsd application ONLINE ONLINE ecsrac01
ora....c01.ons application ONLINE ONLINE ecsrac01
ora....c01.vip application ONLINE ONLINE ecsrac01
ora....02.lsnr application ONLINE ONLINE ecsrac02
ora....c02.gsd application ONLINE ONLINE ecsrac02
ora....c02.ons application ONLINE ONLINE ecsrac02
ora....c02.vip application ONLINE ONLINE ecsrac02
ecsrac02$
在rac环境中如果使用crs_stop -all去停rac服务时,很有可能会出现crs中的一些服务的状态是UNKNOWN,这时我们可以采用两种方法去重新启动crs服务:
1:转换到root用户下用/etc/init.d/init.crs stop先禁用crs,然后再用/etc/init.d/init.crs start去启用crs,启用crs后会自动启动crs的一系列服务,注意此种方法需要在两台节点上都执行;
2:是用crs_stop -f 参数把crs中状态是UNKNOWN的服务关掉,然后再用crs_start -f (加一个-f的参数)启动所有的服务就可以。参考:
要分别在两个节点上执行;
在启动的过程中最好检测着crs、ASM和数据库的日志:
crs日志:
[oracle@rac1 ~]$ tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac1/alertrac1.log
[oracle@rac2 ~]$ tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac2/alertrac2.log
ASM日志:
[oracle@rac1 ~]$ tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM1.log
[oracle@rac2 ~]$ tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM2.log
数据库日志:
[oracle@rac1 ~]$ tail -f /u01/app/oracle/admin/ora9i/bdump/alert_ora9i1.log
[oracle@rac2 ~]$ tail -f /u01/app/oracle/admin/ora9i/bdump/alert_ora9i2.log