Chinaunix首页 | 论坛 | 博客
  • 博客访问: 2154293
  • 博文数量: 157
  • 博客积分: 10047
  • 博客等级: 上将
  • 技术积分: 6757
  • 用 户 组: 普通用户
  • 注册时间: 2005-05-19 11:38
文章分类

全部博文(157)

文章存档

2011年(16)

2010年(50)

2009年(42)

2008年(49)

我的朋友

分类: Oracle

2009-07-22 22:03:11

# crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM 

检查crs进程:

# ps -ef | grep css

root      6929     1  0 19:56 ?        00:00:00 /bin/sh /etc/init.d/init.cssd fatal
root      6960  6928  0 19:56 ?        00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root      6963  6929  0 19:56 ?        00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root      7064  6935  0 19:56 ?        00:00:00 /bin/sh /etc/init.d/init.cssd startcheck

从上可以看出init.cssd停在startcheck中,并没有运行ocssd.bin daemon。

检查crsd.log及ocssd.log,没有发现有用的信息,而在$ORA_CRS_HOME/log/rac1/client中:

[root@rac1 client]# more css339.log
Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 .  All rights reserved.
2009-06-09 20:00:28.799: [ CSSCLNT][2541220896]clsssInitNative: connect failed, rc 9

[root@rac1 client]# more  clsc790.log
Database 10g CRS Release 10.2.0.1.0 Production Copyright 1996, 2005 .  All rights reserved.
2009-06-09 20:39:27.940: [ COMMCRS][2541220896]clsc_connect: (0×66c3d0) no listener at (ADDRESS=(PROTOCOL=IPC)(KEY=CRSD_UI_SOCKET))
2009-06-09 20:39:27.941: [ COMMCRS][2541220896]clsc_connect: (0×6116e0) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=SYSTEM.evm.acceptor.auth))
2009-06-09 20:39:27.941: [ default][2541220896]Terminating clsd session

检查系统日志,在客户报故障的时间范围内发现如下信息:

Jun  9 17:19:39 rac1 logger: Cluster Ready Services starting up automatically.
Jun  9 17:19:39 rac1 init.crs: Startup will be queued to init within 90 seconds.
Jun  9 17:19:39 rac1 rc: Starting init.crs:  succeeded
Jun  9 17:19:39 rac1 readahead: Starting background readahead: 
Jun  9 17:19:39 rac1 rc: Starting readahead:  succeeded
Jun  9 17:19:41 rac1 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.4528.
Jun  9 17:19:41 rac1 logger: Cluster Ready Services waiting on dependencies. Diagnostics in /tmp/crsctl.4531.
Jun  9 17:19:41 rac1 logger: Cluster Ready Services waiting on dependencies. Diagnostics in
/tmp/crsctl.4603.

转到/tmp去查看crsctl.*文件,却发现没有,奇怪!那这些信息又是在哪里产生呢?于是去查看/etc/init.d/init.cssd脚本,该信息在脚本中的如下地方产生:

    # Wait for additional filesystems and objects to become available
    # crsctl should print out a message indicating cause of failure.

    $SU $ORACLE_USER -c “$CRSCTL check boot > $CRSCTLOUT”
    RC=$?
    while [ "$RC" != "0" ]
    do
      $LOGMSG Cluster Ready Services waiting on dependencies. Diagnostics in $CRSCTLOUT.
      $SLEEP $DEP_CHECK_WAIT
      $SU $ORACLE_USER -c “$CRSCTL check boot > $CRSCTLOUT”
      RC=$?
    done

该段脚本正好是在startcheck的选项中执行的,再看看变量$CRSCTLOUT的定义:

# Temp file for crsctl output
CRSCTLOUT=/tmp/crsctl.$$

正是/tmp目录。难道是/tmp目录有问题?手工执行:

[root@rac1 /]# /etc/init.d/init.cssd startcheck
-bash: /tmp/crsctl.31969: No such file or directory

果然是/tmp有问题,检查/tmp的权限:

drwxr-xr-x    7   1003 dba   4096 Jun 10 04:02 tmp

晕,权限被改了,正确的应该是:

drwxrwxrwt   15 root root  4096 Jun 10 09:51 tmp

于是修改/tmp权限:

#chown -R root:root /tmp
#chmod -R 1777 /tmp

然后执行:

[root@rac1 tmp]# /etc/init.d/init.crs stop
Shutting down Cluster Ready Services (CRS):
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.

[root@rac1 tmp]# /etc/init.d/init.crs start
Startup will be queued to init within 90 seconds.

过一会儿查看crs进程,crs成功运行,至此问题解决!

客户环境:RHEL AS 4 +  10g RAC

阅读(3206) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~