网卡故障导致监听程序宕掉

作者：

total 455330
-rw-------   1 root       root       83388480 Oct  9  2007 core.2007-10-09-18:20:12
-rw-------   1 root       root       40824392 Mar 21 18:22 core.2008-03-21-18:27:26
-rw-------   1 root       root       33996800 Mar 21 18:31 core.2008-03-21-18:36:21
-rw-------   1 root       root       48979472 Mar 22 10:40 core.2008-03-22-10:45:09
-rw-r--r--   1 root       root       10500373 Apr 29 19:04 crsd.l01
-rw-r--r--   1 root       root       10519165 Jan 16 22:50 crsd.l02
-rw-r--r--   1 root       root       4875446 Jun 16 14:43 crsd.log
ll
total 186
-rw-r--r--   1 oracle     dba            473 Mar 21 16:32 evtf.log
-rw-r--r--   1 oracle     dba          12322 May  7 16:07 ora.billdb.db.log
-rw-r--r--   1 oracle     dba           9141 May  7 16:06 ora.chnfscc1.gsd.log
-rw-r--r--   1 oracle     dba          10924 May  7 16:06 ora.chnfscc1.ons.log
-rw-r--r--   1 root       sys          19659 Jun 16 08:24 ora.chnfscc1.vip.log
-rw-r--r--   1 root       sys          14778 Mar 22 10:45 ora.chnfscc2.vip.log
drwxrwxrwt   2 oracle     dba             96 Sep 19  2007 racgeut
drwxrwxrwt   2 oracle     dba             96 Sep 20  2007 racgevtf
drwxrwxrwt   2 oracle     dba             96 Sep 19  2007 racgmain

crsd.log错误如下：
2008-06-16 06:09:31.083: [    RACG][1] [25588][1][ora.chnfscc1.vip]: Interface lan0 checked failed (host=CHNFSCC1)
Invalid parameters, or failed to bring up VIP (host=CHNFSCC1)

2008-06-16 06:09:31.091: [    RACG][1] [25588][1][ora.chnfscc1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2008-06-16 06:09:31.091: [    RACG][1] [25588][1][ora.chnfscc1.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check chnfscc1

2008-06-16 06:09:31.097: [    RACG][1] [25588][1][ora.chnfscc1.vip]: clsrcexecut: rc = 1, time = 0.441s

2008-06-16 06:09:31.097: [    RACG][1] [25588][1][ora.chnfscc1.vip]: end for resource = ora.chnfscc1.vip, action = check, status = 1, time = 0.482s

2008-06-16 08:24:14.304: [    RACG][1] [17556][1][ora.chnfscc1.vip]: Interface lan0 checked failed (host=CHNFSCC1)
Invalid parameters, or failed to bring up VIP (host=CHNFSCC1)

2008-06-16 08:24:14.304: [    RACG][1] [17556][1][ora.chnfscc1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/u01/app/oracle/product/crs

2008-06-16 08:24:14.304: [    RACG][1] [17556][1][ora.chnfscc1.vip]: clsrcexecut: cmd = /u01/app/oracle/product/crs/bin/racgeut -e _USR_ORA_DEBUG=0 54 /u01/app/oracle/product/crs/bin/racgvip check chnfscc1

2008-06-16 08:24:14.304: [    RACG][1] [17556][1][ora.chnfscc1.vip]: clsrcexecut: rc = 1, time = 0.334s

2008-06-16 08:24:14.322: [    RACG][1] [17556][1][ora.chnfscc1.vip]: end for resource = ora.chnfscc1.vip, action = check, status = 1, time = 0.381s

ora.chnfscc1.vip.log
错误如下：
2008-06-16 06:09:31.103: [  CRSAPP][379445] CheckResource error for ora.chnfscc1.vip error code = 1
2008-06-16 06:09:31.108: [  CRSRES][379445] In stateChanged, ora.chnfscc1.vip target is ONLINE
2008-06-16 06:09:31.108: [  CRSRES][379445] ora.chnfscc1.vip on chnfscc1 went OFFLINE unexpectedly
2008-06-16 06:09:31.108: [  CRSRES][379445] StopResource: setting CLI values
2008-06-16 06:09:31.111: [  CRSRES][379445] Attempting to stop `ora.chnfscc1.vip` on member `chnfscc1`
2008-06-16 06:09:31.520: [  CRSRES][379445] Stop of `ora.chnfscc1.vip` on member `chnfscc1` succeeded.
2008-06-16 06:09:31.521: [  CRSRES][379445] ora.chnfscc1.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
2008-06-16 06:09:31.534: [  CRSRES][379445] ora.chnfscc1.vip failed on chnfscc1 relocating.
2008-06-16 06:09:31.609: [  CRSRES][379445] StopResource: setting CLI values
2008-06-16 06:09:31.611: [  CRSRES][379445] Attempting to stop `ora.chnfscc1.LISTENER_CHNFSCC1.lsnr` on member `chnfscc1`
2008-06-16 06:10:16.577: [  OCRSRV][29]th_select_handler: Failed to retrieve procctx from ht. constr = [16615904] retval lht [-27] Signal CV.
2008-06-16 06:10:49.588: [  CRSRES][379445] Stop of `ora.chnfscc1.LISTENER_CHNFSCC1.lsnr` on member `chnfscc1` succeeded.
2008-06-16 06:10:49.604: [  CRSRES][379445] Attempting to start `ora.chnfscc1.vip` on member `chnfscc2`
2008-06-16 06:10:56.251: [  CRSRES][379445] Start of `ora.chnfscc1.vip` on member `chnfscc2` succeeded.
2008-06-16 06:20:17.331: [  CRSRES][379543] StopResource: setting CLI values
2008-06-16 06:20:17.353: [  CRSRES][379543] Attempting to stop `ora.chnfscc1.vip` on member `chnfscc2`
2008-06-16 06:20:17.777: [  CRSRES][379543] Stop of `ora.chnfscc1.vip` on member `chnfscc2` succeeded.
2008-06-16 06:20:17.782: [  CRSRES][379543] startRunnable: setting CLI values
2008-06-16 06:20:17.782: [  CRSRES][379543] Attempting to start `ora.chnfscc1.vip` on member `chnfscc1`
2008-06-16 06:20:17.938: [  OCRSRV][29]th_select_handler: Failed to retrieve procctx from ht. constr = [21988752] retval lht [-27] Signal CV.
2008-06-16 06:20:24.373: [  CRSRES][379543] Start of `ora.chnfscc1.vip` on member `chnfscc1` succeeded.

..........

2008-06-16 08:24:14.364: [  CRSRES][379916] In stateChanged, ora.chnfscc1.vip target is ONLINE
2008-06-16 08:24:14.364: [  CRSRES][379916] ora.chnfscc1.vip on chnfscc1 went OFFLINE unexpectedly
2008-06-16 08:24:14.364: [  CRSRES][379916] StopResource: setting CLI values
2008-06-16 08:24:14.376: [  CRSRES][379916] Attempting to stop `ora.chnfscc1.vip` on member `chnfscc1`
2008-06-16 08:24:14.534: [  OCRSRV][29]th_select_handler: Failed to retrieve procctx from ht. constr = [16616864] retval lht [-27] Signal CV.
2008-06-16 08:24:14.825: [  CRSRES][379916] Stop of `ora.chnfscc1.vip` on member `chnfscc1` succeeded.
2008-06-16 08:24:14.826: [  CRSRES][379916] ora.chnfscc1.vip RESTART_COUNT=0 RESTART_ATTEMPTS=0
2008-06-16 08:24:14.840: [  CRSRES][379916] ora.chnfscc1.vip failed on chnfscc1 relocating.
2008-06-16 08:24:14.994: [  CRSRES][379916] StopResource: setting CLI values
2008-06-16 08:24:14.996: [  CRSRES][379916] Attempting to stop `ora.chnfscc1.LISTENER_CHNFSCC1.lsnr` on member `chnfscc1`
2008-06-16 08:25:32.935: [  CRSRES][379916] Stop of `ora.chnfscc1.LISTENER_CHNFSCC1.lsnr` on member `chnfscc1` succeeded.
2008-06-16 08:25:32.957: [  CRSRES][379916] Attempting to start `ora.chnfscc1.vip` on member `chnfscc2`
2008-06-16 08:25:39.616: [  CRSRES][379916] Start of `ora.chnfscc1.vip` on member `chnfscc2` succeeded.
2008-06-16 08:30:31.393: [  CRSRES][379937] StopResource: setting CLI values
2008-06-16 08:30:31.409: [  CRSRES][379937] Attempting to stop `ora.chnfscc1.vip` on member `chnfscc2`
2008-06-16 08:30:31.866: [  CRSRES][379937] Stop of `ora.chnfscc1.vip` on member `chnfscc2` succeeded.
2008-06-16 08:30:31.873: [  CRSRES][379937] startRunnable: setting CLI values
2008-06-16 08:30:31.874: [  CRSRES][379937] Attempting to start `ora.chnfscc1.vip` on member `chnfscc1`
2008-06-16 08:30:38.479: [  CRSRES][379937] Start of `ora.chnfscc1.vip` on member `chnfscc1` succeeded.

分析结果如下：
在时间点2008-06-16 06:09:31.083网卡lan0宕掉，随后在2008-06-16 06:09:31.103时间点出现vip也异常offline
随后Oracle将节点1的vip服务停止，接着将节点1的监听停止，然后在节点2启动vip，将浮动ip迁移到节点2，几分钟后再次在节点2将vip关闭，然后在节点1重新启动

这里面vip可以自动重启和漂移，但是没有看到监听的自动重启