Chinaunix首页 | 论坛 | 博客

liyf0371peter1981.blog.chinaunix.net

首页　| 　博文目录　| 　关于我

liyf0371

  • 博客访问： 3383286
  • 博文数量： 631
  • 博客积分： 10716
  • 博客等级： 上将
  • 技术积分： 8397
  • 用 户 组： 普通用户
  • 注册时间： 2008-04-01 22:35
文章分类

全部博文（631）

文章存档

2020年（2）

2019年（22）

2018年（4）

2017年（37）

2016年（22）

2015年（1）

2013年（12）

2012年（20）

2011年（19）

2010年（20）

2009年（282）

2008年（190）

我的朋友
最近访客
推荐博文
相关博文
ASM 仲裁盘OCR DG损坏，重建步骤

分类： Oracle

2016-07-03 09:57:07

主机启动后发现少了ocrsd.bin进程，检查grid alert日志，发现下面信息。 ASM OCR DG 无法启动。

  1. GMON updating disk modes for group 2 at 10 for pid 27, osid 7121
  2. ERROR: no read quorum in group: required 2, found 0 disks
  3. Fri Jan 15 10:33:21 2016
  4. NOTE: cache dismounting (not clean) group 2/0x676844DF (OCRDG)
  5. NOTE: messaging CKPT to quiesce pins Unix process pid: 7124, image: oracle@cntl202 (B001)
  6. Fri Jan 15 10:33:21 2016
  7. NOTE: halting all I/Os to diskgroup 2 (OCRDG)      >>>>>>>>>>>>>>>OCRDG有IO故障。
  8. Fri Jan 15 10:33:21 2016
  9. NOTE: LGWR doing non-clean dismount of group 2 (OCRDG)
  10. NOTE: LGWR sync ABA=6.60 last written ABA 6.60
  11. WARNING: Offline for disk OCRDG_0000 in mode 0x7f failed.
  12. WARNING: Offline for disk OCRDG_0001 in mode 0x7f failed.
  13. WARNING: Offline for disk OCRDG_0002 in mode 0x7f failed.
  14. Fri Jan 15 10:33:21 2016
  15. NOTE: No asm libraries found in the system

ASM使用ASMLIB，"oracleasm listdisks" 发现少了一个ASM DISKS，
  1. --缺少了一个ASMDISK1
  2. [/root]# oracleasm listdisks
  3. ASMDISK2 
  4. ASMDISK3 
  5. DATADISK1
 [/root]# oracleasm querydisk -v -p -d ASMDISK1
 Disk "ASMDISK1" does not exist or is not instantiated
重建ASMDISK1
  1. --无法创建，提示已存在。
  2. [/root]#oracleasm createdisk ASMDISK1 /dev/sdd1
  3. Device "/dev/sdd1" is already labeled for ASM disk ""

  4.
  5. [/root]#oracleasm deletedisk ASMDISK1
  6. Disk "ASMDISK1" does not exist or is not instantiated
 --清空磁盘头。
  1. [/root]#dd if=/dev/zero of=/dev/sdd1 bs=1024 count=100
  2. 100+0 records in
  3. 100+0 records out
  4. 102400 bytes (102 kB) copied, 0.0812539 s, 1.3 MB/s

  6. [/root]#oracleasm createdisk ASMDISK1 /dev/sdd1
  7. Writing disk header: done
  8. Instantiating disk: done


1. [ALL NODES]停止CRS，crsd.bin进程不存在，无法正常停，需要 "-f" 强制停止
  1. [/root]#crsctl stop crs -f
  2. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cntl069'
  3. CRS-2673: Attempting to stop 'ora.mdnsd' on 'cntl069'
  4. CRS-2673: Attempting to stop 'ora.ctssd' on 'cntl069'
  5. CRS-2673: Attempting to stop 'ora.asm' on 'cntl069'
  6. CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cntl069'
  7. CRS-2677: Stop of 'ora.mdnsd' on 'cntl069' succeeded
  8. CRS-2677: Stop of 'ora.asm' on 'cntl069' succeeded
  9. CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'cntl069'
  10. CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'cntl069' succeeded
  11. CRS-2677: Stop of 'ora.drivers.acfs' on 'cntl069' succeeded
  12. CRS-2677: Stop of 'ora.ctssd' on 'cntl069' succeeded
  13. CRS-2673: Attempting to stop 'ora.cssd' on 'cntl069'
  14. CRS-2677: Stop of 'ora.cssd' on 'cntl069' succeeded
  15. CRS-2673: Attempting to stop 'ora.gipcd' on 'cntl069'
  16. CRS-2677: Stop of 'ora.gipcd' on 'cntl069' succeeded
  17. CRS-2673: Attempting to stop 'ora.gpnpd' on 'cntl069'
  18. CRS-2677: Stop of 'ora.gpnpd' on 'cntl069' succeeded
  19. CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cntl069' has completed
  20. CRS-4133: Oracle High Availability Services has been stopped.

2. 删除CRS  <--root user
  1. [/root]#$CRS_HOME/crs/install/rootcrs.pl -deconfig -force               >>>>>>>>>其他节点
  2. Using configuration parameter file: /grid/11.2/product/crs/install/crsconfig_params
  3. PRCR-1119 : Failed to look up CRS resources of ora.cluster_vip_net1.type type
  4. PRCR-1068 : Failed to query resources
  5. Cannot communicate with crsd
  6. PRCR-1070 : Failed to check if resource ora.gsd is registered
  7. Cannot communicate with crsd
  8. PRCR-1070 : Failed to check if resource ora.ons is registered
  9. Cannot communicate with crsd
  10. ...

  12. [/root]#$CRS_HOME/crs/install/rootcrs.pl -deconfig -force -lastnode        >>>>>>>>>最后一个节点
  13. CRS-4535: Cannot communicate with Cluster Ready Services
  14. CRS-4000: Command Stop failed, or completed with errors.
  15. CRS-4544: Unable to connect to OHAS
  16. CRS-4000: Command Stop failed, or completed with errors.
  17. Removing Trace File Analyzer
  18. error: package cvuqdisk is not installed
  19. Successfully deconfigured Oracle clusterware stack on this node

3. [ALL NODES] 执行root.sh 配置CS  <-- root user
  1. [/root]#CRS_HOME/root.sh
  2. Performing root user operation for Oracle 11g

  4. The following environment variables are set as:
  5.     ORACLE_OWNER= grid
  6.     ORACLE_HOME= /grid/11.2/product

  8. Enter the full pathname of the local bin directory: [/usr/local/bin]:
  9. The contents of "dbhome" have not changed. No need to overwrite.
  10. The contents of "oraenv" have not changed. No need to overwrite.
  11. The contents of "coraenv" have not changed. No need to overwrite.

  13. Entries will be added to the /etc/oratab file as needed by
  14. Database Configuration Assistant when a database is created
  15. Finished running generic part of root script.
  16. Now product-specific root actions will be performed.
  17. Using configuration parameter file: /grid/11.2/product/crs/install/crsconfig_params
  18. User ignored Prerequisites during installation
  19. Installing Trace File Analyzer
  20. OLR initialization - successful
  21. Adding Clusterware entries to upstart
  22. CRS-2672: Attempting to start 'ora.mdnsd' on 'cntl069'
  23. CRS-2676: Start of 'ora.mdnsd' on 'cntl069' succeeded
  24. CRS-2672: Attempting to start 'ora.gpnpd' on 'cntl069'
  25. CRS-2676: Start of 'ora.gpnpd' on 'cntl069' succeeded
  26. CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cntl069'
  27. CRS-2672: Attempting to start 'ora.gipcd' on 'cntl069'
  28. CRS-2676: Start of 'ora.cssdmonitor' on 'cntl069' succeeded
  29. CRS-2676: Start of 'ora.gipcd' on 'cntl069' succeeded
  30. CRS-2672: Attempting to start 'ora.cssd' on 'cntl069'
  31. CRS-2672: Attempting to start 'ora.diskmon' on 'cntl069'
  32. CRS-2676: Start of 'ora.diskmon' on 'cntl069' succeeded
  33. CRS-2676: Start of 'ora.cssd' on 'cntl069' succeeded

  35. ASM created and started successfully.

  37. Disk Group CRS created successfully.

  39. clscfg: -install mode specified
  40. Successfully accumulated necessary OCR keys.
  41. Creating OCR keys for user 'root', privgrp 'root'..
  42. Operation successful.
  43. Successful addition of voting disk 966344bb8c054f61bf7627175f0d53e8.
  44. Successful addition of voting disk 3f3735b4d5f44f79bf27624845c0c81b.
  45. Successful addition of voting disk 8bf524ee376b4ff1bf5c3b7e69e5c98b.
  46. Successfully replaced voting disk group with +CRS.
  47. CRS-4266: Voting file(s) successfully replaced
  48. ## STATE File Universal Id File Name Disk group
  49. -- ----- ----------------- --------- ---------
  50.  1. ONLINE 966344bb8c054f61bf7627175f0d53e8 (ORCL:ASMDISK1) [CRS]
  51.  2. ONLINE 3f3735b4d5f44f79bf27624845c0c81b (ORCL:ASMDISK2) [CRS]
  52.  3. ONLINE 8bf524ee376b4ff1bf5c3b7e69e5c98b (ORCL:ASMDISK3) [CRS]
  53. Located 3 voting disk(s).
  54.   CRS-2672: Attempting to start 'ora.asm' on 'cntl069'
  55. CRS-2676: Start of 'ora.asm' on 'cntl069' succeeded
  56. CRS-2672: Attempting to start 'ora.CRS.dg' on 'cntl069'
  57. CRS-2676: Start of 'ora.CRS.dg' on 'cntl069' succeeded
  58. Preparing packages for installation...
  59. cvuqdisk-1.0.9-1
  60. Configure Oracle Grid Infrastructure for a Cluster ... succeeded

4. 检查CRS配置正常。
  1. [/grid/11.2/product]#crsctl status res -t
  2. --------------------------------------------------------------------------------
  3. NAME TARGET STATE SERVER STATE_DETAILS
  4. --------------------------------------------------------------------------------
  5. Local Resources
  6. --------------------------------------------------------------------------------
  7. ora.CRS.dg
  8.                ONLINE ONLINE cntl069
  9.                ONLINE ONLINE cntl070
  10. ora.asm
  11.                ONLINE ONLINE cntl069 Started
  12.                ONLINE ONLINE cntl070 Started
  13. ora.gsd
  14.                OFFLINE OFFLINE cntl069
  15.                OFFLINE OFFLINE cntl070
  16. ora.net1.network
  17.                ONLINE ONLINE cntl069
  18.                ONLINE ONLINE cntl070
  19. ora.ons
  20.                ONLINE ONLINE cntl069
  21.                ONLINE ONLINE cntl070
  22. ora.registry.acfs
  23.                ONLINE ONLINE cntl069
  24.                ONLINE ONLINE cntl070
  25. --------------------------------------------------------------------------------
  26. Cluster Resources
  27. --------------------------------------------------------------------------------
  28. ora.LISTENER_SCAN1.lsnr
  29.       1 ONLINE ONLINE cntl069
  30. ora.cntl069.vip
  31.       1 ONLINE ONLINE cntl069
  32. ora.cntl070.vip
  33.       1 ONLINE ONLINE cntl070
  34. ora.cvu
  35.       1 ONLINE ONLINE cntl069
  36. ora.oc4j
  37.       1 ONLINE ONLINE cntl069
  38. ora.scan1.vip
  39.       1 ONLINE ONLINE cntl069

5. 确实LISTENER资源，netca创建。

6. 如果已经有数据库，可使用下面命令加入资源
  1. 添加数据库(oracle用户下执行)
  2.  [oracle@node1 ~]$ srvctl add database -d racdb -o /u01/product/oracle/11.2.0/db_1
  3.  [oracle@node1 ~]$
  4.  
  5. 添加实例(oracle用户下执行)
  6.  [oracle@node1 ~]$ srvctl add instance -d racdb -i racdb1 -n node1
  7.  [oracle@node1 ~]$ srvctl add instance -d racdb -i racdb2 -n node2
  8.  
  9. 添加之前数据库的服务(oracle用户下执行)
  10.  [oracle@node1 ~]$ srvctl add service -d racdb -s kettle -r racdb1 -a racdb2 -P BASIC
若还没数据库，可使用dbca重建。                                                                                                                                                                                                                                                                                             

再次检查CRS状态
  1. [/grid/11.2/product]#crsctl status res -t
  2. --------------------------------------------------------------------------------
  3. NAME TARGET STATE SERVER STATE_DETAILS
  4. --------------------------------------------------------------------------------
  5. Local Resources
  6. --------------------------------------------------------------------------------
  7. ora.CRS.dg
  8.                ONLINE ONLINE cntl069
  9.                ONLINE ONLINE cntl070
  10. ora.DATA.dg
  11.                ONLINE ONLINE cntl069
  12.                ONLINE ONLINE cntl070
  13. ora.LISTENER.lsnr
  14.                ONLINE ONLINE cntl069
  15.                ONLINE ONLINE cntl070
  16. ora.asm
  17.                ONLINE ONLINE cntl069 Started
  18.                ONLINE ONLINE cntl070 Started
  19. ora.gsd
  20.                OFFLINE OFFLINE cntl069
  21.                OFFLINE OFFLINE cntl070
  22. ora.net1.network
  23.                ONLINE ONLINE cntl069
  24.                ONLINE ONLINE cntl070
  25. ora.ons
  26.                ONLINE ONLINE cntl069
  27.                ONLINE ONLINE cntl070
  28. ora.registry.acfs
  29.                ONLINE ONLINE cntl069
  30.                ONLINE ONLINE cntl070
  31. --------------------------------------------------------------------------------
  32. Cluster Resources
  33. --------------------------------------------------------------------------------
  34. ora.LISTENER_SCAN1.lsnr
  35.       1 ONLINE ONLINE cntl069
  36. ora.cntl069.vip
  37.       1 ONLINE ONLINE cntl069
  38. ora.cntl070.vip
  39.       1 ONLINE ONLINE cntl070
  40. ora.cvu
  41.       1 ONLINE ONLINE cntl069
  42. ora.ecard2pt.db
  43.       1 ONLINE ONLINE cntl069 Open
  44.       2 ONLINE ONLINE cntl070 Open
  45. ora.oc4j
  46.       1 ONLINE ONLINE cntl069
  47. ora.scan1.vip
  48.       1 ONLINE ONLINE cntl069
阅读(4343) | 评论(0) | 转发(0) |
0

上一篇：ASM文件管理深析

下一篇：solaris系统panics故障实例与解决

给主人留下些什么吧！~~
关于我们 | 关于IT168 | 联系方式 | 广告合作 | 法律声明 | 免费注册

Copyright 2001-2010 ChinaUnix.net All Rights Reserved 北京皓辰网域网络信息技术有限公司. 版权所有

感谢所有关心和支持过ChinaUnix的朋友们

16024965号-6