Chinaunix首页 | 论坛 | 博客
  • 博客访问: 3396658
  • 博文数量: 631
  • 博客积分: 10716
  • 博客等级: 上将
  • 技术积分: 8397
  • 用 户 组: 普通用户
  • 注册时间: 2008-04-01 22:35
文章分类

全部博文(631)

文章存档

2020年(2)

2019年(22)

2018年(4)

2017年(37)

2016年(22)

2015年(1)

2013年(12)

2012年(20)

2011年(19)

2010年(20)

2009年(282)

2008年(190)

分类: Oracle

2017-08-31 10:43:00

先做一些检查:



1.检查ocr的完整性

root@oeltan1 ~]# su - grid
[grid@oeltan1 ~]$ cluvfy comp ocr -n all
  1. Verifying OCR integrity   
  2. Checking OCR integrity...  
  3. Checking the absence of a non-clustered configuration...  
  4. All nodes free of non-clustered, local-only configurations  
  5. ASM Running check passed. ASM is running on all specified nodes  
  6. Checking OCR config file "/etc/oracle/ocr.loc"...  
  7. OCR config file "/etc/oracle/ocr.loc" check successful  
  8. Disk group for ocr location "+CRS" available on all the nodes  
  9. Disk group for ocr location "+DATA1" available on all the nodes  
  10.   
  11. NOTE:   
  12. This check does not verify the integrity of the OCR contents. Execute 'ocrcheck' as a privileged user to verify the contents of OCR.  
  13. OCR integrity check passed  
  14. Verification of OCR integrity was successful.   


[grid@oeltan1 ~]$ ocrcheck
  1. Status of Oracle Cluster Registry is as follows :  
  2.          Version                  :          3  
  3.          Total space (kbytes)     :     262120  
  4.          Used space (kbytes)      :       3320  
  5.          Available space (kbytes) :     258800  
  6.          ID                       : 1360300110  
  7.          Device/File Name         :       +CRS  
  8.                                     Device/File integrity check succeeded  
  9.          Device/File Name         :     +DATA1  
  10.                                     Device/File integrity check succeeded  
  11.   
  12.   
  13.                                     Device/File not configured  
  14.   
  15.   
  16.                                     Device/File not configured  
  17.   
  18.   
  19.                                     Device/File not configured  
  20.   
  21.   
  22.          Cluster registry integrity check succeeded  
  23.   
  24.   
  25.          Logical corruption check bypassed due to non-privileged user  


2.检查votedisk



[grid@oeltan1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   2141c8c7d8904f89bfa45dbaecb827c5 (/dev/asm-diskb) [CRS]

Located 1 voting disk(s).

3看看磁盘使用的情况 

  1* select name,header_status,path from V$asm_disk
SQL> /


NAME                           HEADER_STATU PATH
------------------------------ ------------ ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CRS_0000                       MEMBER       /dev/asm-diskb
DATA1_0002                     MEMBER       /dev/asm-diskg
DATA1_0001                     MEMBER       /dev/asm-diskf
DATA1_0000                     MEMBER       /dev/asm-diske
CRS_0002                       MEMBER       /dev/asm-diskd
CRS_0001                       MEMBER       /dev/asm-diskc
FRI_0000                       MEMBER       /dev/asm-diskh
FRI_0001                       MEMBER       /dev/asm-diski
FRI_0002                       MEMBER       /dev/asm-diskj


做备份 


11g中votedisk不需要备份


1备份ASMspfile                                        

SQL> create pfile='/tmp/asm_pfile_0724.txt' from spfile    


2备份ocr

[root@oeltan1 ~]# ocrconfig -manualbackup

3检查备份:

[root@oeltan1 ~]# ocrconfig -showbackup
  1. oeltan1     2013/07/24 11:18:08     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup00.ocr  
  2.   
  3.   
  4. oeltan1     2013/07/24 07:18:02     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup01.ocr  
  5.   
  6.   
  7. oeltan1     2013/07/24 03:17:54     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup02.ocr  
  8.   
  9.   
  10. oeltan1     2013/07/23 07:17:31     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/day.ocr  
  11.   
  12.   
  13. oeltan1     2013/07/12 07:14:46     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/week.ocr  
  14.   
  15.   
  16. oeltan1     2013/07/23 11:17:30     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130723_111730.ocr  
  17.   
  18.   
  19. oeltan1     2013/07/23 11:15:00     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130723_111500.ocr  
  20.   
  21.   
  22. oeltan1     2013/07/18 10:36:48     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130718_103648.ocr  
  23.   
  24.   
  25. oeltan1     2013/07/18 10:36:30     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130718_103630.ocr  
  26.   
  27.   
  28. oeltan2     2013/06/17 09:04:34     /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130617_090434.ocr  


破坏掉3块CRS盘


开始搞破坏喽。。。


[root@oeltan2 ~]# dd if=/dev/zero of=/dev/asm-dskb bs=1024 count=1000
[root@oeltan2 ~]# dd if=/dev/zero of=/dev/asm-dskc bs=1024 count=1000
[root@oeltan2 ~]# dd if=/dev/zero of=/dev/asm-dskd bs=1024 count=1000

检查一下:
[root@oeltan1 ~]# ocrcheck
PROT-601: Failed to initialize ocrcheck
PROC-22: The OCR backend has an invalid format

开始还原:


1.强制停止CRS

[root@oeltan1 ~]# crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'oeltan1'
CRS-4133: Oracle High Availability Services has been stopped.
上面这个时间有点长,10多分钟

节点2也要停

2.以独占模式启动:

[root@oeltan1 ~]# crsctl start crs -excl -nocrs
  1. CRS-4123: Oracle High Availability Services has been started.  
  2. CRS-2672: Attempting to start 'ora.mdnsd' on 'oeltan1'  
  3. CRS-2676: Start of 'ora.mdnsd' on 'oeltan1' succeeded  
  4. CRS-2672: Attempting to start 'ora.gpnpd' on 'oeltan1'  
  5. CRS-2676: Start of 'ora.gpnpd' on 'oeltan1' succeeded  
  6. CRS-2672: Attempting to start 'ora.cssdmonitor' on 'oeltan1'  
  7. CRS-2672: Attempting to start 'ora.gipcd' on 'oeltan1'  
  8. CRS-2676: Start of 'ora.cssdmonitor' on 'oeltan1' succeeded  
  9. CRS-2676: Start of 'ora.gipcd' on 'oeltan1' succeeded  
  10. CRS-2672: Attempting to start 'ora.cssd' on 'oeltan1'  
  11. CRS-2672: Attempting to start 'ora.diskmon' on 'oeltan1'  
  12. CRS-2676: Start of 'ora.diskmon' on 'oeltan1' succeeded  
  13. CRS-2676: Start of 'ora.cssd' on 'oeltan1' succeeded  
  14. CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'oeltan1'  
  15. CRS-2672: Attempting to start 'ora.ctssd' on 'oeltan1'  
  16. CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'oeltan1' succeeded  
  17. CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'oeltan1'  
  18. CRS-2676: Start of 'ora.ctssd' on 'oeltan1' succeeded  
  19. CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'oeltan1' succeeded  
  20. CRS-2672: Attempting to start 'ora.asm' on 'oeltan1'  
  21. CRS-2676: Start of 'ora.asm' on 'oeltan1' succeeded  



3.创创建CRS磁盘组

[grid@oeltan1 ~]$ asmcmd
ASMCMD> ls
ASMCMD> 
[grid@oeltan1 ~]$ sqlplus / as sysasm


SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 26 09:33:13 2013


Copyright (c) 1982, 2011, Oracle.  All rights reserved

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options


SQL> show parameter spfile


NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string


SQL> create diskgroup crsvote normal redundancy disk '/dev/asm-diskb','/dev/asm-diskc','/dev/asm-diskd'
  2  attribute 'compatible.asm'='11.2.0.0.0','compatible.rdbms'='11.2.0.0.0';



Diskgroup created.

4.还愿spfile

SQL> create spfile='+CRSVOTE' from pfile='/tmp/asm_pfile_0724.txt';


File created.


ASMCMD> pwd  
+CRSVOTE/oeltan-cluster/ASMPARAMETERFILE
ASMCMD> ls
REGISTRY.253.821785057


5. 修改ocr.loc,节点2也要改。

[root@oeltan1 ~]# vi /etc/oracle/ocr.loc


#Device/file +DATA1 being deleted
ocrconfig_loc=+CRSVOTE
local_only=false


6.导入OCR

# ocrconfig -restore /apps/grid/gridhome/11.2.0/grid/cdata/oeltan-cluster/backup_20130723_111730.ocr


ASMCMD> pwd
+CRSVOTE/oeltan-cluster
ASMCMD> ls
ASMPARAMETERFILE/
OCRFILE/


检查OCR
[root@oeltan1 ~]# ocrcheck
  1. Status of Oracle Cluster Registry is as follows :  
  2.          Version                  :          3  
  3.          Total space (kbytes)     :     262120  
  4.          Used space (kbytes)      :       3320  
  5.          Available space (kbytes) :     258800  
  6.          ID                       : 1360300110  
  7.          Device/File Name         :   +CRSVOTE  
  8.                                     Device/File integrity check succeeded  
  9.   
  10.   
  11.                                     Device/File not configured  
  12.   
  13.   
  14.                                     Device/File not configured  
  15.   
  16.   
  17.                                     Device/File not configured  
  18.   
  19.   
  20.                                     Device/File not configured  
  21.   
  22.   
  23.          Cluster registry integrity check succeeded  
  24.   
  25.   
  26.          Logical corruption check succeeded  




7.修复VOTE  DISK

[root@oeltan1 ~]# crsctl query css votedisk
Located 0 voting disk(s).
[root@oeltan1 ~]# 
[root@oeltan1 ~]# crsctl replace votedisk +CRSVOTE
CRS-4602: Failed 27 to add voting file 66d2624128ee4fd2bf882a8a3d295a99.
CRS-4602: Failed 27 to add voting file 1d4cf4ec74794f05bfc8f3f452cddfaf.
CRS-4602: Failed 27 to add voting file 731c37abf7fa4fc0bf6219155cf0e7e0.
Failed to replace voting disk group with +CRSVOTE.
CRS-4000: Command Replace failed, or completed with errors.
[root@oeltan1 ~]# crsctl query css votedisk
Located 0 voting disk(s).
出现上面的错误,是因为asm_diskstring配置不正确:
检查一下:
SQL> show parameter asm


NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups                       string      CRSVOTE
asm_diskstring                       string
asm_power_limit                      integer     1
asm_preferred_read_failure_groups    string
修改
SQL> alter system set asm_diskstring='/dev/asm*' scope=both;
需要重启ASM
上述问题更详细的解决过程:http://blog.csdn.net/tanqingru/article/details/9493703

再次还愿votedisk


[root@oeltan1 ~]# crsctl replace votedisk +CRSVOTE
Successful addition of voting disk 54e90342890c4f58bf319f3c57c4acd7.
Successful addition of voting disk 5103d7e756f14f69bf3935ace34e5de7.
Successful addition of voting disk 370645ed11324fe5bf99eac9bf5efb21.
Successfully replaced voting disk group with +CRSVOTE.
CRS-4266: Voting file(s) successfully replaced
[root@oeltan1 ~]# crsctl  query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   54e90342890c4f58bf319f3c57c4acd7 (/dev/asm-diskb) [CRSVOTE]
 2. ONLINE   5103d7e756f14f69bf3935ace34e5de7 (/dev/asm-diskc) [CRSVOTE]
 3. ONLINE   370645ed11324fe5bf99eac9bf5efb21 (/dev/asm-diskd) [CRSVOTE]
Located 3 voting disk(s).

重启CRS

1.检查一下crs


[root@oeltan1 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager


2.正常重启一次

crsctl stop crs
crsctl start crs

节点2也启动:
[root@oeltan2 tmp]# crsctl start crs

3.检查一下crs


[root@oeltan1 ~]# crsctl check crs


CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

恢复后检查

1最后进行检查:

[root@oeltan1 tmp]# crs_stat -t
  1. Name           Type           Target    State     Host          
  2. ------------------------------------------------------------  
  3. ora.CRS.dg     ora....up.type ONLINE    OFFLINE                 
  4. ora.DATA1.dg   ora....up.type ONLINE    ONLINE    oeltan1       
  5. ora.FRI.dg     ora....up.type ONLINE    ONLINE    oeltan1       
  6. ora....ER.lsnr ora....er.type ONLINE    ONLINE    oeltan2       
  7. ora....N1.lsnr ora....er.type ONLINE    ONLINE    oeltan2       
  8. ora.asm        ora.asm.type   ONLINE    ONLINE    oeltan1       
  9. ora.cvu        ora.cvu.type   ONLINE    ONLINE    oeltan1       
  10. ora.gsd        ora.gsd.type   OFFLINE   OFFLINE                 
  11. ora....network ora....rk.type ONLINE    ONLINE    oeltan1       
  12. ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    oeltan1       
  13. ora....SM1.asm application    ONLINE    ONLINE    oeltan1       
  14. ora....N1.lsnr application    ONLINE    OFFLINE                 
  15. ora....an1.gsd application    OFFLINE   OFFLINE                 
  16. ora....an1.ons application    ONLINE    ONLINE    oeltan1       
  17. ora....an1.vip ora....t1.type ONLINE    ONLINE    oeltan2       
  18. ora....SM2.asm application    ONLINE    ONLINE    oeltan2       
  19. ora....N2.lsnr application    ONLINE    ONLINE    oeltan2       
  20. ora....an2.gsd application    OFFLINE   OFFLINE                 
  21. ora....an2.ons application    ONLINE    ONLINE    oeltan2       
  22. ora....an2.vip ora....t1.type ONLINE    ONLINE    oeltan2       
  23. ora.ons        ora.ons.type   ONLINE    ONLINE    oeltan1       
  24. ora.scan1.vip  ora....ip.type ONLINE    ONLINE    oeltan2       
  25. ora.scan2.vip  ora....ip.type ONLINE    ONLINE    oeltan2       
  26. ora.scan3.vip  ora....ip.type ONLINE    ONLINE    oeltan2       
  27. ora.tantest.db ora....se.type ONLINE    ONLINE    oeltan1       
  28. ora....tag.svc ora....ce.type ONLINE    ONLINE    oeltan2       


[root@oeltan1 tmp]# crsctl stat res -t
  1. --------------------------------------------------------------------------------  
  2. NAME           TARGET  STATE        SERVER                   STATE_DETAILS         
  3. --------------------------------------------------------------------------------  
  4. Local Resources  
  5. --------------------------------------------------------------------------------  
  6. ora.CRS.dg  
  7.                ONLINE  OFFLINE      oeltan1                                        
  8.                OFFLINE OFFLINE      oeltan2                                        
  9. ora.DATA1.dg  
  10.                ONLINE  ONLINE       oeltan1                                        
  11.                ONLINE  ONLINE       oeltan2                                        
  12. ora.FRI.dg  
  13.                ONLINE  ONLINE       oeltan1                                        
  14.                ONLINE  ONLINE       oeltan2                                        
  15. ora.LISTENER.lsnr  
  16.                ONLINE  OFFLINE      oeltan1                                        
  17.                ONLINE  ONLINE       oeltan2                                        
  18. ora.asm  
  19.                ONLINE  ONLINE       oeltan1                  Started               
  20.                ONLINE  ONLINE       oeltan2                  Started               
  21. ora.gsd  
  22.                OFFLINE OFFLINE      oeltan1                                        
  23.                OFFLINE OFFLINE      oeltan2                                        
  24. ora.net1.network  
  25.                ONLINE  ONLINE       oeltan1                                        
  26.                ONLINE  ONLINE       oeltan2                                        
  27. ora.ons  
  28.                ONLINE  ONLINE       oeltan1                                        
  29.                ONLINE  ONLINE       oeltan2                                        
  30. --------------------------------------------------------------------------------  
  31. Cluster Resources  
  32. --------------------------------------------------------------------------------  
  33. ora.LISTENER_SCAN1.lsnr  
  34.       1        ONLINE  ONLINE       oeltan2                                        
  35. ora.cvu  
  36.       1        ONLINE  ONLINE       oeltan1                                        
  37. ora.oc4j  
  38.       1        ONLINE  ONLINE       oeltan1                                        
  39. ora.oeltan1.vip  
  40.       1        ONLINE  INTERMEDIATE oeltan2                  FAILED OVER           
  41. ora.oeltan2.vip  
  42.       1        ONLINE  ONLINE       oeltan2                                        
  43. ora.scan1.vip  
  44.       1        ONLINE  ONLINE       oeltan2                                        
  45. ora.scan2.vip  
  46.       1        ONLINE  ONLINE       oeltan2                                        
  47. ora.scan3.vip  
  48.       1        ONLINE  ONLINE       oeltan2                                        
  49. ora.tantest.db  
  50.       1        ONLINE  ONLINE       oeltan1                  Open                  
  51.       2        ONLINE  ONLINE       oeltan2                  Open                  
  52. ora.tantest.tantag.svc  
  53.       1        ONLINE  OFFLINE                                                     
  54.       2        ONLINE  ONLINE       oeltan2                                        



ASMCMD> ls
CRSVOTE/
DATA1/
FRI/
阅读(1751) | 评论(0) | 转发(0) |
给主人留下些什么吧!~~