How to drop ASM DiskGroup in RAC? (DISKGROUP删除后db资源显示OFFLINE)

11.2.0.4 rac 2nodes

$ crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora.CRS.dg     ora....up.type ONLINE    ONLINE    node1       
ora.DATAVG.dg  ora....up.type ONLINE    ONLINE    node1 ora.FRADG.dg ora....up.type ONLINE    OFFLINE               
ora....ER.lsnr ora....er.type ONLINE    ONLINE    node1       
ora....N1.lsnr ora....er.type ONLINE    ONLINE    node2       
ora.asm        ora.asm.type   ONLINE    ONLINE    node1       
ora.cvu        ora.cvu.type   ONLINE    ONLINE    node1 ora.db.db ora....se.type ONLINE    OFFLINE               
ora.gsd        ora.gsd.type   OFFLINE   OFFLINE               
ora....network ora....rk.type ONLINE    ONLINE    node1       
ora....SM1.asm application    ONLINE    ONLINE    node1       
ora....E1.lsnr application    ONLINE    ONLINE    node1       
ora.node1.gsd  application    OFFLINE   OFFLINE               
ora.node1.ons  application    ONLINE    ONLINE    node1       
ora.node1.vip  ora....t1.type ONLINE    ONLINE    node1       
ora....SM2.asm application    ONLINE    ONLINE    node2       
ora....E2.lsnr application    ONLINE    ONLINE    node2       
ora.node2.gsd  application    OFFLINE   OFFLINE               
ora.node2.ons  application    ONLINE    ONLINE    node2       
ora.node2.vip  ora....t1.type ONLINE    ONLINE    node2       
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    node1       
ora.ons        ora.ons.type   ONLINE    ONLINE    node1       
ora....ry.acfs ora....fs.type ONLINE    ONLINE    node1       
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    node2       

$ crsctl stat res -t
——————————————————————————–
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
Cluster Resources
——————————————————————————–

...
ora.db.db
      1        OFFLINE OFFLINE                               Instance Shutdown
      2        OFFLINE OFFLINE                               Instance Shutdown
...

symptom:

to chose sqlplus “drop diskgroup”. however,the diskgroup that dropped is still listed as a resource. , Actually, the DB is open still, check alert log

ora-00313,ora-00312 ora-17503, ora-15001

The alert log output online log write fails , and friend told me For some reason directly deleted an ASM DiskGroup(FRADG) by someone. Fortunately, the current online logfile group had 2 member distributed in two DiskGroup, so the DB does not Crash.

Solution:

SELECT GROUP#, STATUS, MEMBER 
FROM V$LOGFILE
WHERE STATUS='INVALID';

To drop the log file members in had drop diskgroup(FRADG).
e.g.

ALTER DATABASE DROP LOGFILE MEMBER '+FRADG/db/onlinelog/group_2.258.861051483';

Note:
if throw ORA-01609: log 4 is the current log for thread 2 – cannot drop members error,
just switch online logfile manually using following command and try again:

alter system switch logfile;
alter system checkpoint;

# add log member to a exists log file group
ALTER DATABASE ADD LOGFILE MEMBER ‘+DATAVG’ TO GROUP 2;

Then look back the Cluster Resource Issue

at first ,we try to start res db manually using srvctl .

$ srvctl start database -d db

PRCR-1079 : Failed to start resource ora.db.db
CRS-5017: The resource action "ora.FRADG.dg start" encountered the following error:
ORA-15032: not all alterations performed
ORA-15017: diskgroup "FRADG" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "FRADG"

checked the dependency for the database resource

$ crsctl status resource ora.db.db -f

...
START_DEPENDENCIES=hard(ora.DATAVG.dg,ora.FRADG.dg) weak(type:ora.listener.type,global:type:ora.scan_listener.type,uniform:ora.ons,global:ora.gns) pullup(ora.DATAVG.dg,ora.FRADG.dg)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(intermediate:ora.asm,shutdown:ora.DATAVG.dg,shutdown:ora.FRADG.dg)
STOP_TIMEOUT=600
...

Solution:

Since ASM diskgroup FRADG has been physically dropped. We should to remove FRADG resource from CRS start/stop dependency for database resource ora.db.db.

Cleanup Diskgroup Cluster resources

$su - root
Remove start dependency and stop dependency

$ srvctl modify database -d db -a "DATAVG"

$ crsctl status resource ora.db.db -f |grep DEPENDENCIES

Tip:
START and STOP DEPENDENCIES for DG FRADG are removed now

Finally delete the DG resoure

$ srvctl disable diskgroup -g FRADG
$ srvctl remove diskgroup -g FRADG –f

How to drop asm groups in RAC? sqlplus in asm, asmca, EM. either ones will work.

A FAQ in OTN

Question then is, is there one command that I can run which will do both? drop the diskgroup AND as a RAC resource.
If you use SQLPLUS to drop Diskgroup you must to use "srvctl remove diskgroup -g" to remove diskgroup from OCR.

If you use ASMCA one step remove both.

No, there is no one command to do it.... but is easy with 4 command.
e.g:
oracle@node1 > sqlplus / as sysasm
SQL> sqlplus / as sysasm

SQL> ! srvctl status diskgroup -g DATA
Disk Group DATA is running on node1,node2,node3

SQL> ! srvctl stop diskgroup -g DATA -n node2,node3

SQL> DROP DISKGROUP DATA;

SQL > ! srvctl remove diskgroup -g DATA

On release 11.2.0.2, after drop a diskgroups ASM is not releasing file descriptors on the raw devices.
Document 4693355.8 states that this problem (Bug:4693355 & Bug:7225720) is fixed on release 11.2.0.1.0, but the problem persists on release 11.2.0.2.0. Even if DROP DISKGROUP or DISMOUNT DISK GROUP scenarios.

Note that the forced disk group dismount will cause all datafiles in that database to go offline, which means they will need recovery (and restore if I drop disk group PLAY)

Summary:
Database&ASM diskgroup resource open/close dependency is ASM Auto Maintenance, but some special cases need we to cleanup Diskgroup Cluster resources manually. When you delete DISKGROUP, we must ensure that there is no DB in use.