1. installsfrac
uninstallsfrac
2.VCS control----------
VXDMP
CVM cluster volume manager
CFS cluster file system
RAC extensions
Oracle service groups
3.vcsmm
LMON
DB_SID group
primary member
slave members
group membership information is exchanged cluster wide by GAB port o
4.CVM
VxVM in a cluster environment
A/A A/P support
must support SCSI-III persistent reservation(used for I/O fencing)
R/W set per node,per disk group
Fast Mirror Resync
Disk Group Split and Join
Master/Slave architecture
Use GAB for membership and messaging
VxVM
vxconfigd daemon
#vxconfigd
master vxconfigd ----->slave vxconfigd
use GAB,LLT transport data
CVM master node
#vxclustadm -v nodestate
#vxdclt -c mode
mode: enabled: cluster active - MASTER
All CVM VxVM commands must be done on the master node
change master(not advised)
#vxclustadm stopnode
#vxclustadm -m vcs -t gab startnode
5.CFS
VxFS with cluster license enabled
VxVM--VxFS
CVM---CFS
Master/Slave architecture
GLM--Global lock manager ----lock
use GAB for membership and messaging
6.ODM
oracle data manager
ODM is an oracle API
VERITAS provides a library for oracle use ODM
7.communications
High-Level---RAC,ODM,CFS,CVM
Low-Level
---GAB all processes in sfrac use GAB for communication
#/sbin/gabconfig -a
GAB Port Memberships
======================================================
Port a gen 4a1c0001 membership 01
Port b gen g8ty0002 membership 01
Port d gen 40100001 membership 01
Port f gen f1990002 membership 01
Port h gen d8850002 membership 01
Port o gen f1100002 membership 01
Port q gen 28d10002 membership 01
Port v gen 1fc60002 membership 01
Port w gen 15ba0002 membership 01
8.VCS communication
HAD---High Availability Daemon ---GAB port h--the VCS Engine
agent--control resources,start,stop,monitor and reports status to HAD
VCSMM--VCS Membership Module--kernel layer over GAB---port o
9.CVM communication
kmsg --port v
vxconfigd --port w
10.CFS communication
GLM --port f
QLOG--port q
11.ODM communication
GSM --port d
12.VCS problem
log files
engine log-------/var/VRTSvcs/log/engine_A.log
agent log------$VCS_HOME/log/agent_A.log
#hastatus -sum
#hacf -verify /etc/VRTSvcs/conf/config
#hastart -force
13.CVM recovery
DRL ---dirty region logs
each node writes to its own DRL area.
recovery----after a cluster crash,all individual DRL copies are merged into the master's DRL,and the individual DRL maps are cleared.
then master do write back for the affected areas
CVM joining issues
GAB port u to synchronize the join sequence
fencing---port b is up on all systems
14.CFS recovery
CFS mounting issues
15.ODM issues
/dev/odm is mounted
port d is open
$ORACLE_HOME/bin/gsdctl stop
#hastop -all
#hastatus -sum
#./opt/VRTS/bin/vxlicinst -k xxxx-
edit
/etc/VRTSvcs/conf/config/main.cf
#hacf -verify config
rcp main.cf
edit
/etc/VRTSvcs/conf/config/CVMTypes.cf
rcp CVMTypes.cf
edit
/etc/VRTSvcs/conf/config/OracleTypes.cf
rcp OracleTypes.cf
16.SFRAC install
VSFRAC 4.0 Contents
The following is a list of all the VSFRAC components and their subcomponents.
CVM: VERITAS Clustered Volume Manager
VRTSvxvm VRTSvmman VRTSvmdoc VRTSvmpro
CFS: VERITAS Clustered File System
VRTSvxfs VRTSfsdoc VRSTglm
VEA: VERITAS Enterprise Administrator
VRTSob VRTSobgui VRTSog.exe (Windows UI)
VCS: VERITAS Cluster Server
VRTSvlic VRTSperl VRTSllt VRTSgab VRTSvcs VRTSvcsmg VRTSvcsag VRTSvcsdc VRTSvcsmn VRTSweb VRTSvcsw
VERITAS Cluster Server Agents
VRTScavf VRTSvcsor
VERITAS RAC Support
vxfen driver IPC LMX VCSMM VRTSdbac ODM VRTSodm VRTSgms
VERITAS San Access Layer Server
VRTSsald
onfigure Components
•Verify SCSI 3 PR support
•Setup up coordinator disks
•Start I/O fencing
•Verify configuration
GAB Port Summary
Use gabconfig –a to obtain the port summary.
GAB Port Memberships
Port a gen 4a1c0001 membership 01 (GAB)
Port b gen g8ty0002 membership 01 (FENCE)
Port d gen 40100001 membership 01 (ODM)
Port f gen f1990002 membership 01 (CFS)
Port h gen d8850002 membership 01 (HAD)
Port o gen 243f0002 membership 01 (VCSMM)
Port q gen 28d10002 membership 01 (qlog)
Port v gen 1fc60002 membership 01 (VxVM)
Port w gen 15ba0002 membership 01 (VxCONFIGD)
2Startup Fault Areas
•From /etc/init.d
startupscripts
–LLT
–LMX
–GAB (GAB Port a)
–ODM (odm GAB Port d)
–Fencing (vxfen GAB Port b)
–VCSMM(vcsmm GAB Port o)
–VCS(had GAB Port h)
•VCS started processes
–Port v VxVM
–Port w VxCONFIGD
–Port f CFS
–Port q qlog
/opt/VRTS/install/logs
#/opt/VRTSvcs/bin/chk_sfrac_pkgs
The utility checks for the presence of the following packages:
VRTSvxvm
VRTSvxfs
VRTSglm
VRTScavf
VRTSgms
VRTSodm
VRTSvlic
VRTSllt
VRTSgab
VRTSvcs
VRTSperl
VRTSvcsag
VRTSvcsmg
VRTSdbac
VRTSvcsor
/dev/rdsk/c0t1d1s2
setting up I/O fencing
Colrdinator Disk 1,2,3
#vxdisksetup -i diskx
#vxdg init vxfencoorddg disk1 disk2 disk3
#vxdg deport vxfencoorddg
#vxdg -t import vxfencoorddg
#vxdg deport vxfencoorddg
#echo vxfencoorddg >/etc/vxfendg ---both system
#vxdctl enable ---rescan disk
#/etc/init.d/vxfen start
#cat /etc/vxfentab
#gabconfig -a
#vxfenadm -g all -f /etc/vxfentab
edit VCS
#haconf -dump -makero
#hastop -all
#cd /etc/VRTSvcs/conf/config
#cp main.cf main.orig
edit main.cf,add UseFence = SCSI3
#hacf -verify .
rcp man.cf to the other node
shutdown -y -6
#/sbin/gabconfig -a
17.split brain
preexisting split brain ------repair interconnect cables
partition in time --------will restart node1
