分类： 系统运维
2012-07-16 19:59:12
规划，安装和测试
HACMP规划如下：
HACMP 安装hacmp6.1，并打上最新的补丁。
使用磁盘心跳，作为HACMP的心跳。
主机名： erpapp, erpdb
这里的例子是erpapp, erpdb
IP: (举例)
Service IP:
10.0.10.1 erpapp
10.0.10.2 erpdb
Persistent IP:
10.0.10.11 erpappper
10.0.10.12 erpdbper
boot ip1:
192.168.126.2 erpapp_boot1
192.168.126.3 erpdb_boot1
boot ip2:
192.168.128.2 erpapp_boot2
192.168.128.3 erpdb_boot2
Netmask: 255.255.255.0
GW: 10.0.10.253
配置HACMP前期准备：
网卡配置：
erpdb上：
root@erpdb#/hascripts]#lsdev -Cc adapter
…
ent0 Defined Virtual I/O Ethernet Adapter (l-lan)
ent1 Available Virtual I/O Ethernet Adapter (l-lan)
ent2 Available Logical Host Ethernet Port (lp-hea)
ent3 Available Logical Host Ethernet Port (lp-hea)
…
两块网卡配置成两个不同网段的boot ip:
#ifconfig -a
en2: flags=1e080863,c0
inet 192.168.126.3 netmask 0xffffff00 broadcast 192.168.126.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en3: flags=1e080863,c0
inet 192.168.128.3 netmask 0xffffff00 broadcast 192.168.128.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
erpapp上：
erpapp上的四块网卡配置成etherchannel 方式，分别为ent6,ent7：
#lsdev –Cc adapter
…
ent4 Available Logical Host Ethernet Port (lp-hea)
ent5 Available Logical Host Ethernet Port (lp-hea)
ent6 Available EtherChannel / IEEE 802.3ad Link Aggregation
ent7 Available EtherChannel / IEEE 802.3ad Link Aggregation
…
ent6:
EtherChannel / Link Aggregation ent6
Parent Adapter NONE
EtherChannel / Link Aggregation Adapters ent0,ent1 +
Backup Adapter NONE +
Automatically Recover to Main Channel yes +
Perform Lossless Failover After Ping Failure yes +
Delete Backup Adapter +
Add Main Adapter +
Delete Main Adapter +
Add Backup Adapter +
Enable Alternate Address no +
Alternate Address [0x000000000000] +
Enable Gigabit Ethernet Jumbo Frames no
Mode standard +
IEEE 802.3ad Interval long +
Hash Mode default +
Internet Address to Ping [0]
Number of Retries [3] +#
Retry Timeout (sec) [1] +#
Apply change to DATABASE only no +
ent7也一样，只不过由ent2,ent3构成。
#ifconfig -a
en6: flags=1e080863,c0
inet 192.168.126.2 netmask 0xffffff00 broadcast 192.168.126.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en7: flags=1e080863,c0
inet 192.168.128.2 netmask 0xffffff00 broadcast 192.168.128.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
机器配置：
erpdb: rootvg hdisk0,hdisk1
内存：64 G
erpapp: rootvg hdisk0,hdisk1
内存: 64 G
软件： AIX6100-07 HACMP6.1
使用
#chdev –l hdiskXXXX –a pv=yes
使得两边的主机都能够看到共享的磁盘hdisk2,hdisk3,hdisk4,hdisk5等。
如果此时磁盘被另一台机器所使用，可能会报错。
可以通过lsattr –El hdiskxx来查看每个磁盘的大小。
配置每用户的最大进程数:
#smitty chgsys
Maximum number of PROCESSES allowed per user [2048]
安装HACMP相关软件：
先安装aix6.1系统盘上的
bos.data,
bos.net.tcp.client/server
bos.compat,
bos.perf,
bos.clvm,
bos.rte.SRC,libc,libcfg,libcur,libpthreads,odm
bos.adt
X11.adt
X11.Dt,
X11.compat,
rsct.*
然后使用hacmp6.1的盘，除了cluster.hativoli以外，所有的hacmp的包都要安装。
安装完毕后，mount –v cdrfs –o ro /dev/cd0 /mnt
打补丁。先将补丁拷贝到硬盘，然后使用 smit update_all, 选择[.]作为安装目录。
如果HACMP安装出现无法更正的错误，可以考虑删除。
删除hacmp的方法如下：
首先停止服务：smit clstop.
然后删除所有的以cluster开头的软件，smit remove.
然后检查：lslpp –l |grep cluster.
再重新安装hacmp6.1和fix。安装的过程和初次安装的过程一样。
镜像rootvg：
下面的步骤认为用户有一个包含hdisk0的rootvg卷组并且准备镜像rootvg到一个新磁盘：hdisk1.
1.把hdisk1加进rootvg卷组：
#extendvg rootvg hdisk1
2.关掉rootvg的 QUORUM功能:
#chvg -Qn rootvg
3.镜像 rootvg
#mirrorvg -c 2 rootvg hdisk1
4.重新生成 boot image
#bosboot -ad /dev/hdisk0
5.更新bootlist
#bootlist -m normal –o hdisk0 hdisk1
6.重起系统
#shutdown –Fr
确认磁盘一致：
磁盘列表（两台机器一致）：
#lspv
hdisk2 00c8ad9281bdff92 app2u03vg
hdisk3 00c8ad927d1b8c6a app2u04vg
hdisk4 00c8ad927d1bc8f0 app2u05vg
hdisk5 00f77a0634a33312 db1vgoradata
hdisk6 00f77a0634a3cc81 db1vgoraarch
hdisk7 00f77a0634a41877 db1vgbak
hdisk8 00f77a068536b7df vgHeartBeat
#
如果有不一致，可以使用importvg,exportvg重新导入导出一次进行调整。
其中app2u03vg, app2u04vg, app2u05vg,是属于erpapp这个应用，而db1vgoradata,db1oraarch,db1vgbak是属于dbapp这个应用。
vgHeartBeat为concurrent vg，是用来做磁盘心跳用的。
确保这些卷组都不是自动varyon的，并且两边一致，而且其中的lv所对应的目录也不是自动mount的，否则在HA同步时可能会有问题。
修改/etc/hosts表（两台机器一致）：
#more /etc/hosts
10.0.10.1 erpapp
10.0.10.2 erpdb
192.168.128.2 erpapp_boot2
192.168.128.3 erpdb_boot2
192.168.126.2 erpapp_boot1
192.168.126.3 erpdb_boot1
10.0.10.11 erpappper
10.0.10.12 erpdbper
#
修改两台机器上的/.rhosts文件，包括/usr/es/sbin/cluster/etc/rhosts。
#more /usr/es/sbin/cluster/etc/rhosts
erpappper
erpdbper
erpdb_boot1
erpapp_boot1
erpdb_boot2
erpapp_boot2
erpdb
erpapp
创建启停脚本：
放在两台机器的/hascripts/下，两台机器一致：
root@erpapp#/hascripts]#ls -l
total 32
-rwxr-xr-x 1 root system 189 Jul 14 15:03 start_erpapp.sh
-rwxr-xr-x 1 root system 189 Jul 14 15:04 start_erpdb.sh
-rwxr-xr-x 1 root system 189 Jul 14 15:05 stop_erpapp.sh
-rwxr-xr-x 1 root system 189 Jul 14 15:06 stop_erpdb.sh
使用rlogin测试erpdb,erpapp网络登陆.
smitty hacmp配置过程:
1.创建Cluster :
#smitty hacmp
Extended Configuration
Extended Top Conf
Conf an HACMP Cluster
Add/Change/Show an HACMP Cluster
Cluster Name [erp_db_cluster]
2配置Cluster节点，加入bootip：
#smitty hacmp
Extended Configuration
Extended Top Conf
Conf HACMP Nodes:
Add a Node to HACMP Cluster
NodeName [erpappN]
Communication Path to Node [erpapp_boot1]
同样方法将erpdbN加入。
…
NodeName [erpdbN]
Communication Path to Node [erpdb_boot1]
3．创建IP网络：
# smitty hacmp
Extended Conf
Extended Top Conf
Conf HACMP Networks
Add a Network to the HACMP Cluster
Select a Network Type ----> ether
*NetworkName [net_ether_01]
*NetworkType ether
*Netmask [255.255.255.0]
*Enable IP Address Takeover via IP alias [yes]
IP Address offset for Heartbeating over IP Alias []
然后将两台机器的四块网卡(erpapp_boot1,erpdb_boot1, erpdb_boot2, erpapp_boot2)分别加入到两端网络相应的节点中。
#smitty hacmp Configure HACMP Communication Interfaces/Devices
Add Communication Interfaces/Device
Add Pre-defined Communication Interfaces and Devices
Add Communication Interfaces/Device
Select a Network ---> net_ether_01
Add a Communication Interface
[Entry Fields]
* IP Label/Address [erpapp_boot1] +
* Network Type ether
* Network Name net_ether_01
* Node Name [erpappN] +
Network Interface []
…
[Entry Fields]
* IP Label/Address [erpapp_boot2] +
* Network Type ether
* Network Name net_ether_01
* Node Name [erpappN] +
Network Interface []
….
[Entry Fields]
* IP Label/Address [erpdb_boot1] +
* Network Type ether
* Network Name net_ether_01
* Node Name [erpdbN] +
Network Interface []
….
[Entry Fields]
* IP Label/Address [erpdb_boot2] +
* Network Type ether
* Network Name net_ether_01
* Node Name [erpdbN] +
Network Interface []
做完之后进行一次同步，修改错误。
4.定义永久IP(erpappper,erpdbper):
#smitty hacmp Extended Top Conf
Configure HACMP Persistent Node IP Label/Addresses
Add a Persistent Node IP Label/Address
*Node Name erpappN
*Network Name [net_ether_01]
*Node IP Label/Address [erpappper]
Netmask(IPv4)/Prefix Length(IPv6) []
....
Add a Persistent Node IP Label/Address
*Node Name erpdbN
*Network Name [net_ether_01]
*Node IP Label/Address [erpdbper]
Netmask(IPv4)/Prefix Length(IPv6) []
5 配置Service IP(erpapp,erpdb)到net_ether_01上：
#smitty hacmp Extended Resource Configuration
HACMP Extended Resources Configuration
Configure HACMP Service IP Labels/Addresses
Add a Service IP Label/Address
Select a Service IP Label/Address type ---> Configurable on Multiple Nodes
Network Name -->net_ether_01 (10…./24)
Add a Service IP Label/Address configurable on Multiple Nodes (extended)
*IP Label/Address erpapp +
Netmask(IPv4)/Prefix Length(IPv6) []
* Network Name net_ether_01
Alternate HW Address to accompany IP Label/Address []
….
Add a Service IP Label/Address configurable on Multiple Nodes (extended)
*IP Label/Address erpdb +
Netmask(IPv4)/Prefix Length(IPv6) []
* Network Name net_ether_01
Alternate HW Address to accompany IP Label/Address []
6. 配置通信设备（erpappN,erpdbN）：
增加磁盘心跳：和别的communication interface一样，net_hbdisk_01只是传递心跳的一种方式。
配置共享磁盘：concurrentvg
#smitty mkvg
Add a Scalable Volume Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
VOLUME GROUP name [vgHeartBeat]
Physical partition SIZE in megabytes 32 +
* PHYSICAL VOLUME names [hdisk8] +
Force the creation of a volume group? no +
Activate volume group AUTOMATICALLY no +
at system restart?
Volume Group MAJOR NUMBER [51] +#
Create VG Concurrent Capable? enhanced concurrent +
Max PPs per VG in kilobytes 32 +
Max Logical Volumes xxx
使用export vg然后再另一台机器上importvg，
注意两边(erpapp,erpdb)的该卷组的major number要一样。
在做之前，可能需要重新发现一下：
Extended Config
Discover HACMP-related Information from Configured Nodes
然后：
Extended Config ->
Extended Topo Config
Config HACMP Communication Interface/Devices
Add Discovered Communication Int & Dev
Communication Dev
选择列出的设备。这里选择hdisk8.添加完成后会自动添加网络（包括两个节点）,类似net_ hbdisk _01。
Change/Show a Communication Interface
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Device Name erpdbN_hdisk8_01
New Device Name [erpdbN_hdisk8_01]
* Node Name erpdbN +
* Device Path /dev/hdisk8
Network Type diskhb
* Network Name [net_diskhb_01] +
….
Change/Show a Communication Interface
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Device Name erpappN_hdisk8_01
New Device Name [erpappN_hdisk8_01]
* Node Name erpappN +
* Device Path /dev/hdisk8
Network Type diskhb
* Network Name [net_diskhb_02]
测试：
erpappN:
#/usr/sbin/rsct/bin/dhb_read -p hdisk8 –t
erpdbN:
/usr/sbin/rsct/bin/dhb_read -p hdisk8 –r
7. 增加资源、资源组：
增加两个应用（erpapp,dbapp）:
Extended config
Extended Resource Config
Configure HACMP Application Servers
Add an Application Server
* Server Name [erpapp]
* Start Script [/hascripts/start_erpapp.sh]
* Stop Script [/hascripts/stop_erpapp.sh]
Application Monitor Name(s)
….
Add an Application Server
* Server Name [dbapp]
* Start Script [/hascripts/start_erpdb.sh]
* Stop Script [/hascripts/stop_erpdb.sh]
Application Monitor Name(s)
创建Erp/db资源组：
Extended config
Extended Resource Config
HACMP Extended Resource Group Configuration ->
Change/Show a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Resource Group Name erpRG
New Resource Group Name []
Participating Nodes (Default Node Priority) [erpappN erpdbN ] +
Startup Policy Online On Home Node O> +
Fallover Policy Fallover To Next Prio> +
Fallback Policy Never Fallback +
…
Change/Show a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Resource Group Name oraRG
New Resource Group Name []
Participating Nodes (Default Node Priority) [erpdbN erpappN] +
Startup Policy Online On Home Node O> +
Fallover Policy Fallover To Next Prio> +
Fallback Policy Never Fallback +
8 创建完毕后，
然后修改资源组属性：
Extended config
Extended Resource Config
HACMP Extended Resource Group Configuration
Change/Show All Resources and Attributes for a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
Resource Group Name erpRG
Participating Nodes (Default Node Priority) erpappN erpdbN
Startup Policy Online On Home Node O>
Fallover Policy Fallover To Next Prio>
Fallback Policy Never Fallback
Service IP Labels/Addresses [erpapp] +
Application Servers [erpapp] +
Volume Groups [app2u03vg app2u04vg app2u05vg> +
Use forced varyon of volume groups, if necessary false +
Automatically Import Volume Groups false +
Filesystems (empty is ALL for VGs specified) [ ] +
Filesystems Consistency Check fsck +
Filesystems Recovery Method sequential +
Filesystems mounted before IP configured false +
Filesystems/Directories to Export (NFSv2/3) []
Filesystems/Directories to NFS Mount []
Network For NFS Mount [] +
Tape Resources [] +
Raw Disk PVIDs [] +
Fast Connect Services [] +
Communication Links [] +
Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +
Miscellaneous Data []
WPAR Name []
[BOTTOM]
...
Change/Show All Resources and Attributes for a Resource Group
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
Resource Group Name oraRG
Participating Nodes (Default Node Priority) erpdbN erpappN
Startup Policy Online On Home Node O>
Fallover Policy Fallover To Next Prio>
Fallback Policy Never Fallback
Service IP Labels/Addresses [erpdb] +
Application Servers [dbapp] +
Volume Groups [db1vgoradata db1vgoraarch db1vgbak> +
Use forced varyon of volume groups, if necessary false +
Automatically Import Volume Groups false
Filesystems (empty is ALL for VGs specified) [ ] +
Filesystems Consistency Check fsck +
Filesystems Recovery Method sequential +
Filesystems mounted before IP configured false +
Filesystems/Directories to Export (NFSv2/3) [] +
+
Filesystems/Directories to NFS Mount []
Network For NFS Mount [] +
Tape Resources [] +
Raw Disk PVIDs [] +
Fast Connect Services []
Communication Links [] +
Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +
Miscellaneous Data []
WPAR Name []
[BOTTOM]
9．验证并同步拓扑。使用extend中的菜单项。选择自动纠正错误。
# smitty hacmp
Extended Configuration
Extended Verification and Synchronization
* Verify, Synchronize or Both [Both] +
* Automatically correct errors found during [Yes] +
verification?
* Force synchronization if verification fails? [No] +
* Verify changes only? [No] +
* Logging [Standard] +
10. 启动和停止HACMP:
启动HACMP（快速路径：#smitty clstart）
# smitty hacmp
System Management (C-SPOC)
Manage HACMP Services
Start Cluster Services
* Start now, on system restart or both now
Start Cluster Services on these nodes [erpapp erpdb]
* Manage Resource Groups Automatically
BROADCAST message at startup? true
Startup Cluster Information Daemon? false
Ignore verification errors? false
Automatically correct errors found during Interactively
cluster start?
停止HACMP（快速路径：#smitty clstop）
# smitty hacmp
System Management (C-SPOC)
Manage HACMP Services
Stop Cluster Services
* Stop now, on system restart or both now+
Stop Cluster Services on these nodes [erpapp erpdb]
BROADCAST cluster shutdown? true+
* Select an Action on Resource Groups Move Resource Groups+
查看HA日志
tail -f /var/hacmp/log/hacmp.out
移动资源组：
# smitty hacmp
System Management (C-SPOC)
Resource Group and Applications
Move a Resource Group to Another Node / Site
Move Resource Groups to Another Node
Select a Resource Group -->
erpRG ONLINE erpappN /
Select a Destination Node –>
erpdbN
确认，即可移动资源组erpRG到erpdbN节点。
11．验证HACMP。
A.
Smit clstop takeover 发生接管。 测试正常。
B.
Smit clstop graceful，只是停止hacmp的机器上serviceIP或concurrent VG, 而不发生接管。测试正常。
C.
拔插一台机器的电源，发生接管。
D．
重新启动一台主机。Shutdown –Fr . 发生接管，测试正常。
E．
拔掉一台机器上的服务IP网线。测试正常。
