AIX 6.1 HACMP6.1安装配置

规划，安装和测试

HACMP规划如下：

HACMP 安装hacmp6.1，并打上最新的补丁。

使用磁盘心跳，作为HACMP的心跳。

 

主机名： erpapp, erpdb

这里的例子是erpapp, erpdb

 

IP: (举例)

Service IP:

10.0.10.1 erpapp

10.0.10.2 erpdb

 

Persistent IP:

10.0.10.11 erpappper

10.0.10.12 erpdbper

 

boot ip1:

192.168.126.2 erpapp_boot1

192.168.126.3 erpdb_boot1

 

boot ip2:

192.168.128.2 erpapp_boot2

192.168.128.3 erpdb_boot2

 

Netmask: 255.255.255.0

GW: 10.0.10.253

 

配置HACMP前期准备：

网卡配置：

erpdb上：

root@erpdb#/hascripts]#lsdev -Cc adapter

…

ent0 Defined Virtual I/O Ethernet Adapter (l-lan)

ent1 Available Virtual I/O Ethernet Adapter (l-lan)

ent2 Available Logical Host Ethernet Port (lp-hea)

ent3 Available Logical Host Ethernet Port (lp-hea)

…

 

两块网卡配置成两个不同网段的boot ip:

#ifconfig -a

en2: flags=1e080863,c0

inet 192.168.126.3 netmask 0xffffff00 broadcast 192.168.126.255

tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0

en3: flags=1e080863,c0

inet 192.168.128.3 netmask 0xffffff00 broadcast 192.168.128.255

tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0

 

erpapp上：

erpapp上的四块网卡配置成etherchannel 方式，分别为ent6,ent7：

#lsdev –Cc adapter

…

ent4 Available Logical Host Ethernet Port (lp-hea)

ent5 Available Logical Host Ethernet Port (lp-hea)

ent6 Available EtherChannel / IEEE 802.3ad Link Aggregation

ent7 Available EtherChannel / IEEE 802.3ad Link Aggregation

…

 

ent6:
  EtherChannel / Link Aggregation                     ent6
  Parent Adapter                                      NONE
  EtherChannel / Link Aggregation Adapters            ent0,ent1              +
  Backup Adapter                                      NONE                   +
       Automatically Recover to Main Channel          yes                    +
       Perform Lossless Failover After Ping Failure   yes                    +
  Delete Backup Adapter                                                      +
  Add Main Adapter                                                           +
  Delete Main Adapter                                                        +
  Add Backup Adapter                                                         +
  Enable Alternate Address                            no                     +
  Alternate Address                                  [0x000000000000]        +
  Enable Gigabit Ethernet Jumbo Frames                no
  Mode                                                standard               +
  IEEE 802.3ad Interval                               long                   +
  Hash Mode                                           default                +
  Internet Address to Ping                           [0]
  Number of Retries                                  [3]                     +#
  Retry Timeout (sec)                                [1]                     +#
  Apply change to DATABASE only                       no                     +

 

ent7也一样，只不过由ent2,ent3构成。

#ifconfig -a

en6: flags=1e080863,c0

inet 192.168.126.2 netmask 0xffffff00 broadcast 192.168.126.255

tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0

en7: flags=1e080863,c0

inet 192.168.128.2 netmask 0xffffff00 broadcast 192.168.128.255

tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0

 

机器配置：

erpdb: rootvg hdisk0,hdisk1

内存：64 G

erpapp: rootvg hdisk0,hdisk1

内存: 64 G

软件： AIX6100-07 HACMP6.1

 

使用

#chdev –l hdiskXXXX –a pv=yes

使得两边的主机都能够看到共享的磁盘hdisk2,hdisk3,hdisk4,hdisk5等。

如果此时磁盘被另一台机器所使用，可能会报错。

可以通过lsattr –El hdiskxx来查看每个磁盘的大小。

 

配置每用户的最大进程数:

#smitty chgsys

Maximum number of PROCESSES allowed per user [2048]

 

安装HACMP相关软件：

先安装aix6.1系统盘上的

bos.data,

bos.net.tcp.client/server

bos.compat,

bos.perf,

bos.clvm,

bos.rte.SRC,libc,libcfg,libcur,libpthreads,odm

bos.adt

X11.adt

X11.Dt,

X11.compat,

rsct.*

然后使用hacmp6.1的盘，除了cluster.hativoli以外，所有的hacmp的包都要安装。

安装完毕后，mount –v cdrfs –o ro /dev/cd0 /mnt

打补丁。先将补丁拷贝到硬盘，然后使用 smit update_all, 选择[.]作为安装目录。

如果HACMP安装出现无法更正的错误，可以考虑删除。

删除hacmp的方法如下：

首先停止服务：smit clstop.

然后删除所有的以cluster开头的软件，smit remove.

然后检查：lslpp –l |grep cluster.

再重新安装hacmp6.1和fix。安装的过程和初次安装的过程一样。

 

镜像rootvg：

下面的步骤认为用户有一个包含hdisk0的rootvg卷组并且准备镜像rootvg到一个新磁盘：hdisk1.

1.把hdisk1加进rootvg卷组：

#extendvg rootvg hdisk1

2.关掉rootvg的 QUORUM功能:

#chvg -Qn rootvg

3.镜像 rootvg

#mirrorvg -c 2 rootvg hdisk1

4.重新生成 boot image

#bosboot -ad /dev/hdisk0

5.更新bootlist

#bootlist -m normal –o hdisk0 hdisk1

6.重起系统

#shutdown –Fr

 

确认磁盘一致：

磁盘列表（两台机器一致）：

#lspv

hdisk2 00c8ad9281bdff92 app2u03vg

hdisk3 00c8ad927d1b8c6a app2u04vg

hdisk4 00c8ad927d1bc8f0 app2u05vg

hdisk5 00f77a0634a33312 db1vgoradata

hdisk6 00f77a0634a3cc81 db1vgoraarch

hdisk7 00f77a0634a41877 db1vgbak

hdisk8 00f77a068536b7df vgHeartBeat

#

如果有不一致，可以使用importvg,exportvg重新导入导出一次进行调整。

其中app2u03vg, app2u04vg, app2u05vg,是属于erpapp这个应用，而db1vgoradata,db1oraarch,db1vgbak是属于dbapp这个应用。

vgHeartBeat为concurrent vg，是用来做磁盘心跳用的。

确保这些卷组都不是自动varyon的，并且两边一致，而且其中的lv所对应的目录也不是自动mount的，否则在HA同步时可能会有问题。

 

 

 

修改/etc/hosts表（两台机器一致）：

 

#more /etc/hosts

10.0.10.1 erpapp

10.0.10.2 erpdb

192.168.128.2 erpapp_boot2

192.168.128.3 erpdb_boot2

192.168.126.2 erpapp_boot1

192.168.126.3 erpdb_boot1

10.0.10.11 erpappper

10.0.10.12 erpdbper

#

 

修改两台机器上的/.rhosts文件，包括/usr/es/sbin/cluster/etc/rhosts。

 

#more /usr/es/sbin/cluster/etc/rhosts

erpappper

erpdbper

erpdb_boot1

erpapp_boot1

erpdb_boot2

erpapp_boot2

erpdb

erpapp

 

创建启停脚本：

放在两台机器的/hascripts/下，两台机器一致：

root@erpapp#/hascripts]#ls -l

total 32

-rwxr-xr-x 1 root system 189 Jul 14 15:03 start_erpapp.sh

-rwxr-xr-x 1 root system 189 Jul 14 15:04 start_erpdb.sh

-rwxr-xr-x 1 root system 189 Jul 14 15:05 stop_erpapp.sh

-rwxr-xr-x 1 root system 189 Jul 14 15:06 stop_erpdb.sh

使用rlogin测试erpdb,erpapp网络登陆.

 

smitty hacmp配置过程:

1.创建Cluster :

#smitty hacmp
Extended Configuration
   Extended Top Conf
     Conf an HACMP Cluster
       Add/Change/Show an HACMP Cluster
         Cluster Name  [erp_db_cluster]

 

2配置Cluster节点，加入bootip：

#smitty hacmp
  Extended Configuration
   Extended Top Conf
     Conf HACMP Nodes:
       Add a Node to HACMP Cluster
              NodeName   [erpappN]
              Communication Path to Node  [erpapp_boot1]

同样方法将erpdbN加入。

            …
              NodeName   [erpdbN]
              Communication Path to Node  [erpdb_boot1]

 

3．创建IP网络：

# smitty hacmp
Extended Conf
  Extended Top Conf
    Conf HACMP Networks
      Add a Network to the HACMP Cluster
                Select a Network Type  ----> ether
                    *NetworkName  [net_ether_01]
                    *NetworkType   ether
                    *Netmask       [255.255.255.0]
                    *Enable IP Address Takeover via IP alias [yes]
                      IP Address offset for Heartbeating over IP Alias []

 

然后将两台机器的四块网卡(erpapp_boot1,erpdb_boot1, erpdb_boot2, erpapp_boot2)分别加入到两端网络相应的节点中。

#smitty hacmp
  Extended Configuration

    Configure HACMP Communication Interfaces/Devices
            Add Communication Interfaces/Device
                Add Pre-defined Communication Interfaces and Devices
                    Add Communication Interfaces/Device
                                Select a Network ---> net_ether_01
                        Add a Communication Interface
                                                                  [Entry Fields]
           * IP Label/Address                                   [erpapp_boot1]                      +
           * Network Type                                        ether
           * Network Name                                        net_ether_01
           * Node Name                                          [erpappN]                      +
             Network Interface                                  []    
           …
                                                                  [Entry Fields]
           * IP Label/Address                                   [erpapp_boot2]                      +
           * Network Type                                        ether
           * Network Name                                        net_ether_01
           * Node Name                                          [erpappN]                      +
             Network Interface                                  []    
           ….
                                                                  [Entry Fields]
           * IP Label/Address                                   [erpdb_boot1]                      +
           * Network Type                                        ether
           * Network Name                                        net_ether_01
           * Node Name                                          [erpdbN]                      +
             Network Interface                                  []    
           ….
                                                                  [Entry Fields]
           * IP Label/Address                                   [erpdb_boot2]                      +
           * Network Type                                        ether
           * Network Name                                        net_ether_01
           * Node Name                                          [erpdbN]                      +
             Network Interface                                  []    

 

做完之后进行一次同步，修改错误。

 

4.定义永久IP(erpappper,erpdbper):

#smitty hacmp
  Extended Configuration

   Extended Top Conf
    Configure HACMP Persistent Node IP Label/Addresses
            Add a Persistent Node IP Label/Address
            *Node Name                                            erpappN
            *Network Name                                       [net_ether_01]
            *Node IP Label/Address                              [erpappper]
            Netmask(IPv4)/Prefix Length(IPv6)                  []
....
            Add a Persistent Node IP Label/Address
            *Node Name                                            erpdbN
            *Network Name                                       [net_ether_01]
            *Node IP Label/Address                              [erpdbper]
            Netmask(IPv4)/Prefix Length(IPv6)                  []

 

5 配置Service IP(erpapp,erpdb)到net_ether_01上：

#smitty hacmp
  Extended Configuration

        Extended Resource Configuration
              HACMP Extended Resources Configuration
                Configure HACMP Service IP Labels/Addresses
                    Add a Service IP Label/Address
                        Select a Service IP Label/Address type ---> Configurable on Multiple Nodes
                            Network Name -->net_ether_01 (10…./24)
                                Add a Service IP Label/Address configurable on Multiple Nodes (extended)

               *IP Label/Address                    erpapp                            +
                 Netmask(IPv4)/Prefix Length(IPv6)                  []
               * Network Name                                        net_ether_01
                 Alternate HW Address to accompany IP Label/Address []

            ….
                  Add a Service IP Label/Address configurable on Multiple Nodes (extended)
              *IP Label/Address                    erpdb                            +
                Netmask(IPv4)/Prefix Length(IPv6)                  []
              * Network Name                                        net_ether_01
                Alternate HW Address to accompany IP Label/Address []

 

6. 配置通信设备（erpappN,erpdbN）：

增加磁盘心跳：和别的communication interface一样，net_hbdisk_01只是传递心跳的一种方式。

配置共享磁盘：concurrentvg

#smitty mkvg 
Add a Scalable Volume Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                        [Entry Fields]       
  VOLUME GROUP name                                  [vgHeartBeat]                
  Physical partition SIZE in megabytes                32                     +
* PHYSICAL VOLUME names                              [hdisk8]                +
  Force the creation of a volume group?               no                     +
  Activate volume group AUTOMATICALLY                 no                     +
    at system restart?                                                       
  Volume Group MAJOR NUMBER                          [51]                    +#
  Create VG Concurrent Capable?                       enhanced concurrent    +
  Max PPs per VG in kilobytes                         32                     +
  Max Logical Volumes                                 xxx

 

使用export vg然后再另一台机器上importvg，

注意两边(erpapp,erpdb)的该卷组的major number要一样。

在做之前，可能需要重新发现一下：

Extended Config
    Discover HACMP-related Information from Configured Nodes

然后：

Extended Config ->
    Extended Topo Config
        Config HACMP Communication Interface/Devices
            Add Discovered Communication Int & Dev
              Communication Dev

选择列出的设备。这里选择hdisk8.添加完成后会自动添加网络（包括两个节点）,类似net_ hbdisk _01。

 

Change/Show a Communication Interface

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                        [Entry Fields]
  Device Name                                         erpdbN_hdisk8_01        
  New Device Name                                    [erpdbN_hdisk8_01]       
* Node Name                                          erpdbN                  +
* Device Path                                         /dev/hdisk8
  Network Type                                        diskhb
* Network Name                                       [net_diskhb_01]         +

  ….
Change/Show a Communication Interface

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                        [Entry Fields]
  Device Name                                         erpappN_hdisk8_01     
  New Device Name                                    [erpappN_hdisk8_01]    
* Node Name                                           erpappN               +
* Device Path                                         /dev/hdisk8
  Network Type                                        diskhb
* Network Name                                       [net_diskhb_02]      

 

测试：

erpappN:

#/usr/sbin/rsct/bin/dhb_read -p hdisk8 –t

erpdbN:

/usr/sbin/rsct/bin/dhb_read -p hdisk8 –r

 

7. 增加资源、资源组：

增加两个应用（erpapp,dbapp）:

Extended config
Extended Resource Config
    Configure HACMP Application Servers
        Add an Application Server
            * Server Name                                        [erpapp]
* Start Script                                       [/hascripts/start_erpapp.sh]
* Stop Script                                        [/hascripts/stop_erpapp.sh]
  Application Monitor Name(s)                    
….
    Add an Application Server
            * Server Name                                        [dbapp]
* Start Script                                       [/hascripts/start_erpdb.sh]
* Stop Script                                        [/hascripts/stop_erpdb.sh]
  Application Monitor Name(s)                    

 

创建Erp/db资源组：

Extended config
Extended Resource Config
HACMP Extended Resource Group Configuration  ->
Change/Show a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                        [Entry Fields]
  Resource Group Name                                 erpRG
  New Resource Group Name                            []
  Participating Nodes (Default Node Priority)        [erpappN erpdbN ]     +
  Startup Policy                                      Online On Home Node O> +
  Fallover Policy                                     Fallover To Next Prio> +
  Fallback Policy                                     Never Fallback         +
…
   Change/Show a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                        [Entry Fields]
  Resource Group Name                                 oraRG
  New Resource Group Name                            []
  Participating Nodes (Default Node Priority)        [erpdbN erpappN]      +
  Startup Policy                                      Online On Home Node O> +
  Fallover Policy                                     Fallover To Next Prio> +
  Fallback Policy                                     Never Fallback         +

 

8 创建完毕后，

然后修改资源组属性：

Extended config
Extended Resource Config
  HACMP Extended Resource Group Configuration 
    Change/Show All Resources and Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP]                                                   [Entry Fields]
  Resource Group Name                                 erpRG
  Participating Nodes (Default Node Priority)         erpappN erpdbN
  Startup Policy                                      Online On Home Node O>
  Fallover Policy                                     Fallover To Next Prio>
  Fallback Policy                                     Never Fallback
  Service IP Labels/Addresses                        [erpapp]               +
  Application Servers                                [erpapp]                +
  Volume Groups                                      [app2u03vg app2u04vg app2u05vg> +
  Use forced varyon of volume groups, if necessary    false                  +
  Automatically Import Volume Groups                  false                  +
  Filesystems (empty is ALL for VGs specified)       [ ]                     +
  Filesystems Consistency Check                       fsck                   +
  Filesystems Recovery Method                         sequential             +
  Filesystems mounted before IP configured            false                  +
  Filesystems/Directories to Export (NFSv2/3)        []                  
    Filesystems/Directories to NFS Mount               []                      
  Network For NFS Mount                              []                      +
  Tape Resources                                     []                      +
  Raw Disk PVIDs                                     []                      +
  Fast Connect Services                              []                      +
  Communication Links                                []                      +
  Primary Workload Manager Class                     []                      +
  Secondary Workload Manager Class                   []                      +
  Miscellaneous Data                                 []
  WPAR Name                                          []
[BOTTOM]

...

Change/Show All Resources and Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP]                                                   [Entry Fields]
  Resource Group Name                                 oraRG                  
  Participating Nodes (Default Node Priority)         erpdbN erpappN
  Startup Policy                                      Online On Home Node O>
  Fallover Policy                                     Fallover To Next Prio>
  Fallback Policy                                     Never Fallback
  Service IP Labels/Addresses                        [erpdb]                +
  Application Servers                                [dbapp]                 +
  Volume Groups                                      [db1vgoradata db1vgoraarch db1vgbak> +
  Use forced varyon of volume groups, if necessary    false                  +
  Automatically Import Volume Groups                  false 
  Filesystems (empty is ALL for VGs specified)       [ ]                     +
  Filesystems Consistency Check                       fsck                   +
  Filesystems Recovery Method                         sequential             +
  Filesystems mounted before IP configured            false                  +
  Filesystems/Directories to Export (NFSv2/3)        []                      +
                                                                             +
  Filesystems/Directories to NFS Mount               []                      
  Network For NFS Mount                              []                      +
  Tape Resources                                     []                      +
  Raw Disk PVIDs                                     []                      +
  Fast Connect Services                              []
  Communication Links                                []                      +
  Primary Workload Manager Class                     []                      +
  Secondary Workload Manager Class                   []                      +
  Miscellaneous Data                                 []
  WPAR Name                                          []
[BOTTOM]

9．验证并同步拓扑。使用extend中的菜单项。选择自动纠正错误。

# smitty hacmp                                      
      Extended Configuration
       Extended Verification and Synchronization
       * Verify, Synchronize or Both                            [Both] +
       * Automatically correct errors found during                 [Yes] +
                 verification?                                                              
       * Force synchronization if verification fails?                 [No] +
       * Verify changes only?                                  [No] +
       * Logging                                         [Standard] +

10.  启动和停止HACMP:

启动HACMP（快速路径：#smitty clstart）

# smitty hacmp
    System Management (C-SPOC)
      Manage HACMP Services
        Start Cluster Services
          * Start now, on system restart or both                now  
            Start Cluster Services on these nodes    [erpapp erpdb]                
                    * Manage Resource Groups                  Automatically
                      BROADCAST message at startup?                true
                      Startup Cluster Information Daemon?              false 
            Ignore verification errors?                       false
            Automatically correct errors found during         Interactively         
            cluster start?    

 

停止HACMP（快速路径：#smitty clstop）

  # smitty hacmp
    System Management (C-SPOC)
      Manage HACMP Services
        Stop Cluster Services
          * Stop now, on system restart or both             now+
                      Stop Cluster Services on these nodes    [erpapp erpdb]
                      BROADCAST cluster shutdown?               true+
                    * Select an Action on Resource Groups    Move Resource Groups+

 

查看HA日志

tail -f /var/hacmp/log/hacmp.out

 

移动资源组：

# smitty hacmp
    System Management (C-SPOC)
            Resource Group and Applications
                Move a Resource Group to Another Node / Site
                    Move Resource Groups to Another Node
                    Select a Resource Group -->
                                 erpRG                             ONLINE               erpappN /
                            Select a Destination Node –> 
                                    erpdbN

 

确认，即可移动资源组erpRG到erpdbN节点。

 

11．验证HACMP。

A.

Smit clstop takeover 发生接管。 测试正常。

B.

Smit clstop graceful，只是停止hacmp的机器上serviceIP或concurrent VG, 而不发生接管。测试正常。

C.

拔插一台机器的电源，发生接管。

D．

重新启动一台主机。Shutdown –Fr . 发生接管，测试正常。

E．

拔掉一台机器上的服务IP网线。测试正常。