数据中心数据库集群安装手册(第一部分)

                                            ――RAC安装前的系统配置

 

 

 

 

 

 目录

１.服务器规划... 3

２.安装系统包... 3

３.建立用户... 4

４.建立目录并赋权限... 5

５.配置环境变量: 5

５.１配置grid环境变量... 5

５.１.１配置lyhdb1节点1. 5

５.１.２配置lyhdb1节点2. 7

５.２配置oracle环境变量... 9

５.２.１配置lyhdb1节点1. 9

５.２.２配置lyhdb1节点2. 11

５.３修改用户资源限制... 13

６.配置内核参数... 14

７.修改用户文件数限制... 14

８.网络配置... 15

９.配置ntp服务... 16

１０.安装cvuqdisk. 16

１０.１配置lyhdb1节点1. 16

１０.２配置lyhdb2节点2. 17

１１.安装oracleasm.. 17

１２.配置oracleasm.. 17

１２.１配置lyhdb1节点1. 17

１２.２配置lyhdb2节点2. 18

１３.建立数据库磁盘分区... 19

１３.１分区规划... 19

１３.２操作命令列表... 19

１３.３操作步骤... 20

１３.４分区检查... 22

１３.４.１检查 lyhdb1 节点1 分区列表... 22

１３.４.２检查lyhdb2  节点2 分区列表... 23

１４.建立ASM磁盘分区... 23

１４.１ASM磁盘分区规划列表... 23

１４.２生成ASM磁盘... 23

１４.２.１配置lyhdb1节点1 ASM.. 23

１４.２.２配置lyhdb2节点2 ASM.. 24

１５.建立公共密钥... 24

１５.１grid用户... 24

１５.１.１配置lyhdb1节点1. 24

１５.１.２配置lyhdb2节点2. 26

１５.１.３配置公共密钥... 28

１５.１.４分发公共密钥... 28

１５.１.５测试公共密钥... 29

１５.２oracle用户... 30

１５.２.１配置lyhdb1节点1. 30

１５.２.２配置lyhdb2节点2. 32

１５.２.３配置公共密钥... 33

１５.２.４分发公共密钥... 34

１５.２.５测试公共密钥... 34

１６.grid安装环境测试... 35

１６.１测试结果... 36

１６.２问题总结... 37

１６.３解决办法... 37

１６.３.１grid添加dba组... 37

１６.３.２安装缺失包... 38

 

 

 

 

 

 

 

 

 

注：没有特殊声明的章节，都需要在各节点依次完成。

服务器规划

Server	Hostname	User	Instance	Ip	Scan ip
Database 1	lyhdb1	grid	helath1	172.19.0.65	172.19.0.91 172.19.0.92 172.19.0.93
		oracle		172.19.0.66
Database2	lyhdb2	grid	health2	172.19.0.67
		oracle		172.19.0.68

 

安装系统包

RAC安装所需要的包

make-3.80

libaio-0.3.104

libaio-32bit-0.3.104

binutils-2.16.91.0.5

gcc-4.1.0

gcc-c++-4.1.2

compat-libstdc++-5.0.7 (x86_64)

glibc atleast 2.4-31.54, not between 2.5-18 & 2.5-23 (x86_64)

glibc-devel-2.4

glibc-devel-32bit-2.4

ksh-93r-12.9

libaio-devel-0.3.104

libaio-devel-32bit-0.3.104

libelf-0.8.5

libgcc-4.1.2 (x86_64)

libstdc++-4.1.2 (x86_64)

sysstat-8.0.4

 以上包是oracle11gr2最低版本要求，根据OS版本不同，软件版本会有所不同

建立用户

 

groupadd -g 1000 oinstall

groupadd -g 1200 asmadmin

groupadd -g 1201 asmdba

groupadd -g 1202 asmoper

groupadd -g 1300 dba

groupadd -g 1301 oper

useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle

useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper,dba -d /home/grid -s /bin/bash -c "Grid Infrastructure Owner" grid

配置密码：

lyhdb1:~ # passwd grid

Password changed.

lyhdb1:~ # passwd oracle

Password changed.

口令全部初始化为:xinao

建立目录并赋权限

mkdir -p /u01/app/grid

mkdir -p /u01/app/11.2.0/grid

chown -R grid:oinstall /u01

mkdir -p /u01/app/oracle

chown oracle:oinstall /u01/app/oracle

chmod -R 775 /u01

 

配置环境变量 １配置grid环境变量 １.１配置lyhdb1节点1

修改/home/grid/.bash_profile文件，添加以下内容：

设置grid用户环境变量,各节点除了ORACLE_SID不同，其它都保持一致,注意使用grid用户登录并配置

#lyhdb1 : ORACLE_SID=+ASM1

#lyhdb2 : ORACLE_SID=+ASM2

# ---------------------------------------------------

# .bash_profile

# ---------------------------------------------------

# OS User:      grid

# Application:  Oracle Grid Infrastructure

# Version:      Oracle 11g release 2

# ---------------------------------------------------

JAVA_HOME=/usr/local/java; export JAVA_HOME                                   

#Each RAC node must have a unique ORACLE_SID.

# ---------------------------------------------------

ORACLE_SID=+ASM1; export ORACLE_SID

ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

ORACLE_PATH=/u01/app/oracle/common/oracle/sql; export ORACLE_PATH

SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin

PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

PATH=${PATH}:/u01/app/common/oracle/bin

export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp

export TMPDIR=/tmp

umask 022

１.２配置lyhdb1节点2

修改/home/grid/.bash_profile文件，添加以下内容：

设置grid用户环境变量,各节点除了ORACLE_SID不同，其它都保持一致,注意使用grid用户登录并配置

#lyhdb1 : ORACLE_SID=+ASM1

#lyhdb2 : ORACLE_SID=+ASM2

# ---------------------------------------------------

# .bash_profile

# ---------------------------------------------------

# OS User:      grid

# Application:  Oracle Grid Infrastructure

# Version:      Oracle 11g release 2

# ---------------------------------------------------

JAVA_HOME=/usr/local/java; export JAVA_HOME                                   

#Each RAC node must have a unique ORACLE_SID.

# ---------------------------------------------------

ORACLE_SID=+ASM2; export ORACLE_SID

ORACLE_BASE=/u01/app/grid; export ORACLE_BASE

ORACLE_HOME=/u01/app/11.2.0/grid; export ORACLE_HOME

ORACLE_PATH=/u01/app/oracle/common/oracle/sql; export ORACLE_PATH

SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin

PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

PATH=${PATH}:/u01/app/common/oracle/bin

export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp

export TMPDIR=/tmp

umask 022

２配置oracle环境变量 ２.１配置lyhdb1节点1

设置oracle用户环境变量,各节点除了ORACLE_SID不同，其它都保持一致,注意使用oracle用户登录并配置

/home/oracle/.bash_profile,添加以下变量

#lyhdb1 : ORACLE_SID=health1

#lyhdb2 : ORACLE_SID=health2

# ---------------------------------------------------

# .bash_profile

# ---------------------------------------------------

# OS User:      oracle

# Application:  Oracle Database Software Owner

# Version:      Oracle 11g release 2

# ---------------------------------------------------

JAVA_HOME=/usr/local/java; export JAVA_HOME

#Each RAC node must have a unique ORACLE_SID.

#(i.e. health1, health2,...)

# ---------------------------------------------------

ORACLE_SID=health1; export ORACLE_SID

ORACLE_UNQNAME=health; export ORACLE_UNQNAME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME

ORACLE_PATH=/u01/app/common/oracle/sql; export ORACLE_PATH

SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin

PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

PATH=${PATH}:/u01/app/common/oracle/bin

export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp

export TMPDIR=/tmp

umask 022

 

２.２配置lyhdb1节点2

设置oracle用户环境变量,各节点除了ORACLE_SID不同，其它都保持一致,注意使用oracle用户登录并配置

/home/oracle/.bash_profile,添加以下变量

#lyhdb1 : ORACLE_SID=health1

#lyhdb2 : ORACLE_SID=health2

# ---------------------------------------------------

# .bash_profile

# ---------------------------------------------------

# OS User:      oracle

# Application:  Oracle Database Software Owner

# Version:      Oracle 11g release 2

# ---------------------------------------------------

JAVA_HOME=/usr/local/java; export JAVA_HOME

#Each RAC node must have a unique ORACLE_SID.

#(i.e. health1, health2,...)

# ---------------------------------------------------

ORACLE_SID=health2; export ORACLE_SID

ORACLE_UNQNAME=healthdb; export ORACLE_UNQNAME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1; export ORACLE_HOME

ORACLE_PATH=/u01/app/common/oracle/sql; export ORACLE_PATH

SQLPATH=/u01/app/common/oracle/sql; export SQLPATH

ORACLE_TERM=xterm; export ORACLE_TERM

NLS_DATE_FORMAT="DD-MON-YYYY HH24:MI:SS"; export NLS_DATE_FORMAT

TNS_ADMIN=$ORACLE_HOME/network/admin; export TNS_ADMIN

ORA_NLS11=$ORACLE_HOME/nls/data; export ORA_NLS11

PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin

PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin

PATH=${PATH}:/u01/app/common/oracle/bin

export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib

export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib

CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib

export CLASSPATH

THREADS_FLAG=native; export THREADS_FLAG

export TEMP=/tmp

export TMPDIR=/tmp

umask 022

３修改用户资源限制

配置/etc/profile, 修改用户资源限制

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then 

    if [ $SHELL = "/bin/ksh" ]; then

        ulimit -p 16384

        ulimit -n 65536

    else

        ulimit -u 16384 -n 65536

     fi

fi

配置内核参数

/etc/sysctl.conf添加以下内核参数

# Kernel parameters required by Oracle 11gR2

fs.file-max = 6815744

fs.aio-max-nr = 1048576

kernel.shmall = 2097152

kernel.shmmax = 2147483648

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default = 4194304

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

 

root下执行sysctl -p使修改生效

 

修改用户文件数限制

/etc/security/limits.conf，添加以下参数

oracle           soft    nproc           2047   

oracle           hard    nproc           16384  

oracle           soft    nofile          1024    

oracle           hard    nofile          65536 

grid             soft    nproc           2047

grid             hard    nproc           16384

grid             soft    nofile          1024

grid             hard    nofile          65536

网络配置

修改/etc/hosts文件，添加以下内容

#public ip address

172.19.0.65     lyhdb1

172.19.0.67     lyhdb2

#virtual ip address

172.19.0.66     lyhdb1-vip

172.19.0.68     lyhdb2-vip

#private ip address

192.168.1.1     lyhdb1-priv

192.168.1.2     lyhdb2-priv

#scan ip address

172.19.0.91   racnode-scanip

配置ntp服务

Network Time Protocol（NTP） 使用网络时间协议 (NTP)

修改ntp的配置文件/etc/sysconfig/ntp，添加"-x"选项

#NTPD_OPTIONS="-g -u ntp:ntp"

NTPD_OPTIONS="-x -g -u ntp:ntp"

开启服务:

lyhdb2:~ #  service ntp start

Starting network time protocol daemon (NTPD)                          done

修改系统服务，开机自启动

lyhdb2:/backup # chkconfig --list | grep ntp

ntp                       0:off  1:off  2:off  3:off  4:off  5:off  6:off

lyhdb2:/backup # chkconfig -s ntp 35

lyhdb2:/backup # chkconfig  --list | grep ntp

ntp                       0:off  1:off  2:off  3:on   4:off  5:on   6:off

安装cvuqdisk

cvuqdisk-1.0.7-1.rpm在 linux.x64_11gR2_grid安装文件中，解压即可获得

１配置lyhdb1节点1

lyhdb1:/backup/grid/rpm # export CVUQDISK_GRP=oinstall

lyhdb1:/backup/grid/rpm # rpm -ivh cvuqdisk-1.0.7-1.rpm

Preparing...                ########################################### [100%]

   1:cvuqdisk               ########################################### [100%]

２配置lyhdb2节点2

lyhdb2:/backup/grid/rpm # export CVUQDISK_GRP=oinstall

lyhdb2:/backup/grid/rpm # rpm -ivh cvuqdisk-1.0.7-1.rpm

Preparing...                ########################################### [100%]

   1:cvuqdisk               ########################################### [100%]

安装oracleasm

lyhdb1:/backup/suse_asm # rpm -ivh oracleasm-kmp-default-2.0.5_2.6.32.12_0.7-7.8.70.x86_64.rpm

lyhdb1:/backup/suse_asm # rpm -ivh oracleasm-2.0.5-7.8.70.x86_64.rpm

lyhdb1:/backup/suse_asm # rpm -ivh oracleasmlib-2.0.4-1.sle11.x86_64.rpm

lyhdb1:/backup/suse_asm # rpm -ivh oracleasm-support-2.1.3-1.SLE11.x86_64.rpm

各节点依次安装

配置oracleasm １配置lyhdb1节点1

lyhdb1:/ # /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.

 

This will configure the on-boot properties of the Oracle ASM library

driver.  The following questions will determine whether the driver is

loaded on boot and what permissions it will have.  The current values

will be shown in brackets ('[]').  Hitting without typing an

answer will keep that current value.  Ctrl-C will abort.

 

Default user to own the driver interface []: oracle

Default group to own the driver interface []: oinstall

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

Initializing the Oracle ASMLib driver:                done

Scanning the system for Oracle ASMLib disks:       done

lyhdb1:/ #

２配置lyhdb2节点2

lyhdb2:~ # /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library

driver.  The following questions will determine whether the driver is

loaded on boot and what permissions it will have.  The current values

will be shown in brackets ('[]').  Hitting without typing an

answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle

Default group to own the driver interface []: oinstall

Start Oracle ASM library driver on boot (y/n) [n]: y

Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

Initializing the Oracle ASMLib driver:                            done

Scanning the system for Oracle ASMLib disks:                    done

lyhdb2:~ #

建立数据库磁盘分区 １分区规划

 

分区列表
分区	分区用途	分区大小	剩余
/dev/sdb		11.4TB	5890GB
/dev/sdb1	CRS DATA	2GB
/dev/sdb2	FRA DATA	500GB
/dev/sdb3	DB DATA	5TB

 

２操作命令列表

 

操作列表
操作命令	操作说明
parted /dev/sdb	启动parted分区工具，使用/dev/sdb磁盘阵列
print 或 p free	打印当前分区列表
mklabel gpt	使用gpt分区表(GUID Partition Table)
mkpart primary 0 2G	建立大小为2GB的分区
mkpart primary 2G 502G	建立大小为500GB的分区
mkpart primary 502G 5.502T	建立大小为5TB的分区
quit	退出

３操作步骤

lyhdb1:~ # parted /dev/sdb

GNU Parted 1.8.8

Using /dev/sdb

Welcome to GNU Parted! Type 'help' to view a list of commands.

(parted) print                                                           

Model: LSI INF-01-00 (scsi)

Disk /dev/sdb: 11.4TB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Number  Start  End  Size  Type  File system  Flags

(parted) mklabel gpt

Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?

Yes/No? yes                                                              

(parted) print                                                           

Model: LSI INF-01-00 (scsi)

Disk /dev/sdb: 11.4TB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags

(parted)          

(parted) mkpart primary 0 2G                                             

(parted) print                                                           

Model: LSI INF-01-00 (scsi)

Disk /dev/sdb: 11.4TB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags                

 1      17.4kB  2000MB  2000MB               primary  , , , , , , , , , , ,

(parted) mkpart primary 2G 502G

(parted) p                                                               

Model: LSI INF-01-00 (scsi)

Disk /dev/sdb: 11.4TB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags                

 1      17.4kB  2000MB  2000MB               primary  , , , , , , , , , , ,

 2      2000MB  502GB   500GB                primary  , , , , , , , , , , ,

(parted) mkpart primary 502G 5.502T

(parted) p free                                                          

Model: LSI INF-01-00 (scsi)

Disk /dev/sdb: 11.4TB

Sector size (logical/physical): 512B/512B

Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags                

 1      17.4kB  2000MB  2000MB               primary  , , , , , , , , , , ,

 2      2000MB  502GB   500GB                primary  , , , , , , , , , , ,

 3      502GB   5502GB  5000GB               primary  , , , , , , , , , , ,

        5502GB  11.4TB  5890GB  Free Space                                 

(parted) quit                                                            

Information: You may need to update /etc/fstab.

４分区检查 ４.１检查 lyhdb1 节点1 分区列表

４.２检查lyhdb2  节点2 分区列表

 

建立ASM磁盘分区 １ASM磁盘分区规划列表

ASM磁盘分区规划列表
ASM磁盘分区	对应的磁盘分区	大小	用途
CRSVOL1	/dev/sdb1	2GB	CRS仲裁磁盘分区
FRAVOL1	/dev/sdb2	500GB	数据库快速恢复区
DATAVOL1	/dev/sdb3	5TB	数据库数据

 

２生成ASM磁盘 ２.１配置lyhdb1节点1 ASM

在lyhdb1节点1上面执行以下命令：

lyhdb1:~ # /usr/sbin/oracleasm createdisk CRSVOL1 /dev/sdb1

Writing disk header: done

Instantiating disk: done

lyhdb1:~ # /usr/sbin/oracleasm createdisk FRAVOL1 /dev/sdb2

Writing disk header: done

Instantiating disk: done

lyhdb1:~ # /usr/sbin/oracleasm createdisk DATAVOL1 /dev/sdb3

Writing disk header: done

Instantiating disk: done

lyhdb1:~ # /usr/sbin/oracleasm listdisks

CRSVOL1

DATAVOL1

FRAVOL1

２.２配置lyhdb2节点2 ASM

在lyhdb2节点２上面执行以下命令：

lyhdb2:~ # /etc/init.d/oracleasm scandisks

Scanning the system for Oracle ASMLib disks:                done

lyhdb2:~ # /etc/init.d/oracleasm listdisks

CRSVOL1

DATAVOL1

FRAVOL1

建立公共密钥 １grid用户 １.１配置lyhdb1节点1

grid@lyhdb1:~> /usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/grid/.ssh/id_rsa):

Created directory '/home/grid/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/grid/.ssh/id_rsa.

Your public key has been saved in /home/grid/.ssh/id_rsa.pub.

The key fingerprint is:

ef:1c:52:36:03:0e:8f:00:98:1b:3b:51:e4:a9:58:ad grid@lyhdb1

The key's randomart image is:

+--[ RSA 2048]----+

| =+              |

|=. +             |

| =+ o . .        |

|=o . . = .       |

|o.E   . S =      |

|         + o     |

|        . o      |

|         + .     |

|          o      |

+-----------------+

grid@lyhdb1:~> /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/grid/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/grid/.ssh/id_dsa.

Your public key has been saved in /home/grid/.ssh/id_dsa.pub.

The key fingerprint is:

4e:7f:56:18:99:a3:e6:19:02:33:11:c3:1d:31:bc:d3 grid@lyhdb1

The key's randomart image is:

+--[ DSA 1024]----+

|     .+++o       |

|      .oo.   o   |

|      +  o  =    |

|       +o E. +   |

|        S.+ . .  |

|       o = o .   |

|        . + o    |

|           o     |

|                 |

+-----------------+

１.２配置lyhdb2节点2

lyhdb2:~ # su - grid

grid@lyhdb2:~> /usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/grid/.ssh/id_rsa):

Created directory '/home/grid/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/grid/.ssh/id_rsa.

Your public key has been saved in /home/grid/.ssh/id_rsa.pub.

The key fingerprint is:

74:a3:00:93:e0:e0:2a:56:f9:95:2a:cd:9a:f4:b2:3d grid@lyhdb2

The key's randomart image is:

+--[ RSA 2048]----+

|. ..o.           |

|.o  oo  .        |

| ..o  .o. o      |

|. . + oo o .     |

|o. o =  S        |

|o . =            |

|   + .           |

|    +E           |

|   . ..          |

+-----------------+

 

grid@lyhdb2:~> /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/grid/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/grid/.ssh/id_dsa.

Your public key has been saved in /home/grid/.ssh/id_dsa.pub.

The key fingerprint is:

97:81:b7:62:4a:ae:d9:2b:1e:0c:30:bc:75:6c:d9:f2 grid@lyhdb2

The key's randomart image is:

+--[ DSA 1024]----+

|                 |

|.   . o  .       |

|o. . * .. o      |

| oo o o  . +     |

| ..   .ES +      |

|   o o o o       |

|    o o          |

|    .=           |

|   .+.o.         |

+-----------------+

 

１.３配置公共密钥

以上操作在各节点依次执行，完成后，在lyhdb1上执行下面操作:

grid@lyhdb1:~> ssh lyhdb1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys   

The authenticity of host 'lyhdb1 (172.19.0.65)' can't be established.

RSA key fingerprint is f0:38:97:8a:23:4d:f8:06:ca:60:b7:79:f9:1e:55:47.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb1,172.19.0.65' (RSA) to the list of known hosts.

Password:

grid@lyhdb1:~> ssh lyhdb2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys   

The authenticity of host 'lyhdb2 (172.19.0.67)' can't be established.

RSA key fingerprint is 3f:85:48:72:75:f6:2a:63:57:6e:67:36:69:96:f6:cd.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb2,172.19.0.67' (RSA) to the list of known hosts.

Password:

１.４分发公共密钥

分发公共密钥到各节点：(在lyhdb1上执行)

grid@lyhdb1:~> scp ~/.ssh/authorized_keys lyhdb1:.ssh/authorized_keys

authorized_keys                                                                                                            100%  786     0.8KB/s   00:00   

grid@lyhdb1:~> scp ~/.ssh/authorized_keys lyhdb2:.ssh/authorized_keys

Password:

authorized_keys                                                                                                            100%  786     0.8KB/s   00:00   

grid@lyhdb1:~>

１.５测试公共密钥

在lyhdb1执行：

grid@lyhdb1:~> ssh lyhdb2

grid@lyhdb2:~> ssh lyhdb1

The authenticity of host 'lyhdb1 (172.19.0.65)' can't be established.

RSA key fingerprint is f0:38:97:8a:23:4d:f8:06:ca:60:b7:79:f9:1e:55:47.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb1,172.19.0.65' (RSA) to the list of known hosts.

在lyhdb2执行：

grid@lyhdb2:~> ssh lyhdb1

Last login: Fri Mar 16 18:22:00 2012 from lyhdb2

grid@lyhdb1:~> logout

Connection to lyhdb1 closed.

grid@lyhdb2:~> ssh lyhdb2

The authenticity of host 'lyhdb2 (172.19.0.67)' can't be established.

RSA key fingerprint is 3f:85:48:72:75:f6:2a:63:57:6e:67:36:69:96:f6:cd.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb2,172.19.0.67' (RSA) to the list of known hosts.

Last login: Fri Mar 16 18:11:59 2012 from lyhdb1

grid@lyhdb2:~> logout

Connection to lyhdb2 closed.

注意，以上测试步骤不可省略，必须在各节点执行一次，因为在第一次使用SSH连接会有提示信息，需要确认一次，以后则不再需要。

２oracle用户

操作步骤同grid 用户相同，只需要在oracle用户下重复执行一次。

２.１配置lyhdb1节点1

lyhdb1:/ # su - oracle

oracle@lyhdb1:~> /usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_rsa):

Created directory '/home/oracle/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_rsa.

Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.

The key fingerprint is:

88:42:45:c8:84:10:28:7c:81:2a:63:e6:6c:9d:2d:ec oracle@lyhdb1

The key's randomart image is:

+--[ RSA 2048]----+

|*=o++            |

|+ooo             |

|o o              |

|++   . .         |

|*.+ + . S        |

| + * .           |

|. . .            |

|   E             |

|                 |

+-----------------+

 

oracle@lyhdb1:~> /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

df:0b:1c:b9:10:9c:1f:b5:95:d9:9e:82:97:b2:d7:59 oracle@lyhdb1

The key's randomart image is:

+--[ DSA 1024]----+

|            . .+ |

|       . . . oo .|

|        + . o ...|

|         o = + .E|

|        S + + o o|

|         + = . o |

|          = o    |

|           . .   |

|            .    |

+-----------------+

２.２配置lyhdb2节点2

lyhdb2:~ # su - oracle

oracle@lyhdb2:~> /usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_rsa):

Created directory '/home/oracle/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_rsa.

Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.

The key fingerprint is:

7c:a4:ea:3b:bf:a9:97:ad:7a:30:33:9a:8a:a6:5f:5f oracle@lyhdb2

The key's randomart image is:

+--[ RSA 2048]----+

|                 |

|                 |

|          .      |

|       . o       |

|        S .      |

|      =. .       |

|    .o.=Eo       |

| o .oo..+..      |

|=.o.  *O=o       |

+-----------------+

oracle@lyhdb2:~> /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

f1:94:05:f8:57:8c:e3:48:ef:8e:5f:c7:8e:3a:a2:cf oracle@lyhdb2

The key's randomart image is:

+--[ DSA 1024]----+

|         ....o   |

|        . .oo o  |

|        .oo+ o   |

|         +o +    |

|        S .o     |

|            .  . |

|           o  . o|

|         .o o. + |

|        .oEooo. .|

+-----------------+

２.３配置公共密钥

以上操作在各节点依次执行，完成后，在lyhdb1上执行下面操作:

oracle@lyhdb1:~> ssh lyhdb1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys   

Password:

oracle@lyhdb1:~>

oracle@lyhdb1:~> ssh lyhdb2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

The authenticity of host 'lyhdb2 (172.19.0.67)' can't be established.

RSA key fingerprint is 3f:85:48:72:75:f6:2a:63:57:6e:67:36:69:96:f6:cd.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb2,172.19.0.67' (RSA) to the list of known hosts.

Password:

oracle@lyhdb1:~>

２.４分发公共密钥

分发公共密钥到各节点：(在lyhdb1上执行)

oracle@lyhdb1:~> scp ~/.ssh/authorized_keys lyhdb1:.ssh/authorized_keys

authorized_keys                                                                                                            100%  790     0.8KB/s   00:00   

oracle@lyhdb1:~> scp ~/.ssh/authorized_keys lyhdb2:.ssh/authorized_keys

Password:

authorized_keys

２.５测试公共密钥

在lyhdb1执行：

oracle@lyhdb1:~> ssh lyhdb1

Last login: Fri Mar 16 18:31:24 2012 from lyhdb1

oracle@lyhdb1:~> logout

Connection to lyhdb1 closed.

oracle@lyhdb1:~> ssh lyhdb2

Last login: Fri Mar 16 18:38:20 2012 from lyhdb1

oracle@lyhdb2:~>

在lyhdb2执行：

oracle@lyhdb2:~> ssh lyhdb1

The authenticity of host 'lyhdb1 (172.19.0.65)' can't be established.

RSA key fingerprint is f0:38:97:8a:23:4d:f8:06:ca:60:b7:79:f9:1e:55:47.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb1,172.19.0.65' (RSA) to the list of known hosts.

Last login: Fri Mar 16 18:45:35 2012 from lyhdb1

oracle@lyhdb1:~> logout

Connection to lyhdb1 closed.

oracle@lyhdb2:~> ssh lyhdb2

The authenticity of host 'lyhdb2 (172.19.0.67)' can't be established.

RSA key fingerprint is 3f:85:48:72:75:f6:2a:63:57:6e:67:36:69:96:f6:cd.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'lyhdb2,172.19.0.67' (RSA) to the list of known hosts.

Last login: Fri Mar 16 18:38:40 2012 from lyhdb1

安装环境测试

命令：./runcluvfy.sh stage -pre crsinst -n lyhdb1,lyhdb2 -verbose

该命令在Oracle的集群安装文件里，解压之后在安装文件根目录可以找到。

 在lyhdb1上执行：

grid@lyhdb1:/backup/grid>  ./runcluvfy.sh stage -pre crsinst -n lyhdb1,lyhdb2 -verbose

 

测试结果请查看《grid环境测试》

１测试结果

以下是检测失败信息：

 Check: Membership of user "grid" in group "dba"

  Node Name         User Exists   Group Exists  User in Group  Comment        

  ----------------  ------------  ------------  ------------  ----------------

  lyhdb2            yes           yes           no            failed

lyhdb1            yes           yes           no            failed         

Result: Membership check for user "grid" in group "dba" failed

Check: Package existence for "libstdc++43-devel-32bit-4.3.3_20081022"

  Node Name     Available                 Required                  Comment  

  ------------  ------------------------  ------------------------  ----------

  lyhdb2        missing                   libstdc++43-devel-32bit-4.3.3_20081022  failed

lyhdb1        missing                   libstdc++43-devel-32bit-4.3.3_20081022  failed   

Result: Package existence check failed for "libstdc++43-devel-32bit-4.3.3_20081022"

Check: Package existence for "sysstat-8.1.5"

  Node Name     Available                 Required                  Comment  

  ------------  ------------------------  ------------------------  ----------

  lyhdb2        sysstat-8.1.5-7.9.56      sysstat-8.1.5             passed

lyhdb1        missing                   sysstat-8.1.5             failed   

Result: Package existence check failed for "sysstat-8.1.5"

２问题总结

1. grid用户缺少dba组

2. 缺少libstdc++43-devel-32bit-4.3.3_20081022 双节点都没有安装

3. 缺少sysstat-8.1.5   lyhdb1上没有安装

３解决办法 ３.１grid添加dba组

lyhdb1:/backup # usermod -G asmadmin,asmdba,asmoper,dba grid

lyhdb2:/backup # usermod -G asmadmin,asmdba,asmoper,dba grid

检查用户组是否正确

lyhdb1:/backup # cat /etc/group

修改前：

oinstall:!:1000:

asmadmin:!:1200:grid

asmdba:!:1201:oracle,grid

asmoper:!:1202:grid

dba:!:1300:oracle

oper:!:1301:oracle

修改后：

oinstall:!:1000:

asmadmin:!:1200:grid

asmdba:!:1201:oracle,grid

asmoper:!:1202:grid

dba:!:1300:oracle,grid

oper:!:1301:oracle

３.２安装缺失包

lyhdb1:/media/suse/x86_64 # rpm -ivh libstdc++43-devel-32bit-4.3.4_20091019-0.7.35.x86_64.rpm

Preparing...                ########################################### [100%]

   1:libstdc++43-devel-32bit########################################### [100%]

 

lyhdb2:/media/suse/x86_64 # rpm -ivh libstdc++43-devel-32bit-4.3.4_20091019-0.7.35.x86_64.rpm

Preparing...                ########################################### [100%]

   1:libstdc++43-devel-32bit########################################### [100%]

 

lyhdb1:/media/suse/x86_64 # rpm -ivh sysstat-8.1.5-7.9.56.x86_64.rpm

Preparing...                ########################################### [100%]

   1:sysstat                ########################################### [100%]

 

再次测试，直到问题全部解决。

grid@lyhdb1:/backup/grid>  ./runcluvfy.sh stage -pre crsinst -n lyhdb1,lyhdb2 –verbose

当检测最后出现Pre-check for cluster services setup was successful.

说明系统环境配置正确，可以进行grid的安装