在Solaris 10 X86上安装Oracle 10g RAC详解

概述
本文详细介绍了如何在基于Solaris 10 X86的节点上来安装Oracle 10g RAC的具体步骤和注意事项，同时也提供了具体的命令操作以及安装过程的截屏，最后是创建一个Cluster数据库的具体步骤。
系统环境
两台Sun的基于 AMD64的机器(V20Z) 
一台Sun的Storage(3510)
一个百兆交换机
Solaris 10 X86 1/06
Oracle 10g 10.1.0.3 EE RAC for Solaris X86 with Oracle patch 4186426
其中两个V20Z节点通过光纤卡直接和Storage3510相连接
安装前期配置
本文中的Oracle RAC的安装是基于两个节点的配置，很多步骤需要在两个节点(RAC1 和RAC2）上都要完成,以下是安装Oracle RAC前需要做的准备工作:
1,在所有节点上创建Oracle用户组和帐号
bash-3.00# groupadd -g 100 oinstall
bash-3.00# groupadd -g 101 dba
bash-3.00# mkdir -p /export/home
bash-3.00# useradd -u 200 -g oinstall -G dba -s /usr/bin/bash -d /export/home/oracle -m oracle
bash-3.00# id -a oracle
uid=200(oracle) gid=100(oinstall) groups=101(dba)
添加完用户后需要为oracle用户设置密码，使得帐号生效。
2，在所有节点上验证用户nobody是否存在
bash-3.00# id -a nobody
uid=60001(nobody) gid=60001(nobody) groups=60001(nobody)
3，为所有的节点设置网络地址
对于每个节点，Oracle RAC需要设置一个公共的网络地址，一个虚拟的未用网络地址和一个私有网络地址，对于公共的网络地址，这里假设已经在安装系统时已经配置完成，即将bge0配置为公共地址，以下是配置其他网络地址的具体步骤，
bash-3.00# ifconfig bge0:1 plumb up
bash-3.00# ifconfig bge0:1 10.0.100.212 netmask 255.255.255.0 
bash-3.00# ifconfig bge1 plumb up
bash-3.00# ifconfig bge1 192.168.0.12 netmask 255.255.255.0
可以将这些步骤写入/etc/profile,使得每次系统启动后，这些地址自动就配置完成。最终的网络配置为：
bash-3.00# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000 
bge0: flags=1000843 mtu 1500 index 2
inet 10.0.100.222 netmask ffffff00 broadcast 10.0.100.255
ether 0:9:3d:0:75:20 
bge0:1: flags=1000843 mtu 1500 index 2
inet 10.0.100.212 netmask ffffff00 broadcast 10.0.100.255
bge1: flags=1000803 mtu 1500 index 3
inet 192.168.0.12 netmask ffffff00 broadcast 192.168.0.255
ether 0:9:3d:0:75:21 
bash-3.00$ netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ------ ---------
192.168.0.0 192.168.0.11 U 1 0 bge1
10.0.100.0 10.0.100.221 U 1 47 bge0
10.0.100.0 10.0.100.211 U 1 0 bge0:1
224.0.0.0 10.0.100.221 U 1 0 bge0
127.0.0.1 127.0.0.1 UH 23 25657 lo0
4,在所有节点上，生成文件/etc/hostname.bge0 和/etc/hostname.bge1
具体内容如下，并需将网络配置写入文件/etc/hosts
bash-3.00$ more /etc/host*
::::::::::::::
/etc/hostname.bge0
::::::::::::::
RAC1
::::::::::::::
/etc/hostname.bge1
::::::::::::::
rac_priv_host1
::::::::::::::
/etc/hosts
::::::::::::::
#
# Internet host table
#
127.0.0.1 localhost 
10.0.100.221 RAC1.prc.sun.com RAC1
10.0.100.222 RAC2.prc.sun.com RAC2
10.0.100.211 RAC1-VIR.prc.sun.com RAC1-VIR
10.0.100.212 RAC2-VIR.prc.sun.com RAC2-VIR
192.168.0.11 rac_priv_host1
192.168.0.12 rac_priv_host2
5,配置信任主机
Oracle RAC需要将所有节点配置成为信任主机，即在所有节点上生成具有以下内容的文件/etc/hosts.equiv,并进行测试
bash-3.00$ more /etc/hosts.equiv 
RAC1.prc.sun.com oracle
RAC2.prc.sun.com oracle
RAC1 oracle
RAC2 oracle
rsh RAC2 date
Thu Jan 19 02:12:12 CST 2006
rsh RAC2 hostname
RAC2
6,在所有节点上创建Oracle RAC安装目录并设置权限
bash-3.00# mkdir -p /u01/app/oracle/product/10.1.0.3/oracle
bash-3.00# chown -R oracle:oinstall /u01/app/oracle /product/10.1.0.3/oracle
bash-3.00# chmod -R 755 /u01/app/oracle/product/10.1.0.3/oracle
bash-3.00# mkdir -p /u01/crs/oracle/product/10.1.0.3/crs
bash-3.00# chown -R oracle:oinstall /u01/crs/oracle/product/10.1.0.3/crs
bash-3.00# chmod -R 755 /u01/crs/oracle/product/10.1.0.3/crs
为安装Oracle CRS在所有节点上设置Oracle的环境变量在$HOME/.profile,$HOME为用户oracle的起始目录
bash-3.00$ cat .profile 
ORACLE_BASE=/u01/app/oracle
export ORACLE_BASE
ORACLE_HOME=/u01/crs/oracle/product/10.1.0.3/crs
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin:/usr/bin:/usr/ccs/bin:/bin:/usr/openwin/bin
export PATH
在所有节点上将以下系统设置写入文件/etc/system
set semsys:seminfo_semmni 100
set semsys:seminfo_semmns 1024
set semsys:seminfo_semmsl 256
set semsys:seminfo_semvmx 32767
set shmsys:shminfo_shmmax 4294967295
set shmsys:shminfo_shmmin 100
7,存储设备的配置:
在这里，我们用了Sun Storage3510来做为安装Oracle 10g RAC的设备，并将其配置为raw partitions，以下为具体的需求:

Oracle和Oracle CRS需要的Raw Partitions数目和大小

从以上的表格可以看出，对于两个节点的Oracle RAC,我们需要在存储器上配置17个raw partitions(创建Oracle数据库需要15个，安装Oracle CRS需要2个）,由于每个Solaris的磁盘最多只能有7个slice供使用(slice 2,8,9是不能用做分区的),所以需要在Storage 3510上来分多个磁盘来满足需要,然后在Solaris中对每个盘进行分区，注意的是为了避免CRS或是Oracle数据库文件覆盖磁盘中的分区表，在分slices时不能从cylinder 0(比如slice 2)开始。
由于3510的分区在Solaris中的设备名为一个很长的字符串，为了更方便的表示这些设备，这里采用了创建连接到这些设备的方法，并设置相应的属性和权限，这两个设备是为Oracel CRS安装用的，见以下步骤：
mkdir /ORACLE
cd /ORACLE
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C802d0s0 ocr
chown -RL root:oinstall ocr
chmod -R 640 ocr
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C802d0s1 vote
chown -RL oracle:dba vote 
chmod -R 640 vote
安装Oracle CRS
当以上所有的配置完成后，我们就可以进行Oracle CRS的安装了
启动Oracle CRS安装脚本，注意参数-ignoreSysPrereqs是为了忽略版本检查带来的错误，
./runInstaller -ignoreSysPrereqs
以下是具体的安装步骤和图示,
1,欢迎界面，直接点击next
2,指定详细目录和凭证 

此时会弹出一个提示栏要求在RAC1上运行一个脚本，如下：
/u01/crs/oracle/oraInventory/orainstRoot.sh 
chmod: WARNING: can't access /var/opt/oracle
Creating the Oracle inventory pointer file (/var/opt/oracle/oraInst.loc)
Changing groupname of /u01/crs/oracle/oraInventory to oinstall.
2,指定安装文件的路径，缺省的安装路径来自于我们先前在$HOME/.profile的设置

3,选择语言---->点击下一步
4,配置cluster

5,指定网络接口的用途，这里我们指定bge0做为公共网络接口，bge1做为私有网络接口

6,指定OCR的路径为/ORACLE/ocr
7,指定Vote的路径为 /ORACLE/ocr
此时需要用户在所有节点上以root身份来运行脚本orainstRoot.sh,如下：
/u01/crs/oracle/oraInventory/orainstRoot.sh 
Creating the Oracle inventory pointer file (/var/opt/oracle/oraInst.loc)
Changing groupname of /u01/crs/oracle/oraInventory to oinstall.
/u01/crs/oracle/oraInventory/orainstRoot.sh
chmod: WARNING: can't access /var/opt/oracle
Creating the Oracle inventory pointer file (/var/opt/oracle/oraInst.loc)
Changing groupname of /u01/crs/oracle/oraInventory to oinstall.
8,然后会出现安装配置的小结界面，点击下一步后就进入安装阶段：
9,安装过程,从图中我们看到CRS安装程序自动向另一个节点RAC2拷贝文件

在安装后期，会提示用户在所有节点上以 root身份来运行root.sh，如下：
/u01/crs/oracle/product/10.1.0.3/crs/root.sh 
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/crs/oracle/product/10.1.0.3/crs
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Checking to see if Oracle CRS stack is already up...
Setting the permissions on OCR backup directory
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.1.0.3' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
clscfg: EXISTING configuration version 2 detected.
clscfg: version 2 is 10G Release 1.
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :   
node 1: rac1 rac_priv_host1 rac1
node 2: rac2 rac_priv_host2 rac2
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Adding daemons to inittab
Preparing Oracle Cluster Ready Services (CRS):
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
rac2
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
/u01/crs/oracle/product/10.1.0.3/crs/root.sh 
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/crs/oracle/product/10.1.0.3/crs
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Checking to see if Oracle CRS stack is already up...
Setting the permissions on OCR backup directory
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/crs/oracle/product/10.1.0.3' is not owned by root
WARNING: directory '/u01/crs/oracle/product' is not owned by root
WARNING: directory '/u01/crs/oracle' is not owned by root
assigning default hostname rac1 for node 1.
assigning default hostname rac2 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :   
node 1: rac1 rac_priv_host1 rac1
node 2: rac2 rac_priv_host2 rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /ORACLE/vote
Successful in setting block0 for voting disk.
Format complete.
Adding daemons to inittab
Preparing Oracle Cluster Ready Services (CRS):
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
rac2
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)

以上步骤完成后，整个CRS的安装就完成了

我们可以做一些测试来验证CRS的安装是否正确
cd /u01/crs/oracle/product/10.1.0.3/crs/bin/
pwd
/u01/crs/oracle/product/10.1.0.3/crs/bin
./olsnodes -n
rac1 1
rac2 2
同时要注意，当Oracle CRS安装完成后，以下进程是必须要在运行状态的：
1，oprocd – Cluster的进程监控器
2，evmd – 事件管理守护进程
3，ocssd – Cluster节点关系管理进程
4, crsd -- 高可靠性恢复和管理操作进程
安装Oracle10g数据库
安装好CRS后，就可以在此基础上安装Oracle 10g 了，由于Oracle 10g和Solaris 10 X86在内存管理方式有些不兼容，对于双CPU的V20Z机器，我们需要将一个CPU暂时离线，等Oracle安装好后可以将离线的CPU重新起动,暂停一个CPU的步骤如下：
psradm -f 1
psrinfo 
0 on-line since 01/19/2006 16:47:51
1 off-line since 01/19/2006 18:48:26
为安装Oracle 10g在所有节点上重新设置Oracle的环境变量在$HOME/.profile,$HOME为用户oracle的起始目录
bash-3.00$ cat .profile 
ORACLE_BASE=/u01/app/oracle
export ORACLE_BASE
ORACLE_HOME=/u01/crs/oracle/product/10.1.0.3/oracle
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin:/usr/bin:/usr/ccs/bin:/bin:/usr/openwin/bin
export PATH
开始安装Oralce 10g,如下，同样， 参数-ignoreSysPrereqs是为了忽略版本检查带来的错误
./runInstaller -ignoreSysPrereqs
以下为具体的步骤和截屏
1，欢迎界面后就进入到指定安装路径界面，缺省路径来自于$HOME/.profile的设置

3,制定Cluster安装模式，在这里我们选中两个节点，RAC1和RAC2

4,接下来的界面中选择安装类型为”Enterprise Edition”,在其后的数据库配置中，这里选择先不创建一个起始数据库，本文中将在Oracle 10g安装好后再创建一个数据库

6,接下来就进入到安装界面，可以看到Oracle安装到了所有节点





同样，安装后期会提示用户在所有节点上以root身份运行脚本root.sh,这时在RAC1运行是还要进行VIP配置。
/u01/app/oracle/product/10.1.0.3/oracle/root.sh 
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.1.0.3/oracle
Enter the full pathname of the local bin directory: [/usr/local/bin]: 
Creating /usr/local/bin directory...
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Adding entry to /var/opt/oracle/oratab file...
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
此时Oracle安装程序会进行VIP配置，需要选择网络接口和各个节点的虚拟地址








配置完成后会出现显示整个配置的结果：





同时需要在另一个节点RAC2上运行root.sh
/u01/app/oracle/product/10.1.0.3/oracle/root.sh 
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.1.0.3/oracle
Enter the full pathname of the local bin directory: [/usr/local/bin]: 
Creating /usr/local/bin directory...
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /var/opt/oracle/oratab file...
Adding entry to /var/opt/oracle/oratab file...
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
CRS resources are already configured
以上步骤完成后，整个Oracle 10g的安装就完成了。





下面我们还需要进行Oracle网络服务的配置，具体步骤如下：
netca
Oracle Net Services Configuration:
1, 在网络服务配置时选择”Cluster configuration”,如下图所示：





2,选中需要进行配置的节点，这里选中RAC1和RAC2。





3,以后的配置可以按照缺省项一步步进行，最后直到配置成功完成。
1,在创建数据库之前，我们还需要添加一个Oracle的补丁4186426,这个补丁解决了启动Oracle实例时会失败的错误，用户可以从上得到这一补丁,需要注意的是安装这个补丁时要按照它提供的步骤进行，这里不做详述.
在所有节点上创建数据库

1，由于创建oracle数据库需要多个raw devices,这里为了方便起见，将这些raw devices的设备文件连接成更易懂的文件名,同时需要设置相应的属主和权限，具体步骤如下：
cd ORACLE
mkdir db
chown oracle:root db
cd db
ls
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C800d0s0 system
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C800d0s1 sysaux
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C800d0s3 undotbs1
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C800d0s4 undotbs2
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C800d0s5 temp 
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s0 example
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s1 users 
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s3 redo1_1
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s4 redo1_2
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s5 redo2_1
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s6 redo2_2
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C801d0s7 spfile 
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C802d0s3 control1
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C802d0s4 control2
ln -s /dev/rdsk/c3t600C0FF00000000007D35237AD84C802d0s5 pwdfile 
pwd 
/ORACLE/db
chown -RL oracle:dba *
chmod -R 660 *
将以上的目录做为创建数据库所需raw devices的配置文件，具体步骤为：
mkdir -p /u01/app/oracle/oradata/tree
并在此目录下创建文件tree_raw.conf,具体内容如下：
cat tree_raw.conf 
system=/ORACLE/db/system
sysaux=/ORACLE/db/sysaux
example=/ORACLE/db/example
users=/ORACLE/db/users
temp=/ORACLE/db/temp
undotbs1=/ORACLE/db/undotbs1
undotbs2=/ORACLE/db/undotbs2
redo1_1=/ORACLE/db/redo1_1
redo1_2=/ORACLE/db/redo1_2
redo2_1=/ORACLE/db/redo2_1
redo2_2=/ORACLE/db/redo2_2
control1=/ORACLE/db/control1
control2=/ORACLE/db/control2
spfile=/ORACLE/db/spfile
pwdfile=/ORACLE/db/pwdfile
然后编辑文件/export/home/oracle/.profile,将以下几列加入
DBCA_RAW_CONFIG=/u01/app/oracle/oradata/tree/tree_raw.conf
export DBCA_RAW_CONFIG
2，在各节点上分别设置环境变量
RAC1,
ORACLE_SID=tree1
export ORACLE_SID
RAC2,
ORACLE_SID=tree2
export ORACLE_SID
3,和安装Oracle数据库一样，需要将节点RAC1和RAC2上的另一个CPU离线，可参照安装Oracle数据库一节中的步骤来做。
=====================================================

以oracle用户来创建一个Cluster数据库，具体步骤如下：

dbca
创建数据库步骤：
1,选择数据库的类型,这里需要选择创建Oracle Real Application Clusters database



然后选择创建一个数据库并随后在节点选择时选中两个节点RAC1和RAC2.
4,在选择模板时，这里选择创建”General Purpose”的数据库。
5,随后按照提示输入全局数据库的名字，这里设定名字为”tree”



6,在随后的数据库管理选项中选取Oracle Enterprise Manager Grid Control,即不选中”Configure the Database with Enterprise Manage”的选项



以后的步骤中需要设定密码，设定存储和恢复机制以及数据库内容和服务，这里按照缺省值设定即可。
随后要设置数据库的初始参数，在此，根据机器的内存大小，设定SGA为800M，PGA 为200M，用户可根据具体的环境来设定相应的大小



接下来需要配置数据库的存储并设置创建数据库的一些选项,所有这些设置完成后，就开始创建数据库，最终顺利建立，见下图：



在此我们还可以来起动并验证一下刚刚创建的数据库
------------------------------------------------
oracle:RAC1>srvctl status database -d tree
Instance tree1 is running on node rac1
Instance tree2 is running on node rac2
srvctl status nodeapps -n RAC1
VIP is running on node: RAC1
GSD is running on node: RAC1
Listener is running on node: RAC1
ONS daemon is running on node: RAC1
srvctl status nodeapps -n RAC2
VIP is running on node: RAC2
GSD is running on node: RAC2
Listener is running on node: RAC2
ONS daemon is running on node: RAC2
srvctl stop database -d tree
srvctl status database -d tree
Instance tree1 is not running on node rac1
Instance tree2 is not running on node rac2
srvctl status nodeapps -n RAC1
VIP is running on node: RAC1
GSD is running on node: RAC1
Listener is running on node: RAC1
ONS daemon is running on node: RAC1
srvctl status nodeapps -n RAC2
VIP is running on node: RAC2
GSD is running on node: RAC2
Listener is running on node: RAC2
ONS daemon is running on node: RAC2
srvctl start database -d tree
srvctl status database -d tree
Instance tree1 is running on node rac1
Instance tree2 is running on node rac2
sqlplus /nolog
SQL*Plus: Release 10.1.0.3.0 - Production on Tue Jan 24 17:00:22 2006
Copyright (c) 1982, 2004, Oracle. All rights reserved.
SQL> connect / as sysdba
Connected.
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area 838860800 bytes
Fixed Size 781424 bytes
Variable Size 221254544 bytes
Database Buffers 616562688 bytes
Redo Buffers 262144 bytes
Database mounted.
Database opened.
SQL> SELECT * FROM V$ACTIVE_INSTANCES;
INST_NUMBER INST_NAME
----------- ------------------------------------------------------------
1 RAC1:tree1
2 RAC2:tree2
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
参考文献：
Oracle Real Application Clusters Installation and Configuration Guide 10g Release 1 (10.1)
for AIX-Based Systems, Apple Mac OS X, hp HP-UX, hp Tru64 UNIX, Linux, Solaris Operating System, and Windows Platforms Part No. B10766-08