分类： Oracle
2009-11-24 09:03:06
简单描述一下，在Oracle 10203 for Solaris sparc的RAC双节点环境中，新增一个节点的过程。共享存储已经在第三个节点上配置完成，这里主要介绍操作系统上和Oracle上的配置。
这一篇介绍准备过程。
首先是将主机的IP地址，调整到RAC服务器IP地址附近，方便管理。修改目标主机IP地址到172.25.198.226。
# ifconfig ce0 172.25.198.226 netmask 255.255.0.0 broadcast 172.25.255.255
然后修改/etc/hosts文件中的IP地址。
下面将主机名也修改为配套的主机名信息racnode3：
# hostname
supingsol
# hostname racnode3
利用hostname设置主机名后，还要分别修改/etc/hosts、/ect/nodename、/etc/hostname.ce0、/etc/net/ticlts/hosts、/etc/net/ticots/hosts、/etc/net/ticotsord/hosts这6个文件：
# more /etc/hosts
#
# Internet host table
#
127.0.0.1 localhost
172.25.198.226 racnode3 loghost
192.168.192.109 rw
# more /etc/nodename
racnode3
# more /etc/hostname.ce0
racnode3
# more /etc/net/ticlts/hosts
#
# Copyright 2003 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)hosts 1.3 03/12/07 SMI"
#
# NOTE: This file is no longer consulted and may be removed from a future
# release of Solaris.
#
racnode3
# more /etc/net/ticots/hosts
#
# Copyright 2003 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)hosts 1.3 03/12/07 SMI"
#
# NOTE: This file is no longer consulted and may be removed from a future
# release of Solaris.
#
racnode3
# more /etc/net/ticotsord/hosts
#
# Copyright 2003 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)hosts 1.3 03/12/07 SMI"
#
# NOTE: This file is no longer consulted and may be removed from a future
# release of Solaris.
#
racnode3
下面添加用户和组信息，根据RAC节点的组信息和用户信息在当前节点创建用户和组：
# groupadd -g 100 oinstall
# groupadd -g 101 dba
# mkdir /export/home/oracle
# useradd -u 100 -g oinstall -G dba -d /export/home/oracle oracle
# chown oracle:oinstall /export/home/oracle
# mkdir /data
# chown -R oracle:oinstall /data
# id nobody
uid=60001(nobody) gid=60001(nobody)
# passwd oracle
用Oracle用户登陆，设置环境变量：
# su - oracle
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
$ vi .profile
".profile"［新文件］
umask 022
ORACLE_SID=testrac3
export ORACLE_SID
ORACLE_BASE=/data/oracle
export ORACLE_BASE
ORACLE_HOME=/data/oracle/product/10.2/database
export ORACLE_HOME
NLS_LANG='SIMPLIFIED CHINESE_CHINA.ZHS16GBK'
export NLS_LANG
PATH=$PATH:$ORACLE_HOME/bin
export PATH
DISPLAY=172.25.6.201:1.0
export DISPLAY
修改/etc/hosts文件添加public id, private id和vitual id：
# vi /etc/hosts
"/etc/hosts"［只读取］ 13 行，300 字符
#
# Internet host table
#
127.0.0.1 localhost
172.25.198.226 racnode3 loghost
172.25.198.227 racnode3-vip
10.0.0.3 racnode3-priv
172.25.198.222 racnode1
172.25.198.224 racnode1-vip
10.0.0.1 racnode1-priv
172.25.198.223 racnode2
172.25.198.225 racnode2-vip
10.0.0.2 racnode2-priv
在其他两个节点，分别添加下面的内容：
172.25.198.226 racnode3
172.25.198.227 racnode3-vip
10.0.0.3 racnode3-priv
在节点1、节点2和当前节点，编辑/export/home/oracle/.rhosts文件，内容如下：
racnode1
racnode1-vip
racnode1-priv
racnode2
racnode2-vip
racnode2-priv
racnode1
racnode1-vip
racnode1-priv
racnode2
racnode2-vip
racnode2-priv
racnode3
racnode3-vip
racnode3-priv
在当前服务器上，添加PRIVATE IP地址：
# ifconfig -a
lo0: flags=2001000849
inet 127.0.0.1 netmask ff000000
ce0: flags=1000843
inet 172.25.198.226 netmask ffff0000 broadcast 172.25.255.255
ether 0:3:ba:2c:d8:da
# ifconfig ce1 plumb
# ifconfig ce1 10.0.0.3 netmask 255.255.255.0 broadcast 10.0.0.255 up
为了启动后，ce1可以自动配置，编辑/etc/hostname.ce1文件：
racnode3-priv
然后添加相应的掩码和广播地址，在/etc/netmasks文件后面添加：
10.0.0.0 255.255.0.0
另外需要添加默认的网关/etc/defaultrouter：
172.25.13.1
检查nscd进程是否启动：
# ps -ef|grep nscd
root 162 1 0 3月 12 ? 1:13 /usr/sbin/nscd
root 4498 4418 0 17:33:49 pts/1 0:00 grep nscd
如果没有启动，可以使用/etc/init.d/nscd start来启动。
在两个节点上下面检查软件包是否都已经安装：
# uname -r
5.10
# pkginfo -i SUNWarc SUNWbtool SUNWhea SUNWlibC SUNWlibm SUNWlibms SUNWsprot SUNWtoo SUNWi1of SUNWxwfnt
system SUNWarc Lint Libraries (usr)
system SUNWbtool CCS tools bundled with SunOS
system SUNWhea SunOS Header Files
system SUNWi1of ISO-8859-1 (Latin-1) Optional Fonts
system SUNWlibC Sun Workshop Compilers Bundled libC
system SUNWlibm Math & Microtasking Library Headers & Lint Files (Usr)
system SUNWlibms Math & Microtasking Libraries (Usr)
system SUNWsprot Solaris Bundled tools
system SUNWtoo Programming Tools
system SUNWxwfnt X Window System platform required fonts
编辑/etc/system，设置核心参数：
set noexec_user_stack=1
set semsys:seminfo_semmni=100
set semsys:seminfo_semmns=1024
set semsys:seminfo_semmsl=256
set semsys:seminfo_semvmx=32767
set shmsys:shminfo_shmmax=21474836479
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=100
set shmsys:shminfo_shmseg=10
检查UDP协议配置：
# ndd /dev/udp udp_xmit_hiwat
57344
# ndd /dev/udp udp_recv_hiwat
57344
这个值已经很接近Oracle的推荐值了，这里就不修改了。
下面重启系统，安装的准备工作到此结束。
# init 6
简单描述一下，在Oracle 10203 for Solaris sparc的RAC双节点环境中，新增一个节点的过程。共享存储已经在第三个节点上配置完成，这里主要介绍操作系统上和Oracle上的配置。
这一篇描述CLUSTER软件的建立。
Oracle10203RAC环境添加新节点（一）：http://yangtingkun.itpub.net/post/468/493738
在新增节点racnode3上将共享存储上的裸设备授权给Oracle用户：
bash-3.00# chown oracle:oinstall /dev/rdsk/c1t500601603022E66Ad*
建立和节点1、节点2上一致的/dev/rac/vot和/dev/rac/ocr链接，执行相同的裸设备文件：
bash-3.00# mkdir /dev/rac
bash-3.00# ln -s /dev/rdsk/c1t500601603022E66Ad2s1 /dev/rac/ocr
bash-3.00# ln -s /dev/rdsk/c1t500601603022E66Ad2s3 /dev/rac/vot
在安装节点执行$ORA_CLS_HOME/out/bin/addNode.sh来启动图形化界面添加新的节点：
bash-2.03$ ./addNode.sh
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.8
Passed
Checking Temp space: must be greater than 150 MB. Actual 8715 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
All installer requirements met.
Oracle Universal Installer, Version 10.2.0.1.0 Production
Copyright (C) 1999, 2005, Oracle. All rights reserved.
在图形界面中，填入新增节点的PUBLIC、PRIVATE和VIP对应的名称。
安装过程Oracle跳过了一些远端执行碰到的错误，需要手工更正，在racnode3上执行下面的脚本：
bash-3.00$ /data/oracle/product/10.2/crs/oui/bin/runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=OraCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.10
Passed
Checking Temp space: must be greater than 250 MB. Actual 10167 MB Passed
Checking swap space: must be greater than 500 MB. Actual 7415 MB Passed
All installer requirements met.
'AttachHome' was successful.
在节点2上执行下面的脚本：
bash-2.03$ /data/oracle/product/10.2/crs/oui/bin/runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTROY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode2
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.8
Passed
Checking Temp space: must be greater than 150 MB. Actual 8233 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
All installer requirements met.
'UpdateNodeList' was successful.
节点3上运行类似的脚本：
bash-3.00$ /data/oracle/product/10.2/crs/oui/bin/runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTROY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.10
Passed
Checking Temp space: must be greater than 250 MB. Actual 10167 MB Passed
Checking swap space: must be greater than 500 MB. Actual 7415 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
All installer requirements met.
'UpdateNodeList' was successful.
随后需要依次执行下面的脚本：
1.节点3上的/data/oracle/oraInventory/orainstRoot.sh：
bash-3.00# ./data/oracle/oraInventory/orainstRoot.sh
Creating the Oracle inventory pointer file (/var/opt/oracle/oraInst.loc)
Changing permissions of /data/oracle/oraInventory to 770.
Changing groupname of /data/oracle/oraInventory to oinstall.
The execution of the script is complete
2.节点1上的/data/oracle/product/10.2/crs/install/rootaddnode.sh：
# ./data/oracle/product/10.2/crs/install/rootaddnode.sh
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Attempting to add 1 new nodes to the configuration
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node
node 3: racnode3 racnode3-priv racnode3
Creating OCR keys for user 'root', privgrp 'other'..
Operation successful.
/data/oracle/product/10.2/crs/bin/srvctl add nodeapps -n racnode3 -A racnode3-vip/255.255.255.0/ce0 -o /data/oracle/product/10.2/crs
3.节点3上的/data/oracle/product/10.2/crs/root.sh：
bash-3.00# ./data/oracle/product/10.2/crs/root.sh
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured
OCR LOCATIONS = /dev/rac/ocr
OCR backup directory '/data/oracle/product/10.2/crs/cdata/crs' does not exist. Creating now
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node
node 1: racnode1 racnode1-priv racnode1
node 2: racnode2 racnode2-priv racnode2
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
racnode1
racnode2
racnode3
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
IP address "racnode1-vip" has already been used. Enter an unused IP address.
由于vip的配置出现了错误，需要手工启动vipca来进行配置，在图形化界面用root执行/data/oracle/product/10.2/crs/bin/vipca，输入racnode3对应的VIP名称和ip地址：racnode3-vip，172.25.198.227，然后执行确定。
成功配置后，在添加节点的图形化工具addNode中点击继续。
至此，cluster node节点添加成功。
简单描述一下，在Oracle 10203 for Solaris sparc的RAC双节点环境中，新增一个节点的过程。共享存储已经在第三个节点上配置完成，这里主要介绍操作系统上和Oracle上的配置。
这一篇描述数据库软件的建立并建立ASM实例。
Oracle10203RAC环境添加新节点（一）：http://yangtingkun.itpub.net/post/468/493738
Oracle10203RAC环境添加新节点（二）：http://yangtingkun.itpub.net/post/468/493764
在图形界面环境下进行ORACLE_HOME/oui/bin，运行addNode.sh，来进行新节点的添加过程：
$ cd $ORACLE_HOME/oui/bin
$ ./addNode.sh
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
Oracle Universal Installer, Version 10.2.0.3.0 Production
Copyright (C) 1999, 2006, Oracle. All rights reserved.
选择需要安装的节点，这里是RACNODE3，然后点击继续：
检查汇总界面，Oracle将10.2.0.1版本和10.2.0.3补丁一起安装到新增节点上。
由于执行过程中出现了错误，需要手工执行远端脚本：
bash-3.00$ /data/oracle/product/10.2/database/oui/bin/runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database ORACLE_HOME_NAME=OraDb10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
'AttachHome' was successful.
同样运行updateNodeList的脚本：
bash-3.00$ /data/oracle/product/10.2/database/oui/bin/runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=false "INVENTROY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
'UpdateNodeList' was successful.
运行完成后，需要在新节点上运行root.sh脚本：
# /data/oracle/product/10.2/database/root.sh
点击确定，完成软件的安装。
检查新建节点后台进程：
bash-3.00$ ps -ef|grep ora
oracle 1587 1584 0 4月 01 pts/1 0:00 -sh
oracle 11022 5480 0 14:12:39 pts/2 0:00 ps -ef
oracle 2877 2876 0 4月 01 ? 0:04 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 1247 1244 0 3月 31 pts/2 0:00 -sh
oracle 1987 1 0 4月 01 ? 0:00 sh -c sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode3
root 1996 1 0 4月 01 ? 2:05 /data/oracle/product/10.2/crs/bin/crsd.bin reboot
root 2234 2180 0 4月 01 ? 0:09 /data/oracle/product/10.2/crs/bin/oprocd run -t 1000 -m 500 -f
oracle 2170 1987 0 4月 01 ? 0:23 /data/oracle/product/10.2/crs/bin/evmd.bin
oracle 2492 2170 0 4月 01 ? 0:02 /data/oracle/product/10.2/crs/bin/evmlogger.bin -o /data/oracle/product/10.2/cr
oracle 2263 2262 0 4月 01 ? 0:00 /bin/sh -c ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode3/c
oracle 2264 2263 0 4月 01 ? 3:44 /data/oracle/product/10.2/crs/bin/ocssd.bin
oracle 2262 2181 0 4月 01 ? 0:00 sh -c /bin/sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/rac
oracle 5480 16119 0 11:29:48 pts/2 0:00 bash
oracle 2876 1 0 4月 01 ? 0:00 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 16119 1284 0 10:56:59 pts/2 0:00 -sh
oracle 11023 5480 0 14:12:39 pts/2 0:00 grep ora
下面在节点1上启动dbca来添加racnode3上的asm实例。
首先选项Oracle Real Application Clusters database选项，然后选择配置ASM。
在节点选择出选择全部三个节点，然后点击确定。
DBCA给出提示：ASM is present on the cluster but needs to be extended to the following nodes: [racnode3]. Do you want ASM to be extended?
确认后，dbca会在新增节点上启动ASM实例。
随后弹出对话框：
DBCA could not startup the ASM instance configured on this node. To proceed with ASM diskgroup management you need the ASM instance to be up and running. Do you want to recreate the ASM instance on this node?
确认后，dbca给出对话框输入asm实例的SYS用户密码，以及ASM实例的SPFILE和PFILE文件位置。
检查后台进程，ASM实例已经启动：
bash-3.00$ ps -ef|grep ora
oracle 18894 1 0 14:17:50 ? 0:00 asm_mman_+ASM3
oracle 1587 1584 0 4月 01 pts/1 0:00 -sh
oracle 21601 5480 0 14:59:24 pts/2 0:00 ps -ef
oracle 2877 2876 0 4月 01 ? 0:04 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 1247 1244 0 3月 31 pts/2 0:00 -sh
oracle 1987 1 0 4月 01 ? 0:00 sh -c sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode3
root 1996 1 0 4月 01 ? 2:10 /data/oracle/product/10.2/crs/bin/crsd.bin reboot
oracle 18882 1 0 14:17:50 ? 0:02 asm_diag_+ASM3
root 2234 2180 0 4月 01 ? 0:09 /data/oracle/product/10.2/crs/bin/oprocd run -t 1000 -m 500 -f
oracle 18937 1 0 14:17:51 ? 0:00 asm_gmon_+ASM3
oracle 2170 1987 0 4月 01 ? 0:23 /data/oracle/product/10.2/crs/bin/evmd.bin
oracle 2492 2170 0 4月 01 ? 0:02 /data/oracle/product/10.2/crs/bin/evmlogger.bin -o /data/oracle/product/10.2/cr
oracle 2263 2262 0 4月 01 ? 0:00 /bin/sh -c ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/racnode3/c
oracle 2264 2263 0 4月 01 ? 3:51 /data/oracle/product/10.2/crs/bin/ocssd.bin
oracle 18884 1 0 14:17:50 ? 0:00 asm_psp0_+ASM3
oracle 2262 2181 0 4月 01 ? 0:00 sh -c /bin/sh -c 'ulimit -c unlimited; cd /data/oracle/product/10.2/crs/log/rac
oracle 5480 16119 0 11:29:48 pts/2 0:00 bash
oracle 2876 1 0 4月 01 ? 0:00 /data/oracle/product/10.2/crs/opmn/bin/ons -d
oracle 18890 1 0 14:17:50 ? 0:04 asm_lms0_+ASM3
oracle 16119 1284 0 10:56:59 pts/2 0:00 -sh
oracle 18888 1 0 14:17:50 ? 0:04 asm_lmd0_+ASM3
oracle 18886 1 0 14:17:50 ? 0:04 asm_lmon_+ASM3
oracle 18875 1 0 14:17:49 ? 0:01 asm_pmon_+ASM3
oracle 18896 1 0 14:17:50 ? 0:00 asm_dbw0_+ASM3
oracle 18900 1 0 14:17:50 ? 0:00 asm_lgwr_+ASM3
oracle 18902 1 0 14:17:50 ? 0:00 asm_ckpt_+ASM3
oracle 18913 1 0 14:17:51 ? 0:00 asm_smon_+ASM3
oracle 18928 1 0 14:17:51 ? 0:00 asm_rbal_+ASM3
oracle 19072 1 0 14:17:56 ? 0:01 /data/oracle/product/10.2/database/bin/racgimon daemon ora.racnode3.ASM3.asm
oracle 18968 1 0 14:17:52 ? 0:01 asm_lck0_+ASM3
oracle 20168 1 0 14:58:26 ? 0:00 asm_pz98_+ASM3
oracle 21602 5480 0 14:59:24 pts/2 0:00 grep ora
oracle 19315 1 0 14:57:51 ? 0:00 asm_pz99_+ASM3
简单描述一下，在Oracle 10203 for Solaris sparc的RAC双节点环境中，新增一个节点的过程。共享存储已经在第三个节点上配置完成，这里主要介绍操作系统上和Oracle上的配置。
这一篇添加一个新节点上的数据库实例。
Oracle10203RAC环境添加新节点（一）：http://yangtingkun.itpub.net/post/468/493738
Oracle10203RAC环境添加新节点（二）：http://yangtingkun.itpub.net/post/468/493764
Oracle10203RAC环境添加新节点（三）：http://yangtingkun.itpub.net/post/468/493811
首先确保数据库处于启动状态，如果数据库没有启动：
bash-2.03$ srvctl start inst -d testrac -i testrac1
bash-2.03$ srvctl start inst -d testrac -i testrac2
确保监听处于启动状态，如果没有启动，使用lsnrctl start启动监听：
bash-2.03$ lsnrctl start
LSNRCTL for Solaris: Version 10.2.0.3.0 - Production on 01-2月 -2009 00:22:24
Copyright (c) 1991, 2006, Oracle. All rights reserved.
启动/data/oracle/product/10.2/database/bin/tnslsnr: 请稍候...
TNSLSNR for Solaris: Version 10.2.0.3.0 - Production系统参数文件为/data/oracle/product/10.2/database/network/admin/listener.ora写入/data/oracle/product/10.2/database/network/log/listener.log的日志信息监听: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.225)(PORT=1521)))监听: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))
正在连接到 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.25.198.225)(PORT=1521)))
LISTENER 的 STATUS
------------------------别名 LISTENER版本 TNSLSNR for Solaris: Version 10.2.0.3.0 - Production启动日期 01-2月 -2009 00:22:26正常运行时间 0 天 0 小时 0 分 0 秒跟踪级别 off安全性 ON: Local OS Authentication
SNMP OFF监听程序参数文件 /data/oracle/product/10.2/database/network/admin/listener.ora监听程序日志文件 /data/oracle/product/10.2/database/network/log/listener.log监听端点概要...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.225)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))服务摘要..服务 "PLSExtProc" 包含 1 个例程。
例程 "PLSExtProc", 状态 UNKNOWN, 包含此服务的 1 个处理程序...服务 "testrac" 包含 1 个例程。
例程 "testrac2", 状态 UNKNOWN, 包含此服务的 1 个处理程序...命令执行成功
对于新节点而言，可以首先通过NETCA来配置监听程序：
$ netca
Oracle Net Services Configuration:
首先选择Cluster Configuration，然后选择全表3个节点，选择Listener Configuration，选择ADD，输入监听名称。
在图形界面环境下运行DBCA添加实例报错，由于DBCA对于ASM的支持存在问题，只能选择手工添加实例：
首先建立UNDOTBS3表空间，以及THREAD3对应的REDO.LOG的方法：
SQL> select group#, thread#, sequence#, bytes/1024/1024
2 from v$log;
GROUP# THREAD# SEQUENCE# BYTES/1024/1024
---------- ---------- ---------- ---------------
1 1 1716 512
2 1 1717 512
3 2 839 512
4 2 840 512
SQL> select member from v$logfile;
MEMBER
---------------------------------------------------
+DISK/testrac/onlinelog/group_1.258.633485433
+DISK/testrac/onlinelog/group_2.260.633485441
+DISK/testrac/onlinelog/group_3.268.633485447
+DISK/testrac/onlinelog/group_4.270.633485455
下面手工添加THREAD 3对应的REDO.LOG：
SQL> alter database add logfile thread 3 group 5 '+DISK/testrac/onlinelog/redo5.log' size 512m;
数据库已更改。
SQL> alter database add logfile thread 3 group 6 '+DISK/testrac/onlinelog/redo6.log' size 512m;
数据库已更改。
SQL> select group#, thread#, sequence#, bytes/1024/1024
2 from v$log;
GROUP# THREAD# SEQUENCE# BYTES/1024/1024
---------- ---------- ---------- ---------------
1 1 1716 512
2 1 1717 512
3 2 839 512
4 2 840 512
5 3 0 512
6 3 0 512
已选择6行。
SQL> select member from v$logfile;
MEMBER
----------------------------------------------------------------
+DISK/testrac/onlinelog/group_1.258.633485433
+DISK/testrac/onlinelog/group_2.260.633485441
+DISK/testrac/onlinelog/group_3.268.633485447
+DISK/testrac/onlinelog/group_4.270.633485455
+DISK/testrac/onlinelog/redo5.log
+DISK/testrac/onlinelog/redo6.log
已选择6行。
接着添加UNDOTBS3表空间：
SQL> select file_name from dba_data_files
2 where tablespace_name like 'UNDOTBS_';
FILE_NAME
----------------------------------------------------------------------
+DISK/testrac/datafile/undotbs1.263.618591197
+DISK/testrac/datafile/undotbs2.266.618591249
SQL> create undo tablespace undotbs3
2 datafile '+DISK/testrac/datafile/undotbs3.dbf' size 4096m;
表空间已创建。
检查数据库的SPFILE配置：
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +DISK/testrac/spfiletestrac.ora
由于系统使用的是SPFILE，下面直接修改必要的初始化参数值：
SQL> alter system set cluster_database_instances = 3 scope = spfile;
系统已更改。
SQL> alter system set instance_number = 3 scope = spfile sid = 'testrac3';
系统已更改。
SQL> alter system set local_listener =
2 '(ADDRESS = (PROTOCOL = TCP)(HOST = 172.25.198.227)(PORT = 1521))'
3 scope = spfile sid = 'testrac3';
系统已更改。
SQL> alter system set thread = 3 scope = spfile sid = 'testrac3';
系统已更改。
SQL> alter system set undo_tablespace = 'UNDOTBS3' scope = spfile sid = 'testrac3';
系统已更改。
下面重启数据库的实例，使得数据库加载新的SPFILE：
bash-2.03$ srvctl stop db -d testrac
bash-2.03$ srvctl start inst -d testrac -i testrac1
bash-2.03$ srvctl start inst -d testrac -i testrac2
登陆实例1的节点，ENABLE THREAD 3：
SQL> alter database enable thread 3;
数据库已更改。
在新增节点上编辑PFILE文件，指向SPFILE文件：
bash-3.00$ cd $ORACLE_HOME
bash-3.00$ cd dbs
bash-3.00$ vi inittestrac3.ora
spfile='+DISK/testrac/spfiletestrac.ora'
生成密码文件：
bash-3.00$ orapwd file=orapwtestrac3 password=password
建立adump、bdump、udump和cdump目录：
bash-3.00$ cd $ORACLE_BASE/admin
bash-3.00$ ls -l
total 2
drwxr-xr-x 7 oracle oinstall 512 Jul 1 17:37 +ASM
bash-3.00$ mkdir testrac
bash-3.00$ cd testrac
bash-3.00$ mkdir bdump cdump adump udump
下面就可以登陆sqlplus并启动新实例了：
bash-3.00$ sqlplus "/ as sysdba"
SQL*Plus: Release 10.2.0.3.0 - Production on 星期五 4月 3 15:06:35 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
已连接到空闲例程。
SQL> startup
ORACLE 例程已经启动。
Total System Global Area 2147483648 bytes
Fixed Size 2031480 bytes
Variable Size 503316616 bytes
Database Buffers 1627389952 bytes
Redo Buffers 14745600 bytes数据库装载完毕。数据库已经打开。
SQL> select inst_id, instance_number, instance_name
2 from gv$instance;
INST_ID INSTANCE_NUMBER INSTANCE_NAME
---------- --------------- ----------------
3 3 testrac3
2 2 testrac2
1 1 testrac1
SQL> select instance_name, status from v$instance;
INSTANCE_NAME STATUS
---------------- ------------
testrac3 OPEN
随后还需要将手工添加的实例信息加到srvctl工具中：
$ srvctl add instance -d testrac -i testrac3 -n racnode3
$ srvctl modify instance -d testrac -i testrac3 -s +ASM3
至此，新增实例的操作完成。
还需要修改三个节点上的tnsnames.ora文件，确保通过TESTRAC的访问在3个实例上根据负载均衡的机制进行连接。
TESTRAC =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = racnode1-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = racnode2-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = racnode3-vip)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = testrac)
(FAILOVER_MODE =
(TYPE = SELECT)
(METHOD = BASIC)
(RETRIES = 180)
(DELAY = 5)
)
)
)
比如配置TESTRAC服务名如上所示，下面连接数据库：
SQL> conn test/test@testrac已连接。
SQL> select instance_name from v$instance;
INSTANCE_NAME
----------------
testrac3
SQL> host
$ sqlplus test/test@testrac
SQL*Plus: Release 10.2.0.3.0 - Production on 星期四 4月 9 14:56:34 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
连接到:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> select instance_name from v$instance;
INSTANCE_NAME
----------------
testrac1
SQL> host
$ sqlplus test/test@testrac
SQL*Plus: Release 10.2.0.3.0 - Production on 星期四 4月 9 14:56:46 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
连接到:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> select instance_name from v$instance;
INSTANCE_NAME
----------------
testrac2
第一个错误是在运行ADDNODE.SH工具时碰到的，在节点3的安装过程中出现了空间不足的错误：
INFO: ld.so.1: lsnodes: fatal: libskgxn2.so: open failed: No such file or directory
INFO: Vendor clusterware is not detected.
INFO: Assigning node number '3' to node 'racnode3'
INFO: Setting variable 'CLUSTER_NEW_HOST_NAMES' to 'racnode3,'. Received the value from a code block.
INFO: Setting variable 'CLUSTER_NEW_NODE_NUMBERS' to '3,'. Received the value from a code block.
INFO: Initializing OUI access setup
INFO: Performing operation for OUI access setup
INFO:
*** Cluster Node Addition Summary Page***
m_asNodeArray:racnode2,racnode3,racnode1m_sLocalNode:racnode2
sNativeVolName:/data1/
m_asNodeArray:racnode2,racnode3,racnode1m_sLocalNode:racnode2
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error in checking free space on path '/data1' on node 'racnode3' .
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.throwRemoteOpsException(OiipgClusterOps.java:2456)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.getFreeSpaceOnNode(OiipgClusterOps.java:2236)
at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.addAddNodeSpaceInfos(OiifpAddNodeSummaryPanel.java:439)
at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.populateTree(OiifpAddNodeSummaryPanel.java:246)
at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.init(OiifpAddNodeSummaryPanel.java:227)
at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.createPanelElements(OiifpAddNodeSummaryPanel.java:148)
at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.createWizardPanel(OiifmGraphicPageHandler.java:583)
at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.showWizardPage(OiifmGraphicPageHandler.java:241)
at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.processWizardPage(OiifmGraphicPageHandler.java:193)
at oracle.sysman.oii.oiif.oiifm.OiifmGraphicSummaryPageHandler.processWizardPage(OiifmGraphicSummaryPageHandler.java:90)
at oracle.sysman.oii.oiif.oiifm.OiifmGraphicInterfaceManager.doWizardPageOperation(OiifmGraphicInterfaceManager.java:352)
at oracle.sysman.oii.oiif.oiifb.OiifbWizChainDlgElem.doOperation(OiifbWizChainDlgElem.java:690)
at oracle.sysman.oii.oiif.oiifw.OiifwAddNodeSummaryWCDE.doOperation(OiifwAddNodeSummaryWCDE.java:176)
at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
at oracle.sysman.oii.oiic.OiicPullSession.doOperation(OiicPullSession.java:1273)
at oracle.sysman.oii.oiic.OiicSessionWrapper.doOperation(OiicSessionWrapper.java:289)
at oracle.sysman.oii.oiic.OiicInstaller.run(OiicInstaller.java:546)
at oracle.sysman.oii.oiic.OiicInstaller.runInstaller(OiicInstaller.java:929)
at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: PRKC-1031 : Error checking free space for /data1 on racnode3
df: (/data1 ) ????????????????????????????????
at oracle.ops.mgmt.cluster.ClusterCmd.getFreeSpace(ClusterCmd.java:1697)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.getFreeSpaceOnNode(OiipgClusterOps.java:2218)
... 17 more
解决了空间问题后，图形安装工具自动解除了错误。
第二个问题是在运行root.sh脚本的过程中，安装结束后，需要在新节点上运行root.sh，不过运行出现了下面的错误：
# . /data/oracle/product/10.2/crs/root.sh
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured
Oracle CRS stack is already configured and will be running under init(1M)
检查后发现，是由于手工拷贝Oracle目录的时候，将/var/opt/oracle目录也拷贝过来，导致Oracle检查ocr.loc发现已经配置过CRS了，删除ocr.loc后，再次运行root.sh脚本，恢复正常。
# . /data/oracle/product/10.2/crs/root.sh
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured
OCR LOCATIONS = /dev/rac/ocr
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node
node 1: racnode1 racnode1-priv racnode1
node 2: racnode2 racnode2-priv racnode2
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
racnode1
racnode2
racnode3
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
IP 地址 "racnode1-vip" 已被使用。请输入一个未使用的 IP 地址。
不过这里又出现了第三个错误。由于以前存在的实例现在处于启动状态，VIP已经成功分配，所以这里出现了这个问题，下面用oracle用户切换为root权限，启动vipca进行配置：
$ su root
Password:
# ./vipca
成功配置后，问题解决。
随后碰到的错误仍然和COPY目录有关，由于手工将/data/oracle/oraInventroy目录拷贝到了目标服务器上，因此无论是cluster软件还是database软件安装过程的attach操作都会报错：
INFO: Invoking OUI on cluster nodes racnode3
INFO: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clu
sterparam.ini -silent -ignoreSysPrereqs -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=Or
aCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
-remoteInvocation -invokingNodeName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error occured while trying to run Unix command /data1/data/oracle/prod
uct/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clusterparam.ini -silent -ignor
eSysPrereqs -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=OraCrs10g_home1 CLUSTER_NODES=r
acnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3 -remoteInvocation -invokingNo
deName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM on nodes racnode3. [/usr/bin/sh: /dat
a1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?????? :failed]
.
.
.
at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: /usr/bin/sh: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?
????? :failed
at oracle.ops.mgmt.cluster.ClusterCmd.runCmd(ClusterCmd.java:1959)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:256)
... 13 more
SEVERE: Remote 'AttachHome' failed on nodes: 'racnode3'. Refer to '/data/oracle/oraInventory/logs/addNodeActions2009-01-30_01-01-56A
M.log' for details.
You can manually re-run the following command on the failed nodes after the installation:
/data/oracle/product/10.2/crs/oui/bin/runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_H
OME_NAME=OraCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NOD
E=
DATABASE软件的安装过程碰到同样的问题，如果手工执行命令错误如下：
$ runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=OraCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3正在启动 Oracle Universal Installer...
正在检查安装程序要求...
检查操作系统版本: 必须是5.8, 5.9 or 5.10。 实际为 5.10
通过
检查临时空间: 必须大于 250 MB。 实际为 4149 MB 通过检查交换空间: 必须大于 500 MB。 实际为 4289 MB 通过
所有安装程序要求均已满足。
OUI-10197: 无法在 /data/oracle/product/10.2/crs 创建新的 Oracle 主目录。此位置已存在 Oracle 主目录。请选择其它位置。
bash-3.00$ pwd
/data/oracle/product/10.2/crs/oui/bin
bash-3.00$ cd ../../../database/oui/bin/
bash-3.00$ runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database ORACLE_HOME_NAME=OraDb10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3正在启动 Oracle Universal Installer...
在 oraparam.ini 中未找到先决条件检查, 不执行系统先决条件检查。
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
OUI-10197: 无法在 /data/oracle/product/10.2/database 创建新的 Oracle 主目录。此位置已存在 Oracle 主目录。请选择其它位置。
'AttachHome' 失败。
这是由于拷贝的oraInvertory目录已经包含了CLUSTER和ORACLE软件信息，因此添加会失败，这里简单的忽略这个错误即可。
随后出现了UPDATENODELIST的错误，情况类似：
INFO: Invoking OUI on cluster nodes racnode1
INFO: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clu
sterparam.ini -silent -ignoreSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=r
acnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1 -remoteInvocation -invokingNo
deName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error occured while trying to run Unix command /data1/data/oracle/prod
uct/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clusterparam.ini -silent -ignor
eSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS
=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1 -remoteInvocation -invokingNodeName racnode2 -logFilePath
"/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM on nodes racnode1. [Starting Oracle Universal Installer...Checking
installer requirements...Checking operating system version: must be 5.6, 5.7, 5.8 or 5.9. Actual 5.8
PassedChecking Temp space: must be greater than 250 MB. Actual 8591 MB PassedChecking swap space: must be greater tha
n 500 MB. Actual 11377 MB PassedAll installer requirements met.Checking operating system patches: 108652-66,108921-16,108940-53
,108773-18,111310-01,109147-24,111308-03,111111-03,112396-02,110386-03,111023-02,108987-13,108528-21,108989-02,108993-18Checking for
patch 108652-66; found 108652-98 obsoleted Checking for patch 108921-16; found Checking for patch 108940-53; found Checking for pat
ch 108773-18; found Checking for patch 111310-01; found Checking for patch 109147-24; found Checking for patch 111308-03; found Chec
king for patch 111111-03; found Checking for patch 112396-02; found Checking for patch 110386-03; found Checking for patch 111023-02
; found Checking for patch 108987-13; found Checking for patch 108528-21; found Checking for patch 108989-02; found Checking for pat
ch 108993-18; found Passed'UpdateNodeList' failed. :failed]
.
.
.
at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: Starting Oracle Universal Installer...Checking installer requirements...Checkin
g operating system version: must be 5.6, 5.7, 5.8 or 5.9. Actual 5.8 PassedChecking Temp spa
ce: must be greater than 250 MB. Actual 8591 MB PassedChecking swap space: must be greater than 500 MB. Actual 11377 MB Pa
ssedAll installer requirements met.Checking operating system patches: 108652-66,108921-16,108940-53,108773-18,111310-01,109147-24,11
1308-03,111111-03,112396-02,110386-03,111023-02,108987-13,108528-21,108989-02,108993-18Checking for patch 108652-66; found 108652-98
obsoleted Checking for patch 108921-16; found Checking for patch 108940-53; found Checking for patch 108773-18; found Checking for
patch 111310-01; found Checking for patch 109147-24; found Checking for patch 111308-03; found Checking for patch 111111-03; found C
hecking for patch 112396-02; found Checking for patch 110386-03; found Checking for patch 111023-02; found Checking for patch 108987
-13; found Checking for patch 108528-21; found Checking for patch 108989-02; found Checking for patch 108993-18; found Passed'Up
dateNodeList' failed. :failed
at oracle.ops.mgmt.cluster.ClusterCmd.runCmd(ClusterCmd.java:1959)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:256)
... 13 more
INFO: Running command '/data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/c
rs/oui/bin/../clusterparam.ini -silent -ignoreSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/cr
s CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3 -remoteInvoc
ation -invokingNodeName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM' on the nodes 'racno
de3'.
INFO: Invoking OUI on cluster nodes racnode3
INFO: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clu
sterparam.ini -silent -ignoreSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=r
acnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3 -remoteInvocation -invokingNo
deName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error occured while trying to run Unix command /data1/data/oracle/prod
uct/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clusterparam.ini -silent -ignor
eSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS
=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3 -remoteInvocation -invokingNodeName racnode2 -logFilePath
"/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM on nodes racnode3. [/usr/bin/sh: /data1/data/oracle/product/10.2/c
rs/oui/bin/../bin/runInstaller: ?????? :failed]
at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:262)
.
.
.
at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: /usr/bin/sh: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?
????? :failed
at oracle.ops.mgmt.cluster.ClusterCmd.runCmd(ClusterCmd.java:1959)
at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:256)
... 13 more
SEVERE: Remote 'UpdateNodeList' failed on nodes: 'racnode1,racnode3'. Refer to '/data/oracle/oraInventory/logs/addNodeActions2009-01
-30_01-01-56AM.log' for details.
You can manually re-run the following command on the failed nodes after the installation:
/data/oracle/product/10.2/crs/oui/bin/runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUS
TER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=
INFO: User Selected: Yes/OK
无论是数据库还是CLUSTER的UPDATENODELIST操作也都失败了，不过随后无论是在racnode1上，还是在racnode3上指向的updatenodelist操作都成功了：
$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1
Starting Oracle Universal Installer...
Checking installer requirements...
Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.8
Passed
Checking Temp space: must be greater than 150 MB. Actual 8639 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
All installer requirements met.
'UpdateNodeList' was successful.
下面是racnode3上分别执行cluster和database的updatenodelist操作：
$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3正在启动 Oracle Universal Installer...
正在检查安装程序要求...
检查操作系统版本: 必须是5.8, 5.9 or 5.10。 实际为 5.10
通过
检查临时空间: 必须大于 250 MB。 实际为 4149 MB 通过检查交换空间: 必须大于 500 MB。 实际为 4289 MB 通过检查监视器: 监视器配置至少必须显示 256 种颜色。 实际为 16777216 通过
所有安装程序要求均已满足。
'UpdateNodeList' 成功。
$ /runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=false "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
bash: /runInstaller: 无此文件或目录
bash-3.00$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=false "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3正在启动 Oracle Universal Installer...
在 oraparam.ini 中未找到先决条件检查, 不执行系统先决条件检查。
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
'UpdateNodeList' 成功。
通过手工运行，UPDATENODELIST操作成功执行。
首先碰到一个错误，当新实例ASM实例启动后，在V$ASM_DISK中仅能查询到一个磁盘，而从DBCA的状况看，3个实例只有2个MOUNT到磁盘组。
这个问题以前碰到多，应该是缺少权限造成的：
# cd /dev/rdsk/
# chown oracle:oinstall c1t500601603022*
授权后：
bash-3.00$ export ORACLE_SID=+ASM3
bash-3.00$ sqlplus "/ as sysdba"
SQL*Plus: Release 10.2.0.3.0 - Production on 星期四 4月 2 15:07:23 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
连接到:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
SQL> select count(*) from v$asm_disk;
COUNT(*)
----------
45
在使用dbca添加实例的时候碰到几个问题，所有的问题都是在添加实例9个步骤中的第3个步骤：Select an active cluster database to add on instance。
首先需要确保数据库实例处于启动状态，否则dbca在添加实例的时候会报错。
其次应该保证所有节点的LISTENER处于启动状态，否则dbca添加实例的界面点击NEXT后不会有任何的反应。
即使LISTENER全部启动也是不够的，必须确保LISTENER检测每个节点的PUBLIC IP。对于以及运行的RAC环境而言，LISTENER一般监听的是每个节点的VIP，而如果没有添加PUBLIC IP，那么dbca就会报错，说明监听的对象racnode1:1521:testrac1不存在。
需要手工在所有的节点上添加对PUBLIC IP的监听设置，比如节点1的LISTENER.ORA设置如下：
bash-2.03$ more listener.ora
# listener.ora.racnode1 Network Configuration File: /data/oracle/product/10.2/database/network/admin/listener.ora.racnode1
# Generated by Oracle configuration tools.
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /data/oracle/product/10.2/database)
(PROGRAM = extproc)
)
(SID_DESC =
(GLOBAL_DBNAME = testrac)
(ORACLE_HOME = /data/oracle/product/10.2/database)
(SID_NAME = testrac1)
)
)
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 172.25.198.224)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = 172.25.198.222)(PORT = 1521))
)
(DESCRIPTION =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
)
)
随后重启监听：
bash-2.03$ lsnrctl stop
LSNRCTL for Solaris: Version 10.2.0.3.0 - Production on 01-2月 -2009 00:35:54
Copyright (c) 1991, 2006, Oracle. All rights reserved.
正在连接到 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.25.198.224)(PORT=1521)))命令执行成功
bash-2.03$ lsnrctl start
LSNRCTL for Solaris: Version 10.2.0.3.0 - Production on 01-2月 -2009 00:36:02
Copyright (c) 1991, 2006, Oracle. All rights reserved.
启动/data/oracle/product/10.2/database/bin/tnslsnr: 请稍候...
TNSLSNR for Solaris: Version 10.2.0.3.0 - Production系统参数文件为/data/oracle/product/10.2/database/network/admin/listener.ora写入/data/oracle/product/10.2/database/network/log/listener.log的日志信息监听: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.224)(PORT=1521)))监听: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.222)(PORT=1521)))监听: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))
正在连接到 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=172.25.198.224)(PORT=1521)))
LISTENER 的 STATUS
------------------------别名 LISTENER版本 TNSLSNR for Solaris: Version 10.2.0.3.0 - Production启动日期 01-2月 -2009 00:36:02正常运行时间 0 天 0 小时 0 分 0 秒跟踪级别 off安全性 ON: Local OS Authentication
SNMP OFF监听程序参数文件 /data/oracle/product/10.2/database/network/admin/listener.ora监听程序日志文件 /data/oracle/product/10.2/database/network/log/listener.log监听端点概要...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.224)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=172.25.198.222)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC0)))服务摘要..服务 "PLSExtProc" 包含 1 个例程。
例程 "PLSExtProc", 状态 UNKNOWN, 包含此服务的 1 个处理程序...服务 "testrac" 包含 1 个例程。
例程 "testrac1", 状态 UNKNOWN, 包含此服务的 1 个处理程序...命令执行成功
每个节点执行此操作后，再次点击next，终于可以到达下一个步骤。
最后一个问题在于DBCA对于ASM实例的处理还存在问题。由于使用了ASM，自然也就使用了OMF特性。于是新增的实例所需要添加的UNDOTBS3表空间，以及THREAD3对应的REDO.LOG文件，都采用了OMF方式。如果说DBCA采用了OMF而不运行用户提供存储位置，那么它能在正确的位置存储文件也可以，可惜DBCA存在问题，当最后运行的时候报错：Failed to create directory “” on “racnode3”, “PRKC-1055: Directory name passed was null”. PRKC-1055: Directory name passed was null。
所以只能采用提前建立UNDOTBS3表空间，以及THREAD3对应的REDO.LOG的方法：
SQL> select group#, thread#, sequence#, bytes/1024/1024
2 from v$log;
GROUP# THREAD# SEQUENCE# BYTES/1024/1024
---------- ---------- ---------- ---------------
1 1 1716 512
2 1 1717 512
3 2 839 512
4 2 840 512
SQL> select member from v$logfile;
MEMBER
---------------------------------------------------
+DISK/testrac/onlinelog/group_1.258.633485433
+DISK/testrac/onlinelog/group_2.260.633485441
+DISK/testrac/onlinelog/group_3.268.633485447
+DISK/testrac/onlinelog/group_4.270.633485455
下面手工添加THREAD 3对应的REDO.LOG：
SQL> alter database add logfile thread 3 group 5 '+DISK/testrac/onlinelog/redo5.log' size 512m;
数据库已更改。
SQL> alter database add logfile thread 3 group 6 '+DISK/testrac/onlinelog/redo6.log' size 512m;
数据库已更改。
SQL> select group#, thread#, sequence#, bytes/1024/1024
2 from v$log;
GROUP# THREAD# SEQUENCE# BYTES/1024/1024
---------- ---------- ---------- ---------------
1 1 1716 512
2 1 1717 512
3 2 839 512
4 2 840 512
5 3 0 512
6 3 0 512
已选择6行。
SQL> select member from v$logfile;
MEMBER
----------------------------------------------------------------
+DISK/testrac/onlinelog/group_1.258.633485433
+DISK/testrac/onlinelog/group_2.260.633485441
+DISK/testrac/onlinelog/group_3.268.633485447
+DISK/testrac/onlinelog/group_4.270.633485455
+DISK/testrac/onlinelog/redo5.log
+DISK/testrac/onlinelog/redo6.log
已选择6行。
接着添加UNDOTBS3表空间：
SQL> select file_name from dba_data_files
2 where tablespace_name like 'UNDOTBS_';
FILE_NAME
----------------------------------------------------------------------
+DISK/testrac/datafile/undotbs1.263.618591197
+DISK/testrac/datafile/undotbs2.266.618591249
SQL> create undo tablespace undotbs3
2 datafile '+DISK/testrac/datafile/undotbs3.dbf' size 4096m;
表空间已创建。
随后使用DBCA进行添加节点，不过在最后一步添加表空间和REDO的时候，需要手工将表空间和REDO.LOG删除。不过根据DBCA提供的汇总显示外，仍然存在问题，DBCA认为新节点对应的THREAD为1，而认为对应的UNDO表空间是UNDOTBS4。点击确定后，仍然报错。
看来只能通过手工的方法来新增实例了。具体方法在第四篇文章中已经描述过了，这里就不在重复了。
随后在启动新增实例的时候碰到了一个错误：
bash-3.00$ sqlplus "/ as sysdba"
SQL*Plus: Release 10.2.0.3.0 - Production on 星期五 4月 3 14:53:23 2009
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
已连接到空闲例程。
SQL> startup
ORACLE 例程已经启动。
Total System Global Area 2147483648 bytes
Fixed Size 2031480 bytes
Variable Size 503316616 bytes
Database Buffers 1627389952 bytes
Redo Buffers 14745600 bytes
ORA-01618: ???? 3 ??? - ????
这时由于没有启用对应的THREAD。登陆实例1的节点，ENABLE THREAD 3：
SQL> alter database enable thread 3;
数据库已更改。
重启实例3问题消失。
