veritas VVR介绍与使用

veritas vvr

2011-03-27 18:31:20|  分类： varitas |  标签： |字号大中小

随着全球信息化时代的到来，信息数据越来越成为企业关注的焦点，即在各种自然灾害或者人为破坏的情况下，如何保证生产数据的安全和关键业务的不中断运行。传统的保护方法离线备份、备份介质异地保存在一定程度上可以解决上述问题，但为了能够对业务系统提供更高的实时性和可靠性，保证关键业务7x24不中断，应对激烈的市场竞争和提高客户满意度，企业必须在IT系统围绕“连续”（BC）主题进行构建，实施业务连续/容灾备份计划，包括业务连续性、高可用性管理、容灾、数据复制和恢复方案、安全等，下面就对数据复制技术做一个简单的介绍。

根据数据复制运行的位置，可以将数据复制方式分为以下四种：

基于存储(Storage-Based)：代表厂商有EMC SRDF、IBM PPRC/GDPS 、HP BusinessCopy、HDS TrueCopy等，特点对主机透明，对应用系统的影响较小，技术成熟，有较多的成功案例，但是投资较大，对网络连接的要求也较高。

基于服务器（Host-based）：代表厂商主要是Symantec公司的Veritas Volume Replicator (VVR)，是存储卷的复制技术。需要占用主机一定的系统资源。
基于存储交换机(SAN-Based)：运行于挂接在存储网络上的专用设备中，代表厂商有CISCO的SANTap，FalconStor IPStor，Topio SANsafe，DataCore SANsymphony, Asynchronous Internet Mirroring (AIM)等。

基于数据库/软件应用 :代表厂商有Oracle DataGuard，Quest SharePlex，IBM-DB2的远程Q复制，SYBASE的复制服务器(Replication Server) 等，这些技术都是对数据逻辑操作的复制，属于逻辑数据复制，而其它3种复制的是数据卷中的数据变化，属于物理数据复制。典型的逻辑数据复制是通过数据库日志实现的，优点是硬件投资小，对生产系统的影响比较小，对主、备用系统的硬件平台没有选择性；可以在广域网范围内使用。但它对数据库系统有严格的要求，并且主、备用系统必须使用相同的数据库版本。

上面介绍的主要的各厂商方案和产品可以参见下图：

本文主要介绍基于主机和逻辑卷的数据复制VVR。

VERITAS的VVR（Veritas Volume Replicator）是用于有效的灾难恢复的数据复制软件，可以连续对应用数据进行一个或者多个的拷贝保护。具有的优点主要有：高性能，距离不限，支持异构的存储和操作系统，对数据的连续保护，支持同步和异步复制。

VVR系统架构如下图：   

VVR具有的系统特性如下：

(1) 最高支持32个远程节点，可以通过基于IP的LAN或者WAN网络；

(2) 通过对逻辑卷的同步和异步复制模式，保证数据的完整性和一致性；

(3) 确保备用节点和主节点的数据具有相同的写入顺序；

(4) 支持对网络带宽的管理和定义，支持多链路的连接；

(5) 支持对备用节点的脱机操作，以便在备用节点进行查询（例如DSS决策支持系统）或者数据备份；

(6) 易管理：同时支持命令行和图形界面（VEA）；

(7) 主备节点之间支持多种异构存储；

(8) 支持不同操作系统平台之间的复制；（注：在本文中，因为设计到Oracle数据库的复制，所以主备操作系统必须同一类型，比如必须都是Solaris平台，版本完全相同的Oracle版本）

(9) 支持主流关系数据库系统，包括Oracle,DB2,Sybase和Informix，数据文件的存放格式包括逻辑卷（裸设备）或者文件系统；

(10) 支持共享的集群环境中的逻辑卷复制，比如Oracle RAC (Real Application Clusters) (注：本文中采用的是单节点Oracle复制)；

(11) 支持Veritas volume set；

(12) 支持PDC (Portable Data Container)环境的数据复制；

VVR数据复制系统主要由下面五个组件组成：

■ Replicated Volume Group (RVG) 复制卷组

RVG是位于VxVM 磁盘组(DG)内的一组逻辑卷，RVG隶属于磁盘组的一部分，通过定义这些逻辑卷，
应用对逻辑卷的写操作都可以保持以主节点上同样的顺序，来写到备用节点上。

对于多数数据库系统来说，都是由一些特定的进程完成对磁盘的写操作，比如Oracle系统中的LOWR
和DBWR进程，而这些写操作总是依照着一定顺序，从而来保证实例恢复中数据的一致和完整。比如
数据库变化通常在写入日志文件后才会写入数据文件，所以当日志文件和数据文件属于要复制的不同
逻辑卷的时候，它们必须被定义到同一组RVG中。

每组RVG最多可以包括2048个数据卷。

下面介绍SRL和RLINK，它们也包括在RVG中。

■ Storage Replicator Log (SRL)
SRL用于记录RVG中数据写操作的日志，并循环被使用。每个RVG包括一个SRL。数据在写入RVG中的卷之前，首先要写到主节点的SRL中。

从VxVM的角度来看，SRL是一个单独的卷，因为所有的写操作首先要写到SRL中，这个卷需要有较好的写性能，所以多数情况下需要对这个卷进行条带化，并且在不同的盘上mirror以保护数据，同时数据卷和SRL卷最好分布在不同的物理磁盘上以得到更好的性能。

不允许人工直接对SRL进行访问。

■ Replication Link—RLINK

用于连接主RVG和备用RVG之间的一个“链接”，一个主的RVG最多可以有32个RLINK。可以定义RLINK
的多个属性来调整数据复制的参数，比如指定复制的网络。

■ Data Change Map (DCM)

■ Replicated Data Set (RDS)

一个RDS包括了所有的主节点和备用节点的复制卷组(RVG)。RDS不是一个具体存在的对象，而是一个VVR复制技术中的一个概念。

在异步模式下VVR的数据复制和数据流向  

在同步模式下VVR的数据复制和数据流向

另外，VVR是Veritas Volume Manager (VxVM)的一个单独的option，需要单独的license。

Veritas Volume Replicator 的安装
主备节点硬件和软件配置和版本如下：

主节点 机器名：lxsfrac01

Sun Microsystems Sun Fire X4100 Server

操作系统 solaris10u4 x86_64

VVR版本: Veritas Volume Replicator 5.0 GA

Oracle 版本： Oracle 10gR2 Enterprise Edition

磁盘阵列：SAN/ HDS AMS 1000

Public网卡：e1000g0 私有(复制)网卡：e1000g1

备用节点机器名：lxsfrac02

软硬件环境与主节点相同

两个节点分别安装VVR和vxfs，安装方法相同，以lxsfrac02为例：

放入安装介质，一般Solaris都由vold进程自动mount CD，

如果需要手工mount：mount –F hsfs –o ro /dev/dsk/c0t6d0s2 /cdrom

# cd ./volume_replicator/
root@lxsfrac02 # ls
copyright docs installvvr pkgs release_notes scripts tools uninstallvvr
root@lxsfrac02 # ls -l
total 24
-rw-r--r-- 1 root root 476 Dec 15 2006 copyright
drwxrwxr-x 2 root root 512 Jun 21 10:13 docs
-rwxr-xr-x 1 root root 2283 Apr 14 2007 installvvr
drwxrwxr-x 3 root root 1024 Jun 21 10:13 pkgs
drwxrwxr-x 2 root root 512 Jun 21 10:13 release_notes
drwxrwxr-x 3 root root 512 Jun 21 10:13 scripts
drwxrwxr-x 3 root root 512 Jun 21 10:13 tools
-rwxr-xr-x 1 root root 2283 Apr 14 2007 uninstallvvr
root@lxsfrac02 # ./installvvr lxsfrac02
Veritas Volume Replicator 5.0 Installation Program
Copyright (c) 2007 Symantec Corporation. All rights reserved. Symantec, the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202.
Logs for installvvr are being created in /var/tmp/installvvr-mrk46Y.
Initial system check:
Checking VVR installation on lxsfrac02 ............................................ not installed
Checking architecture on lxsfrac02 ................................................ i386
Veritas Volume Replicator 5.0 Installation Program
Checking system licensing
Installing licensing packages
VVR is not licensed on lxsfrac02
Enter a VVR license key for lxsfrac02: ******************
****************** successfully registered on lxsfrac02
Permanent VVR license registered on lxsfrac02
Do you want to enter another license key for lxsfrac02? [y,n,q] (n)
Veritas Volume Replicator 5.0 Installation Program
Checking installed packages on lxsfrac02
Veritas Volume Replicator 5.0 Installation Program
VVR can be installed without optional packages to conserve disk space.
Additional packages are typically installed to simplify future upgrades.
1) Required Veritas Volume Replicator packages - 855 MB required
2) All Veritas Volume Replicator packages - 861 MB required
3) Storage Foundation Enterprise packages - 911 MB required
Select the packages to be installed on all systems? [1-3,q,?] (3) 2
Veritas Volume Replicator 5.0 Installation Program
The following VVR packages will be installed:
VRTSperl Veritas Perl 5.8.8 Redistribution
VRTSvlic Veritas Licensing
VRTSicsco Symantec Common Infrastructure
VRTSpbx Symantec Private Branch Exchange
VRTSsmf Symantec Service Management Framework
VRTSat Symantec Product Authentication Service
VRTSobc33 Veritas Enterprise Administrator Core Service
VRTSob Veritas Enterprise Administrator Service
VRTSobgui Veritas Enterprise Administrator
VRTSccg Veritas Enterprise Administrator Central Control Grid
VRTSmh Veritas Storage Foundation Managed Host by Symantec
VRTSaa Veritas Enterprise Administrator Action Agent
VRTSspt Veritas Software Support Tools
SYMClma Symantec License Inventory Agent
VRTSvxvm Veritas Volume Manager Binaries
VRTSdsa Veritas Datacenter Storage Agent
VRTSfspro Veritas File System Management Services Provider
VRTSvmman Veritas Volume Manager Manual Pages
VRTSvmdoc Veritas Volume Manager Documentation
Press [Return] to continue:
...continued:
VRTSdcli Veritas Distributed Command Line Interface
VRTSalloc Veritas Volume Manager Intelligent Storage Provisioning
VRTSvmpro Veritas Volume Manager Management Services Provider
VRTSddlpr Veritas Device Discovery Layer Services Provider
VRTSvrpro Veritas Volume Replicator Client Extension and Provider for
Veritas Enterprise Administrator
VRTSvcsvr Veritas Cluster Server Agents for VVR
VRTSjre Veritas Java Runtime Environment Redistribution
VRTSjre15 Veritas Java Runtime Environment Redistribution
VRTSweb Veritas Java Web Server
VRTSvrw Veritas Volume Replicator Web Console
VRTSvrdoc Veritas Volume Replicator Documentation
Press [Return] to continue:
Checking for any AP driver issues on lxsfrac02 ................................... None
A copy of a previous VxVM configuration is present on the system lxsfrac02.
Do you want to restore and reuse the previous VxVM configuration of system lxsfrac02 [y,n,q,?] (y) n
Veritas Volume Replicator 5.0 Installation Program
Installing VVR: 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Installation completed successfully on all systems
It is optional to configure VVR now. If you choose to configure VVR later, you can either do so manually, or run the installvvr -configure command. The product installation scripts can be found in /opt/VRTS/install directory
Are you ready to configure VVR? [y,n,q] (y)
The enclosure-based naming scheme is a feature of Veritas Volume Manager. It allows one to reference disks using a symbolic name that is more meaningful than the operating system's normal device access name. This symbolic name is typically derived from the array name.
Do you want to set up the enclosure-based naming scheme? [y,n,q,?] (n) y
Veritas Volume Replicator 5.0 Installation Program
Veritas Volume Manager default disk group name configuration:
Many Veritas Volume Manager commands affect the contents or configuration of a disk group. Such commands require that the user specify a disk group. This is accomplished by using the -g option of a command or setting the VXVM_DEFAULTDG environment variable. An alternative to these two methods for some commands is to configure the name of the default disk group of a system.
Note: The default disk group feature is not available with the DCLI vxadm(1M) and vxquery(1M) commands. When using these commands, a required disk group must be explicitly identified using the -g option.
Do you want to set up a default disk group for each system? [y,n,q,?] (y) n
Veritas Volume Replicator 5.0 Installation Program
Following are the default ports that will be used by VVR on lxsfrac02 :
Port for VVR heartbeat communication: 4145
Port for vradmind daemon: 8199
Port for vxrsyncd daemon: 8989
Ports for data replication: Anonymous-Ports
The ports indicated for VVR heartbeats and vradmind are IANA assigned.
The port settings should be identical for systems that will be part of the same Replicated Data Set. They should also be identical for all the systems in a cluster.
Do you want to change any of the VVR ports on lxsfrac02? [y,n,q] (n)
Veritas Volume Replicator 5.0 Installation Program
VVR Statistics Collection Tool
The VVR administrative daemon vradmind can collect and maintain various
statistics which are helpful in solving VVR performance issues. This
collection can be tuned using a few tunables like the frequency for
gathering the statistics and the number of days for which the collected
statistics should be preserved, after which the earlier statistics are
automatically deleted.
The frequency of VVR statistics collection on lxsfrac02 is set to per 10 seconds.
Do you want to change the frequency of VVR statistics collection on lxsfrac02 ? [y,n,q] (n)
The maximum number of days for which VVR statistics can be retained is set to 3 on lxsfrac02
Do you want to change the maximum number of days for retaining VVR statistics on lxsfrac02 ? [y,n,q] (n)
Veritas Volume Replicator 5.0 Installation Program
Configuration of VVR tunables (For the advanced user)
Do you want to view or modify VVR tunables on lxsfrac02 ? [y,n,q,?] (n)
Veritas Volume Replicator 5.0 Installation Program
Verify the install systems Fully Qualified Hostnames.
Querying fully qualified domain name of host "lxsfrac02" ............................. fail
Press [Return] to continue:
Unable to find FQHN for lxsfrac02!
Enter the fully qualified hostname of "lxsfrac02" or 'QUIT' to quit install.
Enter the fully qualified host name: lxsfrac02
Veritas Volume Replicator 5.0 Installation Program
Choose how this install will be managed.
This product can configured one of the two following ways:
* Storage Foundation Management Server managed host (Recommended)
* Standalone host
Pre-requisites for Storage Foundation Management Server managed host:
* A Storage Foundation Management Server (SFMS) is configured
* SFMS should be currently running and fully operational.
* Authentication Broker that is used by SFMS should be running.
The following information would be requested during configuration:
* The host name of the SFMS
* Previously set SFMS agent account password
There are no pre-requisites for deploying as a standalone host.
Enabling Storage Foundation Management Server management simplifies and improves management of the complex data center
resources, reducing planned and unplanned down time.
Enable Storage Foundation Management Server Management? [y,n,q] (y) n
Veritas Volume Replicator 5.0 Installation Program
Verify the install systems Fully Qualified Hostnames.
Configuring VEA in STANDALONE mode on "lxsfrac02" ..................................... ok
Configuring gridnode on "lxsfrac02" ................................................... ok
Registering gridnode on "lxsfrac02" ................................................... ok
Configuring actionagent on "lxsfrac02" ................................................ ok
Registering actionagent on "lxsfrac02" ................................................ ok
Registering StorageAgent on "lxsfrac02" ............................................... ok
Do you want to start Veritas Volume Replicator processes now? [y,n,q] (y)
Veritas Volume Replicator 5.0 Installation Program
Starting VVR: 100% ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Startup completed successfully on all systems
You declined to set up the name of the default disk group for lxsfrac02.
Installation log files, summary file, and response file are saved at:
/opt/VRTS/install/logs/installvvr-mrk46Y
To enable Dynamic Multi-Pathing (DMP), MPxIO must be disabled. Refer to the Storage Foundation Release Notes for further details on disabling MPxIO.
检查vradmin deamon：

# ps -ef|grep vrad|grep -v grep
root 9808 1 0 16:24:47 ? 0:00 /usr/sbin/vradmind
安装Veritas File System：安装过程略

# cd file_system/
root@lxsfrac02 # ls -l
total 22
-rw-r--r-- 1 root root 476 Dec 15 2006 copyright
drwxrwxr-x 2 root root 512 Jun 21 10:12 docs
-rwxr-xr-x 1 root root 2283 Apr 14 2007 installfs
drwxrwxr-x 3 root root 1024 Jun 21 10:12 pkgs
drwxrwxr-x 2 root root 512 Jun 21 10:12 release_notes
drwxrwxr-x 3 root root 512 Jun 21 10:12 scripts
-rwxr-xr-x 1 root root 2283 Apr 14 2007 uninstallfs
root@lxsfrac02 # ./installfs lxsfrac02

Veritas Volume Replicator VVR的配置

1. 两个节点上都需要做：

修改两个节点的/etc/hosts

10.198.90.122 lxsfrac01
10.198.90.139 lxsfrac02
192.168.0.122 lxsfrac01-priv
192.168.0.139 lxsfrac02-priv
分别创建私有网络(专有)，建议数据复制使用专用网络。

# ifconfig e1000g1 plumb 192.168.0.122 netmask 255.255.255.0 up
root@lxsfrac02 # ifconfig e1000g1 plumb 192.168.0.139 netmask 255.255.255.0 up
root@lxsfrac01 # ping 192.168.0.139
192.168.0.139 is alive
创建相同名称的磁盘组（含有最少两个磁盘），相同名称数据卷oravol，创建复制日志卷(SRL)，分别在本地安装Oracle 10gR2.(安装Oracle过程略。)

# vxdg init oradg AMS_WMS0_2 AMS_WMS0_3
root@lxsfrac01 # vxassist -g oradg make oravol 10g
root@lxsfrac01 # vxedit -g oradg set user=oracle group=dba mode=644 oravol
root@lxsfrac01 # vxassist -g oradg make srl 800m
2．在备用节点上修改配置文件/etc/vx/vras/.rdg，让其包含主节点上的disk group ID （这里是1193985774.38.lxsfrac01）。当创建备用RVG时，这个文件用来对主节点的授权。

可以通过vxprint –l来得到disk group ID

root@lxsfrac02 # vi /etc/vx/vras/.rdg
………
1193985774.38.lxsfrac01
"/etc/vx/vras/.rdg" 43 lines, 1544 characters
3．在主节点上：

创建vxfs文件系统

mount -F vxfs -o largefiles /dev/vx/dsk/oradg/oravol /oracle
chown oracle:dba /oracle
切换到oracle用户使用dbca建库，并且所有数据文件选择/oracle目录

创建Primary RVG：

root@lxsfrac01 # vradmin -g oradg createpri ora_rvg oravol srl

其中的oravol是要复制的数据卷列表，当多于一个时，用逗号隔开。

创建Secondary RVG：

# vradmin -g oradg addsec ora_rvg lxsfrac01-priv lxsfrac02-priv
设置复制方式为异步：

root@lxsfrac01 # vradmin -g oradg set ora_rvg lxsfrac02-priv synchronous=off
开始复制：

root@lxsfrac01 # vradmin -g oradg -a startrep ora_rvg
Message from Primary:
VxVM VVR vxrlink WARNING V-5-1-3359 Attaching rlink to non-empty rvg. Autosync will be performed.
VxVM VVR vxrlink INFO V-5-1-3614 Secondary data volumes detected with rvg ora_rvg as parent:
VxVM VVR vxrlink INFO V-5-1-6183 oravol: len=20971520 primary_datavol=oravol
VxVM VVR vxrlink INFO V-5-1-3365 Autosync operation has started
root@lxsfrac01 #
root@lxsfrac01 # vradmin -g oradg -f startrep ora_rvg
VxVM VVR vradmin INFO V-5-52-1203 Replication to Secondary host lxsfrac02-priv already started.
检查相关信息和状态，显示数据复制过程，可以通过使用-i参数指定interval

root@lxsfrac01 # vxrvg -g oradg getrlinks ora_rvg
rlk_lxsfrac02-priv_ora_rvg
root@lxsfrac01 # vxrlink -g oradg status rlk_lxsfrac02-priv_ora_rvg
Fri Nov 02 17:01:22 2007
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_lxsfrac02-priv_ora_rvg is in AUTOSYNC. 9888256 Kbytes remaining.
root@lxsfrac01 # vxrlink -g oradg status rlk_lxsfrac02-priv_ora_rvg
Fri Nov 02 17:02:08 2007
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_lxsfrac02-priv_ora_rvg is in AUTOSYNC. 9375488 Kbytes remaining.
root@lxsfrac01 # vxrlink -g oradg status rlk_lxsfrac02-priv_ora_rvg
Fri Nov 02 17:13:26 2007
VxVM VVR vxrlink INFO V-5-1-4464 Rlink rlk_lxsfrac02-priv_ora_rvg is in AUTOSYNC. 1901824 Kbytes remaining.
root@lxsfrac01 # vxrlink -g oradg status rlk_lxsfrac02-priv_ora_rvg
Fri Nov 02 17:19:16 2007
VxVM VVR vxrlink INFO V-5-1-4467 Rlink rlk_lxsfrac02-priv_ora_rvg is up to date
可以手工进行gracefully切换

root@lxsfrac01 # umount /oracle
root@lxsfrac01 # vradmin -g oradg migrate ora_rvg lxsfrac02-priv
VxVM VVR vradmin WARNING V-5-52-93 Make sure applications using Primary data volumes are stopped.
vradmin: Continue with migrate (y/n)? y
root@lxsfrac01 #