使用convirt集中管理kvm虚拟机

官方文档：

kvm中使用kickstart无法获取IP的问题：

http://blog.chinaunix.net/uid-1838361-id-3067686.html

首先，Convirt分为两部分CMS 即Convirt Manage Server,即管理控制端。

另外一部分为Managed Servers即被管理端。被管理端即运行虚拟机的物理服务器。

一、安装CMS（convirt manage server）

尝试在Ccentos5.7上，使用二进制安装的mysql上，安装部署Convirt 2.0.1 CMS，没有成功。

以下为在Centos6.2的虚拟机上安装的步骤。

按照官方文档安装即可。

1、添加convirt的yum源并安装socat。

wget --no-cache  -O /etc/yum.repos.d/convirt.repo

yum makecache

安装socat，socat的主要特点就是在两个数据流之间建立通道，且支持众多协议和链接方式：ip, tcp, udp, ipv6, pipe,exec,system,open,proxy,openssl,socket等

yum install socat

2、下载convirt相关程序

convirt-install：CMS端安装程序

wget --no-cache

convirt：CMS及managed server端安装程序

wget --no-cache

convirture-tools：managed server端安装程序

wget --no-cache

3、使用root用户安装依赖程序

tar -xzf convirt-install-2.0.1.tar.gz

cd convirt-install/install/cms/scripts/
sudo ./install_dependencies

注：建议不要在运行着mysql的服务器上安装cms，因为会rpm安装mysql，并执行/usr/bin/mysql_secure_installation该脚本的作用是加强mysql的安全性，会修改mysql的root密码，清除匿名用户等。如果服务器上已经运行了二进制的mysql，容易冲突及覆盖。

建议启动一个虚拟机来安装运行CMS。

安装过程中提示修改root密码，修改为"convirt"。稍后可以修改。

修改MySQL配置，在/etc/mysql/my.cnf文件的[mysqld]部分添加如下内容：

innodb_buffer_pool_size=1G
innodb_additional_mem_pool_size=20M

重启mysql。

4、安装CMS

默认情况下，将安装convirt到用户的home目录下，需要修改安装路径：

convirt-install/install/cms/scripts/install_config

修改CONVIRT_BASE为要安装的路径

如：

CONVIRT_BASE=/usr/local （不需要在后面添加/）

同时注意修改下面的CONVIRT_TARBALL和CONVIRT_DOWNLOAD_URL的convirt版本号。

*Untar the CMS tarball.

source convirt-install/install/cms/scripts/install_config
tar -xzf ./convirt-2.0.1.tar.gz -C $CONVIRT_BASE

*Setup TurboGears

./convirt-install/install/cms/scripts/setup_tg2

* Setup ConVirt

更改数据库相关信息

src/convirt/web/convirt/development.ini 修改 sqlalchemy.url

 For example in the following url, username = root, password = convirt, server=localhost, database port = 3306 

  and database name is convirt (the one before '?' in the url)

  

  sqlalchemy.url=mysql://root:convirt@localhost:3306/convirt?charset=utf8

注：后台收集的cpu、内存等信息都会保存到数据库中，默认为365天，数据量非常大，造成后期mysql查询很慢，磁盘IO很高，如果机器性能不好，应该修改下面的参数，来减少数据保存的时间：

purge_hr_data = 60
purge_day_data = 30
purge_week_data = 30
purge_month_data = 30
purge_raw_data = 30
task_results_purge_interval=30
TaskPaneLimit=7
task_panel_row_limit=200
notifications_row_limit=200

如何删除以前的数据？

因为我的cms的服务器性能不好，造成删除这些数据都很慢，所以我将数据库dump到一台性能较好的服务器上删除以后再导到cms服务器上。

 

delete from metrics where metric_type=1 and cdate < '2012-06-01 19:24:03';
delete from metrics where metric_type=4 and cdate < '2012-06-01 19:24:03';
delete from metrics where metric_type=7 and cdate < '2012-06-01 19:24:03';

delete from metrics_arch where rollup_type=1 and cdate < '2012-06-01 19:24:03';

DELETE FROM task_results WHERE task_results.task_id in (4,9,10) AND status = 3 and datediff(now() , task_results.timestamp) >= 15;

delete  FROM task_results WHERE task_id in (SELECT tasks.task_id AS tasks_task_id FROM tasks WHERE datediff(now() , tasks.submitted_on) >= 15 AND tasks.parent_task_id in (4,9,10)) and status=3;

delete FROM tasks WHERE datediff(now() , tasks.submitted_on) >= 15 AND tasks.parent_task_id in (4,9,10) and task_id not in (select task_id from task_results);

脚本：

#!/usr/bin/perl
use DBI;
use Date::Format;

my $del_nday_ago=15;
my $del_time=time2str('%Y-%m-%d %H:%d:%S',time - (86400*$del_nday_ago));

my $dbh=connect_mysql();

$dbh->do("delete from metrics where metric_type=1 and cdate < '$del_time';");
$dbh->do("delete from metrics where metric_type=4 and cdate < '$del_time';");
$dbh->do("delete from metrics where metric_type=7 and cdate < '$del_time';");


$dbh->do("delete from metrics_arch where rollup_type=1 and cdate < '$del_time';");

$dbh->do("DELETE FROM task_results WHERE task_results.task_id in (4,9,10) AND status = 3 and datediff(now() , task_results.timestamp
) >= $del_nday_ago;");

$dbh->do("delete FROM task_results WHERE task_id in (SELECT tasks.task_id AS tasks_task_id FROM tasks WHERE datediff(now() , tasks.s
ubmitted_on) >= $del_nday_ago AND tasks.parent_task_id in (4,9,10)) and status=3;");

$dbh->do("delete FROM tasks WHERE datediff(now() , tasks.submitted_on) >= $del_nday_ago AND tasks.parent_task_id in (4,9,10) and tas
k_id not in (select task_id from task_results);");

sub connect_mysql {
    my $host = "127.0.0.1";
    my $port = "3306";
    my $db = "convirt";
    my $user = 'root';
    my $pass = 'password';
    my $dbh = DBI->connect("DBI:mysql:database=$db:host=$host:port=$port",
                           $user, $pass, {"RaiseError" => 1,
                           "AutoCommit" => 1}) or die $!;
                           return $dbh;
                                            }

安装： 

./convirt-install/install/cms/scripts/setup_convirt

过程中会创建cms的ssh key，密码设置为空。

创建的key的文件名为：cms_id_rsa  cms_id_rsa.pub

5、启动CMS

/usr/local/convirt/convirt-ctl start

启动的端口为8081

iptables -I INPUT -p tcp --dport 8081 -j ACCEPT

登陆的初始用户名及密码为 admin/admin

6、VNC

需要为VNC打开6900:6999端口范围，用于convirt允许VNC applet访问虚拟机的concole。

登陆CMS服务器，添加：

 iptables -I INPUT -m state --state NEW -p tcp --dport 6900:6999 -j ACCEPT

二、安装被管理端managed server

安装KVM，对于centos/rhel 5.x，安装工具可以自动配置桥接网络，对于centos/rhel 6.x，需要手动配置。

建议无论centos/rhel 5.x还是6.x都手动配置桥接网络。

安装convirt-tool

For the KVM platform, the command creates appropriate public bridges, required scripts and writes a summary of its operations to the /var/cache/convirt/server_info file.

cd /usr/local/;wget --no-cache
tar -xzf convirture-tools-2.0.1.tar.gz
cd ./convirture-tools/install/managed_server/scripts

查看帮助：

  ./convirt-tool -h

验证platform而不做任何更改：

  ./convirt-tool --detect_only setup

安装所需的依赖

 ./convirt-tool install_dependencies

有可能出现的冲突

Transaction Check Error:

  package tunctl-1.5-3.el5.x86_64 (which is newer than tunctl-1.5-2.el5.i386) is already installed

  file /usr/share/man/man8/tunctl.8.gz from install of tunctl-1.5-2.el5.i386 conflicts with file from package tunctl-1.5-3.el5.x86_64

/usr/local/convirture-tools/common/scripts/RedHat_functions

install_dependencies

   #$SUDO yum -y install dnsmasq tunctl tcp_wrappers socat expect pexpect 

   $SUDO yum -y install dnsmasq  tcp_wrappers socat expect pexpect 

去掉其中的tunctl   

安装

注意，建议先手动配置桥接网络，下面的安装convirt-tool，并不配置桥接网络及防火墙策略。。

./convirt-tool --skip_bridge --skip_firewall setup

关于手动配置桥接网络的配置，可以查看：

或：

http://blog.chinaunix.net/space.php?uid=1838361&do=blog&id=257238

配置bridge network要特别注意，特别是通过远程配置，因为配置错误可能造成不能远程连接。

迁移涉及的端口：

Fox KVM : TCP ports 8002 to 8012 for migration

通过console添加managed server时：

ERROR [convirt.viewModel] [NodeService.py]:489 :EOL while scanning string literal (, line 3)

原因：

/etc/redhat-release文件中包含两行

造成managed server 在使用下面的方法获取系统信息时：

./common/scripts/functions:      CODE_NAME=`cat /etc/fedora-release | sed s/.*\(// | sed s/\)//`

得到类似这样的信息

查看：/var/cache/convirt/server_info

CODE_NAME="Tikanga

Final"

VER="5.7

5.7"

解决办法：

删除一行，重新执行：./convirt-tool --skip_bridge --skip_firewall setup 

添加 cms_id_rsa.pub到被管理server的authorized_keys中

如果出现Key Mismatch 

需要重启CMS服务

三、将现有libvirt管理的KVM虚拟机导入到convirt中。

将xml的配置文件转换成.cfg配置文件，工具：

batch_convert [-h] [] [

./batch_convert.sh /etc/libvirt/qemu/  .

转换完的格式：

kernel = '/usr/lib/xen/boot/hvmloader'
vif = ['mac=52:54:00:55:52:e9,bridge=br0', 'mac=52:54:00:5a:b7:d4,bridge=br1']
on_shutdown = 'destroy'
disk = ['qemu:raw:/usr/local/kvm/rhel6.1/disk.os,vda,w', 'qemu:raw:/usr/local/kvm/rhel6.1/disk.data,vdb,w']
vncunused = 1
apci = 0
uuid = 'dba78c52-a767-2496-742c-5215ffc284fa'
os_version = '1.0'
on_reboot = 'restart'
boot = 'c'
platform = 'kvm'
pae = 1
memory = 2614
acpi = 1
os_name = 'Linux'
vnc = 1
on_crash = 'destroy'
network_mode = 'tap'
bootloader = ''
arch = 'x86_64'
usb = 1
name = 'rhel6.1'
usbdevice = 'tablet'
device_model = '/usr/libexec/qemu-kvm'
builder = 'hvm'
template_version = '1.0'
vfb = ['type=vnc,vncunused=1']
vcpus = 4
os_flavor = 'Linux'

其中需要注意的是磁盘部分，需要修改：

disk = ['qemu:raw:/usr/local/kvm/rhel6.1/disk.os,vda,w', 'qemu:raw:/usr/local/kvm/rhel6.1/disk.data,vdb,w']

修改为：

disk=['file:/usr/local/kvm/rhel6.1/disk.os,/dev/vda,w','file:/usr/local/kvm/rhel6.1/disk.data,/dev/vdb,w']
drive=['file=/usr/local/kvm/rhel6.1/disk.os,if=virtio,format=raw,index=0,boot=on','file=/usr/local/kvm/rhel6.1/disk.data,if=virtio,format=raw,index=1']

找不到bootloader?试试磁盘名前加/dev/，

另外，对于pxe安装，报找不到bootloader的问题，应查看/tftpboot/pxelinux.cfg/default，确认default指向正确的项目。

boot=on

包含bootloader的分区，如果写了多个，会报错：

'TUNSETIFF: Device or resource busy\nTUNSETIFF: Device or resource busy\nqemu: two bootable drives specified\n

使用vim直接修改cfg文件不生效，需要重新导入cfg文件，或者在console中编辑config file。

实际上，上面的drive，boot，disk等参数来源于kvm，可以查看：

http://www.linux-kvm.org/page/Boot_from_virtio_block_device

网卡，使用virtio：

vif = ['mac=52:54:00:55:52:e9,bridge=br0', 'mac=52:54:00:5a:b7:d4,bridge=br1']

修改为

vif = ['mac=52:54:00:55:52:e9,bridge=br0,model=virtio', 'mac=52:54:00:5a:b7:d4,bridge=br1,model=virtio']

关于kernel

kernel = '/usr/lib/xen/boot/hvmloader'

kvm为什么使用xen的？官方的论坛中有提到：

In the the section 'Boot Parameters' of 'VM templates' by 'Kernel' the following pathc is mentioned '/usr/lib/xen/boot/hvmloader'. 

I guess in case using KVM and not Xen a KVM applicable path must be mentioned here e.g. '/usr/lib/kvm/boot/hvmloader', but that doesn't exist. What is the KVM one?

macamba

 

Re: Bootparameters template: Kernel: /usr/lib/xen/boot/hvmlo

by jd_jedi on Wed Feb 23, 2011 10:09 am

We use common template for both xen and kvm. The hvmloader will be ignored for KVM. (just keep it the way it was). 

将配置文件放置到存放配置文件的目录，导入即可。

四、模板

console->Template Library->Common中的模板可以用于KVM。

有关磁盘：

磁盘的可选名称里没有vda、vdb等？直接手动输入即可。

磁盘类型选择File（VDB），file system选none即可。

如何指定创建的磁盘使用virtio？Miscellaneous中添加属性：

file=/usr/local/convirt/vm_disk/$VM_NAME.disk.xm,if=virtio,format=raw,index=0,boot=on

多个磁盘？使用数组：

['file=/usr/local/convirt/vm_disk/$VM_NAME.os.xm,if=virtio,format=raw,index=0,boot=on','file=/usr/local/convirt/vm_disk/$VM_NAME.disk.xm,if=virtio,format=raw,index=0']

如何验证创建的磁盘是raw？

 qemu-img info -f raw ./convrit_test.disk.xm 

image: ./convrit_test.disk.xm

file format: raw

virtual size: 30G (32212254720 bytes)

disk size: 2.9G

有关网络：

对于桥接网络，应该选择br0，br1.

boot parameters中的boot loader为默认/usr/lib/xen/boot/hvmloader即可：

We use common template for both xen and kvm. The hvmloader will be ignored for KVM. (just keep it the way it was). 

关于

Switching kvm binary to 'qemu-kvm'

/usr/local/convirt/src/convirt/core/platforms/kvm/KVMProxy.py

if self.node_proxy.file_exists("/usr/libexec/qemu-kvm"):
            print "Switching kvm binary to 'qemu-kvm'"
            self.kvm_binary = "/usr/libexec/qemu-kvm"

该文件被编译为：/usr/local/convirt/src/convirt/core/platforms/kvm/KVMProxy.pyc

关于数据库，发现数据库增长较快，主要涉及一下表：

tasks、task_results 、metrics（服务器及及虚拟机的各种历史数据，如cpu利用率等）

所以删除了一些数据，结果造成后台报错，前台不能显示系统状态图，建议不要删除数据库的数据：

delete from tasks where submitted_on <'2012-03-01 00:00:00';