Chinaunix首页 | 论坛 | 博客
  • 博客访问: 2621209
  • 博文数量: 258
  • 博客积分: 9440
  • 博客等级: 少将
  • 技术积分: 6998
  • 用 户 组: 普通用户
  • 注册时间: 2009-05-03 10:28
个人简介

-- linux爱好者,业余时间热衷于分析linux内核源码 -- 目前主要研究云计算和虚拟化相关的技术,主要包括libvirt/qemu,openstack,opennebula架构和源码分析。 -- 第五届云计算大会演讲嘉宾 微博:@Marshal-Liu

文章分类

全部博文(258)

文章存档

2016年(1)

2015年(4)

2014年(16)

2013年(22)

2012年(41)

2011年(59)

2010年(40)

2009年(75)

分类: 云计算

2015-01-11 17:29:02

1. 背景:
    一直以为OpenStack的创建快照的操作是在线创建快照(live snapshot), 并且应该是增量的快照,即利用virsh或者qemu的live snapshot来实现的:
         virsh  snapshot-create-as --live ....
    后来发现快照和原始镜像之间并没有依赖关系,感觉OpenStack还做的挺好的,自动解决了增量快照和原始镜像之间的依赖关系;
    但是后来又发现做快照的时候虚拟机竟然会shutoff, 就感觉不对了,于是分析了下源码。

2. 结论:目前OpenStack默认的快照方式都是cold snapshot, 首先先关机,其次执行如下命令生成一个镜像文件,再次开机,最后再调用glance api将镜像上传。
       qemu-img convert -f qcow2 -O qcow2  
       所以目前并不是真正意义的快照,其实和关闭虚拟机,拷贝一份,再上传没有本质区别。

3. 源代码流程分析

3.1 nova/compute/api.py
  1. # NOTE(melwitt): We don't check instance lock for snapshot because lock is
  2.     # intended to prevent accidental change/delete of instances
  3.     @wrap_check_policy
  4.     @check_instance_cell
  5.     @check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.STOPPED,
  6.                                     vm_states.PAUSED, vm_states.SUSPENDED])
  7.     def snapshot(self, context, instance, name, extra_properties=None):
  8.         """Snapshot the given instance.

  9.         :param instance: nova.db.sqlalchemy.models.Instance
  10.         :param name: name of the snapshot
  11.         :param extra_properties: dict of extra image properties to include
  12.                                  when creating the image.
  13.         :returns: A dict containing image metadata
  14.         """
  15.        #调用glance api创建image entry,为后将snapshot上传为镜像做准备虽然镜像和snapshot在可以上传到glance作为镜像启动虚拟机,
  16.        #但是为了区分二者的不同,glance将镜像和snapshot标记卫不同的类型:type=image 和 type=snapshot
  17.         image_meta = self._create_image(context, instance, name,
  18.                                         'snapshot',
  19.                                         extra_properties=extra_properties)

  20.         # NOTE(comstud): Any changes to this method should also be made
  21.         # to the snapshot_instance() method in nova/cells/messaging.py
  22.         # 将任务状态(task state) 设置为:image_snapshot_pending
  23.         instance.task_state = task_states.IMAGE_SNAPSHOT_PENDING
  24.         instance.save(expected_task_state=[None])
  25.         #通过rpc调用nova/compute/rpcapi.py的snapshot_instance函数
  26.         self.compute_rpcapi.snapshot_instance(context, instance,
  27.                                               image_meta['id
3.2 nova/compute/rpcapi.py
  1. #梳理下流程: (1)用户发起create snapshot的请求; (2)nova-api服务接收到这个请求并进行前期处理,即3.1中代码的处理流程;
  2.                             (3)真正的snapshot操作是需要在nova-compute节点上执行的,所以nova-api需要向nova-compute发送message
  3. #由于OpenStack环境中会有很多个nova-compute,所以需要通过server=_compute_host(None, instance)来获取虚拟机所在的host,并向其发送message。
  4. def snapshot_instance(self, ctxt, instance, image_id):
  5.         version = '3.0'
  6.         cctxt = self.client.prepare(server=_compute_host(None, instance),
  7.                 version=version)
  8.         cctxt.cast(ctxt, 'snapshot_instance',
  9.                    instance=instance,
  10.                    image_id=image_id)
3.3 nova/virt/libvirt/driver.py
  1.  def snapshot(self, context, instance, image_id, update_task_state):
  2.         """Create snapshot from a running VM instance.

  3.         This command only works with qemu 0.14+
  4.         """
  5.         try:
  6.             virt_dom = self._get_domain(instance)
  7.         except exception.InstanceNotFound:
  8.             raise exception.InstanceNotRunning(instance_id=instance['uuid'])

  9.         base_image_ref = instance['image_ref']

  10.         base = compute_utils.get_image_metadata(
  11.             context, self._image_api, base_image_ref, instance)

  12.         snapshot = self._image_api.get(context, image_id)

  13.         disk_path = libvirt_utils.find_disk(virt_dom)
  14.         source_format = libvirt_utils.get_disk_type(disk_path)

  15.         image_format = CONF.libvirt.snapshot_image_format or source_format

  16.         # NOTE(bfilippov): save lvm and rbd as raw
  17.         if image_format == 'lvm' or image_format == 'rbd':
  18.             image_format = 'raw'

  19.         metadata = self._create_snapshot_metadata(base,
  20.                                                   instance,
  21.                                                   image_format,
  22.                                                   snapshot['name'])

  23.         snapshot_name = uuid.uuid4().hex

  24.         state = LIBVIRT_POWER_STATE[virt_dom.info()[0]]

  25.         # NOTE(rmk): Live snapshots require QEMU 1.3 and Libvirt 1.0.0.
  26.         # These restrictions can be relaxed as other configurations
  27.         # can be validated.
  28.         # NOTE(dgenin): Instances with LVM encrypted ephemeral storage require
  29.         # cold snapshots. Currently, checking for encryption is
  30.         # redundant because LVM supports only cold snapshots.
  31.         # It is necessary in case this situation changes in the
  32.         # future.
  33.         #这里需要注意,解释了为啥现在是cold snapshot而不是live snapshot:
  34.         # 有人提过live snapshot的bug,社区认为live snapshot目前不稳定,所以默认条件下采用cold snapshot,并且是通过硬编码来实现的
  35.         # 看下面这个判断条件,成立的时候将live_snapshot = true,其中MIN_LIBVIRT_LIVESNAPSHOT_VERSION=1.3.0, 其实现在libvirt的最新版本
  36.         # 才到1.2.11, 所以这个live_snapshot的条件不满足,就变成了cold_snapshot
  37.         if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,
  38.                                        MIN_QEMU_LIVESNAPSHOT_VERSION,
  39.                                        REQ_HYPERVISOR_LIVESNAPSHOT)
  40.              and source_format not in ('lvm', 'rbd')
  41.              and not CONF.ephemeral_storage_encryption.enabled):
  42.             live_snapshot = True
  43.             # Abort is an idempotent operation, so make sure any block
  44.             # jobs which may have failed are ended. This operation also
  45.             # confirms the running instance, as opposed to the system as a
  46.             # whole, has a new enough version of the hypervisor (bug 1193146).
  47.             try:
  48.                 virt_dom.blockJobAbort(disk_path, 0)
  49.             except libvirt.libvirtError as ex:
  50.                 error_code = ex.get_error_code()
  51.                 if error_code == libvirt.VIR_ERR_CONFIG_UNSUPPORTED:
  52.                     live_snapshot = False
  53.                 else:
  54.                     pass
  55.         else:
  56.             live_snapshot = False

  57.         # NOTE(rmk): We cannot perform live snapshots when a managedSave
  58.         # file is present, so we will use the cold/legacy method
  59.         # for instances which are shutdown.
  60.         if state == power_state.SHUTDOWN:
  61.             live_snapshot = False

  62.         # NOTE(dkang): managedSave does not work for LXC
  63.         #注意这里,如果live_snashot目前是false,所以在做snapshot之前先要执行:
  64.         #(1)_detach_pci_devices, 卸载虚拟机挂载的pci设备,比如数据盘
  65.         #(2)  self._detach_sriov_ports, 卸载虚拟机挂载的SRIOV设备,比如支持SRIOV的网卡设备
  66.         if CONF.libvirt.virt_type != 'lxc' and not live_snapshot:
  67.             if state == power_state.RUNNING or state == power_state.PAUSED:
  68.                 self._detach_pci_devices(virt_dom,
  69.                     pci_manager.get_instance_pci_devs(instance))
  70.                 self._detach_sriov_ports(instance, virt_dom)
  71.                 virt_dom.managedSave(0)
  72.         #判断虚拟机的后端存储是什么,不同的后端存储做snapshot是不同的,本地文件系统的化,默认qcow2
  73.         snapshot_backend = self.image_backend.snapshot(instance,
  74.                 disk_path,
  75.                 image_type=source_format)

  76.         if live_snapshot:
  77.             LOG.info(_LI("Beginning live snapshot process"),
  78.                      instance=instance)
  79.         else:
  80.             LOG.info(_LI("Beginning cold snapshot process"),
  81.                      instance=instance)
  82.         #更新任务的状态为:image_pending_upload, 大家都知道做完snapshot要上传
  83.         update_task_state(task_state=task_states.IMAGE_PENDING_UPLOAD)
  84.         #目前做快照的过程是:
  85.         #(1)现在../data/nova/instance/snapshots目录下生成临时目录,比如nova/instances/snapshots/tmptHr585
  86.         #(2)然后将快照生成到这个目录,具体参见snapshot_backend.snapshot_extract(out_path, image_format)这个函数
  87.         #(3)生成完成后,通过glance api上传,具体参见 self._image_api.update
  88.         snapshot_directory = CONF.libvirt.snapshots_directory
  89.         fileutils.ensure_tree(snapshot_directory)
  90.         with utils.tempdir(dir=snapshot_directory) as tmpdir:
  91.             try:
  92.                 out_path = os.path.join(tmpdir, snapshot_name)
  93.                 if live_snapshot:
  94.                     # NOTE(xqueralt): libvirt needs o+x in the temp directory
  95.                     os.chmod(tmpdir, 0o701)
  96.                     self._live_snapshot(context, instance, virt_dom, disk_path,
  97.                                         out_path, image_format, base)
  98.                 else:
  99.                     #这个函数实际执行了一条命令: qemu-img convert -f qcow2 -O qcow2  disk_path out_path,算是生成了快照
  100.                     snapshot_backend.snapshot_extract(out_path, image_format)
  101.             finally:
  102.                 new_dom = None
  103.                 # NOTE(dkang): because previous managedSave is not called
  104.                 # for LXC, _create_domain must not be called.
  105.                 if CONF.libvirt.virt_type != 'lxc' and not live_snapshot:
  106.                     if state == power_state.RUNNING:
  107.                         new_dom = self._create_domain(domain=virt_dom)   ##恢复做快照之前虚拟机的状态
  108.                     elif state == power_state.PAUSED:
  109.                         new_dom = self._create_domain(domain=virt_dom,
  110.                                 launch_flags=libvirt.VIR_DOMAIN_START_PAUSED)
  111.                     if new_dom is not None:
  112.                         self._attach_pci_devices(new_dom,
  113.                             pci_manager.get_instance_pci_devs(instance))
  114.                         self._attach_sriov_ports(context, instance, new_dom)
  115.                 LOG.info(_LI("Snapshot extracted, beginning image upload"),
  116.                          instance=instance)

  117.             # Upload that image to the image service

  118.             update_task_state(task_state=task_states.IMAGE_UPLOADING,
  119.                      expected_state=task_states.IMAGE_PENDING_UPLOAD)
  120.             with libvirt_utils.file_open(out_path) as image_file:       ###将生成的快照上传到glance
  121.                 self._image_api.update(context,
  122.                                        image_id,
  123.                                        metadata,
  124.                                        image_file)
  125.                 LOG.info(_LI("Snapshot image upload complete"),
  126.                          instance=instance)



------------------------------------------------------------
   email: ustc.dylan@gmail.com
   微博:@Marshal-Liu
------------------------------------------------------------


阅读(18076) | 评论(1) | 转发(3) |
给主人留下些什么吧!~~

linkthaha2016-02-23 17:29:22

前辈您好,我最近在用kvm+virsh实践虚拟机管理的时候发现对带有快照的虚拟机做迁移会失败,必须把快照信息删除掉才能成功,我觉得这个逻辑,有点不符合实践需要呢,目前实践中有没有其他的方法可以保留快照信息呢?