Linux CPU Hotplug CPU热插拔（一）

CPU hotplug Support in Linux(tm) Kernel

Linux支持CPU的热插拔功能

                   Maintainers:

                   CPU Hotplug Core:
                             Rusty Russell
                             Srivatsa Vaddagiri
                   i386:
                             Zwane Mwaikambo
                   ppc64:
                             Nathan Lynch
                             Joel Schopp
                   ia64/x86_64:
                             Ashok Raj
                   s390:
                             Heiko Carstens

Authors: Ashok Raj
Lots of feedback: Nathan Lynch ,
               Joel Schopp

Introduction

Modern advances in system architectures have introduced advanced error
reporting and correction capabilities in processors. CPU architectures permit
partitioning support, where compute resources of a single CPU could be made
available to virtual machine environments. There are couple OEMS that
support NUMA hardware which are hot pluggable as well, where physical
node insertion and removal require support for CPU hotplug.

Such advances require CPUs available to a kernel to be removed either for
provisioning reasons, or for RAS purposes to keep an offending CPU off
system execution path. Hence the need for CPU hotplug support in the
Linux kernel.

A more novel use of CPU-hotplug support is its use today in suspend
resume support for SMP. Dual-core and HT support makes even
a laptop run SMP kernels which didn't support these methods. SMP support
for suspend/resume is a work in progress.

现 在，在一些先进的体系结构中处理器引进了一些先进的错误报告和错误修正能力。CPU体系结构上允许CPU分割，把分割的一部分CPU作为一个可用的虚拟机 器环境。有很多NUAM结构的硬件设备支持设备热插拔功能，比如说NUMA结构的计算机一个物理节点的插入和移出需要CPU热插拔功能的支持。

如此先进的功能需要操作系统能够提供必要条件满足CPU的热插拔功能的实现。或者因为远程访问服务器有意去使某个CPU不在系统的执行路径上（即这个CPU不能再次被使用）。这些都要求Linux操作系统支持CPU的热插拔功能。

现在很多对称多处理器的机器（SMP）使用CPU热插拔功能。双核和高可用技术使得便携式电脑都能使用SMP内核的操作系统，但是不能实现CPU热插拔功能。SMP支持CPU暂停或恢复当作一个任务在系统运行过程中。

General Stuff about CPU Hotplug（关于CPU热插拔常常碰到的一些技术与问题）
--------------------------------

Command Line Switches（命令行开关，就是通过命令行的执行，对一些数据项和操作的开\关设置）
---------------------
maxcpus=n     Restrict boot time cpus to n. Say if you have 4 cpus, using
             maxcpus=2 will only boot 2. You can choose to bring the
             other cpus later online, read FAQ's for more info.
                            这个数据项用来限制机器启动时启动CPU的数目。如果你的机器有4个CPU，你将maxcpus设置为2，那么在机器启动时只能启动2个CPU。你可以在机器完全启动后，可以将剩下的CPU热添加进系统。

additional_cpus=n (*)       Use this to limit hotpluggable cpus. This option sets
                           cpu_possible_map = cpu_present_map + additional_cpus
                           使用这个命令行可以限制可以热添加的CPU数目。下面等式表示系统中几个描述CPU位图之间的关系。
             系统中可以允许最大数目CPU=系统中目前现有的CPU数目+添加进去的CPU数目

(*) Option valid only for following architectures
- x86_64, ia64（只针对X86_64和ia64体系结构的CPU）

ia64 and x86_64 use the number of disabled local apics in ACPI tables MADT
to determine the number of potentially hot-pluggable cpus. The implementation
should only rely on this to count the # of cpus, but *MUST* not rely on the
apicid values in those tables for disabled apics. In the event BIOS doesn't
mark such hot-pluggable cpus as disabled entries, one could use this
parameter "additional_cpus=x" to represent those cpus in the cpu_possible_map.

ia64和x86_64芯片使用ACPI的MADT表中大量的被禁用的局部APICs（Advanced Programmable Interrupt Controllers，高级可编程中断控制器）决定这些CPU作为潜在的热插拔CPU。


possible_cpus=n                 [s390 only] use this to set hotpluggable cpus.
                             This option sets possible_cpus bits in
                             cpu_possible_map. Thus keeping the numbers of bits set
                             constant even if the machine gets rebooted.

CPU maps and such
-----------------
[More on cpumaps and primitive to manipulate, please check


cpu_possible_map: Bitmap of possible CPUs that can ever be available in the
system. This is used to allocate some boot time memory for per_cpu variables
that aren't designed to grow/shrink as CPUs are made available or removed.
Once set during boot time discovery phase, the map is static, i.e no bits
are added or removed anytime.   Trimming it accurately for your system needs
upfront can save some boot time memory. See below for how we use heuristics
in x86_64 case to keep this under check.

cpu_possible_map： 系统中可能所有可用的CPU的位图结构变量。一般在系统引导启动的时候为每个可能存在的CPU分配内存给per_cpu变量。这些不是用来描述CPU的热 添加或热移出操作。这个位图是个静态变量，在系统引导启动时候生成后，就不能修改了。

cpu_online_map: Bitmap of all CPUs currently online. Its set in __cpu_up()
after a cpu is available for kernel scheduling and ready to receive
interrupts from devices. Its cleared when a cpu is brought down using
__cpu_disable(), before which all OS services including interrupts are
migrated to another target CPU.

cpu_online_map： 系统当前online的CPU。这个位图在函数__cpu_up()中被设置，这个位图描述了系统有哪些CPU可以用来调度执行任务和可以接受设备中断。 当执行函数__cpu_disable()时，清除指定CPU在位图cpu_online_map对应的位，这就意味着指定的CPU不可用了，不过这个函 数执行之前，要将指定CPU上所有的服务和中断迁移到其他CPU上执行。

cpu_present_map: Bitmap of CPUs currently present in the system. Not all
of them may be online. When physical hotplug is processed by the relevant
subsystem (e.g ACPI) can change and new bit either be added or removed
from the map depending on the event is hot-add/hot-remove. There are currently
no locking rules as of now. Typical usage is to init topology during boot,
at which time hotplug is disabled.

cpu_present_map： 当前系统存在的CPU。但是，并不是所有存在的CPU都处于online。在物理上的CPU热插拔依据相关的子系统处理，会改变位图 cpu_present_map相应位的值。该位图经典地用于在系统启动时描述系统CPU拓扑结构，并且那时候CPU的hotplug功能被禁用了。

You really dont need to manipulate any of the system cpu maps. They should
be read-only for most use. When setting up per-cpu resources almost always use
cpu_possible_map/for_each_possible_cpu() to iterate.

你并不需要熟练使用系统地CPU这些位图。因为他们通常只读。通过cpu_possible_map/for_each_possible_cpu()反复使用为每个CPU建立必需资源。

Never use anything other than cpumask_t to represent bitmap of CPUs.
在系统中所有的CPU位图都是cpumask_t类型的数据结构。

         #include

         for_each_possible_cpu     - Iterate over cpu_possible_map
         for_each_online_cpu       - Iterate over cpu_online_map
         for_each_present_cpu       - Iterate over cpu_present_map
         for_each_cpu_mask(x,mask) - Iterate over some random collection of cpu mask.

         #include
         get_online_cpus() and put_online_cpus():

The above calls are used to inhibit cpu hotplug operations. While the
cpu_hotplug.refcount is non zero, the cpu_online_map will not change.
If you merely need to avoid cpus going away, you could also use
preempt_disable() and preempt_enable() for those sections.
Just remember the critical section cannot call any
function that can sleep or schedule this process away. The preempt_disable()
will work as long as stop_machine_run() is used to take a cpu down.

上 面提到的所有函数（例程）用于阻止CPU的热插拔操作。当cpu_hotplug的引用计数不为0时，位图cpu_online_map不会发生改变。如 果你仅仅只是需要避免CPU的消失，你同样可以调用preempt_disable()和preempt_enable()。记住在临界区不能调用任何函 数因为那样可能会被睡眠或者中断。当执行stop_machine_run()函数使一个CPU不可用时，一定会调用执行 preempt_disable()。

CPU Hotplug - Frequently Asked Questions.（CPU热插拔常见问题）

Q: How to enable my kernel to support CPU hotplug?该怎样激活我的操作系统内核才能支持CPU热插拔？
A: When doing make defconfig, Enable CPU hotplug support

   "Processor type and Features" -> Support for Hotpluggable CPUs

Make sure that you have CONFIG_HOTPLUG, and CONFIG_SMP turned on as well.

You would need to enable CONFIG_HOTPLUG_CPU for SMP suspend/resume support
as well.
在你配置内核选项时，即在终端输入“make menuconfig”，将会出现一个蓝色界面，将需要的选项打开即可。