If you have recent Intel 45nm Core 2 CPUs and were directed to this article by a message in VirtualCenter or vCenter Server, refer to .
In ESX/ESXi 3.5 Update 2 and later, VMware recommends using Enhanced VMotion Compatibility (EVC) to eliminate most VMotion CPU compatibility problems. For more information on EVC, see .
By default, VirtualCenter and vCenter Server allow migrations with VMotion only between compatible source and destination CPUs. This article provides information about when it is appropriate to override CPU-compatibility constraints, how to override them, and provides links to additional information and resources. Topics include:
For VMotion purposes, compatible CPUs means that source and target CPUs must have the same manufacturer (AMD or Intel) and be members of the same basic processor family (for example, Pentium 4 or Core), and have a common set of features implemented.
To determine if the source and target CPUs meet VMotion requirements, VirtualCenter and vCenter Server compare the target CPU to a default bit mask definition to exclude unimportant features from the comparison. As new features are introduced by processor vendors, the bit masks used by VirtualCenter and vCenter Server are updated. For example, in ESX Server 2.x, the NX (AMD) or XD (Intel) feature is not used to compute VMotion compatibility, but in ESX 3.x, it is.
To disable VMotion’s CPU-compatibility checking for a specific feature, you can modify the default bit mask. The bit mask-modification process is sometimes referred to in some VMware documents as “relaxing” a particular constraint.
A full discussion of the underlying components and how VirtualCenter Server and ESX Server host systems handle them is beyond the scope of this article. For more information, see "CPU Compatibility Masks" in Basic System Administration. Another useful resource is the technical paper on .
For specific information about Intel and AMD CPUs and VMotion compatibility, see:
For simplicity, modifications to the default bit mask for CPU compatibility for VMotion are also referred to simply as masks, typically identified by the CPU feature—for example, the SSE3 mask in this and related KB articles 1991 and 1992.
Many masks are not supported. Experimental support is available for some masks in ESX 3.5 and later. Support terms for masks are summarized in the following table.
| Mask |
ESX Server 2.x |
ESX 3.x |
ESX 4.x |
| CMPXCHG16B (AMD) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
| FFXSR (AMD) |
Not supported. |
Not supported. |
Not supported. |
| NX (AMD) |
Supported. |
Supported. |
Supported. |
| RDTSCP (AMD) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
| SSE3 (AMD) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
| SSE3 (Intel) |
Not supported. |
Not supported. |
Not supported. |
| SSE4.1 (Intel) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
| SSE4.2 (Intel) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
| XD (Intel) |
Supported. |
Supported. |
Supported. |
| AES (Intel) |
Not supported. |
Not supported prior to ESX 3.5. Experimentally supported in ESX 3.5 and later. |
Experimentally supported. |
To obtain more information about a host system's CPU, use the CPU Identification Utility. VMware provides this as an ISO image file that can be uncompressed and used to create a bootable CD-ROM that provides CPU information about a host, even before an operating system or ESX is installed. The latest version of this tool can be found on the VMware downloads page at .
There are two ways to modify the default CPU bit masks, depending on the ESX/ESXi version and individual needs.
- With any ESX/ESXi or ESX Server version: You can modify the default bit mask at the VirtualCenter Server or vCenter Server level by processor manufacturer, by version number, and by other criteria. This method is not supported, and applications attempting to use the masked features might crash during migration with VMotion.
- ESX/ESXi 3.x and later: You can modify the bit masks on a per-virtual machine basis using the VI Client or vSphere Client. You must power off the virtual machine before making these modifications due to the risk of application or guest operating system instability. Support terms for this method are listed in (for Intel processors) and (for AMD processors).
In Virtual Center 2.x and vCenter Server, you can modify the default bit mask for individual virtual machines using the VI Client or vSphere Client.
Power off the virtual machine before modifying its bit mask.
To modify the default bit mask for a virtual machine:
- Launch VI Client or vSphere Client and connect to the VirtualCenter Server or vCenter Server as an administrator.
- Select the virtual machine that you want to migrate in the inventory.
- Click Edit Settings on the Summary tab. The Properties window for the virtual machine appears.
-
Click the Options tab in the Properties window.
-
Select the CPUID Mask option to display the CPU Identification Mask information and settings.
- Click Advanced to display several settings-related boxes, including CPU Identification Mask, in the right-hand pane.
From here the mask can be modified appropriately for the selected virtual machine, depending on what needs to be masked. The following sections include steps on modifying the masks for several common CPU registers.
Issues with the NX/XD features on CPUs are common. Ensure that the feature is either enabled (or disabled) in the BIOS of all hosts to avoid these compatibility error messages.
Note: Not all servers have the option to disable or enable NX or XD in the BIOS.
If a CPU feature compatibility issue with the NX/XD bit is encountered, an error similar to the following is generated:
The CPU of the host is incompatible with the cpu feature requirements of virtual machine; problem detected at CPUID level 0x80000001 register 'edx'.
Power off the virtual machine before modifying the NX/XD mask.
To modify the mask to enable or disable the NX/XD CPU bit:
-
Navigate to the CPUID Mask option on the virtual machine Options tab (see steps above, if necessary).
-
Select Hide the NX flag from guest to disable the CPUcompatibility check for the selected virtual machine,or Expose the NX flag to guest to enable this CPU compatibility check for the selected virtual machine.
-
Click OK to save the change.
Masks for several other features can be modified, to provide further compatibility for VMotion. Support terms for these feature masks are listed in . In general, VMware neither supports nor recommends modifying the VMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
Power off the virtual machine before modifying its bit mask.
To modify other CPU masks:
- Navigate to the virtual machine Options tab (see steps above, if necessary).
-
Click Advanced to open the CPU Identification Mask properties dialog box.
Note: The CPU Identification Mask dialog box has two tabs—Virtual Machine Default, and AMD Override. Most modifications for Intel CPU features are made on the Virtual Machine Default tab. Modifications for AMD CPU features are made on the AMD Overrides tab, and are discussed in the next section of this article.
-
Click the Virtual Machine Default tab to activate the dialog box, if necessary.
To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.
| Feature |
Level |
Row |
Mask |
| SSE3 |
1 |
ecx |
---- ---- ---- ---- ---- ---- ---0 -0-0 |
| SSSE3 |
1 |
ecx |
---- ---- ---- -0-- ---- --0- ---0 -0-- |
| 1 |
edx |
---- ---- ---- --0- ---- ---- ---- ---- |
| SSE4.1 |
1 |
ecx |
---- ---- ---- 0--- ---- ---- ---- ---- |
| SSE4.2 |
1 |
ecx |
---- ---- 0--0 ---- ---- ---- ---- ---- |
| 80000001 |
edx |
---- 0--- ---- ---- ---- ---- ---- ---- |
| AES |
a |
ecx |
---- --0- ---- ---- ---- ---- ---- --0- |
When all modifications are complete, click OK and exit the CPU Identification Mask dialog box.
Features that are specific to AMD processors are listed and can be changed on the AMD Override tab. Support terms for these feature masks are listed in . In general, VMware neither supports nor recommends modifying the VMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
Power off the virtual machine before modifying its bit mask.
To modify other CPU masks for AMD Processors:
-
Navigate to the virtual machine Options tab (see steps above, if necessary).
-
Click Advanced to open the CPU Identification Mask properties dialog box.
-
Click the AMD Override tab to activate the dialog box. The AMD Override page displays.
To modify the mask for a specific feature, enter the series of dashes and 0s as shown in the table below.
| Feature |
Level |
Row |
Mask |
| SSE3 |
1 |
ecx |
---- ---- ---- ---- ---- ---- ---- ---0 |
| RDTSCP |
80000001 |
edx |
---- 0--- ---- ---- ---- ---- ---- ---- |
| 80000001 |
ecx |
---- ---- ---- ---- ---- ---- ---- 0--- |
|
CMPXCH16B |
1 |
ecx |
---- ---- ---- ---- --0- ---- ---- ---- |
| FFXSR |
80000001 |
edx |
---- --0- ---- ---- ---- ---- ---- ---- |
When all modifications are complete click OK and exit the CPU Identification Mask dialog box.
You can combine modifications to the default bit mask to allow migration with VMotion between groups that are incompatible based on more than one CPU feature. For example, to filter-out the compatibility check for both SSE3 and SSSE3 combine the following:
---- ---- ---- ---- ---- ---- ---0 -0-0
and:
---- ---- ---- -0-- ---- --0- ---0 -0--
to yield:
---- ---- ---- -0-- ---- --0- ---0 -0-0
for the ecx register mask.
For ESX/ESXi hosts managed by VirtualCenter or vCenter Server, you can modify the bit mask globally by manually editing the VirtualCenter or vCenter Server configuration file (vpxd.cfg ). The configuration file, vpxd.cfg , contains XML tags defining various elements and settings for the VirtualCenter Server or vCenter Server system. Bit-masks can be modified by adding the appropriate XML tags to define the mask in the context of a guest operating system configuration option to the configuration file .
Changes to the global bit mask do not take effect for individual virtual machines until they have been powered off and powered on again.
Note: The vpxd.cfg file is located by default in C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter
Although a full discussion of building these elements is beyond the scope of this article, the section provides a brief overview. See the following section for specific steps related to and XML tags for like, , , and .
Warning: VMware neither supports nor recommends modifying the VMotion constraints for CPU features used by applications, such as SSE3, because of the risk of application or guest operating system failure after migration.
The tags identify the register to which the mask applies, and for which hosts and which versions the mask applies. The tags are direct descendants of the tag, in the vpxd.cfg file. Tags are listed in the order in which they must be nested:
-
— Direct descendant (child) tag of element. The vpxd.cfg can contain a single guestOSDescriptor element.
-
<host-product-and-version> — Between the opening and closing tags, you can nest multiple host-product-and-version-tags that specify the version and host to which the masks you define apply. The tags can be specified generally, as in for an ESX Server 2.x host, for an ESX/ESXi 3.x host or can be specified precisely, as in . For example, the tag identifies various maintenance release levels of the ESX Server to which the subsequently nested mask applies.
-
— Allows for the configuration to be restricted by virtual machine hardware version. For example, , , or . In the example below is specified, meaning that the mask applies to all virtual machine hardware versions.
-
— Allows for a guest version to be specified. For example, , , , and so on. In the example shown below, is specified, and therefore the mask applies to all guest versions.
-
— This tag precedes the actual mask. The mask is then defined for , , or , depending on your needs. Tag elements that are used to define the mask, identify the CPU mask. The element details include vendor, CPU ID level, and the CPU register. The valid choices for these are:
|
CPU vendor |
|
|
CPU ID level |
, , , |
|
CPU register |
, , , |
Tags must be embedded in the order shown in the table. A CPU-vendor tag is followed by a CPU-ID-level tag, followed by a CPU-register tag. Immediately following the CPU-register tag is the sequence of 32 dashes and x-s that represent the actual 32-bit register mask appropriate for the feature being modified. The mask is then followed by the CPU-register-, CPU-ID-level-, and CPU-vendor-closing tags.
Note: A mask defined using tag is used by the system only when no vendor-specific mask has been specified for the same CPU ID level.
Below is an example of the tag order, shown in the context of the vpxd.cfg file. The and opening and closing tags are already in the vpxd.cfg file. The opening tag can be placed directly after the opening tag, embed the appropriate tags to suit your needs, followed by the closing tag.
[Elements and mask definition go in here. Common Mask Patterns can be copy-and-pasted from the below example patterns.]
...
In this example, any mask embedded in the [Elements and mask...] placeholder section applies to all ESX Server 2.x hosts being managed by the VirtualCenter Server, and to all guest operating systems.
To edit the VirtualCenter Configuration File:
-
Navigate to the location of the vpxd.cfg file. By default, the vpxd.cfg file is located in the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter directory.
-
Use a text editor (such as WordPad) to open the vpxd.cfg file. The vpxd.cfg file is an XML-formatted text file that contains various optional settings for the VirtualCenter Server or vCenter Server system.
-
Add opening and closing tags to identify the hosts, versions, and other specifics to which the mask applies.
-
Copy and paste from the examples in Common Mask Patterns, or apply other masks as needed for your deployment.
-
When the appropriate changes are made, restart the VMware VirtualCenter Server Service to enable the new masking.
The following are example masks for several common CPU feature masks. Copy, paste, and modify them as required for the environment. VMware does not support or recommend using the masks shown below, because of the risk of application or guest operating system failure after migration.
----:----:----:----:----:----:---x:-x-x
----:----:----:----:----:----:----:---x
----:----:----:-x--:----:--x-:---x:-x--
----:----:----:--x-:----:----:----:----
----:----:----:-x--:----:--x-:---x:-x-x
----:----:----:--x-:----:----:----:----
----:----:----:----:----:----:----:---x
Revert to the default VMotion compatibility constraints by reverting any changes made to the default mask. This can be accomplished by following the appropriate steps for the type of masking that was applied:
-
When using per-virtual-machine masks, reset any changed rows to the defaults by clicking Reset All to Default on the CPU Identification Mask dialog box.
-
When using global masks: Remove any added tags entered in the vpxd.cfg file and restart the VirtualCenter service.
