Setting the Number of VMware CPU Cores Per Socket: Best Practices
When configuring processor settings for a new virtual machine, there are several key concepts to understand: how to calculate the number of processor cores per CPU and CPU cores per socket and how these settings affect the speed of virtual machines. In addition, it is important to understand what ensures better VM performance: limiting the number of processors and having more CPU cores or having more processors with fewer cores?
Read on to learn about these concepts and the basic principles for CPU configuration for virtual machines on ESXi hosts.
Terminology
Let’s start with the definitions of the terms used when configuring CPU settings for virtual machines. These terms will help with understanding the working principle and avoid confusion about the number of cores per CPU, CPU cores per socket, and the number of CPU cores vs speed.
CPU Socket
A CPU Socket is a physical connector on the motherboard to which a single physical CPU is connected. A motherboard has at least one CPU socket. Server motherboards usually have multiple CPU sockets that support multiple multicore processors. CPU sockets are standardized for different processor series. Intel and AMD use different CPU sockets for their processor families.
CPU
A CPU (central processing unit, microprocessor chip, or processor) is a computer component. It is the electronic circuitry with transistors that is connected to a socket. A CPU executes instructions to perform calculations, run applications, and complete tasks.
When the clock speed of processors came close to the heat barrier, manufacturers changed the architecture of processors and started producing processors with multiple CPU cores. To avoid confusion between physical processors and logical processors or processor cores, some vendors refer to a physical processor as a socket.
CPU Core
A CPU core is the part of a processor containing the L1 cache. The CPU core performs computational tasks independently without interacting with other cores and external components of a “big” processor that are shared among cores. Basically, a core can be considered as a small processor built into the main processor that is connected to a socket. Applications should support parallel computations to use multicore processors rationally.
Hyper-threading
Hyper-threading is a technology developed by Intel engineers to bring parallel computation to processors that have one processor core. The debut of hyper-threading was in 2002 when the Pentium 4 HT processor was released and positioned for desktop computers. An operating system detects a single-core processor with hyper-threading as a processor with two logical cores (not physical cores). Similarly, a four-core processor with hyper-threading appears to an OS as a processor with 8 cores. The more threads run on each core, the more tasks can be done in parallel.
Modern Intel processors have both multiple cores and hyper-threading. Hyper-threading is usually enabled by default and can be enabled or disabled in BIOS. AMD simultaneous multi-threading (SMT) is the analog of hyper-threading for AMD processors.
vCPU
A vCPU is a virtual processor that is configured as a virtual device in the virtual hardware settings of a VM. A virtual processor can be configured to use multiple CPU cores. A vCPU is connected to a virtual socket.
CPU overcommitment
CPU overcommitment is the provisioning of more logical processors (CPU cores of the physical host) to VMs than the total number of logical processors available on the physical host. CPU overcommitment allows for increased flexibility and efficiency in resource utilization but can lead to performance degradation if the total demand exceeds the available physical CPU capacity.
NUMA
NUMA (non-uniform memory access) is a computer memory design used in multiprocessor computers. The idea is to provide separate memory for each processor (unlike UMA, where all processors access shared memory through a bus). At the same time, a processor can access memory that belongs to other processors by using a shared bus (all processors access all memory on the computer). A CPU has the performance advantage of accessing its own local memory faster than other memory on a multiprocessor computer.
These basic architectures are mixed in modern multiprocessor computers. Processors are grouped on a multicore CPU package or node. Processors that belong to the same node share access to memory modules as with the UMA architecture. Also, processors can access memory from the remote node via a shared interconnect. Processors do so for the NUMA architecture but with slower performance. This memory access is performed through the CPU that owns that memory rather than directly.
NUMA nodes are CPU/Memory couples that consist of a CPU socket and the closest memory modules. NUMA is usually configured in BIOS as the node interleaving or interleaved memory setting.
Example: An ESXi host has two sockets (two CPUs) and 256 GB of RAM. Each CPU has 6 processor cores. This server contains two NUMA nodes. Each NUMA node has 1 CPU socket (one CPU), 6 Cores, and 128 GB of RAM.
ESXi always tries to allocate memory for a VM from a native (home) NUMA node. A home node can be changed automatically if there are changes in VM loads and ESXi server loads.
vNUMA
Virtual NUMA (vNUMA) is the analog of NUMA for VMware virtual machines. A vNUMA consumes hardware resources of more than one physical NUMA node to provide optimal performance. The vNUMA technology exposes the NUMA topology to a guest operating system. As a result, the guest OS is aware of the underlying NUMA topology for the most efficient use. The virtual hardware version of a VM must be 8 or higher to use vNUMA.
Handling of vNUMA was significantly improved in VMware vSphere 6.5, and this feature is no longer controlled by the CPU cores per socket value in the VM configuration. By default, vNUMA is enabled for VMs that have more than 8 logical processors (vCPUs). You can enable vNUMA manually for a VM by editing the VMX configuration file of the VM and adding the line numa.vcpu.min=X, where X is the number of vCPUs for the virtual machine.
Calculating the Number of Cores
Let’s find out how to calculate the number of physical CPU cores, logical CPU cores, and other parameters on a server.
- The total number of physical CPU cores on a host machine is calculated with the formula:
(The number of Processor Sockets) x (The number of cores/processor) = The number of physical processor cores
Only processor sockets with installed processors must be considered.
- If hyper-threading is supported, calculate the number of logical processor cores by using the formula:
(The number of physical processor cores) x (2 threads/physical processor) = the number of logical processors
- Finally, use a single formula to calculate available processor resources that can be assigned to VMs:
(CPU sockets) x (CPU cores) x (threads)
For example, if you have a server with two processors each having 4 cores and supporting hyper-threading, then the total number of logical processors that can be assigned to VMs is:
2(CPUs) x 4(cores) x 2(HT) = 16 logical processors
One logical processor can be assigned as one processor or one CPU core for a VM in VM settings.
As for virtual machines, due to hardware emulation features, they can use multiple processors and CPU cores in their configuration for operation. One physical CPU core can be configured as a virtual CPU or a virtual CPU core for a VM.
- The total amount of clock cycles available for a VM is calculated as:
(The number of logical sockets) x (The clock speed of the CPU)
For example, if you configure a VM to use 2 vCPUs with 2 cores when you have a physical processor whose clock speed is 3.0 GHz, then the total clock speed is 2x2x3=12 GHz. If CPU overcommitment is used on an ESXi host, the available frequency for a VM can be less than calculated if VMs perform CPU-intensive tasks.
Limitations for CPU Cores Per Socket
The CPU/core limitations in vSphere 7.0 Update 1 and vSphere 8.0 are listed below.
- The maximum number of virtual processor sockets assigned to a VM is 768. If you want to assign more than 768 virtual processors, configure a VM to use multicore processors.
- The maximum number of processor cores that can be assigned to a single VM is 768. A virtual machine cannot use more CPU cores than the number of logical processor cores on a physical machine.
- CPU hot add. If a VM has 128 vCPUs or less than 128 vCPUs, then you cannot use the CPU hot-add feature for this VM and edit the CPU configuration of a VM while a VM is in the running state.
- OS CPU restrictions. If an operating system has a limit on the number of processors, and you assign more virtual processors for a VM, the additional processors are not identified and used by a guest OS. Limits can be caused by OS technical design and OS licensing restrictions. Note that there are operating systems that are licensed per socket and per CPU core (for example, Windows Server 2019).
CPU support limits for some operating systems:
- Windows 10 Pro – 2 CPUs
- Windows 10 Home – 1 CPU
- Windows 10 Workstation – 4 CPUs
- Windows Server 2019 Standard/Datacenter – 64 CPUs
- Windows XP Pro x64 – 2 CPUs
- Windows 7 Pro/Ultimate/Enterprise – 2 CPUs
- Windows Server 2003 Datacenter – 64 CPUs
You can check configuration maximums for the vSphere version used in your environment on the VMware website.
Configuration Recommendations
For older vSphere versions, we recommend using sockets over cores in VM configuration. At first, you might not see a significant difference in CPU sockets or CPU cores in VM configuration for VM performance. Be aware of some configuration features that make a difference. Remember about NUMA and vNUMA when you consider setting multiple virtual processors (sockets) for a VM to have optimal performance.
If vNUMA is not configured automatically, mirror the NUMA topology of the physical server. Here are some recommendations for VMs in VMware vSphere 6.5 and later:
- When you define the number of logical processors (vCPUs) for a VM, prefer the cores-per-socket configuration. Continue until the count exceeds the number of CPU cores on a single NUMA node on the ESXi server. Use the same logic until you exceed the amount of memory that is available on a single NUMA node of your physical ESXi server
- Sometimes, the number of logical processors for your VM configuration is more than the number of physical CPU cores on a single NUMA node, or the amount of RAM is higher than the total amount of memory available for a single NUMA node. Consider dividing the count of logical processors (vCPUs) across the minimum number of NUMA nodes for optimal performance.
- Don’t set an odd number of vCPUs if the CPU count or amount of memory exceeds the number of CPU cores. The same applies in case memory exceeds the amount of memory for a single NUMA node on a physical server.
- Don’t create a VM that has a number of vCPUs larger than the count of physical processor cores on your physical host.
- If you cannot disable vNUMA due to your requirements, don’t enable the vCPU Hot-Add feature.
If vNUMA is enabled in vSphere prior to version 6.5, and you have defined the number of logical processors (vCPUs) for a VM, select the number of virtual sockets for a VM while keeping the cores-per-socket amount equal to 1 (that is the default value). This is because the one-core-per-socket configuration enables vNUMA to select the best vNUMA topology for the guest OS automatically. This automatic configuration is optimal on the underlying physical topology of the server. If vNUMA is enabled, and you’re using the same number of logical processors (vCPUs) but increase the number of virtual CPU cores and reduce the number of virtual sockets by the same amount, then vNUMA cannot set the best NUMA configuration for a VM. As a result, VM performance is affected and can degrade.
If a guest operating system and other software installed on a VM are licensed on a per-processor basis, configure a VM to use fewer processors with more CPU cores. For example, Windows Server 2012 R2 is licensed per socket, and Windows Server 2016 is licensed on a per-core basis.
If you use CPU overcommitment in the configuration of your VMware virtual machines, keep in mind these values:
- 1:1 to 3:1 – There should be no problems in running VMs
- 3:1 to 5:1 – Performance degradation is observed
- 6:1 – Prepare for problems caused by significant performance degradation
CPU overcommitment with normal values can be used in test and dev environments without risks.
Configuration of VMs on ESXi Hosts
First, determine the number of logical processors (Number of CPUs) of your physical host that is needed for a virtual machine for proper work with sufficient performance. Then define the number of virtual sockets with processors (Number of Sockets in vSphere Client) and the number of CPU cores (Cores per Socket) you should set for a VM keeping in mind previous recommendations and limitations. The table below can help you select the needed configuration.
If you need to assign more than 8 logical processors for a VM, the logic remains the same. To calculate the number of logical CPUs in vSphere Client, multiply the number of sockets by the number of cores. For example, if you need to configure a VM to use 2-processor sockets, each having 2 CPU cores, then the total number of logical CPUs is 2*2=4. It means that you should select 4 CPUs in the virtual hardware options of the VM in vSphere Client to apply this configuration.
Configuring Cores per CPU in vSphere Client
Let’s explain how to configure CPU options for a VM in VMware vSphere Client. Enter the IP address of your vCenter Server in a web browser, and open VMware vSphere Client. In the navigator, open Hosts and Clusters, and select the needed virtual machine that you want to configure. Make sure that the VM is powered off to be able to change CPU configuration. The configuration process is identical in vSphere 6.x, 7.0, and 8.0.
- Right-click the VM, and in the context menu, hit Edit Settings to open virtual machine settings.
- Expand the CPU section in the Virtual Hardware tab of the Edit Settings window.
- CPU. Click the drop-down menu in the CPU string, and select the total number of needed logical processors for this VM. In this example, we select 4 logical processors for the Ubuntu VM (blog-Ubuntu1).
- Cores per Socket. In this string, click the drop-down menu, and select the needed number of cores for each virtual socket (processor).
- CPU Hot Plug. If you want to use this feature, select the Enable CPU Hot Add checkbox. Remember limitations and requirements.
- Reservation. Select the guaranteed minimum allocation of CPU clock speed (frequency, MHz, or GHz) for a virtual machine on an ESXi host or cluster.
- Limit. Select the maximum CPU clock speed for a VM processor. This frequency is the maximum frequency for a virtual machine, even if this VM is the only VM running on the ESXi host or cluster with more free processor resources. The set limit is true for all virtual processors of a VM. If a VM has 2 single-core processors, and the limit is 1000 MHz, then both virtual processors work with a total clock speed of one million cycles per second (500 MHz for each core).
- Shares. This parameter defines the priority of resource consumption by virtual machines (Low, Normal, High, Custom) on an ESXi host or resource pool. Unlike Reservation and Limit parameters, the Shares parameter is applied for a VM only if there is a lack of CPU resources within an ESXi host, resource pool, or DRS cluster.
Available options for the Shares parameter:
- Low – 500 shares per virtual processor
- Normal – 1000 shares per virtual processor
- High – 2000 shares per virtual processor
- Custom – set a custom value
The higher the Shares value is, the higher the amount of CPU resources provisioned for a VM within an ESXi host or a resource pool.
- Hardware virtualization. Select this checkbox to enable nested virtualization. This option is useful if you want to run a VM inside a VM for testing or educational purposes.
- Performance counters. This feature is used to allow an application installed within the virtual machine to be debugged and optimized after measuring CPU performance.
- Scheduling Affinity. This option is used to assign a VM to a specific processor. The entered value can be like this: “0, 2, 4-7”.
- I/O MMU. This feature allows VMs to have direct access to hardware input/output devices such as storage controllers, network cards, and graphic cards (rather than using emulated or paravirtualized devices). I/O MMU is also called Intel Virtualization Technology for Directed I/O (Intel VT-d) and AMD I/O Virtualization (AMD-V). I/O MMU is disabled by default. Using this option is deprecated in vSphere 7.0. If I/O MMU is enabled for a VM, the VM cannot be migrated with vMotion and is not compatible with snapshots, memory overcommit, suspended VM state, physical NIC sharing, and NSX network virtualization.
If you use a standalone ESXi host and use VMware Host Client to configure VMs in a web browser, the configuration principle is the same as for VMware vSphere Client.
Configuring Cores per CPU in VMware Workstation
If you connect to vCenter Server or ESXi host in VMware Workstation and open VM settings of a vSphere VM, you can edit the basic configuration of virtual processors.
- Click VM > Settings, select the Hardware tab, and click Processors.
- Select the number of virtual processors (sockets) and the number of cores per processor.
In the following screenshot, you see the processor configuration for the same Ubuntu VM that was configured before in vSphere Client.
The number of total processor cores (logical cores of physical processors on an ESXi host or cluster) is calculated and displayed below automatically. In comparison, in the interface of vSphere Client, you set the number of total processor cores (the CPUs option), select the number of cores per processor, and then the number of virtual sockets is calculated and displayed.
Configuring VM Processors in PowerCLI
If you prefer using the command-line interface to configure components of VMware vSphere, use PowerCLI to edit the CPU configuration of VMs. Let’s find out how to edit VM CPU configuration for a VM which is Ubuntu 19 in PowerCLI. The commands are used for VMs that are powered off.
- To configure a VM to use two single-core virtual processors (two virtual sockets are used), use the command:
get-VM -name Ubuntu19 | set-VM -NumCpu 2Enter another number if you want to set another number of processors (sockets) to a VM.
- In the following example, you see how to configure a VM to use two dual-core virtual processors (2 sockets are used):
$VM=Get-VM -Name Ubuntu19$VMSpec=New-Object -Type VMware.Vim.VirtualMachineConfigSpec -Property @{ "NumCoresPerSocket" = 2}$VM.ExtensionData.ReconfigVM_Task($VMSpec)$VM | Set-VM -NumCPU 2
- Once a new CPU configuration is applied to the virtual machine, this configuration is saved in the VMX configuration file of the VM. In our case, we check the Ubuntu19.vmx file located in the VM directory on the datastore (/vmfs/volumes/datastore2/Ubuntu19/). Lines with new CPU configuration are located at the end of the VMX file.
numvcpus = “2”
cpuid.coresPerSocket = “2”
- If you need to reduce the number of processors (sockets) for a VM, use the same command as shown before with less quantity. For example, to set one processor (socket) for a VM, use this command:
get-VM -name Ubuntu19 | set-VM -NumCpu 1
The main advantage of using PowerCLI is the ability to configure multiple VMs in bulk. Task automation is important and convenient if the number of virtual machines to configure is high. Use VMware cmdlets and syntax of Microsoft PowerShell to create scripts.
Conclusion
The performance of applications running on a VM depends on the correct CPU and memory configuration. In VMware vSphere 6.5 and later versions, set more cores in CPU for virtual machines and use the CPU cores per socket approach. If you use vSphere versions older than vSphere 6.5, configure the number of sockets without increasing the number of CPU cores for a VM due to the different behavior of vNUMA in newer and older vSphere versions. Take into account the licensing model of software you need to install on a VM. If the software is licensed per CPU, configure more cores per CPU in VM settings.
When using virtual machines in VMware vSphere, don’t forget to use VMware backup tools. Use NAKIVO Backup & Replication to back up your virtual machines, including VMs that have multiple cores per CPU. Regular backup helps you protect your data and recover the data in case of a disaster.
