VMware vSphere 7.0 Update 3: An Overview of the New Features

VMware vSphere 7.0 Update 3 is the latest vSphere version at the moment of writing this blog post, and it provides a set of new features and enhancements. VMware has released new updates every six months starting from vSphere 7.0. VMware has been listening to its customers and providing improvements with new updates. I recently reviewed the new features in vSphere 7.0 Update 2, and now vSphere 7.0 Update 3 is ready. What’s new in vSphere 7.0 Update 3?

VMware vSphere 7.0 U3 was released on October 5, 2021, and provides many useful features, such as:

• Kubernetes improvements
• Improved usability
• New storage options
• Advanced security
• vSAN 7 Update 3

Read this blog post to learn more about VMware vSphere 7.0 Update 3 (or vSphere 7U3).

Important Update: Critical Issues Found in VMware vSphere 7 U3

Since the initial publication of this post, VMware recalled this VMware vSphere 7 U3 release. In November 2021 VMware removed ESXi 7.0 Update 3 (including 7.0 U3a and 7.0 U3b) and vCenter 7.0 Update 3 from the official website because critical issues were found after the release. Customers identified these issues after installing new vSphere 7U3 versions in their environments.

Known VMware vSphere 7 Update 3 critical issues and situations when they can occur are listed below.

• An ESXi host crash with a PSOD (Purple Screen of Death) occurs when performing UNMAP/TRIM operations by VMs on VMFS 6 datastores.
• The network driver changed from i40enu to i40en can result ESXi upgrade failure with the error message: “host returned esxupdate code –1”.
• If you try to enable VMware High Availability (HA), enabling HA can never complete or fail.
• There are conditions resulting in “Failed to lock the file” conflicts when an ESXi host tries to start a VM or migrate a VM with vMotion. It is presumed that the VM uses RDM LUNs in this case.
•  The ESXi hostd service crashes due to the memory corruption in a function responsible for time service event monitoring.
• Updating with Lifecycle Manager is not available in VMware vSphere Client.
• You may be not available to patch vCenter to version 7.0 U3 because of failure when using older configuration. If there is a stale extension in the vPostgres database which was in use by configuration of the previous vCenter High Availability, upgrading also fails.
• SMB protocol can be blocked due to the Federal Information Processing Standards (FIPS) compliance enabled by default. vCenter Appliance Management Interface (VAMI) backup can fail with the error: “Path not exported by remote file system”.
• A file based backup failure occurs when there is a false result which says that an unhealthy database is detected in environments with vSAN.

You can find the most recent information on the VMware website. As of January 2022, There has been no new release of VMware vSphere 7.0 since the recall. VMware provides technical support for customers using vSphere 7 U3, and you can find more information about official workarounds here. Below you can find my original post detailing the 7 Update 3 release.

Kubernetes Improvements

The use of Kubernetes to run containers is spreading across organizations. VMware has been taking this fact into account and has improved VMware Tanzu to run Kubernetes with each new release. In VMware vSphere 7.0 Update 3, the Kubernetes setup becomes more convenient compared to previous versions of vSphere with Tanzu.

Streamlined network setup for Kubernetes clusters

Flexible DHCP support has been added. Now you can use a configuration from a DHCP server to obtain network settings (IP addresses, DNS servers, NTP, and other values) with more convenience for workload networks and management networks. The new DHCP network mode is integrated into the existing environment. You can use DHCP for the workload cluster and static IP configuration for the management cluster.

Human-readable and more understandable error messages

Kubernetes is a complex environment, and errors can occur. Error messages are now more detailed and contain the necessary information to help administrators and users. Error messages contain more detailed error descriptions that can help you detect what’s broken and fix the related issue. VMware vSphere checks the Tanzu configuration to provide more accurate reports. Using information from new error messages can be more effective than reading pages of error logs manually each time that you need to deal with an error.

Simplifying Operations

If you use a complex VMware configuration, the VMware update to vSphere 7.0 Update 3 can simplify a couple of operations when administering the vSphere environment.

Viewing memory statistics is more convenient

In previous vSphere versions, you had to install additional plug-ins and command-line tools to view memory statistics. In VMware vSphere 7 Update 3, the detailed memory statistics are now part of vSphere and can be easily accessed in the user interface. Troubleshooting and monitoring of memory resources at the host level and VM level are now more convenient. Read more about the importance of infrastructure monitoring in this blog post.

vCenter plugin for NSX

This plugin allows system administrators to install and configure NSX networking in VMware vSphere Client more easily. There is no need to open and log in to the NSX manager when using this vCenter plugin for NSX due to the seamless authentication between vCenter and NSX-T. Integration of vSphere and NSX and the ability to manage settings in a single pane of glass is more user-friendly. Note that an NSX-T license is required to use the plugin.

Improved network management with NSX-T

You can use vSphere distributed virtual switches for distributed firewalling in NSX-T environments. This approach requires fewer network changes in the configuration of vSphere and NSX.

Better logic for maintenance mode and DRS improvements

The logic of trying to enter maintenance mode again if the previous attempt was not successful is better in VMware vSphere 7 U3. The logic of migrating workloads by Distributed Resource Scheduler (DRS) has also been improved. In previous VMware vSphere versions, DRS moved VMs to different ESXi hosts without taking into account the configuration of these VMs. However, migrating VMs with heavy I/O (highly loaded CPU, intensive changes in memory, highly loaded network bandwidth) needs more time for vMotion and more CPU resources.

In VMware vSphere 7 Update 3, DRS is more focused on large workloads that are heavy to move such as large VMs and VMs with a large number of input/output operations (I/O). The logic behind DRS now is to move heavy VMs as few times as possible. This principle helps reduce risks of failure during maintenance operations, upgrades, and VM migrations. DRS tries to migrate a large VM from the host that is put into maintenance mode to an ESXi host of the latest version (that has been already updated). Then the VM remains running on the host to which this large VM was migrated without migration back (now it is not necessary to move this VM back in VMware vSphere 7.0 Update 3).

Read more about VMware DRS and HA clusters here.

PowerCLI 12.4

The PowerCLI update includes the new vSphere Automation API SDK Module and vSphere Management Module (6 new cmdlets for certificate management are included). PowerCLI 12.4 provides the following changes:

• Additional support for vSAN in PowerCLI
• There are new SDK modules for direct access to VMware vSphere Automation API via PowerShell. New API bindings can directly invoke REST API via PowerCLI.
• vSphere API documentation has been updated.
• New PowerCLI logo

Lifecycle Management

Lifecycle management has changed, and new features have been added for higher reliability and a faster vSphere update process.

Depot Editing

In VMware vSphere 7.0 Update 2, VMware added a software depot to store ESXi configurations in a single database. In VMware vSphere 7.0 Update 3, you can edit the image depot and also edit drivers and components that may be changed. You can manage patch, update, and recall objects. You can also delete unnecessary objects in the depot. This functionality is available only via API, but VMware will probably provide this functionality in the user interface later.

Firmware compatibility

Hardware compatibility has been extended to include firmware for drives in addition to storage controllers. This approach can help improve supportability and compatibility to make them more granular, especially for vSAN. VMware can add more hardware support into the VMware image depot.

VMware vSphere Lifecycle Manager now can validate Non-Volatile Memory Express (NVMe) storage devices according to the hardware compatibility list (HCL) to deploy a vSAN cluster.

Less downtime for vCenter upgrade

The downtime when upgrading vCenter is reduced. This news is good for those who use VMware Cloud Services. The steps to upgrade vCenter include:

• A new instance of vCEnter Server Appliance (VCSA) is created at the start of the vCenter upgrade process.
• Configuration and database are copied to the new VCSA instance with further synchronization.
• The VCSA switchover process is performed. VSCA services on the new instance of VCSA are started. The downtime occurs at this step.

vSAN witness management

VMware has added support for dedicated witness host appliances for vSAN topologies in vSphere 7 U3. vSphere Lifecycle Manager (vLCM) now manages standalone vSAN witness nodes. You can use vLCM to patch the witness node and save time when patching vSAN clusters. This feature was not supported for shared vSAN witness nodes before.

vSAN 7 Update 3

VMware tries to make vSAN more user-friendly in each new VMware vSphere release. Let’s look at the new interesting features in VMware vSAN 7 Update 3.

VM I/O Trip Analyzer to understand performance bottlenecks

Finding performance bottlenecks in distributed storage systems is not easy. To simplify this process a new tool has been added to vSAN 7 U3 in VMware vSphere 7.0 Update 3. VM I/O trip analyzer is integrated into vCenter Server to help administrators diagnose issues related to the I/O latency of VMs (for example, latency and low performance caused by network issues or disk issues). VM I/O trip analyzer has a convenient graphical user interface that can display a visualized I/O topology. This I/O analyzer tool checks latencies at each layer to detect the source of the problem.

Note: Trip Analyzer is not supported in the two-node cluster, stretched vSAN cluster, and virtual disks in a remote vSAN datastore.

New metrics and health checks for network monitoring can help configure vSAN better and make troubleshooting easier.

Resilient 2-node clusters

A secondary level of resilience is added to a 2-node cluster in cases when each ESXi host contains three or more groups of disks. As a result, if one host fails with the consequent failure of a witness and a disk group on a surviving host, full data availability is still provided. This functionality is especially useful for small organizations and makes clusters more affordable for them.

Using a vSAN stretched cluster for a Kubernetes cluster

VMware vSAN stretched clusters are used when a vSphere environment is geographically distributed. In previous versions, you could run only traditional VMs in stretched vSAN clusters. In VMware vSphere 7U3, you can use the storage of a stretched vSAN cluster to deploy a Kubernetes cluster.

Better encryption support

Trusted Platform Modules (TPMs) on ESXi hosts are fully supported by vSAN 7U3 to avoid issues related to missing communication with the key provider. You can use TPMs with the vSphere Native Key Provider (NKP) or external Key Management Service (KMS).

vSAN cluster shutdown

The full graceful shutdown of a vSAN cluster has now been simplified. It was difficult to completely stop a vSAN cluster before. The data and control/management plane are now halted in the desired state intelligently to get predictable results.

Storage

Storage is a crucial component of each platform. There are changes and improvements for storage configuration in VMware vSphere 7 U3. Some configuration limits have been changed. Let’s look at the most important changes in terms of storage in the latest version of VMware vSphere.

More hosts can be connected to a shared datastore

In vSphere 7.0 Update 3, you can connect up to 128 ESXi hosts to the same NFS or VMFS 6 datastore without special approval. This option allows you to avoid needing Storage vMotion in some situations, makes the upgrade operations more convenient, etc. This improvement doesn’t increase the maximum number of hosts in the cluster.

sNVMe over TCP support

Solid State Drive Non-Volatile Memory Express (SSD NVMe or sNVMe) storage has become more popular in data centers. VMware vSphere 7 Update 3 supports the NVMe over TCP storage protocol that makes using modern storage technologies more efficient and affordable.

Support for NVMe over fabrics (NVMe-oF) allows you to use traditional network adapters to access these storage devices even if you don’t have Fibre Channel host bus adapters (HBA) and Remote Direct Memory Access (RDMA) capable adapters. While non-NVMe protocols result in significant latency, the NVMe-oF increases performance and reduces latency when accessing NVMe storage via networks.

Improved vVols

Taking large volumes of snapshots for virtual volumes (vVols) becomes better in VMware vSphere 7 Update 3. The snapshot operations are performed as a batch and are processed more intelligently.

Installing ESXi on USB devices is deprecated

Installing ESXi on a USB flash drive or SD flash card was the option for previous ESXi versions in VMware vSphere. Some administrators install ESXi on USB flash drives to save drive bays in the server and reduce costs spent on buying disks to install ESXi. However, this installation method has a list of limitations and disadvantages. The endurance of USB devices is much lower than the endurance of server-class hard disk drives (HDD) or solid-state drives (SSD). This fact has a negative impact on the reliability of an ESXi server. Not all partitions are created when ESXi is installed on a USB flash drive (some partitions on which storage-intensive read-write operations are performed are not created to avoid the fast failure of a USB drive). ESXi 7.0 Update 3 has also architectural changes in how the system partitions are used (the new layout configuration of boot storage is used and USB flash devices can wear out more quickly now). The config store is stored on boot media with ESXi configuration.

For these reasons, installing ESXi on SD cards and USB drives in VMware vSphere 7.0 Update 3 is deprecated. You can still do this but with some difficulty. The warning message that ESXi is running in the degraded mode is displayed if the boot partition is located on an SD card or USB drive.

ALERT: No persistent storage is available for system logs and data. ESX is operating with limited system storage space, logs and system data will be lost on reboot.

Thus the ESXi operating system must now be stored on reliable persistent disks. The minimum storage requirement to install ESXi 7.0 Update 3 is 32 GB.

Security in VMware vSphere 7 Update 3

There are some updates in the category of security in VMware vSphere 7 U3.

Full support for AMD Virtualization Based Security (VBS)

This feature is also known as Microsoft Device Guard and Credential Guard and can help protect credentials in VMs running Windows operating systems. Hyper-V is activated in Windows VM to store secrets in a protected isolated area in memory. When running such VMs in vSphere, nested virtualization is used. VBS was supported for Intel processors in previous versions of vSphere. Now, this feature is fully supported for AMD processors.

UEFI 2.4

Unified Extensible Firmware Interface (UEFI) was updated to version 2.4 for virtual machines to support running the latest versions of operating systems.

Guest Data Publisher was updated

Guest Data Publisher is a feature released in vSphere 7 Update 2 that allows users to transfer data from storage on an ESXi host to a guest operating system inside a virtual machine.

Updated documentation

VMware has updated its documentation to provide more information on how to deploy and configure VMware vSphere with a high level of security.

Improvements for AI/ML

Due to the partnership of VMware and NVIDIA, the latest vSphere version has new improvements in the artificial intelligence and machine learning categories (AI/ML). You can use GPU hardware, GPU virtualization technologies, pre-built tools powered by NVIDIA, and VMware virtual infrastructure to deploy the environment for running AI/ML tasks more quickly. Some main improvements are:

• APIs for cuDNN 8.1.1, CUDA 11.2.2, and NCCL 2.8.4 are supported.
• You can use Kubernetes commands to provision VMs on ESXi hosts with GPUs. As a result, customers can run their AI-based applications on the hardware with GPU by using the self-service model.

Bitfusion 4.0 support

Bitfusion shares Graphics Processing Units (GPUs) and provides a pool of shared resources that can be used by AI and ML workloads. AI frameworks are used for the operation of Bitfusion and working with VMs, containers, and other environments. There are improvements for work with client authentication tokens, retention policies are added, scheduling for GPU workloads is optimized, and monitoring plugins are added. The dark view of the Bitfusion plugin is now available.

Other Improvements in VMware vSphere 7 U3

There are also other improvements in the latest vSphere version (VMware vSphere 7.0 U3 at this moment) that should be mentioned.

NTP as a backup protocol if PTP issues occur

In the previous vSphere version (7.0 U2), VMware released the support of Precision Time Protocol (PTP), which allows VMs to synchronize time with high precision. Synchronization is performed with an ESXi host by using VMware Tools installed in a guest OS of the VM.

In VMware vSphere 7.0 Update 3, VMware provides a failback mechanism if for any reason the PTP sources are not available to synchronize the time. In this case, time is synchronized with the NTP sources. When the PTP sources are back online, then PTP is preferred. You can use vmware.pool.ntp.org and other VMware servers as failback servers for time synchronization via NTP.

Another improvement is that PTP can be configured in two ways now: by using a VM kernel adapter or a dedicated pass-through interface.

Smarter Agent VMs in a cluster

VMware introduced the new vSphere Cluster Services (vCLS) in VMware vSphere 7.0 Update 1. VMware vCLS VMs are run in vSphere for this reason (to take some services previously provided by vCenter only and enable these services on a cluster level). These agent VMs are mandatory for the operation of a DRS cluster and are created when you enable DRS. In previous vSphere versions, you could not select the storage to run vCLS VMs. In VMware vSphere 7.0 Update 3, you can select a datastore in which you want to run vCLS VMs in vSphere and edit some affinity settings. Brackets are removed from the names of vCLS agent VMs for better support and compatibility. Moreover, VM unique identifiers (UUIDs) are now used to identify which VM belongs to which cluster in a user interface.

Conclusion

VMware vSphere 7.0 Update 3 is the latest version of VMware vSphere as of Autumn 2021. This version of vSphere contains new useful features that deserve your attention. Using Kubernetes and vSAN clusters becomes more convenient in vSphere 7 U3. There are usability improvements and some configuration limits are increased. Despite the version of VMware vSphere used in your environment, it is recommended that you back up your VMs. NAKIVO Backup & Replication v.10.4.1 supports VMware vSphere 7.0 Update 3 and can help you protect your VMware VMs. Watch this webinar to learn more about VM backup with NAKIVO Backup & Replication.