LAN-Free Backup Technologies for VMware vSphere
During a backup, large amounts of data are transferred through the networks, slowing down business-critical operations in the production environment. Network administrators use various approaches to minimize the impact of backups on daily operations. These approaches include scheduling backups off-business hours and using traffic-reducing techniques such as data compression and incremental backups. However, one of the best ways to optimize backup traffic is the segregation of production and backup networks. And one of the possible approaches to do this is LAN-free backup.
In this post, we guide you through the two most common LAN-free backup techniques used in VMware virtual environments: direct SAN access and HotAdd. We’ll also explain how NAKIVO Backup & Replication, a comprehensive backup and disaster recovery solution, leverages these techniques for more efficient VM backup.
LAN-Free Transport Methods in VMware
VMware provides multiple data transport methods for copying data from VM virtual disks when creating image-level backups. These methods allow backup applications to copy data in the most rational way to achieve higher data transfer speeds and lower the load on production hardware.
When copying data over a local area network (LAN), the NBD (Network Block Device) and NBDSSL (NBD with SSL) transport modes are used. However, in this blog post, we are interested in LAN-free transport modes like Direct SAN and HotAdd. These modes allow backup software to read data directly from storage attached to ESXi hosts bypassing LANs.
To access storage directly, storage-aware applications such as backup software use the vSphere API for Data Protection (VADP) and Virtual Disk Development Kit (VDDK) provided by VMware. This allows these applications to transfer VM data in the most resource-efficient way and without the need for agents to be installed on source machines.
LAN-Free Backup Method 1: Direct SAN Access
A storage area network (SAN) is a network of disk drives set up as a single entity. Servers in a SAN have access to the storage capacities in this shared pool of storage drives without being tied to a specific HDD or SSD.
Physically, storage arrays, servers, and switches in a SAN can be connected with Fibre Channel or Ethernet cables. Data transfer inside the SAN can be implemented with iSCSI, Fibre Channel, or Fibre Channel over Ethernet protocols. Within a SAN, each storage device (either physical or virtual) is given a unique logical unit number (LUN) to identify it, and this LUN is used to connect the device to a server. Data inside the SAN flows without exiting into the LAN, and direct SAN access leverages this to bypass the LAN when copying data to a backup repository.
Generally, system administrators use SAN to improve data storage efficiency, scalability, and redundancy. By connecting a LUN/device to more than one server, you can ensure business continuity. Even if the primary server goes down, the data will not be lost because it is possible to switch to another server.
How Direct SAN access works
When leveraging direct SAN access in VMware environments, the backup application reads data directly from the LUN where the virtual disk of the source VM is located without transferring it through the ESXi hosts or the LAN. To do this, the backup proxy must be installed on a physical machine connected to the SAN datastores via the storage network (can also be a virtual machine connected to the SAN datastore). A backup proxy is a backup application component for transferring VMware VM data between the source and the backup repository. In the diagram below, the backup proxy is represented as Backup Application.
Note: The backup application uses the virtual disk API and Virtual Disk Development Kit (VDDK), which facilitate getting information about the LUN configuration and accessing VM virtual disks. By using the VMware VDDK library, the backup can be completely agentless, so the backup process does not impact the consistency of data of the source VM or affect the operations running on it. The backup storage is connected to the machine on which the backup application is running.
Benefits
- The Direct SAN transport mode is the fastest way to back up VMware VMs deployed on ESXi hosts connected to a SAN. This mode also reduces the load on production networks and hardware.
Constraints
- A backup application (the backup proxy component) must be installed on either a physical server connected to the SAN or a VM running on the ESXi host physically connected to the SAN. Note that the ESXi host running the VM with the backup application must be a standalone ESXi host. This means that the ESXi host must not be managed by vCenter Server, whose inventory (that is, the VMs managed by this vCenter Server) you want to back up.
- The SAN transport mode doesn’t work with VMware vVols and VMware vSAN. The performance is higher for thick provisioned virtual disks when restoring VMs using the SAN transport mode.
- VMware Tools must be installed on VMs to use features such as device hot add and quiesced snapshots (application-aware snapshots) for data backup.
- The SAN transport mode works with storage devices that can be accessed as a LUN at the driver level (not the file system level, such as NTFS or EXT4). The LUN must be accessed as a raw device.
Ensure that the operating system on which the backup application (or backup proxy) is installed doesn’t initialize (re-signature) a SAN volume with an VMFS file system. VMFS is not supported in operating systems other than ESXi, and such initialization can make a VMFS volume unreadable by an ESXi host. You may need to edit SAN policy settings in the operating system with a backup application installed. The iSCSI initiator must be started on this machine to access a LUN on SAN via iSCSI.
Direct SAN Access in NAKIVO Backup & Replication
NAKIVO Backup & Replication is a data protection solution designed for efficient data protection in VMware vSphere environments. The solution leverages VMware APIs to create agentless image-based backups as well as native VMware technologies like Changed Block Tracking (CBT) and LAN-free data transfer modes for more efficient data protection.
You can use the Direct SAN access mode when creating a VMware vSphere VM backup in the NAKIVO solution. The Transporter is the component responsible for transferring data from vSphere environments to backup storage, replicating VMs between ESXi hosts, etc., and the Director is the component that provides the web interface. The Transporter acts as a backup proxy, to use the terminology of VMware transport methods. The NAKIVO solution can be installed on a physical machine or as VMware vSphere virtual appliance on a VM.
LAN-Free Backup Method 2: HotAdd
HotAdd, also called SCSI HotAdd, is a VMware capability that allows attaching a physical or virtual device while a VM is running (without any downtime). You can hot add devices such as virtual processors, memory, and disks in a VMware VM while the VM is running. In the case of using this VMware technology to create backups while bypassing LAN, a snapshot of a source VM is mounted (that is, “hot added”) as a virtual disk to the VM on which the backup application is installed (the target VM).
Note: A linked clone is a copy of a VM created from the snapshot of the source VM. A linked clone shares the virtual disk of the source (also called parent) VM and can write new data to the differenced disk created with the snapshot.
Running a backup application (or backup proxy) on a virtual machine as a virtual appliance provides you with the advantage of mounting a virtual disk directly to this target VM. Mounting the virtual disk can be done after creating the snapshot of the source VM. In this case, the target VM (with the backup application) can read the contents of the virtual disk of the linked clone and copy data to the backup destination.
When the HotAdd mode is used, a backup solution can read VM data directly from the VM datastores through the storage I/O stack, bypassing the host’s TCP/IP stack that would otherwise impact every VM hosted on the server and slow down the data transfer.
This transport mode is usually used when direct SAN access is not available, for example, if you simply do not use a SAN device. HotAdd yields almost the same performance as the direct SAN access, but it has limitations as well.
Constraints
- The backup application must be installed on a virtual machine to allow a virtual disk to be mounted to it.
- The VM with the backup application installed must have direct access to the storage on which the source VM resides. This storage may be the same physical disk if the backup application and the source VM are on the same host. Such storage can also be a shared one, like NAS or even SAN.
- The HotAdd transport mode is not supported for virtual disks connected to VMs via the IDE virtual disk controller. Virtual disks must be connected to a VM via a SCSI virtual disk controller. SCSI disks support hot add/hot plug, while IDE disks don’t support this feature. A paravirtualized SCSI controller is recommended.
- The maximum number of virtual disks that can be attached to a virtual SCSI controller for VMware VMs is 15. If the number of virtual disks you need to back up at once exceeds the limit (note that at least one virtual disk is used by a backup solution to run), you may need to add one more virtual SCSI disk controller.
How HotAdd transport works
Below, you can see a diagram with two VMware ESXi hosts and virtual machines running on them. Both ESXi hosts are connected to the shared storage via the storage network and to the LAN (production network).
- The backup application (backup proxy) requests the creation of a snapshot of the source virtual disk (.vmdk) of the VM to back up (whether crash or application consistent backups) and the creation of a linked clone. The application uses the virtual disk API and VDDK with VixTransport and VixDiskLib to handle the linked clone.
- This virtual disk located on the ESXi storage is hot added to the target VM with the backup application, which also has access to the same ESXi storage. ESXi storage can be shared storage (as shown in the diagram below).
- The backup application copies virtual disk data from the disks attached to it without using the production network (LAN in the image below).
- Once the backup process is completed, the virtual disk is detached and the snapshot is deleted.
Note: Just as with the direct SAN access mode, VM backup is agentless when using the HotAdd transport mode.
HotAdd in NAKIVO Backup & Replication
As we mentioned above, the NAKIVO solution allows you to maintain the benefits of virtualization with backup and recovery that rely on VMware APIs and native technologies for faster and more resource-efficient backups. In addition to allowing you to use direct SAN access for VMware vSphere data protection, the solution can use the HotAdd technology to bypass LAN and unload production resources.
The following diagram shows how NAKIVO Backup & Replication runs VMware VM backup by using the HotAdd mode for LAN-Free data transfer. The Transporter, the solution component responsible for data transfers, acts as the backup proxy. All VMs in this example are located on the same ESXi host.
- Snapshots of source or target VM disks are attached to the VM with the Transporter installed by using VADP and VDDK (a target VM is a VM replica or a VM to which we restore data). The Transporter can be connected to another Transporter installed on the machine with the backup repository.
- Data is copied without using LAN.
- After the job is completed, these virtual disk snapshots are detached from the VM with the Transporter.
LAN-Free Backup Example: HotAdd with the NAKIVO Solution
For a better understanding how LAN-free backup is implemented in backup software, let’s look at an example of a real VMware vSphere backup job in NAKIVO Backup & Replication. In this example, we use one ESXi host managed by vCenter, a virtual appliance with the NAKIVO solution installed (all solution components including the Transporter are installed on the same VM), and a Windows Server 2019 VM that we want to back up (Win19-02). Both VMs are located on the same datastore attached to the ESXi host.
In the NAKIVO solution, you can select the transport mode in the VMware backup job creation wizard. The transport mode can be selected automatically, or you can set it manually (Automatic is the default option). Our VMware vSphere configuration only supports the HotAdd mode for LAN-free VM backup.
Let’s log in to Linux installed on the NAKIVO solution virtual appliance and check block storage devices connected to this machine with the commands:
ls /dev/disk/by-path
lsblk
Remember the block devices attached before running the backup job (see the screenshot below).
- sda is a system disk with two partitions.
- sdb is used for a built-in backup repository.
- sr0 is a virtual DVD-ROM drive.
When we run the backup job, we see that the HotAdd transport mode is selected for LAN-free VM backup. The VM we are backing up contains one virtual disk, whose size is 40 GB. The used space on this disk is 18.1 GB.
While the backup job is running, we check the block devices connected to our Linux-based virtual appliance once again by using the same commands.
You can notice that a new block device was added. This is because a temporary virtual disk snapshot was created of the source VM (the VM that is being backed up), and it is attached to our virtual appliance as a SCSI disk with the VMware HotAdd feature.
The attached SCSI disk named sdc contains 5 partitions. You can see that the disk size is 40 GB, which is equal to the size we checked before in the web interface when managing the backup job.
If we open VMware vSphere Client, we can see the temporary snapshot created for our Win19-02 VM, which is being backed up. A description is provided for the snapshot with information about the application that triggered its creation.
There are also temporary files stored in the directory with VM files of the NAKIVO backup appliance. These files are related to the mounted virtual disk snapshot that is used for LAN-free VM backup with the HotAdd transport mode.
After the backup job is completed, the SCSI disk is unmounted from the Linux-based virtual appliance, and the temporary virtual disk snapshot is unmounted and deleted for the source VM (Win19-02). The workflow is completed.
