Comparing Citrix XenServer, Microsoft Hyper-V and VMware ESX

Solution Comparison
Functionality CoreTechnology "Bare metal" deployment Core features (vSMP) vLAN Support Management & Administration Role-based administration Physical to virtual (P2V) migration tool Auto-VM placement at startup No-downtime platform updates, host maintenance Auto-VM placement at start-up VM Load Management Business Continuity Automated VM high availability (HA) Policy-based HA Fault tolerance (zero downtime HA) Base support for DR via storage replication Disaster recovery workflow automation Workload Delivery VM Live Migration Virtual machine interoperability (coming soon) Dynamic workload streaming to physical/virtual Shared image streaming to VMs Storage Management Support for local storage Support for shared storage (SAN, NAS) Live storage migration Access to native storage features Backup Management Backup and recovery support Backup proxy server Access to native storage snapshot features (Workflow Studio) (Site Recovery Manager) (with add-on) (via Microsoft Cluster Services) (Alert-based) (Alert-based or Automated) (with HP OEM edition) Citrix XenServer 5.0 VMware ESX-VI 3.5 Microsoft Hyper-V 1.0

Overview
The following is a comparison of bare-metal server virtualization offerings from Citrix, Microsoft and VMware, widely accepted as the industrys three leading server virtualization technologies for x86 servers. Product Offerings Each vendors offering includes a hypervisor, a set of tools to manage virtual environments, and a multi-server management console to manage virtual machines across a group of host servers. Citrix XenServer is a virtualization platform based on the open-source Xen hypervisor and includes XenCenter, a multi-server management console. XenServer is available in Platinum, Enterprise and Standard Editions as well as a free Express Edition. All XenServer editions, including Express, include the XenCenter management console. VMware Infrastructure is a virtualzation platform that includes the VMware ESX Server hypervisor. VirtualCenter, VMwares multi-server management console, is licensed separately. Virtual Infrastructure is offered in three editions: Enterprise, Standard, and Foundation. VMware has introduced a free, lightweight hypervisor called VMware ESXi that can be used in place of ESX. Microsoft Hyper-V is part of the latest Windows Server operating system, Windows Server 2008. Microsoft System Center Virtual Machine Manager (SCVMM) offers mulit-server management capability, and is licensed separately per managed physical host. SCVMM comes in a Workgroup Edition for managing 5 or fewer host servers and an Enterprise Edition for larger environments.

systems and processors (i.e. Intel VT, AMD-V). ESXs approach to virtualization can best be described as binary translation. In effect, ESX tricks guest operating systems into thinking they are running on physical hardware. Furthermore, ESX manages all processor instruction requests from guest VMs, to ensure that OS calls that are intended for real hardware are handled safely. Due to the need to perform so much of this work in software, ESX is a very sophisticated and complex system.

VMware ESX Binary Translation XenServer and Hyper-V have very similar architectures, which differ significantly from VMware. Both solutions employ a combination of paravirtualization and hardware-assisted virtualization. XenServer was the first solution on the market to employ paravirtualization, which allows guest operating systems to be fully aware that they are being run on virtualized hardware. This collaboration between the OS and the virtualization platform enables the development of a simpler, leaner hypervisor, as well as highly optimized performance. Linux was the first OS to be paravirtualized, and was enabled to cooperate with XenServer to ensure optimal performance. XenServer virtual machines use paravirtualized open-source device drivers (i.e. storage, networking) installed into the Domain 0 control domain, which runs a hardened instance of Linux. For guest OSs that cant be fully paravirtualized (i.e. closed-source OSs), XenServer was designed to leverage hardware virtualization assist technologies, available on modern Intel and AMD processors (Intel VT and AMD-V). Intel Research was involved in the early development stages of the open source Xen project.

Comparisons
Hypervisor Architecture All three vendors offerings are bare metal hypervisor technologies that are installed directly onto physical servers without requiring a host operating system (OS), unlike solutions such as Microsoft Virtual Server or VMware Server (GSX). This bare-metal approach, employed by all three solutions, is widely accepted as offering significantly better performance than solutions reliant on a host OS. VMware ESX has been on the market the longest, and its first generation architecture predates virtualization-aware operating

XenServer Architecture As mentioned, Hyper-Vs design architecture mirrors the architecture of Citrix XenServer, also using a combination of paravirtualization and hardware-assisted virtualization. The key difference between Xen and Hyper-V is the OS used in the control domain. Hyper-Vs parent partition runs a hardened, stripped-down version of Windows Server 2008 called Server Core. Like XenServer, Hyper-V device drivers are installed into the control domain and device driver interactions with VMs are managed through the control domain. VMware, by comparison, leverages proprietary device drivers specifically developed for use with ESX. Despite initially downplaying the merits of paravirtualization, VMware has started to adopt paravirtualization into ESX in the form of its VMI technology. The release of VMI performance white paper further highlights VMwares belief in paravirtualization, and its merits over binary translation. Thus far, a few Linux operating systems (i.e. specific versions of Fedora and SuSE) are enabled to run with VMI and Virtual Infrastructure 3.5. Installation, Configuration, and Administration XenServer has a straightforward installation process, often referred to 10 minutes to Xen. XenServer is installed on the host systems using CD or network-based installation process. The XenCenter administration console is then installed on any Windows PC or server. System configuration information is kept in an internal data store on the XenServer that is replicated across all servers in a resource pool, and no thirdparty database is required. ISO and VM storage can be performed on local disks or shared storage (SAN or NAS). Microsoft Hyper-V can be installed either as a role within the full Windows Server 2008, or using the Server Core edition. In either case, the installation process runs from the Windows

Hyper-V Architecture Server 2008 install media. System Center Virtual Machine Manager (SCVMM) is somewhat more complex. SCVMM relies on Active Directory membership of all the Hyper-V host servers, and requires a SQL Server-based configuration database. Library servers, which are used for ISO storage and VM disks, need to be members of the Active Directory as well. Similar to Hyper-V and XenServer, the VMware ESX hypervisor is installed on the host servers. VMware uses Virtual Center for multi-server management and configuration. Virtual Center runs as a Windows service on a separate management server and uses a third-party database for storage and management of host system configurations. For redundancy and fault tolerance of the management database, VMware recommends traditional high-availability database techniques (i.e. SQL Server with Microsoft Cluster Server). XenServer and VMware are also available onboard the firmware of many x86 servers, including Dell and HP. In this case, no installation is required. Hyper-V is not currently available in a firmware-embedded form. Guest OS Support VMware and XenServer offer the broadest guest operation system support, including multiple versions of Windows and a number of Linux distributions. In addition, VMware ESX supports some flavors of BSD, Sun Solaris, and Novell Netware. Hyper-V is focused primarily on Windows guest support. As of September 2008, Microsoft documentation also noted beta-level Linux support for SUSE Linux Enterprise Server 10. Storage Integration Storage is one of the most important considerations for server virtualization deployments. All three solutions offer support for

4 local disk storage of VMs, iSCSI or Fibre Channel-based Storage Area Network (SAN), or Network-attached storage (NAS). Use of a SAN or NAS is required for advanced features of any of the platforms, such as Live Migration and High Availability. VMware typically leverages its proprietary VMFS file system for storage, although raw disk options are available. VMFS is the default storage system for virtual machine files, both with local and shared (SAN or NAS) storage. When used with SAN arrays, use of VMFS allows VMware to take control of certain storage functions, including provisioning and snapshotting, no matter what vendors array is being used. VMFS is a clustered file system which enables advanced features such as VMotion and Storage Vmotion. Storage Vmotion performs live migrations of VMs from one array to another without downtime. Citrix XenServer offers support for for several different underlying storage approaches, including use of Linux LVM, Microsoft VHD, and NFS. XenServers most unique storage feature comes in the form of special plug-ins for advanced array integration. These plug-ins allow XenServer to offload storage tasks to the array, directly leveraging the features of the array, including thick/thin provisioning, snapshotting, and deduplication. For example, a VM snapshot executed within XenServer would actually call the SANs API to perform this action on behalf of XenServer. In this manner, XenServer can optimize performance and permit customers to use the advanced features of the array exactly as they were intended. Features like XenMotion take advantage of a special resource control technology that works on top of all of the different file and storage systems that XenServer supports. Storage management with Hyper-V is largely handled by Windows itself, given that Windows Server runs in the Hyper-V control domain. As a result, Hyper-V inherits a number of general-purpose storage capabilities inherent in Windows. VM Backup/Recovery All three solutions offer support for traditional backup/recovery techniques using industry-standard tools. VMware includes a backup proxy server, referred to as VMware Consolidated Backup (VCB). VCBs purpose is to offload backup processing from the virtual machine hosts to ensure optimal performance. Likewise, XenServer is designed to offload backup processing from hosts by directly leveraging the snapshot capabilites present on nearly all SAN or NAS products. XenServer also includes VM export/import and host backup features within the XenCenter console. Hyper-V inherits many of the generalpurpose backup/recovery features in Windows Server itself, as well as support from its ecosystem.

Disaster Recovery Disaster recovery (DR) involves recreating virtual server infrastructure and data at remote DR facilities. Virtualization simplifies disaster recovery in many ways as virtual machines package server workloads so that they are easier to transport and restart on remote systems. All three vendors support multi-site deployments, whereby VMs can be made available in primary and DR sites. In each case, the virtualization solution relies on SAN-based replication technologies to keep VM files and configuration data current at a backup location. VMware offers Site Recovery Manager, which is essentially a workflow engine for orchestration of DR for virtualized systems. Similarly, Citrix offers Workflow Studio, which can complement XenServer for automation of DR for both virtualized and physical servers. Also, Citrix NetScaler can work in concert with XenServer to redirect traffic to a DR site should a failure occur at the primary site. Microsoft currently lacks a workflow engine specifically for virtualization and DR. High Availability High availability enables virtual machines to be restarted on another physical host, should the primary host unexpectedly fail. All three vendors include high availability (HA) software with the purchase of their higher end virtualization offerings. Both VMware and XenServer have high availbility features that offer granular policies governing the behavior of specific VMs after a host failure. Microsoft Hyper-Vs HA features are largely based on the general purpose Microsoft Cluster Server (MSCS) technology within Windows Server 2008 (Enterprise and Data Center Editions). XenServer is currently the only offering that can be enhanced to offer fault-tolerant HA capabilities resulting in zero-downtime fail-over. This capability is provided by Marathon Technologies. In a fault-tolerant fail-over scenario two or more physical servers would keep virtual machines synchronized at all times. If one server should fail, the other maintains the running workload without loss of data or interruption to users. Live Migration Both Citrix and VMware offer the ability to move running VMs from one host to another, without incurring any downtime on the VMs. Citrix calls this feature XenMotion and VMware refers to its feature as VMotion.

5 Microsoft Hyper-V currently lacks live migration features. Virtual machines running on Hyper-V can be moved, with a small amount of downtime, using Quick migration capabilities. Essentially, quick migration suspends the VM on one host, makes the VM file accessible to another host, and then unsuspends the VM on that second host. and includes automated update features in XenCenter for patching the hypervisor. XenServer does not include patching features for guests, instead leaving this task for the tools provided by the OS vendors (and their ecosystem partners). Microsoft Hyper-V has not been on the market long enough to determine how much regular maintenance is required in terms of updates and patching. Microsoft has indicated that Hyper-V would not require as much regular maintenance as the full version of Windows Server 2008. Because Hyper-V lacks live migration, hypervisor-level patching will require some VM downtime. Dynamic Workload Delivery Citrix is the only vendor to include dynamic workload streaming as part of their server virtualization offering. This technology allows users to setup a library of server workloads (operating system images, applications, configurations) and stream them on-demand to multiple virtual or physical servers within the data center. Ideal for server farms, including Citrix XenApp, dynamic workload streaming allows multiple servers to share a single physical server workload image resulting in significant storage savings as well as streamlined administration of managing server images in large, homogeneous server environments. Dynamic workload streaming can be used to rapidly deploy server workloads to the most appropriate server resources (physical or virtual) at any time during the week, month, quarter or year. This is particularly useful for applications that may be regularly migrated between testing and production environments or for systems that might require physical deployments for peak user activity during the business cycle. Dynamic workload delivery also supports VMware ESX and Microsoft Hyper-V, so customers with mixed environments can take advantage of this unique and innovative feature of Citrix XenServer.
Citrix, Xen and Citrix XenServer are trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the U.S. Patent and Trademark Office and in other countries. VMware and VMotion are registered trademarks or trademarks of VMware, Inc. Microsoft, Windows and Hyper-V are trademarks of Microsoft Corporation. registered trademarks or

VM Load Management VM load management involves two capabilites. First, it ensures that when a VM is started, it does so on the host that is most available for the VM load. Second, it ensures ongoing management of the location of VMs running in a pool of host servers. Hyper-V / SCVMM has intelligent placement of VMs via the Performance & Resource Optimization (PRO) features that will select the optimal host for a new VM when it boots. Because Hyper-V lacks live migration capability, however, it cannot perform ongoing load management while VMs are running. SCVMM can do notification of over-subscribed hosts via System Center Operations Manager, which can prompt administrators to manually address the placement of VMs on the hosts. XenServer selects the optimal host for a new VM at startup, based on available resources. In addition, the built-in alerting and performance monitoring informs administrators when load management actions may need to be performed. VMware offers intelligent placement on boot, as well as automated or semi-automated load management of VMs using its Distributed Resource Scheduler (DRS) feature. System Maintenance Like any system software, all three solutions require a certain level of regular maintenance to apply software updates and patches. Because VMware and XenServer both offer live migration features, patching and updates to the hypervisor can be performed without incurring any downtime for VMs. VMwares product includes automated patching for the hypervisor as well as guests. As a complex and sophisticated software system, VMware requires significantly more patching and updates than other system software such as the BIOS or device drivers. The VMware VI 3.5 updates site lists over 150 patches that have been issued since the product was launched in December 2007, and the automated update process aids in managing the application of these updates. Citrix has issued 4 hotfixes for XenServer 4.0 since its release in August 2007,

Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein are the property of their respective owners