Virtualize More with VM-aware Flash Storage

Tintri VMstore Overview

Technical White Paper

Tintri VMstore Overview

Flash and Virtualization: Storage, Interrupted

Enterprise storage is confronted by two revolutionizing technologies at once:
n

 irtualization is the new normal. More than 50 percent of new workloads are virtualized, and V companies are beginning to make significant investments in virtual desktop infrastructure (VDI).  ommodity flash storage is quickly becoming a significant part of both local and shared storage C infrastructure.

Existing storage offerings are poorly adapted to both virtualization and flash:
n

 irtualization benefits significantly from shared storage, but todays general-purpose shared V storage was designed 20 years before VMware popularized virtualization with a very different set of workloads in mind.  lash storage which is about 400 times faster than disk must be treated very differently F than the rotating magnetic disks most storage systems were designed to use. As a result, most solutions use an expensive and complex bolt-on approach with flash-as-cache.

One hundred percent of flash arrays can address the second problem, but are not designed to run virtual workloads efficiently, so costly space and performance are wasted on idle data. Its the equivalent of using a commercial jet to commute 30 miles it may be slightly faster than driving, but the fuel costs are prohibitively expensive. Consequently, customers struggle with existing storage systems that are poorly adapted to both flash and virtualization, inhibiting the fundamental IT goals of lower cost and greater business agility. Systems purpose-built for both virtualization and flash can overcome these issues. As data centers move from about 30 percent virtualized to well over 50 percent virtualized, deploying storage specifically designed for these environments provides substantially more value. This paper will explore the challenges of designing storage systems using flash for virtualization, and describe Tintris approach.

Flash: Speed Has Its Challenges

Flash storage can deliver 400-times greater raw performance than spinning disk, but introduces fundamental architectural changes. For comparison, the speed of sound 768 mph at sea level is only 250 times faster than the average speed of walking. To travel at supersonic speeds, engineers designed sophisticated aircraft systems specifically for high speeds. It may be possible to strap a rocket motor to ones back and attempt to travel at 768 mph, but the end result would be less than ideal. Flash poses similar challenges to existing storage systems. Multilevel cell (MLC) solid-state drives (SSDs) are the most cost-effective approach and provide excellent random IO performance, but have

TECHNICAL WHITE PAPER / 2

Tintri VMstore Overview

several idiosyncrasies which make MLC unsuitable as a simple drop-in replacement for rotating magnetic disks:
n

 ost-efficiency: Although MLC is two to four times cheaper than its cousin SLC, its still about C 20 times more expensive than SATA disks. To use flash cost-efficiently, technologies like inline deduplication and compression are critical.  atency spikes: Flash drives are programmed at the page level (512B to 4KB), but can only L be erased at the block level (512KB to 2MB) sizes much larger than average IO requests. This asymmetry in write vs. erase sizes leads to write amplification which, if not managed appropriately, creates latency spikes.  urability: MLC flash in particular can be vulnerable to durability and reliability problems in the D underlying flash technology. Each MLC cell can be overwritten only 5,000 to 10,000 times before wearing out, so the file system must account for this and write evenly across cells.

Disk-based systems were created more than 20 years ago to cope with a very different set of problems. Adapting these systems to use flash efficiently is comparable to attempting to adapt an old 8-bit singlethreaded operating system to efficiently use todays multicore 64-bit architectures.

Tintris Flash-Based Architecture

The Tintri VMstore appliance is designed from scratch to fully exploit flash technology for virtual environments. The custom Tintri OS is specifically designed to ensure robust data integrity, reliability and durability in flash, and operates at the virtualization layer (more on this later). MLC flash is a key technology that enables Tintri to meet the intense random IO required to aggregate hundreds or even thousands of VMs on a single appliance. The Tintri OS leverages flash in several ways:
n

 ost-efficiency: By design, nearly all active data will live exclusively in flash. To maximize flash C usage, Tintri combines fast inline dedupe and compression with a hybrid file system that automatically moves only cold data to SATA. Inline dedupe and compression are also highly effective in virtualized environments where many VMs are deployed by cloning existing VMs, or have the same operating system and applications installed. Tintri VMstore flash is neither a pure read cache nor a separate preallocated storage tier. Instead, flash is intelligently assigned where its high performance will provide the most benefit.

Figure 1: Tintri employs sophisticated, patent-pending technology to eliminate both write amplification and latency spikes.

TECHNICAL WHITE PAPER / 3

Tintri VMstore Overview

n

 atency management: Tintri employs sophisticated patent-pending technology to eliminate both L the write amplification and latency spikes characteristic of MLC flash Technology (Figure 1). This approach delivers consistent sub-millisecond latency from the more cost-effective MLC flash.  lash durability: Tintri uses an array of technologies including deduplication, compression, F advanced transactional and garbage collection techniques, combined with SMART (Self-Monitoring, Analysis and Reporting Technology) monitoring of flash devices to intelligently maximize the durability of MLC flash. Tintri also employs RAID 6, eliminating the impact of potential latent manufacturing or internal software defects from this new class of storage devices.

Tintri VMstore leverages the strengths of MLC flash, while negating its weaknesses, providing a highly reliable and durable storage system suitable for enterprise applications.

The Virtualization and Storage Mismatch

Virtualization introduces an element of simplicity and agility the physical world lacks, with a single view of resources under hypervisor control (CPU, memory, and networking resources). But there is a language barrier. Virtualization owes its success in transforming data centers with the powerful virtual machine (VM) abstraction. An application in a virtual infrastructure is, for the first time, a truly logical object. Virtual applications can be copied, reconfigured, redeployed, analyzed, and managed in ways that are very difficult for physical machines. Virtualization provides not just the benefits of server and desktop consolidation, but also simplifies data-center management, deployment, and maintenance. However, most existing IT infrastructure and tools including storage dont speak virtualization as their native language. This obscures the relationship between the virtualized application and the underlying infrastructure. The industry has had to rethink traditional functions like monitoring and troubleshooting to account for virtualization. But not every element of the infrastructure has adapted. Virtualization has improved the cost and efficiency of managing servers but has significantly increased the time and complexity of diagnosing and fixing performance problems with storage. Designed before the widespread adoption of virtualization, legacy shared storage systems provide little help resolving performance problems with individual VMs. The result is a suboptimal infrastructure dominated by ever-escalating storage costs. According to VMwares own estimates in 2010, storage accounted for up to 60 percent of virtualization deployment costs. In fact, traditional shared storage actually amplifies troubleshooting issues, via multiple opaque layers hidden from the VM administrator. Existing storage systems carve out LUNs or volumes, which are mismatched with virtual resources VMs and vDisks. Because of its compatibility with virtualization, adoption of shared storage both SAN (Fibre Channel or iSCSI) and NAS (NFS) has accelerated. However, traditional shared storage products present barriers to virtualization: They manage objects such as LUNs, volumes, or tiers, which have no intrinsic meaning for VMs (Figure 2). Legacy storage will not monitor, snapshot, set policies or replicate data at the same granular level as VMs.
TECHNICAL WHITE PAPER / 4

Tintri VMstore Overview

Host Servers Resource Pool

CPU

Memory

Memory

Figure 2: Traditional shared storage maps VMs to LUNs or volumes, rather than managing at the VM and vDisk level.

Storage Arrays

Platinum

Gold

Silver

This mismatch increases cost and complexity. Each new VM instance must be assigned a specific storage LUN or volume. When IO requirements and VM behavior are not well understood, this becomes a painful trial-and-error process. Storage and VM administrators must coordinate to ensure each application has not only the space it needs, but also sufficient IO performance for the expected load. Usually, multiple VMs occupy the same volume or LUN to reduce mapping complexity and space overhead; however, this complicates IO performance problems. A storage-centric view of performance data means administrators must work backward to determine which VMs are affected and which VMs are generating load. Even technologies such as auto-tiering, which seek to reduce storage management overhead, operate at the wrong level. Without the ability to report behavior on a per-VM or per-virtual disk level, advanced storage technology increases complexity and risk. Instead of the simple VM model provided by hypervisors, legacy storage responds with a blizzard of options and interfaces. In these situations, the complexity of configuring, managing and tuning traditional storage for VMs is costly, and ultimately limits the adoption of virtualization. In fact, many applications cannot be costeffectively virtualized with legacy shared storage.

TECHNICAL WHITE PAPER / 5

Tintri VMstore Overview

Overcoming the Limits of Traditional Storage with VM-Aware Storage
Purpose-built for VMs and focused specifically on the problems of VM storage, Tintri VMstore provides management at the same level as the rest of the virtual infrastructure (Figure 3).

Legacy Shared Storage

LUN Volume Disk RPM Block Size Stripe Width Sequential / Random Pathing RAID level WW Names Cache Size Dedupe Snapshot Frequency Inner / Outer Tracks Degraded Performance
Figure 3: Tintri eliminates unnecessary abstractions

Tintri
VM vDisk

Tintri incorporates advances in flash technology, file system architecture, and user interface design to make storage for virtual applications simple and efficient. Tintri VMstore is designed from the ground up exclusively for VMs by experts in both virtualization and storage. Tintri VMstore is managed in terms of VMs and virtual disks, not LUNs or volumes. The Tintri OS is built from scratch to meet the demands of a VM environment, and to provide features relevant to VMs. It is designed to use flash efficiently and reliably, leveraging technologies like deduplication, compression and automatic data placement to deliver 99 percent of IO from flash. These innovations shift the focus from managing storage as a separately configured component to managing VMs as a whole. This overcomes the performance, management and cost obstacles that prevent virtualization of more of the computing infrastructure. Tintris sharp focus on creating a better storage system for VMs enables us to build a fundamentally new type of product. Building a VM-focused management interface relies on far more than just an attractive GUI. The underlying storage system is designed to natively understand and support storage management operations such as performance and capacity monitoring, snapshots, quality of service (QoS) management, and replication at the VM level.

TECHNICAL WHITE PAPER / 6

Tintri VMstore Overview

Focusing exclusively on VMs enables Tintri to eliminate unnecessary levels of mapping and complexity required by general purpose storage systems (Figure 4). Decision-making is delegated to lower levels of the system, and achieves much higher levels of automation and optimization than possible for generalpurpose storage systems. The result is an agile architecture with much simpler abstractions and interfaces, which in turn facilitates further automation and optimization.
Figure 4: Tintri VMstore maps directly to the VM and vDisk abstractions.

Tintris focus on VMs is most apparent in the VMstore management interface, which presents VMs as the basic units of management, rather than LUNs, volumes, or files. Every object in the interface is familiar to VM administrators (Figure 5). The interface is simple enough for VM administrators to manage storage directly, yet sophisticated enough for storage administrators to leverage their expertise in managing storage for huge numbers of VMs.

TECHNICAL WHITE PAPER / 7

Tintri VMstore Overview

Instant Bottleneck Visualization

Administrators dread troubleshooting storage performance problems. A VM complaint may be due to a problem with the storage, but how do you verify this when the VM is sharing a LUN with a dozen other VMs, and the LUN is a slice of a RAID array that contains many other LUNs? Unfortunately, the legacy array provides no statistics on a per-VM basis. And the problem could instead lie with the ESX host or the storage network, or even the users application. Identifying performance bottlenecks is a time consuming, frustrating and sometimes inconclusive process that requires iteratively gathering data, analyzing the data to form a hypothesis, and testing. In large enterprises, this process often involves coordination between several people and departments, and can span many days or even weeks.

Tintris Bottleneck Visualization

Tintris instant bottleneck visualization automates this troubleshooting process. For each VM and vDisk stored on the system, Tintri displays a breakdown of the end-to-end latency all the way from the guest OS down to the disks within the Tintri appliance. For any VM or vDisk, you can see at a glance how much of the latency was spent in the ESX host, the network, the Tintri file system, or accessing the disk (Figure 6). Moreover, a history of this information is automatically stored and can be displayed as a graph, so you can see the bottleneck for each VM at any given point over the last seven days. This visualization is generated by automatically collecting per-VM hypervisor latency stats and correlating them with per-VM storage stats Tintri VMstore collects for each VM (see diagram, below). Hypervisor latencies are obtained using standard vCenter APIs, while the network, file system and disk latencies are provided by Tintri VMstore, which knows, for each IO request, the identity of the corresponding VM. Tintri provides latency statistics in an intuitive format (Figure 7). In an instant, you can see the bottleneck rather than trying to deduce where it is, based on indirect measurements and time-consuming detective work.

TECHNICAL WHITE PAPER / 8

Tintri VMstore Overview

Figure 7: Tintri provides insight in to the historical latency statistics at the VM level.

TECHNICAL WHITE PAPER / 9

Tintri VMstore Overview

VM Alignment
VM alignment is the daunting to-do item. And its a problem that poses real challenges as virtualization spreads into more mainstream workloads. Misaligned VMs magnify IO requests, consuming extra IOPS on the storage array. At a small scale, the impact is small. However the impact snowballs as the environment grows, with a single array supporting hundreds of VMs. At this size, performance impact estimates range from 10 percent to more than 30 percent. Every guest OS writes data to disk in logical chunks. Storage arrays also represent data in logical blocks. When a VM is created, the block boundaries on the guest OS and storage dont always align automatically. If the blocks are not aligned, guest requests span two storage blocks, requiring additional IO (see Figure 8 and Figure 9). A VM runs a guest OS that creates one or more virtual disks to store state. The guest OS typically defines the layout of each virtual disk with a common partition layout, such as a master boot record (MBR). The MBR stores information about how each virtual disk is partitioned into smaller regions, with its size and location. Except for Windows Server 2008 and Windows 7, blocks defined by the guest OS file system (NTFS, EXT3, etc.) do not typically align with the underlying datastore block layout.

Tintris VM Auto-alignment
So why are VMs misaligned? Certainly administrators attempt to address the issue by using a variety of utilities to manually align VMs and reduce performance demand. Numerous blogs, whitepapers and knowledgebases describe why VMs should be aligned and provide step-by-step instructions. But as administrators know, realigning a VM is a manual process. Worse, it generally requires substantial downtime.

TECHNICAL WHITE PAPER / 10

Tintri VMstore Overview

Tintris VM-aware file system intrinsically understands each virtual disk. Building on this foundation, Tintri VMstore offers VM auto-alignment. Rather than the conventional disruptive approach of realigning each guest, Tintri VMstore dynamically adapts to the guest layout (Figure 10 and Figure 11). Nothing changes from the guest OS point of view. Tintri VMstore automatically aligns all VMs as they are migrated, deployed, cloned or created with zero downtime. A VM administrator can now eliminate this arcane task and enjoy performance gains from 10 percent to more than 30 percent with no VM downtime, and zero user interaction.

Summary
Storage remains the primary obstacle to accelerating virtualization growth. Tintri VMstore allows you to overcome the complexity, performance and cost obstacles that prevent virtualization of more of your computing infrastructure. Instant bottleneck visualization and VM auto-alignment are a direct outgrowth of Tintris custom VM-aware file system. Tintris VM-aware appliance eliminates major storage issues in virtualized environments and leverages flash to provide sub-millisecond latency for hundreds of VMs on a single device.

TECHNICAL WHITE PAPER / 11