The Ultimate in Storage Performance
NetApp V-Series and TMS RamSan-500

In the fall of 2008, NetApp articulated its strategy for the use of flash memory devices to accelerate performance with a multifaceted approach that includes:

• Use of flash-based read cache
• Support for third-party flash memory arrays
• Incorporation of solid state disks (SSDs) into NetApp® disk shelves

This approach was described in detail in a NetApp white paper, “Flash Memory Technology in Enterprise Storage” NetApp will be rolling out support for elements of this strategy throughout 2009. The DRAM-based Performance Acceleration Module (PAM) began shipping in 2008 as a precursor to similar flash-based caching products. You can read more about PAM in a recent Tech OnTap article, “Boost Performance Without Adding Drives.”

On February 3, 2009, NetApp announced support for the Texas Memory Systems (TMS) RamSan-500 SSD array in conjunction with the NetApp V-Series open storage controller. This solution delivers significant enhancements in both raw performance and cost performance. In terms of raw performance, it delivers incredibly low latencies for storage access (~1 millisecond) and very high I/O throughput. In terms of cost/performance, the solution delivers the same number of I/O operations per second (IOPS) at a significantly lower cost versus a disk-based solution with equivalent performance, and with a much smaller physical and carbon footprint.

Latency is often the single most important determinant of performance for critical business processes, analyses, and interactive applications. Unacceptable storage latencies are frequently the motivator for architectural, design, and implementation changes in IT environments.

In this article, we describe the elements of this new solution, illustrate the potential performance benefits, and offer some guidelines that you can use to determine whether your applications can benefit from this solution.

The V-Series/RamSan Solution

The V-Series/RamSan solution combines a NetApp V3170 open storage controller with the TMS RamSan-500, an enterprise-class, flash-based RAID array with optimum performance and reliability. The combination creates an ideal storage solution for applications where latency is critical and where operational costs and data center footprint are also important.

The TMS solution is a mature, enterprise-tested product based on a long history of delivering high-performance, block-based storage. The raw speed and high reliability of the TMS RamSan-500, in conjunction with NetApp data protection, data management, and storage efficiency features, results in a flexible, high-performance solution.

The RamSan-500 provides 1 to 2 terabytes (usable capacity) of flash memory arrayed in RAID-protected hot-swappable modules in a compact, 4U enclosure. The reliability of each module is further enhanced with ECC memory, wear-leveling, and bad-block retirement. A battery-backed DDR RAM cache (16-64GB) accelerates data access and shelters the flash memory from small, random data accesses. The RamSan provides Fibre Channel Protocol (FCP) block-level interfaces to LUNs. A single RamSan-500 is capable of sustaining up to 100,000 random IOPS with approximately 1 millisecond response times.

The V3170/RamSan solution is designed to boot from NetApp disk, reserving space in the RamSan for user data. Up to two RamSan-500 arrays are supported with a single or clustered V3170 controller in the first release. Larger configurations may be supported on an as-needed basis, subject to additional testing.

The NetApp V-Series controller allows you to immediately leverage the full range of NetApp data management and storage efficiency capabilities with data stored in the RamSan. LUNs configured on the RamSan are mounted by the V3170 and grouped into aggregates, just as with any other third-party array, so you can immediately create flexible volumes (FlexVol® volumes) and use them with any supported NAS or SAN protocol. Snapshot™, FlexClone®, SnapMirror®, thin provisioning, deduplication, and other NetApp technologies work the same with the RamSan as they would on any NetApp FAS or V-Series system.

V3170/RamSan Performance

A simple assessment of performance was done to compare the performance of the V3170/RamSan configuration versus a V3170 equipped with 15K RPM Fibre Channel disks. We looked at two scenarios:

• Equal capacity
• Equal throughput (IOPS)

The workload used was a simple 70/30 read/write mix of random 4K I/Os across a 1TB+ working set generated by using the NetApp I/O load generator, SIO (Simulated I/O). The following table compares the V3170/RamSan with 2TB of flash and a V3170 with 2TB of disk.

Table 1) Comparison of V3170/RamSan and V3170 with 15K RPM disks (equivalent capacity).

	V3170/RamSan-500	V3170/FC Drives
Capacity	Equal (2TB)	Equal (2TB)
IOPS	50,000	3,600
Latency	1ms	10ms
Power	1,100W	1,150W
Rack Space	10U	9U

The two solutions provide similar capacity but markedly different IOPS and latency performance. In the disk solution, the number of spindles directly dictates I/O performance, as measured in IOPS.

The second example compares the V-Series/RamSan with a V-Series/disk configuration that is capable of sustaining the same number of IOPS.

Table 2) Comparison of V3170/RamSan and V3170/15K RPM FC disks (equivalent throughput in IOPS).

	V3170/RamSan-500	V3170/FC Drives
IOPS	50,000	50,000
Capacity	2TB	27TB
Latency	1ms	10ms
Power	1,100W	5,900W
Rack Space	10U	48U

In the first comparison, a disk configuration with similar capacity delivered only about 7% of the throughput (in IOPS), while latency was an order of magnitude slower. Table 2 illustrates that it takes 27TB of disk capacity (14 disk shelves) to deliver the same number of IOPS, but the large number of disks does nothing to improve latency due to the mechanical limitations inherent in disk drives. This is a good representation of the situation with many high-performance database applications. A large number of spindles (with correspondingly high space, energy, and cooling needs) must be deployed to provide the necessary I/O throughput—even when the capacity is not necessary.

The message is that for applications in which throughput and/or latency are the limiting factors, the RamSan offers clear advantages. Later in the article, we’ll discuss how to tell if your application is I/O bound.

Which Applications Can Benefit from V-Series with RamSan?

You’re probably already starting to guess which applications are likely to benefit from the reduced latency and impressive throughput that the VSeries/RamSan combination can provide.

In general, applications with random I/O workloads benefit from the use of flash memory devices, while sequential workloads do not. Online Transaction Processing (OLTP) and other database applications are fairly obvious candidates. If an entire database fits within RamSan storage capacity, it will immediately benefit from the solution’s latency and throughput. For larger databases, you can leverage the improved latency of RamSan by storing just your hot files—such as redo logs, indexes, and temp space—on the RamSan and putting the rest of the database on hard disk drives.

Generally, any application where the entire data set, or the active data set, fits in RamSan memory may be a good candidate, especially when that data set is accessed by multiple servers working in parallel. The render farms used in computer animation are a good example. The key, of course, is whether or not disk I/O is the limiting factor in application performance.

Is Your Application I/O Bound?

When it comes to improving application performance, storage is often the last link in the performance chain to be investigated.

Figure 1) Typical approaches to improve application performance.

This is in part because the methods for understanding I/O performance are not that widely understood. However, a number of OS-level, storage-system-level, and application-level I/O performance analysis tools are available to help you investigate possible I/O problems.

Operating System Level Tools for Investigating I/O
For UNIX® and Linux® operating systems, a number of common utilities such as top, iostat, and sar (system activity reporter) can help you understand the potential impact of I/O on your server. If the server is (or can be) dedicated to the single application of interest, these statistics can be useful. For instance, the iostat command on a Linux system shows “%iowait” the percentage of time that the system spends waiting for I/O. (One caveat with this command is that it presents a single point-in-time view.)

For Microsoft® Windows®, the best tool for system performance analysis is Performance Monitor. Unfortunately, Performance Monitor does not provide explicit I/O Wait Time statistics. It does, however, include real-time processor performance levels and disk queue statistics. “Processor: % Processor Time” measures the actual work being done by the processor, and “Avg Disk Queue Length” shows the number of in- process I/O operations. If a system hit hard by transactions shows significant disk queue levels and yet “% Processor Time” is well under 100%, you can assume that server I/O wait time is high.

Storage System Tools
If you are using an intelligent back-end storage system, it may provide additional information about I/O. For instance, by using NetApp Operations Manager, you can graphically review a variety of storage metrics, including volume latencies, operations per second, and so on. By focusing specifically on the volumes used by a particular application, you can tell whether those volumes are experiencing excessive transaction rates and/or high latencies.

You can find out more about using these NetApp tools in an earlier Tech OnTap article, “Monitor, Troubleshoot, and Improve NetApp Storage Performance” and in NetApp Technical Report 3525. “Storage Performance Management.”

Application Tools
For the greatest possible application specificity, you need I/O instrumentation embedded in your application to tell you exactly how the application is spending its time. Many popular database and business applications contain this type of instrumentation. Oracle, for instance, comes with the Statspack utility to monitor database performance. In Oracle10g™, Oracle introduced the AWR (Automatic Workload Repository), along with ADDM (Automatic Database Diagnostic Monitor) as an extra-cost option to their Enterprise Manager Tool.

The Statspack report contains a “Top 5 Timed Events” section that is the first place you would look to begin to understand whether or not your database is I/O bound Figure 2).

Figure 2) Part of the Oracle Statspack report showing Top 5 Timed Events (15 minute interval).

Reviewing the example output, it is immediately obvious that during this interval the database is spending 83% of total elapsed time reading. Dividing the total number of waits into the time spent in seconds shows an average latency of 5.25ms per wait. Although this latency is respectable, the only way to get significant performance improvement out of this database would be to reduce it further. The V-Series/RamSan combination is an excellent solution for that purpose. You can find out more about Oracle® I/O performance in a recent white paper from Texas Memory Systems, “Faster Oracle Performance with Solid State Disks.”

A Final Deployment Scenario: FlexCache with RamSan

As with any large storage deployment, a NetApp V3170 combined with TMS RamSan is a substantial investment, especially if you dedicate that investment to accelerate a single application. An alternative approach for NFS-based environments is to deploy the solution by using NetApp FlexCache software to create a caching architecture.

With a caching architecture, a fast storage cache resides logically between your primary storage systems and your compute servers or desktop clients. Data is automatically copied into this caching tier the first time it is read; subsequent reads are satisfied from the cache rather than the origin storage system. By investing in a centralized caching tier with NetApp V-Series and TMS RamSan, you can leverage that investment across multiple storage systems and applications. You can also deploy more economical, high- capacity disk drives in your primary storage without adversely affecting performance. Because hot data is automatically cached, manual data management or migration software is not needed.

Figure 3) A FlexCache architecture showing the V3170/RamSan in use as a high-performance caching tier.

This approach has demonstrated significant benefits for accelerating the speed of parallel software builds as well as compute-intensive applications such as animation rendering, electronic design automation, seismic analysis, and financial market simulation. Any application in which the same data is read in parallel by multiple compute servers can benefit from such an approach.

Conclusion

If you have an application that is I/O bound or where disk latency has become the limiting factor, the combination of the NetApp V3170 and the TMS RamSan-500 can reduce latency by an order of magnitude (from 10 milliseconds down to 1 millisecond), while delivering up to 50,000 IOPS. This exceptional performance makes the solution highly cost effective for critical applications, and enterprise-class data management features make it an easy fit in your existing environment.

Got opinions about V-Series with TMS RamSan? Ask questions, exchange ideas, and share your thoughts online in NetApp communities.

	Jamon Bowen Enterprise Architect Texas Memory Systems Jamon joined TMS as a software engineer in 2004. Today he serves as enterprise architect for the financial industry. By leading information sessions worldwide and maintaining a strong online presence, Jamon has established himself as a thought leader in the solid state disk storage community. His extensive engineering experience began with an Electrical Engineering degree from the Colorado School of Mines.
	Steven Lusnia V-Series Solution Architect NetApp Steve has been with NetApp for more than eight years. In mid-2003, he started working with large NetApp customers who wanted to leverage NetApp features in their traditional SAN environments. This was the genesis of the V-Series product line. In 2006, Steve took on a joint role with engineering and product management, using his field experience to help drive the evolution of the V-Series product line, including V-Series with TMS RamSan.
	Darrell Suggs Technical Director NetApp Darrell joined NetApp Performance Engineering in 2001. He has authored several key performance tools, guided performance enhancements to NFS environments, led the product development effort for FlexShare™ to enhance storage quality of service, and currently provides technical leadership for NetApp caching products. Darrell received a PhD in Computer Science from Clemson University in 1993 and has over 15 years of experience in the IT industry.