Compression: Yet Another NetApp Efficiency Technology

A Conversation with Jay Kidd

Storage efficiency has become the number-one buying criterion for many organizations today, particularly as they continue to wrestle increasing data with ever smaller budgets.

If you’ve not already heard us beat this drum, know that it is absolutely, positively possible to boost utilization of the storage you already own and defer new capacity purchases for months with NetApp. Our innovative shared storage efficiency capabilities work together, across your organization, to help you deliver against increasingly aggressive business goals with fewer resources.

We continue to innovate to better put our current and new storage efficiency innovations to work throughout your data center to immediately help lower the cost of IT while speeding the pace of your business. Among prevalent efficiency technologies, compression has been a rather hot topic lately. Brought into the spotlight by a couple of notable acquisitions, compression is an important tool, particularly with secondary storage.

Compression’s value lies not in its singular application. The power and potential of compression really pay off when it is used in combination with a host of other storage efficiency features and technologies. Selecting efficiency technologies and features from an integrated portfolio enables you to flexibly and effectively manage the right balance between technology and business needs.

We sat down with NetApp’s Jay Kidd, Senior Vice President, Product Strategy and Development, Storage Solutions Group, to discuss NetApp’s new data compression release.

NetApp offers a pretty compelling value proposition for compression. Can you tell us about this latest addition to our storage efficiency family?
Jay: Compression has been around for quite a while and has been implemented in many forms. The challenge with it has always been to balance the associated performance impact because there is some CPU required to compress and decompress the data. That’s why-until recently-compression has typically been done in hardware for tape drives and VTLs. What we’ve done is develop a way to do compression in software, while mitigating the compute resources required.

So this is zero-penalty compression?
Jay: No software compression is zero impact, but we think we’ve struck a good balance, enabling maximum compression savings and a minimal performance penalty by leveraging existing capabilities of NetApp® Data ONTAP® and WAFL® (Write Anywhere File Layout).

Specifically, as data is written to the system, we collect a small number of 4K WAFL blocks and create a compression group. Once the group is formed, we do a test while the data is still in memory, and we decide if the data is compressible. If it isn’t, we just pass it through to the disk drives. If the test says that it is compressible, we store the data in its compressed form on disk. There is some other Data ONTAP stuff going on, but in a nutshell that’s how we do it.

We briefly touched on deduplication. Can compression and deduplication run together?
Jay: Yes. Compression is a companion to dedupe and other storage efficiency technologies. Users have the choice on a NetApp volume or LUN to run compression, deduplication, both, or neither.

One distinct advantage NetApp offers is in providing an integrated storage efficiency portfolio, where the technologies and features are designed to work on their own or together for compounded savings. Our ultimate goal is to wring every bit of utilization from your IT resources, thus helping lower your TCO.

Compression, coupled with efficiency technologies such as thin provisioning, efficient RAID, and deduplication, minimizes the disk capacity required to deliver a given workload service level. For example, as shown in the table below, combined savings from use of dedupe and compression can be significantly higher than savings through either alone. It then becomes a tradeoff between increased CPU utilization and increased space savings.

Percent of Storage Saved with:
Application Type Compression Only (average) Dedupe Only
Dedupe +
File Services:
Home Directories
50% 30% 65%
File Services:
Engineering Data
55% 30% 75%
File Services:
75% 3% 75%
Virtual Services 55% 70% 70%
Oracle ERP
65% 0% 65%
70% 15% 70%
Exchange 2010
35% 15% 40%

How much performance overhead can users expect?
Jay: Since compression involves doing extra processing, there will be some impact. The actual impact will depend on the type of data (compressibility) and the load on the system at the time. We have developed best practices to guide our customers through such sizing activities to help them optimize their implementations.

Customers will find compression particularly compelling in secondary storage for backup and archive, especially if they tap into the combined savings potential from coupling deduplication and compression.

Take databases, for instance. Many of our customers tell us they would like to reduce the amount of storage required to back up their databases. Our tests have shown up to 70% space savings after using compression followed by dedupe. There are many users out there that will trade increase in CPU utilization in return for 70% capacity savings.

When we released deduplication several years ago, we had a restricted access program just for early adopters. Are we doing the same thing for compression?
Jay: Yes. There is a process our customers can follow to request a compression license. We’ll evaluate their environment and provide our best practice recommendations. That way, we can make sure the use case matches the capabilities we offer and there are no surprises for anyone.

Basically, we want to make sure we stay close to our compression users during the early stages. Eventually, we’ll open up the feature to everyone; you won’t need any special review.

Does a NetApp customer need to buy anything in order to add compression?
Jay: No. We don’t charge for our compression feature. Compression comes with the upgrade to our latest version of Data ONTAP.

Thank you, Jay. Our conversation has been very enlightening. Is there anything else you’d like to add?
Jay: I see some strong parallels for NetApp between compression and deduplication about four years ago. While we were first to market with deduplication integrated into a mainstream storage controller, we supported deduplication for primary storage use cases gradually. We wanted to understand the impact and the benefits given such a wide range of data types. We’re taking that same conservative approach to bringing compression to market and are starting with secondary storage for the same reasons. Our goal is to help make sure our users can predict the benefit they will see and the resources required for all of our storage efficiency capabilities and apply them in the manner that maximizes that benefit at the lowest operational cost.

“NetApp has grown from a startup to a $4B company by providing technology that lets people buy less storage—Snapshot™, thin provisioning, flexible volumes, dedupe, RAID-DP® solution, and so on. Those technologies all help you use less storage. And we continue to innovate and introduce new technologies that let people buy still less. Our company is growing and taking share because we deliver value to our customers. When you help customers save money, they’re more likely to put more of their data on your storage.”
- Jay Kidd, NetApp

So, yes, many other vendors offer compression, but no one is even close in the breadth of storage efficiency and data reduction technologies we offer for both primary and secondary storage. Competitive offerings often aren’t as well integrated, don’t have the breadth and depth of complementary capabilities, or are only file level, which limits the application. NetApp builds these efficiency technologies integrated directly into all our data storage systems, so they work everywhere on file and block data. There’s no need for a separate, single-purpose appliance that requires additional management and toolsets to manage.

We got a lot of skepticism when we released our dedupe feature. We proved that deduplication is compelling on primary storage for a wide range of workloads and especially in conjunction with VMware®. Today, NetApp is a leading provider of deduplication technology in systems, in customers, and in system capacity. We’ll take the same approach with data compression. It might be databases or some other application, but over time, we will prove that software compression can also be handled effectively on all storage tiers: primary, secondary, and archival.

Thanks for taking the time to speak to us today, Jay.

If you want to maximize efficiency of your IT environment, integrated, complementary storage efficiency technologies are your best bet. Efficient IT can minimize the time, effort, and resources of running today’s IT, freeing you to innovate for the future.

Learn more about compression at NetApp and our family storage efficiency technologies.

If you haven’t already, check out our latest information on how to build a future-ready , Flexible IT environment based on shared IT infrastructure .