Case Study: Boosting Storage Efficiency with Deduplication, Thin Provisioning, and FlexClone

At San Diego Data Processing Corporation (SDDPC) our job is to meet the IT needs of various city and county organizations as efficiently as possible. We’ve relied primarily on NetApp to meet our storage needs since 1999, and today we’ve grown to over 300TB of NetApp® storage.

In the years since we acquired our first NetApp system, we’ve moved quickly to adopt the latest NetApp technologies to improve efficiency and keep costs down, so when Tech OnTap asked me to write an article on storage efficiency for this month’s special issue, several of the key technologies that we rely on every day sprang to mind:

• Deduplication for our VMware® environment and home directories gives us up to 65% space saving.
• Thin provisioning for all volumes reduces our storage requirement by 27% or more.
• The use of FlexClone® in our application test/dev environment lets us recover from programming mistakes in a matter of seconds, without consuming additional storage.

Alone or together, these technologies reduce the total amount of storage you need, in turn reducing space, power, and cooling requirements.

Deduplication for VMware and Home Directories

You’ve probably already read a lot about NetApp deduplication in Tech OnTap, so I won’t spend a lot of time explaining the technology. For SDDPC, we like NetApp deduplication because it can be applied to primary storage, you can use it with any storage protocol (NFS, CIFS, iSCSI, FC), it’s application transparent, and it’s built into Data ONTAP®, so there’s no extra charge associated with it.

Figure 1) Typical space savings from deduplication for various applications.

We currently use deduplication for our VMware environment (you can read more about our virtualization efforts in a recent success story) and also for home directories. For VMware, we’ve got 19 of our 25 VMware ESX 3.5 servers all accessing NetApp storage via NFS; some of the rest are connected to NetApp storage via FC. By deduplicating our VMware environment, we’re getting about 65% space savings. In other words, we’re only using a little more than a third of the space for VMware that we would need if we didn’t have deduplication.

We also currently have deduplication turned on for five home directory volumes. Space savings on those volumes range from 12% to 30%.

For our home directory volumes, we have NetApp deduplication configured to run automatically whenever those volumes grow by 20%; for our VMware environment, since that data does not change very frequently, we run the deduplication process once per week every Sunday night.

Using a few simple commands, you can easily determine how much data was processed, when deduplication last ran, and how much space you are saving.

Over time we’ll be enabling deduplication for more applications—such as our Exchange archives—as we upgrade Data ONTAP on storage systems that have been installed for a while. Ultimately, we plan to run deduplication almost everywhere, with the possible exception of applications that have the highest I/O and lowest latency requirements, although, truthfully, the performance impact we’ve seen has been minimal (a few percent for reading, no impact on writes).

If you’re wondering whether your data sets can benefit from deduplication, NetApp offers a Space Savings Estimation Tool (SSET) that will crawl through an NFS or a CIFS volume and estimate about how much space you can save; this can help you decide whether it is worth deduplicating a particular volume.

Thin Provisioning Everywhere

Thin provisioning is another NetApp technology that we use to improve storage efficiency. As in most IT shops, our application administrators almost always overestimate how much storage they will need for a new application up front. An admin may ask for 1TB and end up needing 100GB. Thin provisioning allows us to present them with that 1TB volume without allocating and committing the storage space up front. For this reason, we’ve been using it from day one on all our storage volumes.

Using thin provisioning in combination with the ability to grow or shrink NetApp FlexVol® volumes on the fly gives us much greater flexibility than we would have with traditional storage. We can present an admin with the 1TB volume that he or she has requested, knowing that actual storage will only be consumed as it is used. With multiple thin-provisioned volumes on a storage system, all volumes draw storage from a single pool of available storage only as necessary.

Figure 2) Illustration of the effect of NetApp Thin Provisioning.

In practice, even though a volume is thin provisioned, we typically wait for the storage usage for a new application to stabilize and then go back and shrink the volume down so that the application has a 20–30% buffer over its current usage. This way, we’re never overcommitted, and we ensure that any single application can’t consume all the available free space in an aggregate.

Despite this conservative approach, our use of thin provisioning demonstrates some impressive space savings. For our database volumes (both Oracle® and Microsoft® SQL Server™) we present a total of 4.7TB, but our actual allocated space is only 3.4TB, so our storage requirement is reduced by about 27% versus traditional provisioning.

For our home directory volumes, we are currently saving a more modest 13% with thin provisioning. This is in part because we have aggressively shrunk the size of these volumes to meet actual requirements. The situation is also slightly more complicated because, for historical reasons, these volumes are configured as LUNs that are mounted by our Windows® file and print servers using iSCSI. Thin provisioning works with LUNs in a similar fashion as for NAS volumes. However, to get the maximum benefit, you have to periodically do space reclamation on each LUN. This was described in detail in a previous Tech OnTap article.

We also find that thin provisioning has great synergy with deduplication.

For example, one of our VMware environments supports over 100 virtual machines on 4 ESX 3.5 servers using 4 thin-provisioned volumes, presenting a total capacity of 1.8TB. Our actual utilized capacity, however, is only 841GB, meaning that 41% of the provisioned capacity is actually in use.

In this case we have left a somewhat larger buffer than we might otherwise do because our VMware environment is growing rapidly. Also, the NetApp SnapManager® for Virtual Infrastructure (SMVI) software uses VMware snapshot technology as part of the process of capturing a consistent NetApp Snapshot™ copy. Although SMVI doesn’t hold VMware snapshots for a long time, we reserve the extra space to accommodate the VMware delta files that result from the use of SMVI or for other circumstances in which we might need a VMware snapshot.

Note that, with deduplication, we already reduced our VMware storage footprint by 65%, and now, with thin provisioning, we reduce it a further 59% from what it would be with traditional provisioning. As we expand our use of deduplication, we expect to be able to further leverage thin provisioning and volume resizing to recover additional space.

FlexClone for Test/Dev

A final way that we squeeze extra storage efficiency from our NetApp storage is through the use of FlexClone® in our SAP® test and development environment. When development is in full swing, I get five requests a week on average to restore a development environment in which some unidentified code change had broken an application.

With traditional tape backup, I would have had to locate the backup of the environment from the previous night and restore from tape—setting the developer back a full day and wasting hours of my time and further development time. With NetApp we create a Snapshot copy of all virtual machines used for development every four hours. When a restore request comes in, I can simply locate the correct development environment from the most recent Snapshot copy, create a FlexClone volume of that volume, register the cloned VM as a new virtual machine, and restart it. The process takes minutes and the developer is back in business with minimal time lost.

The storage efficiency comes in when you consider that a typical development environment is about 450GB. If I had to restore 5 such environments a week using traditional methods, I would need 2TB of additional disk space. With FlexClone, I accomplish the same thing without consuming any additional disk space.

Conclusion

By themselves, each of these technologies offers great improvements in storage efficiency. As we’ve already seen, deduplication helps SDDPC get back up to 65% of storage space, even a conservative approach to thin provisioning saves us 30% or more, and FlexClone lets us recover quickly with no additional storage needed. By our estimation, NetApp storage efficiency has reduced our overall storage purchasing requirement by at least 5TB, providing significant savings in terms of capital costs, ongoing management costs, power, and cooling.

Perhaps even more important is the way NetApp technologies work together. Using deduplication, thin provisioning, and volume resizing, you can dramatically reduce your storage requirement, and, when you deduplicate a volume, the space savings is automatically inherited when you replicate the volume with NetApp SnapMirror®—saving you not only storage space but network bandwidth as well.

For example, in a recent pilot project, we used SnapMirror to replicate our virtual machines (already deduplicated) to a second location. Then I used NetApp FlexClone to clone all the replicated virtual machines to test the resulting disaster recovery capability—without using any additional storage space or interrupting the replication schedule. Although we haven’t put this DR scheme into effect yet, the efficiency that NetApp provides makes it more cost effective (and therefore more feasible) than it otherwise would be.

Rick Scherer UNIX System Administrator San Diego Data Processing Corporation Since joining SDDPC in 2001, Rick has demonstrated expertise in a variety of areas beyond the scope of his job title, including Windows, network design, security, SAP and VMware. Since obtaining his VCP in 2006, he has worked to deploy a robust virtual environment at SDDPC, with 25 VMware servers and over 300 virtual machines running mission-critical applications such as SAP NetWeaver®, Microsoft SQL Server, Symantec™ Altiris, and Citrix Presentation Server. Rick has been working with NetApp storage since 1999.