Exchange 2010 on VMware
When you think about running Microsoft® Exchange 2010 on VMware®, it’s easy to come up with good reasons for doing so:
- Better utilize your processor cores. Large, multicore servers are becoming the norm, but most applications cannot take advantage of all the cores in a physical server.
- Isolate roles without adding hardware expense. Exchange 2010 has evolved into a modular architecture with distinct server roles, including mailbox, edge transport, hub transport, and client access. Isolating these roles can make it much easier to troubleshoot Exchange problems, but that would require a lot of physical servers, especially in smaller environments.
- Avoid overprovisioning. It’s always hard to know what the future is going to mean in terms of your Exchange requirements. Running Exchange on VMware means you can provide the right resources now and add them incrementally as you need them, avoiding expensive overprovisioning upfront.
- More easily meet your business requirements. The way you design your Exchange deployment might depend on your specific requirements. Do you need to have a separate mailbox server for each department or business unit? Virtualization makes it simple to accomplish that without raising your hardware expenses and physical server count.
- Provision a new Exchange server rapidly. Provisioning a physical server is time consuming, even when you already have the hardware. With VMware it’s easy to create a template for each type of Exchange server and save them to speed the deployment of servers of the same type in the future.
- Maintain an Exchange lab. Need to maintain a lab to test and troubleshoot Exchange? VMware virtualization is a great foundation for your Exchange 2010 evaluation and testing processes.
- Clone for testing and troubleshooting. Snapshot™ copies and clones give you the power to rapidly clone a particular Exchange VM for troubleshooting.
Despite these and other advantages, there are several questions that I hear almost every time I talk to people about running Exchange on VMware. In this article, I’m going to try and address the most common concerns and provide some best practices associated with running Exchange in joint VMware and NetApp® environments.
Can I Get Support?
The question of support comes up a lot, since Microsoft offers competing virtualization products. Microsoft made two recent changes to its licensing and support policies for VMware that address many of the lingering concerns:
- Support for Exchange 2010 running on VMware Infrastructure/vSphere™: VMware ESX™ 3.5 Update 2 was the first hypervisor to be listed under the Microsoft Server Virtualization Validation Program (SVVP). This certification gives customers who run ESX 3.5 Update 2 or later (including vSphere), Windows® Server 2008, and Exchange Server 2007 SP1 or Exchange 2010 access to cooperative technical support from Microsoft and VMware.
- Relaxed policies for application license mobility: Microsoft updated its licensing policy for Exchange to better accommodate use in a virtual environment. The application licenses are still tied to a physical server; however, Microsoft has removed the clause that restricted reassignment of an application license to once every 90 days. This allows you to remain compliant while performing virtual machine migration and high availability in a virtual environment.
In addition to SVVP, you can also qualify for direct support of virtualized Microsoft applications if you contract for the Microsoft Services Premier Support Program. Support might also be available through your server OEM vendor, VMware Global Support Services (GSS), and Technical Support Alliance Network (TSANet).
You can learn more about how to get support in the Microsoft Exchange 2010 on VMware Support and Licensing Guide.
Will Exchange on VMware Perform?
For critical applications such as Exchange, everyone knows what performance to expect from a physical server, but concerns about performance on virtualized servers still linger. The best way to satisfy yourself that you won’t have any performance surprises when you virtualize Exchange is to look at the extensive performance testing that has been done by both VMware and NetApp. Most of this work was done with Exchange 2007, but I think—especially given the significant I/O reduction from Exchange 2007 to Exchange 2010—that you can be pretty confident that if performance was fine for Exchange 2007, it will be good for Exchange 2010.
Figure 1 provides a summary of Exchange on VMware performance. As you can see, virtual performance is always within 5% of physical performance. Even with 4,000 users, CPU load only reached 25%. The number of heavy users scaled linearly with the addition of CPUs in both the physical and virtual cases.
Figure 1) Summary of Exchange 2007 performance in virtual versus physical environments.
For more details on performance measurements, you can refer to a recent VMware white paper.
VMware Best Practices
VMware has developed an extensive body of best practices for using Exchange in VMware environments. These best practices are summarized here. For full details, refer to the Microsoft Exchange 2010 on VMware Best Practices Guide.
Best Practices for Virtual CPUS (vCPUs)
- Only allocate multiple vCPUs to a virtual machine if the anticipated Exchange workload can truly take advantage of all the vCPUs.
- If the exact workload is not known, size the virtual machine with a smaller number of vCPUs initially and increase the number later if necessary.
- For performance-critical Exchange virtual machines (for example, production systems), try to make sure the total number of vCPUs assigned to all the virtual machines is equal to or less than the total number of cores on the ESX host machine.
Best Practices for Virtual Memory
- Do not overcommit memory until vCenter™ reports that steady-state usage is below the amount of physical memory on the server.
- Set the memory reservation to the configured size of the VM, resulting in a per-VM vmkernel swap file of zero bytes. Remember that setting reservations could limit VMotion™.
- It is important to “right-size” the configured memory of a VM. Memory will be wasted if the Exchange VMs are not utilizing the configured memory.
- Enable VMware Distributed Resource Scheduling (DRS) to make sure workloads are balanced in the ESX cluster. DRS and reservations can guarantee critical workloads have the resources they require to operate optimally.
- To minimize guest OS swapping, the configured size of the VM should be greater than the average memory usage of Exchange running in the guest. Follow Microsoft guidelines for memory and swap/page file configuration of Exchange VMs.
Best Practices for Networking
- Allocate separate NICs/networks for VMotion, FT logging traffic, and ESX console access management; or use VLAN tagging.
- Allocate at least two NICs for Exchange production traffic to leverage NIC teaming capabilities. Generally, allocate at least four NICs per ESX host.
- Use the VMXNET3: a paravirtualized vNIC installed with VMware Tools. VMXNET3 is optimized for virtual environments and designed to provide high performance.
- To support VLANs in vSphere, the virtual or physical network must tag the Ethernet frames with 802.1Q tags using virtual switch tagging (VST), virtual machine guest tagging (VGT), or external switch tagging (EST). VST mode is the most common.
- Follow the networking design guidelines in VMworld 2009 session TA2105: Virtual Networking Concepts and Best Practices. This includes designs to efficiently manage multiple networks and redundancy of network adaptors on ESX hosts.
Best Practices for Resource Management and DRS
- The source and target ESX hosts must be connected to the same gigabit network and the same shared storage.
- A dedicated gigabit network for VMware VMotion is recommended.
- The destination host must have enough resources.
- The VM must not use physical devices such as CD ROM or floppy disk.
- The source and destination hosts must have compatible CPU models, or migration with VMware VMotion will fail.
- To minimize network traffic, it is best to keep VMs that communicate with each other together (for example, mailbox and GCs) on the same host machine.
- VMs with smaller memory sizes are better candidates for migration than larger ones.
Storage Best Practices
- Deploy Exchange VMs on shared storage to allow VMotion, HA, and DRS.
- Storage multipathing: Set up a minimum of four paths from an ESX Server to a storage array (requires at least two HBA ports).
- Create VMFS file systems from VirtualCenter to get best partition alignment.
Best Practices for Performance with NetApp Storage
NetApp has developed additional best practices for using Exchange 2010 with NetApp storage. These were described in a recent Tech OnTap article as well as a detailed technical report and include the best ways to take advantage of NetApp storage efficiency capabilities.
For instance, combining the use of NetApp deduplication and thin provisioning can produce from 40% to 60% storage savings in Exchange 2010 environments.
Additional recent work focused on ways to optimize Exchange performance with NetApp storage. In benchmarking with Microsoft Exchange 2010, the addition of Flash Cache doubled the number of IOPs achieved and increased the supported number of mailboxes by 67%. These results will be described in TR-3865: “Using Flash Cache for Exchange 2010,” scheduled for publication in September 2010.
How Do I Do HA and DR?
Creating flexible high-availability (HA) and disaster recovery (DR) configurations for Exchange 2010 is actually much simpler and less expensive in a virtualized environment. You have more options and more flexibility for data protection at all levels.
In order to configure HA and DR, you first have to understand the significant changes that Microsoft made in its native resiliency options for Exchange 2010.
To replace the server and data resiliency options of earlier versions of Exchange, Microsoft implemented the database availability group (DAG). The DAG uses the same log shipping capability that was used in cluster continuous replication (CCR) in Exchange 2007. A DAG consists of 2 to 16 mailbox servers. Each mailbox server can hold one or more active or passive copies of a database. Each database has a separate status, so one server can host copies of multiple databases and only have some of those copies active at one time.
The DAG uses a new Exchange component called Active Manager. Active Manager facilitates failover and failback. In the event of a failure (including a failure in underlying storage or storage connectivity), Exchange 2010 "promotes" one of the copies of the database to active status; the mailbox role then takes up the task of serving up the mailboxes on that database. Failover occurs in less than 30 seconds.
Protecting Against Localized Failures
VMware provides a number of possible mechanisms for protecting Exchange 2010 availability from local failures.
- VMware HA by itself can protect against downtime due to local hardware failures.
- Combining VMware with the use of a DAG protects against both hardware and software failures, while reducing the time the database is in an unprotected state versus the use of a DAG by itself.
Protecting Against Sitewide Failures
To protect against site failures, VMware suggests combining VMware Site Recovery Manager (SRM) with the DAG capability. The DAG provides local site protection, while storage-based replication such as NetApp SnapMirror® is used to keep a DR facility in sync with the main site. NetApp SnapManager® for Exchange can provide application-aware replication to make sure of consistency. If recovery is initiated at the DR facility, SRM automates and accelerates the recovery process. Contributors from both VMware and NetApp discussed the details of DR for Microsoft applications (including Exchange) using SRM in a recent Tech OnTap Roundtable.
Figure 2) Replication architecture for SRM with NetApp SnapMirror and SnapManager software.
These and other methods for HA and DR are described in detail in Microsoft Exchange 2010 on VMware: Availability and Recovery Options.
NetApp Best Practices
NetApp has established a number of best practices to keep in mind with Exchange environments that use DAGs:
- Microsoft recommends a minimum of three copies of each mailbox database to limit exposures due to possible storage failures, including double disk failures. NetApp recommends deployments on NetApp storage using RAID-DP®, which can protect against double disk failures while reducing the number of mailbox database copies. We recommend two copies of each mailbox database when the copies are on RAID-DP.
- Each DAG copy is up to date and as such does not take the place of backup. To allow for point-in-time restores, Microsoft recommends an additional "lag" database copy that allows point-in-time restores up to 14 days back. As an alternative, NetApp provides SnapManager for Exchange, which allows you to create space-efficient Snapshot copies and restore to any point in time without the need for a lag copy.
- Storage for active and passive copies should be identical in capacity and performance.
- Active and passive copies should be placed in separate volumes.
- Perform backups on one of the passive nodes.
Adoption rates of Exchange 2010 have been high, and a lot of attention is being paid to virtualization. VMware has worked to develop an extensive collection of reference material and other resources to help you succeed when you virtualize Exchange 2010. You can access the full collection here. There’s a lot of information there on capacity planning, sizing, and many other topics.
VMware has also made significant efforts to test and develop Exchange solutions in conjunction with important partners such as NetApp. Tech OnTap has featured a number of recent articles that focus on virtualization of Exchange and other Microsoft applications. (See sidebar.) You can find the full collection of NetApp resources on the NetApp Exchange page.
| || |
Lead Architect-Microsoft Solutions
Scott is part of VMware’s Solution Readiness group, focused on virtualization best practices for Microsoft enterprise applications. Scott has been with VMware since May 2007 and has authored many of the Exchange and SQL Server® white papers on VMware’s Business Critical Applications website. Prior to VMware, Scott spent 11 years as a customer-facing professional services consultant, helping large customers design and deploy solutions focused on VMware virtualization, Microsoft applications, and identity/access management.