FCoE: The Future of Fibre Channel?

Given the industry buzz and genuine excitement that Fibre Channel over Ethernet (FCoE) has generated—as well as support from pretty much everyone in the industry—it seems likely that this converged transport will ultimately supplant existing Fibre Channel networks. The standards effort should be completed and the standard ratified around April 2009. The current draft standard is not expected to change between now and then, and many vendors are preparing products for the market, including NetApp (see sidebar).

If you are a user of Fibre Channel technology today, you should understand and prepare for the arrival of this technology. In this article, I’ll try to answer some important questions you might have about FCoE technology, including:

• What is FCoE?
• Why FCoE?
• What are the FCoE Implementation options?
• What should you do to prepare?

What Is FCoE?

Fibre Channel over Ethernet, or FCoE, is a new protocol (transport) currently being standardized by the T11 committee. (T11 is the committee within the International Committee for Information Technology Standards— INCITS —responsible for Fibre Channel interfaces.) FCoE transports Fibre Channel frames over Ethernet by encapsulating Fibre Channel frames within jumbo Ethernet frames. Although the standard is not expected to be ratified until the end of 2008, the expectation is that no specification changes will occur between now and then.

Why FCoE?

The premise behind FCoE is to provide I/O consolidation by allowing the safe coexistence of different types of traffic on the same wire, thus reducing and simplifying cabling, reducing the number of required adapters per host, and reducing power requirements.

Figure 1) Reduction of host complexity with FCoE.

The force that will drive FCoE adoption is the need to reduce total cost of ownership (TCO) while preserving existing infrastructure investments and maintaining backward compatibility with familiar procedures and processes. By converging Fibre Channel and Ethernet and eliminating the need for different networking technologies, FCoE promises to significantly reduce network complexity.
Initially, most FCoE deployments will probably occur at the host and switch layers, while the back-end arrays will continue to run native Fibre Channel instead of FCoE. This will help preserve the large infrastructure investments that have been made in FC over the years.

The great thing about FCoE is that it allows a slow migration from FC to Ethernet. It’s possible to expand or replace part of your FC network with Ethernet switches, allowing you to make an orderly transition from one networking technology (FC) to the other (Ethernet) as it makes sense.
Longer term, If FCoE is successful, as you consider upgrades to your infrastructure and/or build new data centers, you’ll want to consider storage arrays with native FCoE support. NetApp will support native FCoE and continue to support Fibre Channel on all storage systems.

A recent joint paper from NetApp, QLogic, and Nuova Systems (now part of Cisco) discusses the business benefits of network convergence in detail.

FCoE Implementations

There are two possible ways to implement FCoE:

• Using a hardware initiator with a converged network adapter (CNA), and hardware target similar to the existing Fibre Channel model.






Figure 2) A CNA supports both FC and Ethernet from a single device, reducing the number of network interfaces you need.

• The CNA vendors are the typical FC HBA suppliers such as Qlogic, Emulex, and Brocade, but I fully expect to see traditional NIC vendors such as Intel and Broadcom jump on the bandwagon. In fact, both of them have been actively engaged at the T11 (FC-BB-5) working group to define the FCoE standard.

• Using a software initiator and target with regular 10 Gigabit Ethernet (10GbE) NICs.
  
  In December 2007, Intel released a software initiator package to aid the development of FCoE solutions for Linux®. The expectation is that various Linux distributions will ship with the FCoE software initiator drivers within the next 12 months. The idea is that Linux distributions will ship FCoE ready similar to the way that all OS platforms ship iSCSI ready today.
  
  
• The thinking is that such software implementations will provide high enough performance at a lower cost versus a hardware implementation. As far as the overhead imposed on the server by a software implementation, the theory is that since we buy servers like we buy clothes for our kids, hoping the application(s) will grow into them, there are going to be plenty of CPU cycles available. The iSCSI market proved that theory to be correct even in server virtualization implementations.



Figure 3) Software initiator stack for FCoE.

What Stays the Same?

For those who are already using Fibre Channel, the day-to-day tasks with FCoE such as zoning and LUN mapping, as well as familiar fabric events such as registered state change notification (RSCN) and link state path selection (FSPF), still apply. This means that FCoE is a relatively simple protocol migration. Given that change can be painful, having a new protocol that leverages existing know-how, processes, and investments and makes the transition to Ethernet simpler is a huge benefit.

How Does FCoE Differ from iSCSI?

FCoE replaces the TCP/IP layer used in iSCSI and relies on improvements in the Ethernet layer such as:

• Proper pause frame implementation
• Per priority pause
• No TCP retries (timeouts)
• No IP routing
• No broadcast storms (no ARPs)

Figure 4) Block-level comparison of various storage protocols.

Since there isn’t an IP layer in FCoE, it means that FCoE is not intrinsically routable. However, that doesn't mean it can't be routed. FCoE routing can be performed using already established protocols such as FCIP.

The iSCSI protocol can be implemented in networks that are subject to packet loss, and it doesn't require 10GbE. FCoE requires 10GbE, and a lossless network with infrastructure components that properly implement pause frame requests and per priority pause flow control (PFC) based on separate traffic classes that map to different priorities. The idea behind PFC is that during periods of congestion, high-priority traffic will be allowed to continue while lower priority traffic will be paused.

Your 10GbE switches will also need support for Data Center Ethernet (DCE), an enhanced Ethernet that includes a range of enhancements such as classes of service, better congestion control, and improved management. FCoE also requires Jumbo Frame support because the FC payload is 2,112 bytes and cannot be broken up; iSCSI does not require Jumbo Frames.

What Will Happen to Fibre Channel?

With all the buzz about FCoE, what will happen to Fibre Channel? Will the transition to 16Gb technology still occur, or will FCoE be in the driver's seat? Will Ethernet dictate further developments (40GbE and 100GbE)? As it stands right now, 16Gb FC is planned for 2011. In a recent press release the FCIA stated that it "strongly" supports the development of 16Gb FC and it also supports FCoE. I believe 16Gb FC will become reality, but a larger question will be its adoption rate relative to FCoE. That remains to be seen.

What Should You Do?

What you do now depends on your situation. For those with large investments in Fibre Channel who have no need for upgrades in the next couple of years, your best bet is probably to do nothing. If you are planning upgrades in that time frame, then you should take a serious look at FCoE. Given that the current FC switch vendors will be migrating most of their customer base to Ethernet, at some point you'll probably end up making the switch.

Technology can solve a lot of problems; however, group alignment within an organization is not one of them. One of the challenges iSCSI has faced in the enterprise is squabbling between the storage team and the network team as to who owns the network. In FC implementations the storage guys own the fabric; with iSCSI it's the networking group. In order for FCoE to succeed, these two groups will have to align and work closer together than ever before, and that might be the biggest hurdle for FCoE to overcome.

If you want to learn more about FCoE, my colleague Silvano Gai recently wrote a book on the subject entitled Data Center Networks and Fibre Channel over Ethernet (FCoE). The book covers the protocol in great technical depth and describes the function of each component of the FCoE stack.

Nick Triantos Consulting Systems Engineer NetApp A member of NetApp’s Consulting Systems Engineering Group, Nick is focused on NetApp Partner development and enablement of NetApp® virtualization products and SAN solutions. Nick has been in systems or support engineering roles for nearly 15 years, including positions at HP as an account support engineer (server group) and presales technical consultant (storage group). Nick maintains a blog and has been a frequent contributor to Tech OnTap.