Toby Smith Stock Review

[MLD38]

FCoE/iSCSI debate


Posted by Alacritech's Larry Boucher ( blog Larry's Thoughts) on July 19, 2007
http://larrysthoughts.com/blog/2007/07/post.html

..."What is the story with FCoE? Why has it suddenly become the darling of the Fiber Channel world? A closer look at this protocol and where it might take us is probably a worthwhile, and certainly timely, topic for the storage community.

FCoE can be likened to UDP. UDP is a lightweight transport protocol that was used to move data more efficiently than TCP. However, it doesn’t guarantee delivery of data, so to use it, one must wish to send data that if lost, will not be missed or which the application can re-transmit (certainly not practical in the wide area). Whether or not one can accept this tradeoff largely depends upon the application.

In the case of broadcast audio, the use of UDP is frequently not a problem, as in general, our ears are capable of integrating or smoothing out small drop-outs. Video is much more of a problem, as the encoding schemes are not forgiving of drop-outs, and our eyes are far less forgiving as well. In the past, UDP was used for local area transmissions largely because local area connections were of sufficient quality that errors occurred very infrequently. Users could accept the much less efficient means of guaranteeing delivery implemented at the application layer because errors occurred so infrequently.

Today however, with corporate data being transported extensively over both the local and wide area, users simply can’t tolerate inconsistencies in delivery associated with UDP, and its use has consequently been sharply curtailed. In addition, faster processors and TCP offload have obviated the need for a protocol more efficient than TCP. In fact, using TCP offload is more processor efficient than UDP.

The major argument for FCoE is exactly that of UDP—efficiency. FCoE provides Fiber Channel functionality over Ethernet. This means that the robust qualities of TCP are traded off for the less functional but lighter weight upper layer Fiber Channel protocols. In addition to claims regarding Fiber Channel’s efficiency, arguments have been made that FCoE preserves the “look and feel” of Fiber Channel, which is understood by enterprise storage professionals that presently manage the storage infrastructure.

Depending on the agenda of the person or organization that is pushing FCoE, I would add a few other objectives that can be attributed to those advocating its use. At a high level, storage management can be broken into two areas: the management of storage devices and data, and the management of the storage infrastructure. If standard Ethernet protocols were used (as is the case with iSCSI) the storage infrastructure effectively is merged with the Ethernet infrastructure.

As long as sufficient differences exist from Layer 3 and up, between the standard networking infrastructure and the storage infrastructure, separate people and products can be justified to build and support the storage infrastructure. This allows the storage individuals, many of whom have labored hard to master the complexities of Fiber Channel, to feel secure in their jobs, and the storage network suppliers to continue to sell high-margin relatively low-volume products.

I think that one thing that both sides of the FCoE/iSCSI debate would agree on is that FCoE is not designed for use in a large network, and certainly not for the wide area. I suspect that everyone would also agree that like TCP and UDP, there is no good way for a server to support the same data under both protocols simultaneously. Therefore, as remote archival, synchronized remote facilities and consumer demand for remote block level data increase, it will become more and more difficult to use FCoE effectively, and like UDP, it will slowly atrophy.

Gateways from FCoE to iSCSI (or FCIP) will add cost and performance penalties, while iSCSI will have the flexibility to provide any cost/performance required, from a basic NIC to a sophisticated Chimney based accelerator. The network infrastructure in an iSCSI environment can be maintained completely separately from the remaining corporate network if desired, but it is still comprised of components that are understood by standard networking personnel, and are far less expensive than the low-volume storage only infrastructure products.

Because the low cost and high level of functionality of iSCSI, it is hard to imagine how, long term, all storage will not be delivered over conventional Ethernet and TCP/IP. This is likely to happen regardless of the Fiber Channel communities’ move to FCoE. However, it looks to me like FCoE is the camel’s nose under the iSCSI tent. That, combined with the fact that iSCSI is growing very rapidly and will ramp that much faster with the advent of ten gigabit Ethernet, while FCoE is still two years from seeing any product at all would lead me to believe that there is a good chance that FCoE will be still-born."...

*******************************************************

Alacritech's Larry Boucher profile:
http://www.alacritech.com/Company/Boucher.aspx

Jan 2002 Q&A with Larry Boucher by r. Koprowski (Koprowski has been a journalist for 17 years, covering technology and computing for PBS-TV, The Wall Street Journal, and Forbes ASAP, among others.)
http://www.smartcomputing.com/editorial/article.asp?article=articles/archive/c0201/54c01/54c01.asp
...Back in 1979 at Shugart Associates, Larry Boucher invented the SCSI standard, the famed "scuzzy" interface. That interface permits PCs to connect to peripherals and storage devices....

Larry Boucher & others in a 2007 exchange re FCoE and iSCSI
http://www.ietf.org/mail-archive/web/ips/current/msg02357.html


--------------------------------------------------------------------------------

May 2007: "Will server virtualization end the FC-iSCSI debate? There are pros and cons on both sides of the argument -- and the winning answer depends on your enterprise's storage strategy." by Mario Apicella
http://www.infoworld.com/article/07/05/04/19OPstorinside_1.html

includes:

..."Back to our two transport protocols. Both FC and iSCSI have many appealing points; the former is field-proven and mature, while the latter claims ease of use and a better future (up to 100G). There is obviously more to both technologies than that succinct comparison, but a conversation I had last week with Dell Storage Technologist Matt Baker reminded me of another selling point that iSCSI storage solutions can claim over rival FC-based products. It has to do with server virtualization and how its requirements may impact your storage acquisitions.
“One of the areas that not very many people talk about is storage for server virtualization,” Baker says, adding that many Dell customers are concerned about the intersection points of server virtualization and storage area networks. “Part of what we found [in our customers’ concerns] is the complexity associated with managing a FC-based shared storage solution in a virtual environment."

Baker enumerates the major challenges that admins face when combining server virtualization and SANs, including facilitating VM mobility while maintaining data integrity and data security. What's the best way to back up data, and how can you take advantage of the benefits of a SAN in a virtualized world?

“As we dug more into it, we started to discover that iSCSI, in many ways, helps to alleviate those challenges,” Baker says.

From a storage network admin point of view, I couldn’t agree more. For example, "...

***************************************

April 2007: "A single protocol in your SAN? New FCoE standard proposes to extend native FC transport to Ethernet." also by Mario Apicella
http://www.infoworld.com/infoworld/article/07/04/13/16OPstorinside_1.html

includes:
..."What FCoE proposes is essentially running FC directly over Ethernet, without using another transport layer (TCP/IP serves as this transport layer for FCIP). With FCoE, the FC frame is wrapped inside an Ethernet frame, all with minimal overhead as compared to FCIP.

One of the problems that the new standard has to solve is that it's perfectly acceptable to discard a frame in the Ethernet world, while similar behavior is a mortal sin in FC networks. For example, Ethernet drops frames when the amount of incoming traffic exceeds the capacity of the buffers in a switch. Dropping a frame doesn't cause data loss, because TCP will retransmit frames for which there is no acknowledgement, but that retransmission causes a delay.

By contrast, the obsessively efficient FC protocol never drops a frame. Each end of a FC connection keeps the other party informed of how many buffers are available, and frames are sent only if there is enough buffer capacity at the destination port.

FCoE aims to mimic that behavior via PAUSE frames, a little-known and rarely used extension to the Ethernet protocol that, as its name implies, suspends the flow when there is a congestion point.

It may take years before FCoE products reach our SANs, but I'll speculate that the new standard will make it easier and more affordable to connect your servers to the fabric without doubling with another transport protocol such as iSCSI. You may still need FCIP to bridge remote fabrics, but local connections can speak FC regardless of the medium.

Is the future of iSCSI threatened by FCoE? It's hard to tell at this point, but I think the usual factors such as cost and friendliness of the solutions will determine the outcome.

Perhaps the most interesting aspect of FCoE is that it will make it possible to consolidate FC and Ethernet into a single, reliable, fast fabric -- 10 Gig Ethernet, anyone? "...