INTERNET-DRAFT                                     David L. Black (ed.)
PWE3 WG                                                 EMC Corporation
Intended Status: Standard Track                       Linda Dunbar(ed.)
Expires: November 2011                              Huawei Technologies
                                                             Moran Roth
                                                               Infinera
                                                          Ronen Solomon
                                                       Orckit-Corrigent
                                                            May 3, 2011


                  Encapsulation Methods for Transport of
                 Fibre Channel Traffic over MPLS Networks

                      draft-ietf-pwe3-fc-encap-16.txt


Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with
   the provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on November 3, 2011.

Copyright Notice

   Copyright (c) 2011 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with



Black and Dunbar        Expires November 2011                  [Page 1]

Internet-Draft             FC Encapsulation                    May 2011


   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Abstract

   A Fibre Channel pseudowire (PW) is used to carry Fibre Channel
   traffic over an MPLS network. This enables service providers to take
   advantage of MPLS to offer "emulated" Fibre Channel services. This
   document specifies the encapsulation of Fibre Channel traffic within
   a pseudowire. It also specifies the common procedures for using a PW
   to provide a Fibre Channel service.

Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC-2119].

Table of Contents

   1. Introduction...................................................3
      1.1. Transparency..............................................3
      1.2. Bandwidth Efficiency......................................4
      1.3. Reliability...............................................5
   2. Reference Model................................................5
   3. Encapsulation..................................................8
      3.1. The Control Word.........................................10
      3.2. MTU Requirements.........................................11
      3.3. Mapping of FC traffic to PW packets......................11
         3.3.1. FC Data Frames (PT=0) and FC Login Frames (PT=1)....11
         3.3.2. FC Primitive Sequences and Primitive Signals (PT=2).12
         3.3.3. FC PW Control Frames (PT=6).........................14
      3.4. PW failure mapping.......................................15
   4. Signaling of FC Pseudowires...................................15
   5. Timing Considerations.........................................15
   6. Security Considerations.......................................17
   7. Applicability Statement.......................................17
   8. IANA Considerations...........................................18
   9. Acknowledgments...............................................20
   10. Normative References.........................................20
   11. Informative references.......................................21
   Authors' Addresses...............................................22





Black and Dunbar        Expires November 2011                  [Page 2]

Internet-Draft             FC Encapsulation                    May 2011


1. Introduction

   Fibre Channel (FC) is a high-speed communications technology, used
   primarily for Storage Area Networks (SANs). Within a single site
   (e.g., data center), an FC-based SAN connects servers to storage
   systems, and FC can be extended across sites. When FC is extended
   across multiple sites, the most common usage is storage replication
   in support of recovery from disasters (e.g., flood or fire that takes
   a site out of operation). This is particularly the case over longer
   distances where network latency results in unacceptable performance
   for a server whose storage is not at the same site. Fibre Channel is
   standardized by INCITS Technical Committee T11 [T11] and multiple
   methods for encapsulating and transporting FC traffic over other
   networks have been developed [FC-BB-6].

   FCIP, as described in [RFC3821] and [FC-BB-6], interconnects
   otherwise isolated FC SANs over IP Networks. FCIP uses FC Frame
   Encapsulation [RFC3643] to encapsulate FC frames for tunneling over
   an IP-based network. Since IP networks may drop or reorder packets,
   FCIP relies on TCP to retransmit dropped frames and restore the
   delivery order of reordered frames. Due to possible delay variation
   and TCP timeouts, special timing mechanisms are required to ensure
   correct Fibre Channel operation over FCIP [FC-BB-6].

   MPLS networks can be provisioned and operated with very low loss
   rates and very low probability of reordering, making it possible to
   directly interconnect Fibre Channel ports over MPLS. A Fibre Channel
   pseudowire (FC PW) is a method to transparently transport FC traffic
   over an MPLS network resulting in behavior similar to a pair of FC
   ports that are directly connected by a physical FC link. The result
   is simpler control processing by comparison to FCIP.

   This document specifies the encapsulation of FC traffic into an MPLS
   pseudowire and related PW procedures to transport FC traffic over
   MPLS PWs. The complete FC pseudowire specification consists of this
   document and the FC PW portion of the T11 [FC-BB-6] standard. The
   following subsections describe some of the requirements for
   transporting FC traffic over an MPLS network.

1.1. Transparency

   Transparent extension of an FC link is a key requirement for
   transporting FC traffic over a PW. This requires the FC PW to emulate
   an FC Link between two FC ports, similar to the approach defined for
   FC over GFPT in [FC-BB-6]. GFPT is an Asynchronous Transparent
   Generic Framing Procedure specified by ITU-T, see [FC-BB-6] for
   details and reference to the ITU-T specifications. This results in


Black and Dunbar        Expires November 2011                  [Page 3]

Internet-Draft             FC Encapsulation                    May 2011


   transparent forwarding of FC traffic over the MPLS network from both
   the FC Fabric and the network operator points of view.

   Transparency distinguishes the FC PW approach from FCIP. An FC PW
   logically connects the FC port on the FC link attached to one end of
   the PW directly with the FC port on the far end of the FC link
   attached to the other end of the PW, whereas FCIP introduces FC
   B_Ports at both ends of the extended FC link; each FC B_Port is
   connected to an FC E_Port in an FC switch on the same side of the
   link extension.

1.2. Bandwidth Efficiency

   The bandwidth allocated to a PW may be less than the rate of the
   attached FC port. When there is no data exchange on a native FC link,
   Idle Primitive Signals are continuously exchanged between the two FC
   ports. In order to improve the bandwidth efficiency across the MPLS
   network, it is necessary for the FC PW PE to suppress (or drop) the
   Idle Primitive signals generated by its adjacent FC ports. The far
   end FC PW PE regenerates Idle Primitive signals to send to its
   adjacent FC port as required, see [FC-BB-6].

   FC link control protocols require an FC port to continuously send the
   same FC Primitive Sequence [FC-FS-2] until a reply is received or
   some other event occurs. To improve bandwidth efficiency, the FC PW
   PE encapsulates a subset of repeated FC Primitive Sequences to send
   across the WAN [FC-BB-6]. For example, in a sequence of identical
   received primitives, only every fourth primitive may be sent across
   the MPLS network. Alternatively, a time-based approach may be used to
   send a copy of the repeated FC Primitive Sequence once every few
   milliseconds. The far end FC PW PE regenerates the FC link behavior
   by continuously sending the Primitive Sequence most recently received
   from the WAN until a new primitive signal, primitive sequence or data
   frame is received from the WAN.

   The sending FC PW PE may unilaterally choose any convenient subset
   for sending the same FC Primitive Sequence. This is acceptable
   because the receiving FC PW PE generates a continuous stream of the
   most recently received FC Primitive Sequence on the outgoing native
   FC link, independent of the arrival rate of that FC Primitive
   Sequence from the WAN. In practice, a 10:1 reduction in FC Primitive
   Sequence transmission rate achieves 90% of the bandwidth benefits
   without loss of FC functionality and sending a copy every few
   milliseconds does not pose a serious risk of exceeding the timeouts
   specified in Section 5 below.




Black and Dunbar        Expires November 2011                  [Page 4]

Internet-Draft             FC Encapsulation                    May 2011


   These bandwidth efficiency techniques may cause changes in the FC
   traffic that traverses an FC PW (e.g., number of IDLE signals or
   number of identical Primitive Sequences), but the far end FC PW PE's
   regeneration of FC link behavior on the attached FC port is
   transparent to the FC ports connected to each PW PE.

1.3. Reliability

   Fibre Channel does not employ a native frame retransmission protocol,
   and treats most frame delivery failures as errors. FC SAN traffic
   requires a very low frame loss rate because the typical result of a
   failure to deliver a frame is an I/O operation failure. Recovery from
   such I/O failures involves I/O operation retries after what may be a
   significant delay (30 second and 60 second timeouts are common).  In
   addition, such retries are likely to be logged as errors indicating
   possible problems with FC equipment or cables.  Hence, drops, errors
   and discards of FC frames must be very rare for an FC PW.

   FC SAN implementations have limited tolerance for frame reordering.
   Any reordering affecting more than a few frames within a single
   higher level operation (e.g., a read or write I/O) is usually treated
   as an error by the destination FC port, resulting in discards of the
   frames involved; some deployed FC implementations treat all such
   within-operation frame reordering as errors that result in frame
   discards. As a result, FC frame reordering must be minimized for an
   FC PW.

   The FC PW does not compensate for frame drops, discards or
   reordering.  The MPLS network that hosts the FC PW is expected to be
   designed and operated in a fashion that makes such events very rare.

   In contrast to the TTL field in an IP packet, FC uses a constant
   delivery timeout value (R_A_TOV) for which 10 seconds is the default.
   Each FC frame must be delivered or discarded within that timeout
   period after it is sent, see Section 5.

  2. Reference Model

   An FC PW extends a native FC link over an MPLS network. This document
   specifies the PW encapsulation for FC. Figure 1 describes the
   reference models (derived from [RFC3985]) that support the FC PW. FC
   traffic is received by PE1's FC attachment channel, encapsulated at
   PE1, transported across MPLS network, decapsulated at PE2, and
   transmitted onward via the PE2's FC attachment channel. This document
   assumes that a pseudowire can be provisioned statically or via a
   signaling protocol as defined in [RFC4447].



Black and Dunbar        Expires November 2011                  [Page 5]

Internet-Draft             FC Encapsulation                    May 2011


         |<-------------- Emulated Service ----------------->|
         |                                                   |
         |          |<------- Pseudowire -------->|          |
         |          |                             |          |
         |          |    |<-- MPLS Tunnel -->|    |          |
         |          V    V                   V    V          |
         V   AC     +----+                   +----+    AC    V
   +-----+    |     | PE1|===================| PE2|     |    +-----+
   |     |----------|............PW1..............|----------|     |
   | CE1 |    |     |    |                   |    |     |    | CE2 |
   |     |----------|............PW2..............|----------|     |
   +-----+  ^ |     |    |===================|    |     | ^  +-----+
         ^  |       +----+                   +----+     | |  ^
         |  |   Provider Edge 1          Provider Edge 2  |  |
         |  |                                             |  |
   Customer |                                             | Customer
   Edge 1   |                                             | Edge 2
            |                                             |
            |                                             |
     Native FC service                             Native FC service

         Figure 1: PWE3 FC Interface Reference Configuration

   The following reference model describes the termination point of each
   end of the PW within the PE:

              +-----------------------------------+
              |                PE                 |
      +---+   +-+  +-----+  +------+  +------+  +-+
      |   |   |P|  |     |  |PW ter|  | MPLS |  |P|
      |   |<==|h|<=| NSP |<=|minati|<=|Tunnel|<=|h|<== From network
      |   |   |y|  |     |  |on    |  |      |  |y|
      | C |   +-+  +-----+  +------+  +------+  +-+
      | E |   |                                   |
      |   |   +-+  +-----+  +------+  +------+  +-+
      |   |   |P|  |     |  |PW ter|  | MPLS |  |P|
      |   |==>|h|=>| NSP |=>|minati|=>|Tunnel|=>|h|==> To network
      |   |   |y|  |     |  |on    |  |      |  |y|
      +---+   +-+  +-----+  +------+  +------+  +-+
              |                                   |
              +-----------------------------------+
              Figure 2: PW reference diagram







Black and Dunbar        Expires November 2011                  [Page 6]

Internet-Draft             FC Encapsulation                    May 2011


The Native Service Processing (NSP) function includes the following
functionality:

   o  Idle Suppression: any FC Idle signals received from the source
      PE's attached FC port are suppressed and re-generated at the
      destination PE to send on its attached FC port when there is no
      other FC traffic to send;

   o  FC Primitive Sequence Reduction: a subset of repetitive FC
      Primitive Sequences received from the attached FC port at the
      source PE is selected for WAN transmission, with the destination
      PE sending the FC Primitive Sequence most recently received from
      the WAN on the destination PE's attached FC port continuously
      until a new packet is received from the WAN; and

   o  Flow Control: the Alternate Simple Flow Control (ASFC) protocol is
      used for buffer management in concert with the peer PW PE's NSP
      function so that FC traffic is not dropped. ASFC is a simple
      pause/resume protocol that allows operation repetition; the
      receiver responds to the first pause or resume operation in an
      identical sequence of operations, and ignores the rest of the
      sequence.

   The NSP flow control functionality is required to extend FC's credit-
   based flow control to address situations where the number of buffer
   credits available to an FC link is insufficient to utilize the
   available bandwidth over the additional distance and latency
   represented by the FC pseudowire.  The NSPs avoid this problem by
   inserting ASFC into FC's link flow control used on the attached FC
   ports, see [FC-BB-6].

   In contrast, Idle Suppression and FC Primitive Sequence Reduction are
   bandwidth optimizations that are included in the NSP for clarity in
   this document.  Analogous optimizations are not treated as part of
   the NSP by other pseudowires (e.g., ATM idle frame suppression is not
   considered to be an NSP function by [RFC4717]).

   The NSP function is specified in detail by [FC-BB-6].











Black and Dunbar        Expires November 2011                  [Page 7]

Internet-Draft             FC Encapsulation                    May 2011


3. Encapsulation

   This specification provides port to port transport of FC encapsulated
   traffic. There are a number of port types defined by Fibre Channel,
   including:

   o  An N_port is a port on the node (e.g. host or storage device) used
      with both FC-P2P (Point to Point) or FC-SW (Switched fabric)
      topologies. Also known as a Node port.

   o  An NL_port is a port on the node used with an FC-AL (Arbitrated
      Loop) topology. Also known as a Node Loop port.

   o  An F_port is a port on the switch that connects to a node point-
      to-point (i.e. connects to an N_port). Also known as a Fabric
      port. An F_port is not loop capable.

   o  An FL_port is a port on the switch that connects to a FC-AL loop
      (i.e. to NL_ports). Also known as Fabric Loop port.

   o  An E_port is a port used to connect two Fibre Channel switches.
      Also known as an Expansion port. When E_ports between two switches
      are connected to form a link, that link is referred to as an
      inter-switch link (ISL).

   Among the port types listed above, only the following FC connections
   (as specified in [FC-BB-6]) are supported by an FC PW over MPLS:

          - N_Port to N_Port, established by an FC PLOGI (Port Login)
              operation

          - N_Port to F_Port, established by an FC FLOGI (Fabric Login)
              operation

          - E_Port to E_Port, established by an FC ELP (Exchange Link
              Parameters) operation













Black and Dunbar        Expires November 2011                  [Page 8]

Internet-Draft             FC Encapsulation                    May 2011


   FC traffic flowing over an FC PW is subdivided into four payload
   types (PT) that are encoded in the PW Control Word (see Section 3.1):

   1. FC login traffic (PT = 1): FC login operations and responses that
      establish connections between FC ports.  The three FC login
      operations are PLOGI, FLOGI, and ELP. These operations and their
      responses may require the NSP to allocate buffer resources, see
      the specification of Login Exchange Monitors in [FC-BB-6].

   2. FC data traffic (PT = 0): All FC frames other than those involved
      in an FC login operation.

   3. FC Primitive Sequences and Signals (PT = 2): Native FC link
      control operations - 4-character primitive sequences and signals
      that are not encapsulated in FC frames. See [FC-BB-6] and
      [FC-FS-2].

   4. FC PW Control (PT = 6): FC PW control operations exchanged only
      between the endpoints of the PW. FC PW control operations are used
      for ASFC flow control, ping (e.g., for round trip latency
      measurement) and reporting native FC link errors, see [FC-BB-6].

   This FC PW specification is limited to use with FC service classes 2,
   3 and F (see [FC-FS-2]).  Other FC service classes (e.g., 1, 4 and 6)
   MUST NOT be used with an FC PW.  Numbered FC service classes are used
   for end-to-end FC traffic, whereas service class F is used for inter-
   switch traffic in an FC switched fabric.

   This FC PW specification is limited to native FC attachment links
   that employ an 8b/10b transmission code (see [FC-FS-2]).  The
   protocol specified in this document converts a received 10b code to
   its 8b counterpart for PW encapsulation, and hence does not support
   attached FC links that use a 64b/66b transmission code (e.g., 10GFC,
   16GFC); such links MUST NOT be attached to an FC PW PE unless their
   link speed can be negotiated to one that uses 8b/10b encoding. If an
   invalid 10b code that cannot be converted to an 8b code is received
   from an FC link, the PE sends an FC PW control frame to report the
   error, see [FC-BB-6].











Black and Dunbar        Expires November 2011                  [Page 9]

Internet-Draft             FC Encapsulation                    May 2011


3.1. The Control Word

   The Generic PW Control Word, as defined in "PWE3 Control Word"
   [RFC4385] MUST be used for FC PW to facilitate the transport of short
   packets (by setting the Length field as detailed below), and convey
   the flag bits defined below. The structure of the Control Word for
   the FC PW is as follows:

                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |0 0 0 0| PT  |X|0 0|  Length   |     Sequence Number           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                     Figure 3 - Control Word Structure

   The first four bits of the PW Control Word MUST be set to 0 by the
   ingress PE to indicate PW data.

   Three of the four Flags bits are used to convey the PT - Payload Type
   indication. The 3-bit binary value in this field identifies the
   payload type carried by a PW packet. The following types are defined:

           PT = 0: FC data frame.

           PT = 1: FC login frame.

           PT = 2: FC Primitive Sequence(s) and/or Primitive Signal(s).

           PT = 6: FC PW Control Frame (refer to [FC-BB-6] for usage).

   Packets with other values in the PT field are not valid for the FC PW
   and MUST be discarded by the receiving FC PW PE.

   X - This flag bit is not used by this version of the protocol. It
   SHOULD be set to zero by the sender and MUST be ignored by the
   receiver.

   The fragmentation bits (bits 8-9) are not used by the FC PW protocol.
   These bits may be used in the future for FC specific indications as
   defined in [RFC4385]. The fragmentation bits SHOULD be set to zero by
   the ingress PE and MUST be ignored by the egress PE.

   The Length field enables recovery of the original pseudowire packet
   when a short packet is padded to the minimum 64 octet packet size
   required for Ethernet, see [RFC4385].  The Length field MUST be used


Black and Dunbar        Expires November 2011                 [Page 10]

Internet-Draft             FC Encapsulation                    May 2011


   for packets shorter than 64 octets, MUST be set to zero for longer
   packets, and MUST be processed according to the rules specified in
   [RFC4385].

   The sequence number is not used for the FC PW and MUST be set to 0 by
   the ingress PE, and MUST be ignored by the egress PE.

3.2. MTU Requirements

   The MPLS network MUST be able to transport the largest Fibre Channel
   frame after encapsulation, including the overhead associated with the
   encapsulation. The maximum FC frame size is 2164 octets without PW
   and MPLS labels (refer to Figure 4); this maximum size is a constant
   value that is required for all FC implementations [FC-FS-2]. The MPLS
   network SHOULD accommodate frames of up to 2500 octets in order to
   support possible future increases in the maximum FC frame size.

   Fragmentation, as described in [RFC4623], SHALL NOT be used for an FC
   PW, therefore the network MUST be configured with a minimum MTU that
   is sufficient to transport the largest encapsulated FC frame.

3.3. Mapping of FC traffic to PW packets

   FC frames, Primitive Sequences, and Primitive Signals are transported
   over the PW. All packet types are carried over a single PW. In
   addition to the PW Control Word, an FC Encapsulation Header is
   included in the PW packet. This FC Encapsulation Header is not used
   in this version of the protocol; it SHOULD be set to zero by the
   sender and MUST be ignored by the receiver.

3.3.1. FC Data Frames (PT=0) and FC Login Frames (PT=1)

   FC data frames and FC login frames share a common encapsulation
   format, except that the PT field in the FC PW control word is set to
   0 for data frames and is set to 1 for login frames. An FC login frame
   contains an FC PLOGI, FLOGI or ELP operation or response that
   requires special processing by the NSP in support of flow control,
   see [FC-BB-6].

   Each FC data frame or login frame is mapped to a PW packet, including
   the Start Of Frame (SOF) delimiter, frame header, CRC field and the
   End Of Frame (EOF) delimiter, as shown in figure 4.







Black and Dunbar        Expires November 2011                 [Page 11]

Internet-Draft             FC Encapsulation                    May 2011


                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +---------------------------------------------------------------+
      |                    FC PW Control Word                         |
      +---------------------------------------------------------------+
      |                  FC Encapsulation Header                      |
      +---------------+-----------------------------------------------+
      |     SOF Code  |             Reserved                          |
      +---------------+-----------------------------------------------+
      |                                                               |
      +-----                        FC Frame                      ----+
      |                                                               |
      +---------------------------------------------------------------+
      |                              CRC                              |
      +---------------+-----------------------------------------------+
      |     EOF Code  |                Reserved                       |
      +---------------+-----------------------------------------------+

      Figure 4 - FC frame (SOF/Data/CRC/EOF) encapsulation in PW packet

   The SOF and EOF frame delimiters are each encoded into a single octet
   as specified in [RFC3643], except that the codes for delimiters that
   apply only to FC service class 4 (SOFi4, SOFc4, SOFn4, EOFdt, EOFdti,
   EOFrt, EOFrti - see [FC-FS-2]) MUST NOT be used.

   The CRC in the frame is obtained directly from the FC attachment
   channel, so that the PW PE is not required to re-calculate the CRC or
   to check the CRC in the received frame. The CRC will be checked by
   the FC port that receives the frame, ensuring that coverage is
   provided for data errors that occur between the PW endpoints.  This
   CRC behavior differs from the FCS retention technique for PWs defined
   in [RFC4720] which states that "as usual, the FCS MUST be examined at
   the ingress PE, and errored frames MUST be discarded."


3.3.2. FC Primitive Sequences and Primitive Signals (PT=2)

   FC Primitive Sequences and Primitive Signals are FC ordered sets. On
   an 8b/10b-coded FC link, an ordered set consists of four 10b
   characters, starting with the K28.5 character, followed by three
   Dxx.y data characters. All FC ordered sets start with a K28.5 control
   character, but the three following Dxx.y data characters differ
   depending on the ordered set. A Kxx.y control character has a
   different 10b code from the corresponding Dxx.y data character, but
   uses the same 8b code (e.g., K28.5 and D28.5 both use the 8b code
   0xBC). Here are two examples of ordered sets:


Black and Dunbar        Expires November 2011                 [Page 12]

Internet-Draft             FC Encapsulation                    May 2011


   o  Idle(IDLE) is K28.5 - D21.4 - D21.5 - D21.5. This FC primitive
      signal is sent when the FC link is idle; it is suppressed by the
      FC PW NSP and not sent over the WAN.

   o  Link Reset Response(LRR) is K28.5 - D21.1 - D31.5 - D9.2 (this FC
      primitive sequence is used as part of FC link initialization and
      recovery).

   Each ordered set is encapsulated in a PW packet containing the
   encoded K28.5 control character [FC-BB-6], followed by three encoded
   data characters, as shown in Figure 5.

                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +---------------------------------------------------------------+
      |                    FC PW Control Word                         |
      +---------------------------------------------------------------+
      |                  FC Encapsulation Header                      |
      +---------------+---------------+---------------+---------------+
      |     K28.5     |     Dxx.y     |     Dxx.y     |     Dxx.y     |
      +---------------+---------------+---------------+---------------+
      |                                                               |
      +----                                                       ----+
      |                                                               |
      +---------------+---------------+---------------+---------------+
      |     K28.5     |     Dxx.y     |     Dxx.y     |     Dxx.y     |
      +---------------+---------------+---------------+---------------+

             Figure 5 - FC Ordered Sets encapsulation in PW packet

   The K28.5 10b control character received from the PE's attached FC
   link is encoded for the FC PW as its 8b counterpart (0xBC).  Because
   the same 8b value (0xBC) is used to encode a D28.5 data word, the
   receiving FC PW PE:

   o  MUST check for presence of an 8b K28.5 value (0xBC) at the start
      of each ordered set (see Figure 5), and MUST send that value as a
      10b K28.5 character on the attached FC link.

   o  MUST send the following three Dxx.y 8b values as Dxx.y 10b
      characters on the attached FC link and MUST NOT send any of these
      Dxx.y 8b values as 10b Kxx.y characters on the attached FC link.

   A PW packet may contain one or more encoded FC Ordered sets [FC-BB-
   6]. The Length field in the FC PW Control Word is used to indicate
   the packet length when the PW packet contains multiple Ordered Sets.



Black and Dunbar        Expires November 2011                 [Page 13]

Internet-Draft             FC Encapsulation                    May 2011


   For this reason, FC PW packets that contain FC Ordered Sets MUST NOT
   be larger than 60 octets (8 octets of header words plus at most 13
   ordered sets), in order to ensure that the Length field contains a
   non-zero value, see [RFC4385].

   Idle Primitive Signals could be carried over the PW in the same
   manner as Primitive Sequences. However, [FC-BB-6] requires that Idle
   Primitive Signals be dropped by the Ingress PE and re-generated by
   the egress PE in order to reduce bandwidth consumption (see [FC-BB-6]
   for further details).

   The egress PE extracts the Primitive Sequence or Primitive Signal
   from the received PW packet. For a Primitive Sequence, the PE
   continues transmitting the same FC Ordered Set to its attached FC
   port until an FC frame or another ordered set is received over the
   PW; see Section 1.2 above for discussion of ingress PE transmission
   behavior for Primitive Sequences.  A Primitive Signal is sent once,
   except that Idle Primitive Signals are sent continuously when there
   is nothing else to send.

3.3.3. FC PW Control Frames (PT=6)

   FC PW Control Frames are transported over the PW, by encapsulating
   each frame in a PW packet with PT=6 in the Control Word. FC PW
   Control Frame payloads are generated and terminated by the
   corresponding FC entity. FC PW Control frames are used for FC PW flow
   control (ASFC), ping and transmission of error indications. [FC-BB-6]
   specifies the generation and processing of FC PW Control Frames.  FC
   PW Control Frames are always shorter than 64 octets, and hence the
   Length field in the FC Control Word indicates their length.

                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +---------------------------------------------------------------+
      |                    FC PW Control Word                         |
      +---------------------------------------------------------------+
      |                  FC Encapsulation Header                      |
      +---------------------------------------------------------------+
      |                                                               |
      +-----              FC PW Control Frame                     ----+
      |                                                               |
      +---------------------------------------------------------------+

             Figure 6 - FC PW Control frame encapsulation in PW packet





Black and Dunbar        Expires November 2011                 [Page 14]

Internet-Draft             FC Encapsulation                    May 2011


3.4. PW failure mapping

   PW failures are detected through PW signaling failure, PW status
   notifications as defined in [RFC4447], or through PW OAM mechanisms
   and MUST be mapped to emulated signal failure indications. Sending
   the FC link failure indication to its attached FC link is performed
   by the NSP, as defined by [FC-BB-6].

4. Signaling of FC Pseudowires

   RFC4447 specifies the use of the MPLS Label Distribution Protocol,
   LDP, as a protocol for setting up and maintaining pseudowires. This
   section describes the use of specific fields and error codes used to
   control FC PW.

   The PW Type field in the PWid FEC element and PW generalized ID FEC
   elements MUST be set to the "FC Port Mode" value in section 8 below.

   The Control Word is REQUIRED for FC pseudowires.  Therefore the C-Bit
   in the PWid FEC element and PW generalized ID FEC elements MUST be
   set. If the C-Bit is not set, the pseudowire MUST NOT be established
   and a Label Release MUST be sent with an "Illegal C-Bit" status code
   [RFC4447].

   The Fragmentation Indicator (Parameter ID = 0x09) is specified in
   [RFC4446] and its usage is defined in [RFC4623]. Since fragmentation
   is not used in FC PW, the fragmentation indicator parameter MUST be
   omitted from the Interface Parameter Sub-TLV.

   The Interface MTU Parameter (Parameter ID = 0x01) is specified in
   [RFC4447]. Since all FC interfaces have the same MTU, this parameter
   MUST be omitted from the Interface Parameter Sub-TLV.

   The FCS Retention Indicator (Parameter ID = 0x0A) is specified in
   [RFC4720]. Since the CRC treatment defined in this document differs
   from one that is specified in [RFC4720], this parameter MUST be
   omitted from the Interface Parameter Sub-TLV.

5. Timing Considerations

   Correct Fibre Channel link operation requires that the FC link
   latency between CE1 and CE2 (refer to Figure 1) be:

   o  no more than one-half of the R_T_TOV (Receiver Transmitter Timeout
      Value, default value: 100 milliseconds) of the attached devices
      for Primitive Sequences;



Black and Dunbar        Expires November 2011                 [Page 15]

Internet-Draft             FC Encapsulation                    May 2011


   o  no more than one-half of the E_D_TOV (Error Detect Timeout Value,
      default value: 2 seconds) of the attached devices for frames; and

   o  within the R_A_TOV (Resource Allocation Timeout Value, default
      value: 10 seconds) of the attached fabric(s), if any. The FC
      standards require that the E_D_TOV value for each FC link be set
      so that the R_A_TOV value for the fabric is respected when the
      worst case latency occurs for each link, see [FC-FS-2].

   An FC PW MUST adhere to these three timing requirements and MUST NOT
   be used in environments where high or variable latency may cause
   these requirements to be violated.

   These three timeout values are ordered (R_T_TOV < E_D_TOV < R_A_TOV),
   so adherence to one-half of R_T_TOV for all FC PW traffic is
   sufficient. See [FC-FS-2] for definitions of the FC timeout values.

   The R_T_TOV is used by the FC link initialization protocol. If an FC
   PW's latency exceeds one-half R_T_TOV, initialization of the FC link
   that is encapsulated by the FC PW may fail, leaving that FC link in a
   non-operational state.

   The E_D_TOV is used to detect failures of operational FC links. If an
   FC PW's latency exceeds the one-half E_D_TOV requirement, the FC link
   that is encapsulated by the FC PW may fail. The usual FC response to
   such a link failure is to attempt to recover the FC link by
   initializing it.  That initialization will also fail if the FC PW
   latency exceeds one-half R_T_TOV (a tighter requirement).

   The R_A_TOV is used to determine when FC communication resources
   (e.g., values that identify FC frames) may be reused. If an FC PW's
   violation of the one-half E_D_TOV requirement is sufficient to also
   cause the FC fabric to violate the R_A_TOV requirement, then FC reuse
   of frame identification values after an R_A_TOV timeout may result in
   multiple FC frames with the same identification values, causing
   incorrect Fibre Channel operation. For example, if two such frames
   are swapped between I/O operations, the result may corrupt data in
   the I/O operations.

   The PING and PING_ACK FC PW control frames defined in Section 6.4.7
   of [FC-BB-6] SHOULD be used to measure the current FC pseudowire
   latency between the CE devices. If the measured latency violates any
   of the timing requirements, then the FC PW PE MUST generate a WAN
   Down event as specified in [FC-BB-6].

   The WAN Down event causes the PE to continuously send NOS (an FC
   primitive sequence) on the native FC link to the FC Port at the other


Black and Dunbar        Expires November 2011                 [Page 16]

Internet-Draft             FC Encapsulation                    May 2011


   end of that link (typically an E_Port on a switch in this case).
   This immediately causes the FC link that is carried by the PW to
   become non-operational, halting transmission of FC traffic.  However,
   it is not necessary to tear down the pseudowire itself in this
   situation (e.g., destroy the MPLS path set up by LDP).

   The Transparent FC-BB initialization state machine in [FC-BB-6]
   specifies the protocol used to attempt to recover from a WAN Down
   event (i.e., bring the WAN back up).  If that protocol brings the WAN
   back up, FC traffic will resume and the standard FC link recovery
   protocol will bring the encapsulated FC link back up. If the previous
   pseudowire was destroyed, attempts will be made to re-establish the
   path via LDP as part of recovering from the WAN Down event. If the PW
   round-trip latency remains above R_T_TOV, the initialization protocol
   for the FC PW will repeatedly time out in attempting to recover from
   the WAN Down event, preventing recovery of the FC link carried by the
   PW, see [FC-BB-6].

6. Security Considerations

   The FC PW is an MPLS pseudowire; for MPLS pseudowire security
   considerations, see the security considerations sections of [RFC3985]
   and [RFC4385].

   The protocols used to implement security in a Fibre Channel fabric
   are defined in [FC-SP]. These protocols operate at higher layers of
   the FC hierarchy and are transparent to the FC PW.

   The FC timing requirements (see Section 5) create an exposure of the
   FC PW to inserted latency. Injection of latency sufficient to cause
   the round trip time for an FC PW to exceed R_T_TOV (default: 100ms)
   may cause the FC PW to fail in an active fashion because the FC link
   initialization protocol repeatedly times out. OAM functionality for
   deployed FC PWs SHOULD monitor for persistence of this situation and
   respond accordingly (e.g., shut down the FC PW in order to avoid
   wasting WAN bandwidth on an FC PW whose FC link cannot be
   successfully initialized due to excessive latency).

7. Applicability Statement

   FC PW allows the transparent transport of FC traffic between Fibre
   Channel ports while saving network bandwidth by removing FC Idle
   Signals and reducing the number of FC Primitive Sequences.

   o  The pair of CE devices operates as if they were directly connected
      by an FC link. In particular they react to Primitive Sequences on
      their local FC links as specified by the FC standards.


Black and Dunbar        Expires November 2011                 [Page 17]

Internet-Draft             FC Encapsulation                    May 2011


   o  The FC PW carries only FC data frames, FC Primitive Signals and a
      subset of the copies of an FC Primitive Sequence. Idle Primitive
      Signals are suppressed, and long streams of the same Primitive
      Sequence are reduced over the PW thus saving bandwidth.

   o  The PW PE MUST generate Idle Primitive Signals to the attached FC
      link when there is no other traffic to transmit on the attached FC
      link [FC-FS-2].

   o  The PW PE MUST send Primitive Sequences continuously to the
      attached FC port, as required by the FC standards [FC-FS-2].

   FC PW traffic should only traverse MPLS networks that are provisioned
   based on traffic engineering to provide dedicated bandwidth for FC PW
   traffic. The MPLS network should enforce ingress traffic policing so
   that delivery of FC PW traffic can be assured. To extend FC across a
   network that does not satisfy these requirements, FCIP SHOULD be used
   instead of an FC PW, see [RFC3821] and [FC-BB-6].

   This document does not provide any mechanisms for protecting an FC PW
   against network outages. As a consequence, resilience of the emulated
   FC service to such outages is dependent upon the underlying MPLS
   network, which should be protected against failures. When a network
   outage is detected, the PE SHOULD use a WAN Down event (as specified
   in [FC-BB-6]) to convey the PW status to the CE, to enable faster
   outage handling.

8. IANA Considerations

   IANA is requested to assign a new MPLS Pseudowire (PW) type as
   follows:

      PW type      Description           Reference
      --------     --------------        ----------
      0x001F       FC Port Mode          RFC XXXX

   The above value is suggested as the next available value and has been
   reserved for this purpose by IANA.

   RFC Editor: Please replace RFC XXXX above with the RFC number of this
   document and remove this note.

   IANA should reserve the following Pseudowire Interface Parameters
   Sub-TLV Types that were tentatively allocated for FC PW and restrict
   them to prevent future allocation, citing this RFC as the reference
   for that reservation and restriction. These Sub-TLV types were used
   for the FC PW Selective Retransmission protocol, which the working


Black and Dunbar        Expires November 2011                 [Page 18]

Internet-Draft             FC Encapsulation                    May 2011


   group has decided to eliminate. This action prevents future use of
   these values for other purposes, as there is at least one
   implementation of the Selective Retransmission protocol that has been
   deployed.

       Parameter  ID Length        Reference
      ---------  ---------         ----------
      0x12          4               RFC XXXX
      0x13          4               RFC XXXX
      0x14          4               RFC XXXX
      0x15          4               RFC XXXX

   RFC Editor: Please replace RFC XXXX above with the RFC number of this
   document and remove this note.



































Black and Dunbar        Expires November 2011                 [Page 19]

Internet-Draft             FC Encapsulation                    May 2011


9. Acknowledgments

   Previous versions of this document were authored by Moran Roth, Ronen
   Solomon and Munefumi Tsurusawa; their efforts and contributions are
   gratefully acknowledged. The authors would like to thank Stewart
   Bryant, Elwyn Davies, Steve Hanna, Dave Peterson, Yaakov Stein,
   Alexander Vainshtein, and the members of the IESG for helpful
   comments on this document.

   The protocol specified in this document is intended to be used in
   conjunction with the Fibre Channel pseudowire portion of the FC-BB-6
   specification developed by INCITS Technical Committee T11. The
   authors would like to thank the members of both the IETF and T11
   organizations who have supported and contributed to this work.

   This document was prepared using 2-Word-v2.0.template.dot.

10. Normative References

      [RFC3643]  Weber, R., et al, "Fibre Channel (FC) Frame
                 Encapsulation", RFC 3643, December 2003.

      [RFC3985]  Bryant, S., et al, "Pseudo Wire Emulation Edge-to-Edge
                 (PWE3) Architecture", RFC 3985, March 2005.

      [RFC4446]  Martini, L., "IANA Allocations for Pseudowire Edge to
                 Edge Emulation (PWE3)", RFC 4446, April 2006.

      [RFC4447]  Martini, L., et al, "Pseudowire Setup and Maintenance
                 using the Label Distribution Protocol (LDP)", RFC4447,
                 April 2006.

      [RFC4385]  Bryant, S., et al, "Pseudowire Emulation Edge-to-
                 Edge(PWE3) Control Word for use over an MPLS PSN",
                 RFC4385, February 2006.

      [RFC4623]  Malis, A., Townsley, M., "PWE3 Fragmentation and
                 Reassembly", RFC 4623, August 2006.

      [RFC4720]  Malis, A., et al, "Pseudowire Emulation Edge-to-Edge
                 (PWE3) Frame Check Sequence Retention", RFC 4720,
                 November 2006.







Black and Dunbar        Expires November 2011                 [Page 20]

Internet-Draft             FC Encapsulation                    May 2011


      [FC-BB-6]  "Fibre Channel Backbone-6" (FC-BB-6), T11 Project
                 2159-D, Rev 1.02, October 2010.

RFC Editor: FC-BB-6 is a work in progress. Please treat [FC-BB-6] as a
normative reference to a work in progress, and proceed as follows:
  1. Assign an RFC number to this draft and communicate that
      number to the authors of this draft, one of whom (David Black)
      is the T11 designated liaison to IETF.
  2. Place a reference hold on this draft until FC-BB-6 is published
      as an ANSI standard.
  3. When FC-BB-6 is published as an ANSI standard, the draft authors
      will provide an update to the FC-BB-6 reference that includes
      an ANSI standard number.  Update the FC-BB-6 reference using
      that information, remove the reference hold due to FC-BB-6, and
      remove this note.

     [RFC-2119]  Bradner, S., "Key words for use in RFCs to Indicate
                 requirement Levels", BCP 14, RFC 2119, March 1997.

     [FC-FS-2]   "Fibre Channel - Framing and Signaling-2 (FC-FS-2)",
                 ANSI INCITS 424:2007, August 2007.

11. Informative references

      [RFC3821]  M. Rajogopal, E. Rodriguez, "Fibre Channel over TCP/IP
                 (FCIP)", RFC 3821, July 2004.

      [RFC4717]  Martini, L., et al, "Encapsulation Methods for
                 Transport of Asynchronous Transfer Mode (ATM) over
                 MPLS Networks", RFC 4717, December 2006.

      [T11]      INCITS Technical Committee T11, http://www.t11.org,
                 visited January, 2011.

      [FC-SP]    "Fibre Channel - Security Protocols" (FC-SP), ANSI
                 INCITS 426:2007, February 2007.













Black and Dunbar        Expires November 2011                 [Page 21]

Internet-Draft             FC Encapsulation                    May 2011


Authors' Addresses

      David L. Black (ed.)
      EMC Corporation
      176 South Street
      Hopkinton, MA 01748
      Phone: +1 (508) 293-7953
      Email: david.black@emc.com

      Linda Dunbar (ed.)
      Huawei Technologies
      1700 Alma Drive, Suite 500
      Plano, TX 75075, USA
      Phone: +1 (972) 543-5849
      Email: ldunbar@huawei.com

      Moran Roth
      Infinera Corporation
      169 Java Drive
      Sunnyvale, CA 94089
      Phone: (408) 572-5200
      Email: MRoth@infinera.com

      Ronen Solomon
      Orckit-Corrigent Systems
      126, Yigal Alon st.
      Tel Aviv, ISRAEL
      Phone: +972-3-6945316
      Email: ronens@corrigent.com




















Black and Dunbar        Expires November 2011                 [Page 22]

Internet-Draft             FC Encapsulation                    May 2011


Intellectual Property Statement

   The IETF Trust takes no position regarding the validity or scope of
   any Intellectual Property Rights or other rights that might be
   claimed to pertain to the implementation or use of the technology
   described in any IETF Document or the extent to which any license
   under such rights might or might not be available; nor does it
   represent that it has made any independent effort to identify any
   such rights.

   Copies of Intellectual Property disclosures made to the IETF
   Secretariat and any assurances of licenses to be made available, or
   the result of an attempt made to obtain a general license or
   permission for the use of such proprietary rights by implementers or
   users of this specification can be obtained from the IETF on-line IPR
   repository at http://www.ietf.org/ipr

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   any standard or specification contained in an IETF Document. Please
   address the information to the IETF at ietf-ipr@ietf.org.

Disclaimer of Validity

   All IETF Documents and the information contained therein are provided
   on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
   IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
   WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
   WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE
   ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
   FOR A PARTICULAR PURPOSE.

Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.











Black and Dunbar        Expires November 2011                 [Page 23]

