InfiniBand
From Linux-iSCSI
 | |
|---|---|
|
Mellanox Technologies, Ltd. Mellanox Infiniband SRP fabric module | |
| Original author(s) |
Vu Pham Bart Van Assche Nicholas Bellinger |
| Developer(s) | Mellanox Technologies, Ltd. |
| Initial release | February 15, 2011 |
| Preview release | 4.1.0-rc1 / February 21, 2011 |
| Development status | Production |
| Written in | C |
| Operating system | Linux |
| Type | Fabric module |
| License | GNU General Public License |
| Website | mellanox.com |
- See Target for a complete overview over all fabric modules.
The InfiniBand architecture is an industry standard, channel-based, switched-fabric, interconnect architecture for servers. It is used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices.
InfiniBand forms a superset of the Virtual Interface Architecture.
The InfiniBand/SRP fabric module went upstream into the Linux 3.3 kernel on 1/18/2012.[1]
Contents |
Protocols
A brief overview over relevant or related protocols:
- Converged Enhanced Ethernet (CEE): A set of standards that allow enhanced communication over an Ethernet network. CEE is typically called Data Center Bridging (DCB).
- Data Center Bridging (DCB): A set of standards that allow enhanced communication over an Ethernet network. DCB is sometimes called Converged Enhanced Ethernet (CEE).
- Fibre Channel over Infiniband (FCoIB): The SCSI protocol is embedded into the Fibre Channel interface, which is in turn run as a virtual interface inside of InfiniBand. This does not leverage RDMA.
- InfiniBand over Ethernet (IBoE): A technology that makes high-bandwidth low-latency communication possible over DCB Ethernet networks. Typically called RDMA over Converged Enhanced Ethernet (RoCEE).
- Internet Protocol over InfiniBand (IPoIB): This transport is accomplished by encapsulating IP packets of InfiniBand packets.
- Internet Wide Area RDMA Protocol (iWARP): A technology that makes high-bandwidth low-latency communication possible over TCP/IP networks. iWARP is an Internet Engineering Task Force (IETF) update of the RDMA Consortium's RDMA over TCP standard. iWARP is a superset of the VIA. It is often compared to InfiniBand, also based on VIA, and distinguishes itself by being an implementation on top of IP networks (typically Ethernet), rather than specialized hardware.
- iSCSI Extensions for RDMA (iSER): A protocol model defined by the IETF that maps the iSCSI protocol directly over RDMA and is part of the "Data Mover" architecture. A user-space iSER fabric module can be used through tcm_loop.
- Mellanox fabric module, status: planned.
- RDMA over Converged Enhanced Ethernet (RoCEE): A technology that makes high-bandwidth low-latency communication possible over DCB Ethernet networks. RoCEE allows the deployment of RDMA semantics on lossless Ethernet fabrics by running the IB transport protocol using Ethernet frames. RoCEE packets consist of standard Ethernet frames with an IEEE assigned Ethertype, a GRH, unmodifiedIB transport headers and payload.[2] RoCEE is sometimes also called InfiniBand over Ethernet (IBoE).
- Remote Direct Memory Access (RDMA): Peer-to-peer, memory-to-memory access, very low latency/low overhead, high operation rate, high bandwidth.
- Mellanox fabric module, status: planned.
- SCSI RDMA Protocol (SRP): Defined by ANSI T10, latest draft is rev. 16a (6/3/02) - never ratified as a formal standard.
- Mellanox fabric module, status: Beta
- Sockets Direct Protocol (SDP): A transaction protocol enabling emulation of sockets semantics over RDMA. This allows applications to gain the performance benefits of RDMA without changing application code that relies on sockets. Version 1.0 of the SDP specification was publicly released by the RDMA Consortium in October 2003.
- Virtual Interface Architecture (VIA): Permits zero-copy transmission over TCP and SCTP.
Glossary
- Host Channel Adapter (HCA): provides the mechanism to connect InfiniBand devices to processors and memory.
- Target Channel Adapter (TCA): endpoint of an InfiniBand fabric, typically provides additional I/O functionality.
- Vitual Lane (VL): support multiple logical channels on the same physical link, the actual logical lane used on a given point-to-point link.
RFCs
- RFC 4297: Remote Direct Memory Access (RDMA) over IP Problem Statement
- RFC 4390: Dynamic Host Configuration Protocol (DHCP) over InfiniBand
- RFC 4391: Transmission of IP over InfiniBand (IPoIB)
- RFC 4392: IP over InfiniBand (IPoIB) Architecture
- RFC 4755: IP over InfiniBand: Connected Mode
- RFC 5040: A Remote Direct Memory Access Protocol Specification
- RFC 5045: Applicability of Remote Direct Memory Access Protocol (RDMA) and Direct Data Placement Protocol (DDP)
- RFC 5046: Internet Small Computer System Interface (iSCSI) Extensions for Remote Direct Memory Access (RDMA)
- RFC 5047: DA: Datamover Architecture for the Internet Small Computer System Interface (iSCSI)
Timeline
| History of LIO Target | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Release | Details | 2008 | 2009 | 2010 | 2011 | 2012 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |||||
| 2.x | Version | 2.9 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Feature | GPL release | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 3.x | Version | 3.0 | 3.1 | 3.2 | 3.4 | 3.5 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Feature | ConfigFS control plane, tcm_loop | PR ALUA | Cleanups | Open-FCoE | Backports | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 4.x | Version | 4.0 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Feature | Target Core | tcm_loop | FCoE | iSCSI | Perf. | IB/SRP | FC vSCSI | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Linux | 2.6.38 | 2.6.39 | 3.0 | 3.1 | 3.2 | 3.3 | 3.4 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
See also
- Mellanox InfiniBand fabric modules: SRP, iSER
- Other fabric modules: iSCSI, Fibre Channel, FCoE, IBM vSCSI, tcm_loop
- Target
Notes
- ↑ Linus Torvalds (1/18/2012). "InfiniBand/SRP merge". lkml.org.
- ↑ Tom Talpey, et al. (8/26/2009). "Remote Direct Memory Access over the Converged Enhanced Ethernet Fabric: Evaluating the Options". IEEE Hot Interconnects 17.
Wikipedia entries
- InfiniBand (IB)
- Internet Wide Area RDMA Protocol (iWARP)
- iSCSI Extensions for RDMA (iSER)
- Remote direct memory access (RDMA)
- SCSI RDMA Protocol (SRP)
- Virtual Interface Architecture (VIA)
External links
- Ann Silverthorn (11/1/2006). "InfiniBand edging into storage market". dentistryiq.com.
- Ed Koehler (2/20/2010). "Infiniband and it’s unique potential for Storage and Business Continuity". edkoehler.wordpress.com.
- Odysseas Pentakalos (02/04/2002). "An Introduction to the InfiniBand Architecture". oreillynet.com.
- The InfiniBand Trade Association homepage
- OpenFabrics Alliance
