SCSI RDMA Protocol

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
m (→Cookbook)
m
 
(31 intermediate revisions not shown)
Line 1: Line 1:
{{Infobox software
{{Infobox software
-
| name                  = LIO Target
+
| name                  = {{Target}}
-
| logo                  = [[Image:RisingTide_Logo_small.png|180px|Logo]]
+
| logo                  = [[Image:Corp_Logo.png|180px|Logo]]
| screenshot            = {{RTS screenshot|iSER}}
| screenshot            = {{RTS screenshot|iSER}}
| caption                = [[InfiniBand]] SRP fabric module
| caption                = [[InfiniBand]] SRP fabric module
| collapsible            =  
| collapsible            =  
-
| author                = {{Vu Pham}}<br/>{{Bart Van Assche}}<br>{{Nicholas Bellinger}}
+
| author                = {{Vu Pham}}<br/>{{Bart Van Assche}}<br/>{{Nicholas Bellinger}}
-
| developer              = Mellanox Technologies, Ltd.
+
| developer              = Mellanox Technologies, Ltd.<br/>{{RTS legal}}
| released              = {{RTS releases|SRP-Mellanox|initial_date}}
| released              = {{RTS releases|SRP-Mellanox|initial_date}}
| discontinued          = {{RTS releases|SRP-Mellanox|eol_date}}
| discontinued          = {{RTS releases|SRP-Mellanox|eol_date}}
Line 21: Line 21:
| language              =
| language              =
| genre                  = Fabric module
| genre                  = Fabric module
-
| license                = GNU General Public License
+
| license                = {{GPLv2}}
| website                = {{RTS website}}
| website                = {{RTS website}}
}}
}}
-
:''See [[Target]] for a complete overview over all fabric modules.''
+
:''See [[LIO]] for a complete overview over all fabric modules.''
-
In computing the '''SCSI RDMA Protocol''' ('''SRP''') is a protocol that allows one computer to access [[SCSI]] devices attached to another computer via remote direct memory access ([[RDMA]]).<ref name="SRP">[http://www.t10.org/drafts.htm#SCSI3_SRP ANSI T10 SRPr16a, www.t10.org].</ref><ref name="SRP-alternative">[http://web.archive.org/web/20080422013122/http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf ANSI T10 SRPr16a, web.archive.org]</ref>
+
The '''SCSI RDMA Protocol''' ('''SRP''') is a network protocol that allows one computer system to access [[SCSI]] devices attached to another computer system via [[RDMA]].<ref name="SRP">[http://www.t10.org/drafts.htm#SCSI3_SRP ANSI T10 SRPr16a, www.t10.org].</ref><ref name="SRP-alternative">[http://web.archive.org/web/20080422013122/http://www.t10.org/ftp/t10/drafts/srp/srp-r16a.pdf ANSI T10 SRPr16a, web.archive.org]</ref>
== Overview ==
== Overview ==
-
The SRP protocol is also known as the SCSI Remote Protocol. The use of RDMA makes higher throughput and lower latency possible than what is possible through e.g. the TCP/IP communication protocol. RDMA is only possible with network adapters that support RDMA in hardware. Examples of such network adapters are [[InfiniBand]] HCAs and 10 GbE network adapters with [[iWARP]] support. While the SRP protocol has been designed to use RDMA networks efficiently, it is also possible to implement the SRP protocol over networks that do not support RDMA.
+
SRP was designed to use RDMA networks efficiently. RDMA allows lower latencies and higher throughput than TCP/IP protocols, but requires network adapters with native RDMA support, e.g., [[HCA]]s for [[InfiniBand]], [[RNIC]]s on "lossless" ([[DCB]]) Ethernet, or [[TOE]] [[NIC]]s with [[iWARP]] for standard Ethernet.
-
As with the [[iSCSI Extensions for RDMA]] (iSER) communication protocol, there is the notion of a target (a system that stores the data) and an initiator (a client accessing the target) with the target performing the actual data movement. In other words, when a user writes to a target, the target actually executes a read from the initiator and when a user issues a read, the target executes a write to the initiator.  
+
SRP is based on the [[SCSI]] protocol, which is a point-to-point protocol with corresponding design limitations. In contrast, [[iSER]] is based on [[iSCSI]], and thus better accommodates modern network requirements, including complex topologies, multipathing, target discovery, etc. Hence, iSER is most likely the best choice for [[InfiniBand]] networks going forward.
-
While the SRP protocol is easier to implement than the iSER protocol, iSER offers more management functionality, e.g. the target discovery infrastructure enabled by the iSCSI protocol.
+
SRP never became an official standard: the latest draft of the SRP protocol, revision 16a, dates from July 3, 2002.<ref name="SRP"/>
-
The SRP protocol never made it into an official standard. The latest draft of the SRP protocol, revision 16a, dates from July 3, 2002.<ref name="SRP"/>
+
The InfiniBand/SRP fabric module ({{RTS releases|SRP-Mellanox|module_repo}}, {{RTS releases|SRP-Mellanox|module_info}}) for the {{Target}} was released with Linux kernel {{RTS releases|SRP-Mellanox|kernel_ver}} on {{RTS releases|SRP-Mellanox|initial_date}}.<ref>{{RTS releases|SRP-Mellanox|kernel_rel}}</ref>
-
 
+
-
The InfiniBand/SRP fabric module ({{RTS releases|SRP-Mellanox|module_repo}}, {{RTS releases|SRP-Mellanox|module_info}}) for the Unified Target was released with Linux kernel {{RTS releases|SRP-Mellanox|kernel_ver}} on {{RTS releases|SRP-Mellanox|initial_date}}.<ref>{{RTS releases|SRP-Mellanox|kernel_rel}}</ref>
+
== targetcli ==
== targetcli ==
-
''[[targetcli]]'' from {{RTS full}} is used to configure InfiniBand targets. ''targetcli'' aggregates service modules via a core library, and exports them through an API to the Unified [[Target]], to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
+
''[[targetcli]]'' from {{RTS full}} is used to configure InfiniBand targets. ''targetcli'' aggregates {{T}} service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
-
 
+
-
{{Ambox| type=info| head=RTS OS Admin Manual| text=The [[RTS OS]] [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf Admin Manual] provides comprehensive background and many examples on using ''targetcli'' and on programming the RTS library.}}
+
-
=== Cookbook ===
+
{{Ambox| type=info| head=[[LIO]] Admin Manual| text=The {{LIO Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the [[Datera]] library.}}
-
The following is a cheat sheet for reference purposes.
+
=== Cheat sheet ===
{| class="table-left"
{| class="table-left"
Line 61: Line 57:
| Create an SRP target
| Create an SRP target
|-
|-
-
| In ''/ib_srpt/<WWPN>'': ''luns/ create /backstores/iblock/my_disk''
+
| In /ib_srpt/<WWPN>:<br/>''luns/ create /backstores/iblock/my_disk''
| Export the LUN ''my_disk''
| Export the LUN ''my_disk''
|-
|-
-
| In ''/ib_srpt/<WWPN>'': ''acls/ create <Initiator WWPN>''
+
| In /ib_srpt/<WWPN>:<br/>''acls/ create <Initiator WWPN>''
| Allow access for the initiator at ''<WWPN>''
| Allow access for the initiator at ''<WWPN>''
|-
|-
Line 74: Line 70:
=== Startup ===
=== Startup ===
-
[[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying OS shell.
+
[[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying {{OS}} shell.
<pre>
<pre>
Line 80: Line 76:
Welcome to targetcli:
Welcome to targetcli:
-
  Copyright (c) 2012 by RisingTide Systems LLC.
+
  Copyright (c) 2014 by Datera, Inc.
  All rights reserved.
  All rights reserved.
-
Visit us at http://www.risingtidesystems.com.
+
Visit us at http://www.datera.io.
Using ib_srpt fabric module.
Using ib_srpt fabric module.
Line 170: Line 166:
</pre>
</pre>
-
Alternatively, any LVM logical volume can be used as a backstore, please refer to the [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf RTS OS Admin Manual] on how to create them properly.
+
Alternatively, any LVM logical volume can be used as a backstore, please refer to the {{LIO Admin Manual}} on how to create them properly.
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
Line 278: Line 274:
=== Persist the configuration ===
=== Persist the configuration ===
-
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the target configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.}}
+
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the {{T}} configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.}}
-
Use ''saveconfig'' from the root context to persist the target configuration across OS reboots:
+
Use ''saveconfig'' from the root context to persist the {{T}} configuration across {{OS}} reboots:
<pre>
<pre>
Line 305: Line 301:
=== Spec file ===
=== Spec file ===
-
RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
+
[[Datera]] spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
In particular, the InfiniBand spec file ''/var/target/fabric/ib_srpt.spec'' is included via RTSlib. WWN values are extracted via ''/sys/class/infiniband/*/ports/*/gids/0'' in ''wwn_from_files_filter'' below, and are presented in the [[targetcli]] [[WWN]] context to register individual InfiniBand port GUIDs.
In particular, the InfiniBand spec file ''/var/target/fabric/ib_srpt.spec'' is included via RTSlib. WWN values are extracted via ''/sys/class/infiniband/*/ports/*/gids/0'' in ''wwn_from_files_filter'' below, and are presented in the [[targetcli]] [[WWN]] context to register individual InfiniBand port GUIDs.
Line 327: Line 323:
# The configfs group is standard
# The configfs group is standard
configfs_group = srpt
configfs_group = srpt
 +
</pre>
 +
 +
== Scripting with RTSlib ==
 +
 +
=== Setup script ===
 +
 +
The following Python code illustrates how to setup a basic SRP target and export a mapped LUN:
 +
 +
<pre>
 +
#!/usr/bin/python
 +
# InfiniBand setup script example with RTSlib
 +
from rtslib import *
 +
 +
# Setup an IBLOCK backstore
 +
backstore = IBlockBackstore(3, mode='create')
 +
try:
 +
    so = IBlockStorageObject(backstore, "fioa", "/dev/fioa", gen_wwn=True)
 +
except:
 +
    backstore.delete()
 +
    raise
 +
 +
# Create an IB target endpoint using an ib_srpt WWPN
 +
fabric = FabricModule('ib_srpt')
 +
target = Target(fabric, '0x00000000000000000002c903000e8acd')
 +
tpg = TPG(target, 1)
 +
 +
# Export LUN 0 via the 'so' StorageObject class
 +
lun0 = tpg.lun(0, so, "my_lun")
 +
 +
# Setup the NodeACL for an IB initiator, and create MappedLUN 0
 +
node_acl = tpg.node_acl('0x00000000000000000002c903000e8acd')
 +
mapped_lun = node_acl.mapped_lun(0, 0, False)
 +
</pre>
 +
 +
Note that while SRP [[TPG]]s are masked by [[targetcli]], they are not masked by RTSlib.
 +
 +
=== Object tree ===
 +
 +
The resulting object tree looks as follows:
 +
 +
<pre>
 +
o- / ..................................................................... [...]
 +
  o- backstores .......................................................... [...]
 +
  | o- fileio ............................................... [0 Storage Object]
 +
  | o- iblock ............................................... [1 Storage Object]
 +
  | | o- my_disk .......................................... [/dev/sdb activated]
 +
  | o- pscsi ................................................ [0 Storage Object]
 +
  | o- rd_dr ................................................ [0 Storage Object]
 +
  | o- rd_mcp ............................................... [0 Storage Object]
 +
  o- ib_srpt ........................................................ [1 Target]
 +
    o- 0x00000000000000000002c903000e8acd ............................ [enabled]
 +
      o- acls .......................................................... [1 ACL]
 +
      | o- 0x00000000000000000002c903000e8be9 ................... [1 Mapped LUN]
 +
      |  o- mapped_lun0 ........................................... [lun0 (rw)]
 +
      o- luns .......................................................... [1 LUN]
 +
        o- lun0 .................................... [iblock/my_disk (/dev/sdb)]
 +
  o- iscsi .......................................................... [0 Target]
 +
  o- loopback ....................................................... [0 Target]
 +
  o- qla2xxx ........................................................ [0 Target]
</pre>
</pre>
Line 347: Line 402:
== See also ==
== See also ==
-
* [[RTS OS]], [[targetcli]]
+
* [[{{OS}}]]
-
* [[Target]]
+
* {{Target}}, [[targetcli]]
-
* [[Fibre Channel]], [[Fibre Channel over Ethernet|FCoE]], [[iSCSI]], [[iSER]], [[tcm_loop]], [[vHost]]
+
* [[FCoE]], [[Fibre Channel]], [[iSCSI]], [[iSER]], [[tcm_loop]], [[vHost]]
== Notes ==
== Notes ==
Line 362: Line 417:
== External links ==
== External links ==
-
* [[RTS OS]] [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf Admin Manual]
+
* {{LIO Admin Manual}}
-
* RTSlib Reference Guide [[http://www.risingtidesystems.com/doc/rtslib-gpl/html/ HTML]][[http://www.risingtidesystems.com/doc/rtslib-gpl/pdf/rtslib-API-reference.pdf PDF]]
+
* RTSlib Reference Guide {{Lib Ref Guide HTML}}{{Lib Ref Guide PDF}}
* [http://en.wikipedia.org/wiki/SCSI_RDMA_Protocol SRP] Wikipedia entry
* [http://en.wikipedia.org/wiki/SCSI_RDMA_Protocol SRP] Wikipedia entry
* {{cite web| url=http://www.youtube.com/watch?v=4UVRuq-lgKo| title=SRP Update and Directions| date=2012-05-27| publisher=openfabrics.org}}
* {{cite web| url=http://www.youtube.com/watch?v=4UVRuq-lgKo| title=SRP Update and Directions| date=2012-05-27| publisher=openfabrics.org}}

Latest revision as of 02:42, 7 August 2015

LinuxIO
Logo
LIO 150513.png
InfiniBand SRP fabric module
Original author(s) Vu Pham
Bart Van Assche
Nicholas Bellinger
Developer(s) Mellanox Technologies, Ltd.
Datera, Inc.
Initial release March 18, 2012 (2012-03-18)
Stable release 4.1.0 / June 20, 2012;
8 years ago (2012-06-20)
Preview release 4.2.0-rc5 / June 28, 2012;
8 years ago (2012-06-28)
Development status Production
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License, version 2 (GPLv2)
Website datera.io
See LIO for a complete overview over all fabric modules.

The SCSI RDMA Protocol (SRP) is a network protocol that allows one computer system to access SCSI devices attached to another computer system via RDMA.[1][2]

Contents

Overview

SRP was designed to use RDMA networks efficiently. RDMA allows lower latencies and higher throughput than TCP/IP protocols, but requires network adapters with native RDMA support, e.g., HCAs for InfiniBand, RNICs on "lossless" (DCB) Ethernet, or TOE NICs with iWARP for standard Ethernet.

SRP is based on the SCSI protocol, which is a point-to-point protocol with corresponding design limitations. In contrast, iSER is based on iSCSI, and thus better accommodates modern network requirements, including complex topologies, multipathing, target discovery, etc. Hence, iSER is most likely the best choice for InfiniBand networks going forward.

SRP never became an official standard: the latest draft of the SRP protocol, revision 16a, dates from July 3, 2002.[1]

The InfiniBand/SRP fabric module (srpt.ko, Linux kernel driver database) for the LinuxIO was released with Linux kernel 3.3 on March 18, 2012 (2012-03-18).[3]

targetcli

targetcli from Datera, Inc. is used to configure InfiniBand targets. targetcli aggregates LIO service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).

LIO Admin Manual
The LIO Admin Manual provides comprehensive background and many examples on using targetcli and on programming the Datera library.

Cheat sheet

Command Comment
/backstores/iblock create my_disk /dev/sdb Create the LUN my_disk on the block device /dev/sdb
/ib_srpt create <WWPN> Create an SRP target
In /ib_srpt/<WWPN>:
luns/ create /backstores/iblock/my_disk 		Export the LUN my_disk
In /ib_srpt/<WWPN>:
acls/ create <Initiator WWPN> 		Allow access for the initiator at <WWPN>
/saveconfig Commit the configuration

Startup

targetcli is invoked by running targetcli as root from the command prompt of the underlying LIO shell. 

# targetcli
Welcome to targetcli:

 Copyright (c) 2014 by Datera, Inc.
 All rights reserved.

Visit us at http://www.datera.io.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using loopback fabric module.

/> ib_srpt/ info
Fabric module name: ib_srpt
ConfigFS path: /sys/kernel/config/target/srpt
Allowed WWN list type: free
Fabric module specfile: /var/target/fabric/ib_srpt.spec
Fabric module features: acls
Corresponding kernel module: ib_srpt
/>

Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding ConfigFS mount points (at /sys/kernel/config/target/<fabric>), as specified by the associated spec files (located in /var/target/fabric/fabric.spec).

Display the object tree

Use ls to list the object hierarchy, which is initially empty: 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>
Global parameter auto_cd_after_create
After the creation of a new object, automatically enter its object context.

Per default, auto_cd_after_create is set to true, which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.

Optionally, set auto_cd_after_create=false to prevent targetcli from automatically entering new object contexts after their creation: 

/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Create a backstore

Create a backstore using the IBLOCK or FILEIO type devices.

For instance, enter the top-level backstore context and create an IBLOCK backstore from a /dev/sdb block device: 

/> cd backstores/
/backstores> iblock/ create name=my_disk dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object my_disk using /dev/sdb.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

The resulting object hierarchy looks as follows (displayed from the root object): 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Alternatively, any LVM logical volume can be used as a backstore, please refer to the LIO Admin Manual on how to create them properly.

For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>): 

/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
Generating a wwn serial.
Created iblock storage object my_disk using /dev/vg0/lv1.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

Again, targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

More backstore examples
More examples on creating backstores can be found in targetcli.

Instantiate a target

The InfiniBand ports that are available on the storage array are presented in the WWN context with the following WWPNs, for instance:

Instantiate an InfiniBand target, in this example for SRP over Mellanox Connect-X HCAs, on the existing IBLOCK backstore device my_disk (as set up in targetcli): 

/backstores/iblock/my_disk> /ip_srpt create 0x00000000000000000002c903000e8acd
Created target 0x00000000000000000002c903000e8acd.
Entering new node /ib_srpt/0x00000000000000000002c903000e8acd.
/ib_srpt/0x00...2c903000e8acd>

targetcli automatically changes the working context to the resulting tagged Endpoint.

Export LUNs

Declare a LUN for the backstore device, to form a valid SAN storage object: 

/ib_srpt/0x00...2c903000e8acd> luns/ create /backstores/iblock/my_disk
Selected LUN 0.
Successfully created LUN 0.
Entering new node /ib_srpt/0x00000000000000000002c903000e8acd/luns/lun0.
/ib_srpt/0x00...acd/luns/lun0>

targetcli per default automatically assigns the default ID '0' to the LUN, and then changes the working context to the SAN storage object. The target is now created, and exports /dev/sdb as LUN 0.

Return to the underlying Endpoint as the working context, as no attributes need to be set or modified for standard LUNs: 

/ib_srpt/0x00...act/luns/lun0> cd <
Taking you back to /ib_srpt/0x00000000000000000002c903000e8acd.
/ib_srpt/0x00...2c903000e8acd>

Define access rights

Configure the access rights to allow logins from initiators. This requires setting up individual access rights for each initiator, based on its WWPN.

Determine the WWPN for the respective InfiniBand initiator. For instance, for Linux initiator systems, use: 

# cat /sys/class/infiniband/*/ports/*/gids/0 | sed -e s/fe80/0x0000/ -e 's/\://g'

For a simple setup, allow access to the initiator with the WWPN as determined above: 

/ib_srpt/0x00...2c903000e8acd> acls/ create 0x00000000000000000002c903000e8be9
Successfully created Node ACL for 0x00000000000000000002c903000e8be9.
Created mapped LUN 0.
Entering new node /ib_srpt/0x00000000000000000002c903000e8acd/acls/
0x00000000000000000002c903000e8be9.
/ib_srpt/0x00...2c903000e8be9> cd /

targetcli per default automatically adds the appropriate mapped LUNs.

Display the object tree

The resulting InfiniBand SAN object hierarchy looks as follows (displayed from the root object): 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [1 Target]
  | o- 0x00000000000000000002c903000e8acd ............................ [enabled]
  |   o- acls .......................................................... [1 ACL]
  |   | o- 0x00000000000000000002c903000e8be9 ................... [1 Mapped LUN]
  |   |   o- mapped_lun0 ........................................... [lun0 (rw)]
  |   o- luns .......................................................... [1 LUN]
  |     o- lun0 .................................... [iblock/my_disk (/dev/sdb)]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Persist the configuration

Don't forget to use saveconfig!
Without saveconfig, the LIO configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.

Use saveconfig from the root context to persist the LIO configuration across LIO reboots: 

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/>

Spec file

Datera spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.

In particular, the InfiniBand spec file /var/target/fabric/ib_srpt.spec is included via RTSlib. WWN values are extracted via /sys/class/infiniband/*/ports/*/gids/0 in wwn_from_files_filter below, and are presented in the targetcli WWN context to register individual InfiniBand port GUIDs. 

# WARNING: This is a draft specfile supplied for demo purposes only.

# The ib_srpt fabric module uses the default feature set.
features = acls

# The module uses hardware addresses from there
wwn_from_files = /sys/class/infiniband/*/ports/*/gids/0

# Transform 'fe80:0000:0000:0000:0002:1903:000e:8acd' WWN notation to
# '0x00000000000000000002c903000e8acd'
wwn_from_files_filter = "sed -e s/fe80/0x0000/ -e 's/\://g'"

# Non-standard module naming scheme
kernel_module = ib_srpt

# The configfs group is standard
configfs_group = srpt

Scripting with RTSlib

Setup script

The following Python code illustrates how to setup a basic SRP target and export a mapped LUN: 

#!/usr/bin/python
# InfiniBand setup script example with RTSlib
from rtslib import *

# Setup an IBLOCK backstore
backstore = IBlockBackstore(3, mode='create')
try:
    so = IBlockStorageObject(backstore, "fioa", "/dev/fioa", gen_wwn=True)
except:
    backstore.delete()
    raise

# Create an IB target endpoint using an ib_srpt WWPN
fabric = FabricModule('ib_srpt')
target = Target(fabric, '0x00000000000000000002c903000e8acd')
tpg = TPG(target, 1)

# Export LUN 0 via the 'so' StorageObject class
lun0 = tpg.lun(0, so, "my_lun")

# Setup the NodeACL for an IB initiator, and create MappedLUN 0
node_acl = tpg.node_acl('0x00000000000000000002c903000e8acd')
mapped_lun = node_acl.mapped_lun(0, 0, False)

Note that while SRP TPGs are masked by targetcli, they are not masked by RTSlib.

Object tree

The resulting object tree looks as follows: 

o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [1 Target]
    o- 0x00000000000000000002c903000e8acd ............................ [enabled]
      o- acls .......................................................... [1 ACL]
      | o- 0x00000000000000000002c903000e8be9 ................... [1 Mapped LUN]
      |   o- mapped_lun0 ........................................... [lun0 (rw)]
      o- luns .......................................................... [1 LUN]
        o- lun0 .................................... [iblock/my_disk (/dev/sdb)]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]

Specifications

SRP was not approved as an official standard. The following specifications are available as available as T10 Working Drafts:

RFCs

See also

Notes

  1. ↑ a b ANSI T10 SRPr16a, www.t10.org.
  2. ↑ ANSI T10 SRPr16a, web.archive.org
  3. ↑ Linus Torvalds (2012-03-18). "Linux 3.3". lkml.org. 

Wikipedia entries

External links

Timeline of the LinuxIO
Release Details 2011 2012 2013 2014 2015
123456789101112 123456789101112 123456789101112 123456789101112 123456789101112
4.x Version 4.0 4.1
Feature LIO Core Loop back FCoE iSCSI Perf SRP
 		CM WQ FC
USB
1394 		vHost Perf Misc 16 GFC iSER Misc VAAI Misc DIF Core
NPIV 		DIF iSER DIF FC vhost TCMU Xen Misc Misc virtio 1.0 Misc NVMe OF
Linux 2.6.38 2.6.39 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22
Retrieved from "http://www.linux-iscsi.org/wiki/SCSI_RDMA_Protocol"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense