Building your own FC SAN with Linux

by Michele Baldessari

silkworm

With the Linux kernel 3.5 release the Fibre Channel SCSI Target support has been merged. This means that given a couple of QLogic cards and a Fibre Channel SAN switch you can actually build your own cheap FC SAN environment. I thought of giving it a shot and hit eBay to get:

  • Brocade Silkworm 3250 with 4 SFPs
  • 2 x QLogic ISP2432 HBAs
  • Fibre Optic cables

The small challenge was to actually reset the configuration of the Brocade switch as I kept receiving no output from the serial console. A quick google showed that the DB-9 serial cable needs to have pins 2,3 and 5 connected straight between the connectors. After a little soldering I finally got proper serial output and managed to reset it. Following this excellent guide I reset the password in no time.

Here is the info regarding my setup:

  • marquez – Fedora 19 (as of 20th of May). The target system containing the LUN. Its qla2xxx has WWN: 0x210000e08b943494
  • h2 - RHEL 6.4. The “client” system. WWN: 0x210000e08b947193
  • FC Switch – Fabric OS 4.2.2. WWN: 10:00:00:05:1e:35:d9:0a

FC Switch

Let’s start to configure the FC Switch. We’ll create a simple ZONE adding the WWN’s of the involved parties.

> cfgClear
> aliCreate HBA_MARQUEZ, 21:00:00:e0:8b:94:71:93
> aliCreate HBA_H1, 21:00:00:e0:8b:94:34:94
> zoneCreate "SAN", "HBA_MARQUEZ; HBA_H1"
> cfgDisable
 Updating flash ...
> cfgshow
 Defined configuration:
 zone: SAN HBA_MARQUEZ; HBA_H1
 alias: HBA_H1 21:00:00:e0:8b:94:34:94
 alias: HBA_MARQUEZ 21:00:00:e0:8b:94:71:93
Effective configuration:
 no configuration in effect
> cfgcreate SAN_CFG, "SAN" 
> cfgenable SAN_CFG
 zone config "SAN_CFG" is in effect
Updating flash ...

With the above we created a single zone called ‘SAN’. The two HBAs are members of this zone.

Target

We need to disable the initiator mode from the qla2xxx driver:

# echo 'options qla2xxx qlini_mode="disabled"' > /usr/lib/modprobe.d/qla2xxx.conf
# rmmod qla2xxx
# modprobe qla2xxx

(Note that it’s better to put it under /usr/lib… as most likely qla2xxx will be loaded from your initramfs. Don’t forget to rebuild the initram with ‘dracut -f’. Then we install targetcli and start configuring the LUN and the ACLs on the LUN:

# yum install -y targetcli
# targetcli
/> qla2xxx/ info
Fabric module name: qla2xxx
ConfigFS path: /sys/kernel/config/target/qla2xxx
Allowed WWN types: naa
Allowed WWNs list: naa.210000e08b943494
Fabric module features: acls
Corresponding kernel module: tcm_qla2xxx

/> /qla2xxx create naa.210000e08b943494
Created target naa.210000e08b943494.

/> /backstores/block create lun1 /dev/sdb
Created block storage object lun1 using /dev/sdb.

/> /qla2xxx/naa.210000e08b943494/luns create /backstores/block/lun1
Created LUN 0.

/> /qla2xxx/naa.210000e08b943494/acls create 210000e08b947193

Note that if there are any errors in the FC Switch configuration we will see errors like ‘SNS scan failed — assuming zero-entry result.’ in your messages.

Initiator

We can just issue a LIP or rescan the scsi device and we will see the LUN:

# echo 1 > /sys/class/fc_host/host9/issue_lip

# tail -f /var/log/messages
May 20 22:49:24 h2 kernel: qla2xxx [0000:40:00.0]-505f:9: Link is operational (2 Gbps).
May 20 22:49:27 h2 kernel: scsi 9:0:0:0: Direct-Access     LIO-ORG  lun1             4.0  PQ: 0 ANSI: 5
May 20 22:49:27 h2 kernel: sd 9:0:0:0: Attached scsi generic sg2 type 0
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] 293046768 512-byte logical blocks: (150 GB/139 GiB)
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write Protect is off
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
May 20 22:49:27 h2 kernel: sdb: sdb1
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Attached SCSI disk

At this point we can use /dev/sdb as a normal block device.