Skip to main content

Michele's Blog

Building your own FC SAN with Linux

Michele Baldessari

silkworm

With the Linux kernel 3.5 release the Fibre Channel SCSI Target support has been merged. This means that given a couple of QLogic cards and a Fibre Channel SAN switch you can actually build your own cheap FC SAN environment. I thought of giving it a shot and hit eBay to get:

  • Brocade Silkworm 3250 with 4 SFPs
  • 2 x QLogic ISP2432 HBAs
  • Fibre Optic cables

The small challenge was to actually reset the configuration of the Brocade switch as I kept receiving no output from the serial console. A quick google showed that the DB-9 serial cable needs to have pins 2,3 and 5 connected straight between the connectors. After a little soldering I finally got proper serial output and managed to reset it. Following this excellent guide I reset the password in no time.

Here is the info regarding my setup:

  • marquez - Fedora 19 (as of 20th of May). The target system containing the LUN. Its qla2xxx has WWN: 0x210000e08b943494
  • h2 - RHEL 6.4. The "client" system. WWN: 0x210000e08b947193
  • FC Switch - Fabric OS 4.2.2. WWN: __ 10:00:00:05:1e:35:d9:0a

FC Switch

Let's start to configure the FC Switch. We'll create a simple ZONE adding the WWN's of the involved parties.

> cfgClear
> aliCreate HBA_MARQUEZ, 21:00:00:e0:8b:94:71:93
> aliCreate HBA_H1, 21:00:00:e0:8b:94:34:94
> zoneCreate "SAN", "HBA_MARQUEZ; HBA_H1"
> cfgDisable
 Updating flash ...
> cfgshow
 Defined configuration:
 zone: SAN HBA_MARQUEZ; HBA_H1
 alias: HBA_H1 21:00:00:e0:8b:94:34:94
 alias: HBA_MARQUEZ 21:00:00:e0:8b:94:71:93

Effective configuration:
 no configuration in effect

> cfgcreate SAN_CFG, "SAN" 
> cfgenable SAN_CFG
 zone config "SAN_CFG" is in effect
Updating flash ...

With the above we created a single zone called 'SAN'. The two HBAs are members of this zone.

Target

We need to disable the initiator mode from the qla2xxx driver:

# echo 'options qla2xxx qlini_mode="disabled"' > /usr/lib/modprobe.d/qla2xxx.conf
# rmmod qla2xxx
# modprobe qla2xxx

(Note that it's better to put it under /usr/lib... as most likely qla2xxx will be loaded from your initramfs. Don't forget to rebuild the initram with 'dracut -f'. Then we install targetcli and start configuring the LUN and the ACLs on the LUN:

# yum install -y targetcli
# targetcli
/> qla2xxx/ info
Fabric module name: qla2xxx
ConfigFS path: /sys/kernel/config/target/qla2xxx
Allowed WWN types: naa
Allowed WWNs list: naa.210000e08b943494
Fabric module features: acls
Corresponding kernel module: tcm_qla2xxx

/> /qla2xxx create naa.210000e08b943494
Created target naa.210000e08b943494.
/> /backstores/block create lun1 /dev/sdb
Created block storage object lun1 using /dev/sdb.
/> /qla2xxx/naa.210000e08b943494/luns create /backstores/block/lun1
Created LUN 0.
/> /qla2xxx/naa.210000e08b943494/acls create 210000e08b947193

Note that if there are any errors in the FC Switch configuration we will see errors like 'SNS scan failed -- assuming zero-entry result.' in your messages.

Initiator

We can just issue a LIP or rescan the scsi device and we will see the LUN:

# echo 1 > /sys/class/fc_host/host9/issue_lip

# tail -f /var/log/messages
May 20 22:49:24 h2 kernel: qla2xxx [0000:40:00.0]-505f:9: Link is operational (2 Gbps).
May 20 22:49:27 h2 kernel: scsi 9:0:0:0: Direct-Access     LIO-ORG  lun1             4.0  PQ: 0 ANSI: 5
May 20 22:49:27 h2 kernel: sd 9:0:0:0: Attached scsi generic sg2 type 0
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] 293046768 512-byte logical blocks: (150 GB/139 GiB)
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write Protect is off
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
May 20 22:49:27 h2 kernel: sdb: sdb1
May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Attached SCSI disk

At this point we can use /dev/sdb as a normal block device.