Building your own FC SAN with Linux
by Michele Baldessari
With the Linux kernel 3.5 release the Fibre Channel SCSI Target support has been merged. This means that given a couple of QLogic cards and a Fibre Channel SAN switch you can actually build your own cheap FC SAN environment. I thought of giving it a shot and hit eBay to get:
- Brocade Silkworm 3250 with 4 SFPs
- 2 x QLogic ISP2432 HBAs
- Fibre Optic cables
The small challenge was to actually reset the configuration of the Brocade switch as I kept receiving no output from the serial console. A quick google showed that the DB-9 serial cable needs to have pins 2,3 and 5 connected straight between the connectors. After a little soldering I finally got proper serial output and managed to reset it. Following this excellent guide I reset the password in no time.
Here is the info regarding my setup:
- marquez – Fedora 19 (as of 20th of May). The target system containing the LUN. Its qla2xxx has WWN: 0x210000e08b943494
- h2 - RHEL 6.4. The “client” system. WWN: 0x210000e08b947193
- FC Switch – Fabric OS 4.2.2. WWN: 10:00:00:05:1e:35:d9:0a
Let’s start to configure the FC Switch. We’ll create a simple ZONE adding the WWN’s of the involved parties.
> aliCreate HBA_MARQUEZ, 21:00:00:e0:8b:94:71:93 > aliCreate HBA_H1, 21:00:00:e0:8b:94:34:94 > zoneCreate "SAN", "HBA_MARQUEZ; HBA_H1"
> cfgDisable Updating flash ... > cfgshow Defined configuration: zone: SAN HBA_MARQUEZ; HBA_H1 alias: HBA_H1 21:00:00:e0:8b:94:34:94 alias: HBA_MARQUEZ 21:00:00:e0:8b:94:71:93
Effective configuration: no configuration in effect
> cfgcreate SAN_CFG, "SAN" > cfgenable SAN_CFG zone config "SAN_CFG" is in effect Updating flash ...
With the above we created a single zone called ‘SAN’. The two HBAs are members of this zone.
We need to disable the initiator mode from the qla2xxx driver:
# echo 'options qla2xxx qlini_mode="disabled"' > /usr/lib/modprobe.d/qla2xxx.conf # rmmod qla2xxx # modprobe qla2xxx
(Note that it’s better to put it under /usr/lib… as most likely qla2xxx will be loaded from your initramfs. Don’t forget to rebuild the initram with ‘dracut -f’. Then we install targetcli and start configuring the LUN and the ACLs on the LUN:
# yum install -y targetcli # targetcli /> qla2xxx/ info Fabric module name: qla2xxx ConfigFS path: /sys/kernel/config/target/qla2xxx Allowed WWN types: naa Allowed WWNs list: naa.210000e08b943494 Fabric module features: acls Corresponding kernel module: tcm_qla2xxx /> /qla2xxx create naa.210000e08b943494 Created target naa.210000e08b943494. /> /backstores/block create lun1 /dev/sdb Created block storage object lun1 using /dev/sdb. /> /qla2xxx/naa.210000e08b943494/luns create /backstores/block/lun1 Created LUN 0. /> /qla2xxx/naa.210000e08b943494/acls create 210000e08b947193
Note that if there are any errors in the FC Switch configuration we will see errors like ‘SNS scan failed — assuming zero-entry result.’ in your messages.
We can just issue a LIP or rescan the scsi device and we will see the LUN:
# echo 1 > /sys/class/fc_host/host9/issue_lip # tail -f /var/log/messages May 20 22:49:24 h2 kernel: qla2xxx [0000:40:00.0]-505f:9: Link is operational (2 Gbps). May 20 22:49:27 h2 kernel: scsi 9:0:0:0: Direct-Access LIO-ORG lun1 4.0 PQ: 0 ANSI: 5 May 20 22:49:27 h2 kernel: sd 9:0:0:0: Attached scsi generic sg2 type 0 May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] 293046768 512-byte logical blocks: (150 GB/139 GiB) May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write Protect is off May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA May 20 22:49:27 h2 kernel: sdb: sdb1 May 20 22:49:27 h2 kernel: sd 9:0:0:0: [sdb] Attached SCSI disk
At this point we can use /dev/sdb as a normal block device.