We setup udev to make the iSCSI devices a consistent name across all the machines, and you don't end up with /dev/sda on one machine and /dev/sdb on another.

On each of the clients do the following:

dangermouse:~# echo "BUS==\"scsi\", SYSFS{vendor}==\"IET\", SYSFS{model}==\"VIRTUAL-DISK\", \
KERNEL==\"sd*\", NAME=\"%k\", PROGRAM=\"/lib/udev/getlun.sh $id\", SYMLINK+=\"iscsi_%c%n\"" \
>>/etc/udev/rules.d/10_persistant_scsi.rules
dangermouse:~# cat <<! >> /lib/udev/getlun.sh
#!/bin/bash
echo $1 | awk -F":" '{print $NF}'
!

What this will do is create a link in /dev called iscsi_01 for instance, where 0 is the LUN number, and 1 is the first partition, which might be a link to say /dev/sdb1.

Now we need to set up the target on the machine which won't run the virtual machine. This is penfold.

I used a block device for this on a SATA raid controller, but you could just use an ordinary file or a normal disk, or logical volume. The iSCSI Target documentation will give you more information on setting up something other than a block device.

penfold:~# apt-get install linux-headers-2.6.16-1-xen-686 openssl libssl-dev

Get the latest IET source code from your nearest sourceforge mirror, then uncompress and extract the files from the archive ready to build.

penfold:~# cd /usr/src
penfold:~# wget http://sf.net...../iscsitarget-0.4.13.tar.gz
penfold:~# tar -zxf iscsitarget-0.4.13.tar.gz
penfold:~# cd iscsitarget-0.4.13
penfold:~# make
penfold:~# make install

edit /etc/ietd.conf and make it look like the following.

IncomingUser joe secret
OutgoingUser jack secret2
    Target iqn.2006-05.com.performancemagic:penfold.disk1
	IncomingUser jim othersecret
	OutgoingUser james yetanothersecret
	Lun 0 Path=/dev/sdb,Type=fileio
	Alias Test
	HeaderDigest None
	DataDigest None
	MaxConnections 1
	InitialR2T Yes
	ImmediateData No
	MaxRecvDataSegmentLength 8192
	MaxXmitDataSegmentLength 8192
	MaxBurstLength 262144
	FirstBurstLength 65536
	DefaultTime2Wait 2
	DefaultTime2Retain 20
	MaxOutstandingR2T 8
	DataPDUInOrder Yes
	DataSequenceInOrder Yes
	ErrorRecoveryLevel 0

Now start the iSCSI target.

penfold:~# /etc/init.d/iscsi-target start
Starting iSCSI enterprise target service: succeeded.

(dmesg output abreviated)

penfold:~# dmesg
iSCSI Enterprise Target Software - version 0.4.13
iotype_init(90) register fileio
iotype_init(90) register nullio
sess_param(173) 1 1 1 8192 8192 262144 65536 2 20 1 1 1 0 1 1
sess_param(173) 1 1 1 8192 8192 262144 65536 2 20 1 1 1 0 1 1

Now lets build and setup the iSCSI initiator.

dangermouse:~# cd /usr/src
dangermouse:~# svn co svn://svn.berlios.org/open-iscsi/

The revision I got was 581

dangermouse:~# cd open-iscsi
dangermouse:~# make
dangermouse:~# make install
dangermouse:~# echo "iqn.2006-05.com.performancemagic:dangermouse.01" > /etc/initiatorname.iscsi

Change dangermouse in the above example to buggles for the other XEN host.

Edit /etc/iscsid.conf and make it look like the following.

node.active_cnx = 1
node.startup = manual
node.session.auth.username = jim
node.session.auth.password = othersecret
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 10
node.session.err_timeo.reset_timeout = 30
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Wait = 0
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.MaxConnections = 1
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.DataDigest = None
node.conn[0].iscsi.MaxRecvDataSegmentLength = 65536
discovery.sendtargets.auth.username = joe
discovery.sendtargets.auth.password = secret

Start the initiator.

dangermouse:~# /etc/init.d/open-iscsi start
Starting iSCSI initiator service: succeeded.

(dmesg output abreviated)

dangermouse:~# dmesg
SCSI subsystem initialized
iscsi: registered transport (tcp)

Now we have the target setup and the initiator started, we need to find the targets on the initiator.

dangermouse:~# iscsiadm -m discovery -t sendtargets -p 127.0.0.1:3260
[0f6cce] 127.0.0.1:3260,1 iqn.2006-05.com.performancemagic:penfold.disk1

now take the number in the [ ] brackets and login.

dangermouse:~# iscsiadm -m node -r 0f0cce -l

(dmesg output abreviated)

dangermouse:~# dmesg
scsi0 : iSCSI Initiator over TCP/IP, v1.0-574
sess_param(173) 1 0 1 8192 8192 262144 65536 2 0 1 1 1 0 1 1
Vendor: IET Model: VIRTUAL-DISK Rev: 0
Type: Direct-Access ANSI SCSI revision: 04
SCSI device sda: 10000 512-byte hdwr sectors (5 MB)
sda: Write Protect is off
sda: Mode Sense: 77 00 00 08
SCSI device sda: drive cache: write back
SCSI device sda: 10000 512-byte hdwr sectors (5 MB)
sda: Write Protect is off
sda: Mode Sense: 77 00 00 08
SCSI device sda: drive cache: write back
sda: sda1
sd 0:0:0:1: Attached scsi disk sda