Storage Provisioning with iSCSI for Virtualized Environments

Shyam Iyer

Situation

Linux Engineering

Widespread adoption of iSCSI Technology

Simpler to manage than FibreChannel System Admins familiar with TCP/IP-ethernet Benefits from the Ethernet economies of scale Benefits from Ethernet technology roadmap (10Gb, 40Gb, 100Gb)

Linux Engineering

Storage Administration-Generic Considerations

Volume Management Authentication ACLs RAID Snapshot/Backups Thin provisioning I/O Bandwidth, QOS Cluster-capable

Linux Engineering

Complication

Confidential

Linux Engineering

Storage Provisioning for VMs

VMs manage their storage
Pros
Limited Hypervisor configuration required Minimal Network Mgmt Direct VM and storage relationship

Cons
Multiple Management Window VMs can't make use of underlying technologies

Pic Source: DELL.COM/downloads/global/power/ps3q0720070401-Baker.pdf.

Linux Engineering

Problem with VMs managing the storage..

Multiple Management Windows Inefficient use of underlying technologies Ever Increasing Use Cases

Linux Engineering

Technology Advancements

HW

SR-IOV, VT-D, iSCSI offload, Data Centre Bridging, Multiqueues, VLAN segmentation

Software

Cgroups, io-controller, RPS, RFS, Macvtap, Bonding, DmMultipath

Linux Engineering

iSCSI and VM Migration

VM1 VM2 VM3 VM1
iqn1

VM2
iqn2

VM3
iqn3

Migrate

Hypervisor
iSCSI Initiator - iqn1

Hypervisor
NIC

Network

iqn.1

iqn.1

iqn.2

iqn.3

Network

Storage mapped to Initiator

Storage
iSCSI initiator in Hypervisor

Storage
iSCSI initiator in VM
Linux Engineering

Solution

Linux Engineering

Multi-IQN Concept
VM1
Lend IQN iqn.1 iSCSI_1 VF_1 iqn.2 iqn.2 iSCSI_2 VF_2 iqn.3

VM2 VM1

VM3
Lend IQN

iSCSI_X

iSCSI_3

iSCSI Offload Hardware

Hypervisor

iqn.1

iqn.2

Storage

Linux Engineering

Storage Provisioning for VMs

Hypervisor manages the storage
Pros
Reduced Management windows architecturally Host/Hypervisor taps the advantages of underlying hardware/OS technologies

Bridging The Gap

The Hypervisor needs to administer tools for all the underlying H/W and OS technologies

Linux Engineering

Implementation

Linux Engineering

Implementation Details
Open-iscsi
#iscsiadm -m iface -o new -I initiatorIqnA #iscsiadm -m iface -I initiatorIqnA -o update -n iface.initiator -v <IQNname> #iscsiadm -m discovery -t st -p <IP> -I initiatorIqnA #iscsiadm -m node -l

Libvirt Create MultiIQN iface abstraction Extended ISCSI Storage Pool XML Libvirt APIs to create multiiqn pools

Linux Engineering

Implementation Details
Virt-manager

Management Window TODO Provide QOS to pool

Display attributes of the storage pool

Linux Engineering

Steps
# iscsiadm -m discovery -t sendtargets -p 10.66.90.100 10.66.90.100:3260,1 iqn.2001-05.com.equallogic:0-8a0906-26f1f7d03-1c7267779594c286-test-1 # virsh pool-define pool-multiiqn.xml # virsh pool-start multiiqn Pool multiiqn started

Linux Engineering

Use Cases

Linux Engineering

Traditional VM iSCSI Usage

Each VM owns an initiatiator IQN and logs into different target IQNs VM1 - > <Init iqn1> VM2 - > <Init iqn2> <-----> <-----> <Target iqn1> <Target iqn2>

However certain target have more multiple LUNs behind the same target IQN so policies can't be made based on target IQNs ACLs and policies are based on initiator IQN Bottom Line Initiator IQN can uniquely determine the path to a LUN

Linux Engineering

Target IQN LUN Mapping Scenario 1

ISCSI Target1 IQN D

IQN A

ISCSI Initiator

ISCSI Target2 IQN E

IQN B

ISCSI Target3 IQN F

Linux Engineering

Target IQN LUN Mapping Scenario 2 (Cluster Shared Storage)

ISCSI Target1 IQN D

IQN A

ISCSI Initiator

ISCSI Target2 IQN E

IQN B

ISCSI Target3 IQN F

Linux Engineering

Target IQN LUN Mapping Scenario 3

IQN A

ISCSI Target
IQN C

ISCSI Initiator

IQN B

Multi-IQN Architecture Will Still Hold

Linux Engineering

Architecture Summary

Enables per VM QOS Makes efficient use of underlying HW, OS technologies Supports New Usecases Allows efficient LUN mapping and path policy decisions

Linux Engineering

References
http://www.mail-archive.com/libvir-list@redhat.com/msg19253.html

Architecture Credits - Gaurav Chawla, Jacob Cherian, Matt Domsch

Linux Engineering

QThank You &A

Linux Engineering

Backup

Linux Engineering

Storage Provisioning with iSCSI

Administrative tasks to provision storage with iSCSI

Discovery Mechanisms DHCP iSNS Autologin SCN events Sendtargets Is the iSCSI storage going to be provisioned for SAN-boot? Initiator and Target IQN identifications TCP/IP networking(This is the most simple)

Linux Engineering