iSCSI Protocol Analysis and Tools

1
iSCSI Protocol Analysis and Tools

• Neeraj Reddy Kuppam neeraj_at_netapp.com

2
What are our goals today?

• Learn iSCSI Protocol
• Setup and play with iSCSI SANs
• Learn basic iSCSI troubleshooting skills
• Have Fun while learning!!!!

3
Blocks protocols

• What is the relationship between SCSI, FC and
  iSCSI?

4
SCSI-3 - A path is a path

• SCSI-3 is a client-server I/O protocol
• Targets Data Provider/Server
• 1 or more data containers called Logical Units
  (LUNs)
• Initiators Data Consumer/Client
• Reads/writes data in units of blocks
• SCSI-3 is transport-independent Underlying
  transport mechanism and physical layer is not
  specified

5
SCSI-3 is Transport independent
Target
Initiator
LUN 0
LUN 1
Transport Mechanism
6
SCSI-3 CDBs Responses
Target
Initiator

SCSI_CMD LUN3 CDB28 00 00 10

DATA_IN Length4KB Data01 02
Transport Mechanism
7
FCP-SCSI Transport for SCSI-3

Target
Initiator

FC Frame FCP-SCSI Subtype SCSI_CMD LUN3
CDB28 00 00
FC Frame FCP-SCSI Subtype DATA_IN Length4KB
Data01 02
FC Port
FC Port
FC Fabric
8
iSCSI SCSI-3 over TCP/IP

• iSCSI is a transport protocol for conducting
  SCSI-3 over standard TCP/IP networks
• Defines message formats and rules
• Operates as an Upper-Level Protocol within the
  TCP/IP networking framework
• Peer to NFS, CIFS, HTTP, ...
• Everything is a message Protocol Data Unit (PDU)

9
iSCSI Transport for SCSI-3
Target
Initiator


IP hdr TCP hdr iSCSI PDU SCSI_CMD
LUN3 CDB28 00 00
IP hdr TCP hdr iSCSI PDU DATA_IN
Len4KB Data01 02
Ethernet
Ethernet
GbE
10 GbE
10/100 Mbps
IP Network
10
Why do we need the iSCSI Protocol?

• Benefits of SANs...
• Familiar block-level access
• Dedicated appliance for storage management
• Shared access to centrally managed storage
• without drawbacks of FC
• No expensive dedicated FC infrastructure
• Standard well-understood IP network routers,
  switches, cables, etc.

11
Before we jump in to iSCSI concepts.

• Lets learn a little bit about
• iSCSI analyzers.

12
iSCSI Protocol Analyzers

• Hardware Analyzers
• Finisar www.finisar.com
• I-Tech www.i-tech.com
• Agilent www.agilent.com
• Software Analyzers
• Ethereal www.ethereal.com
• Network Associates SnifferPro www.mcafee.com
• My Favorite
• Ethereal www.ethereal.com
• Its Free !!

13
iSCSI Protocol Analyzers Ethereal

• Ethereal
• Open Source Network Analyzer
• Under GPL
• Windows and Unix executables available
• Supports traces captured by other analyzers
• To name a few
• tcpdump (libpcap), NAI's Sniffer Pro, NetXray,
  Sun snoop and atmsnoop, Shomiti/Finisar Surveyor,
  AIX's iptrace, Microsoft's Network Monitor,
  Novell's LANalyzer, RADCOM's WAN/LAN Analyzer,
  HP-UX nettl, i4btrace from the ISDN4BSD project,
  Cisco Secure IDS iplog, the pppd log
  (pppdump-format), the AG Group's/WildPacket's
  EtherPeek/TokenPeek/AiroPeek.

14
iSCSI Protocol by Example

• iSCSI Protocol concepts

15
iSCSI Concepts Sessions

• iSCSI Session
• communication path and state between initiator
  and target
• iSCSI realization of a SCSI-3 I_T_NEXUS
• consists of one or more TCP connections
• PLUS state of in-progress commands

Host
Target
Disk class driver
I_T_Nexus
SCSI Layer
SCSI layer (Target)
iSCSI Session
iSCSI Initiator
iSCSI Target
CMD1
TCP connections
CMD2
16
iSCSI Concepts Nodename, Aliases

• iSCSI Nodename
• Worldwide-unique nodename, prescribed format
• Examples
• iqn.1992-08.com.netappsn.35780522
• iqn.2001-01.com.vendorhostname
• eui.123456789abcdef0
• iSCSI Target Alias, Initiator Alias
• For convenient identification
• Free format
• Examples
• oracle-host
• qa-storage

17
PDUs

• Each iSCSI message (I-gtT or T-gtI) is called a
  Protocol Data Unit (PDU)
• 48-byte Basic Header Segment (BHS),
• Variable-length data segment
• PDU format

Opcode
Params
Data Segment Length
BHS
0
Params (e.g. CDB for SCSI_CMD PDU)
48
Data Segment
Data
18
iSCSI Protocol Stack the complete picture
19
iSCSI PDU types

• LOGIN, LOGOUT create/shutdown session
• SCSI_CMD, R2T, DATA_IN, DATA_OUT, SCSI_RESP
  messages for SCSI-3 command phases
• TASKMGMT_CMD, RESP for AbortTask, ResetLUN,
  etc.
• NOP_IN, NOP_OUT ping
• ASYNCMSG Target reports event (e.g. abrupt
  shutdown)

20
Example iSCSI Write

• Initiator Target

SCSI_CMD 16K LUN SCSI
CDB WRITE_EXT, 32KB First 16KB of DATA
R2T 0 Offset
16KB Length 16KB
DATA_OUT 16K Offset
16KB Last 16KB of DATA
SCSI_RESP 0 Response
SCSI_GOOD
21
iSCSI Target Discovery

• Types of Discovery
• Static Discovery
• SendTargets Discovery
• Dynamic Discovery
• Lets look at a SendTargets Discovery trace

22
iSCSI Target Authentication

• CHAP
• Challenge Handshake Authentication Protocol
• All iSCSI targets and initiators MUST support
  CHAP.
• Users are likely to make mistakes in CHAP
  configuration.
• Lets troubleshoot some authentication problems
  using traces.

23
After Target Discovery

• Normal Session
• Scanning for LUs
• Lets look at a Bus Rescan Trace

24
Resources

• Use the iSCSI trace reading ability you picked up
  today to troubleshoot common problems.
• NetApp iSCSI Products
• http//www.netapp.com/products/storage-networking
  /ipsan/index.html

25
I need your feedback.

• Tell me now(or)
• Send me an email.

26
Storage Simplified
27
Whats a FAS270c?


Heads
Shelves
Storage Controller

FAS270c HA in a box FCP!
28
FAS270c
Head 1
Head 2
Console port
2XGbE Cu NICS
FC (Fib)
FC (Cu) Shelf Expansion Port
29
NetApp Unified Storage Architecture
SAN
NAS
Departmental
CorporateLAN
NAS(File)
NetApp Fabric Attached Storage