Extending SANs over IP Networks

1
Extending SANs over IP Networks
Addy Lam Professional Services
2
Agenda

• Nishan, The Company
• Evolution of Storage Area Network
• Nishan Vision
• Why deploy IP Storage and Why go with Nishan?
• SAN over IP applications
• Customer Profiles
• Nishan Products
• Comparison between Nishan and Cisco Products
• IP SAN reference material
• Questions and Answers

3
Nishan, The Company
Founded October 1998
Headquarters 3850 North First Street San Jose, CA 95134 [phone]
Staff 90 Employees
Company Privately Held
Investors LightSpeed, ComVentures, Dell, Quantum, Siemens, Sun, others
4
The Value andEvolutionof SAN
5
Value of Storage Area Networking

• Storage Consolidation
• Reduce administrative overhead
• Maximize storage utilization
• High Availability
• Server clusters accessing shared storage
• Mirroring terabytes of data/common accessibility
• Disaster Recovery / Metro and Campus
• Remote mirroring within the SAN
• Tape backup / restore

6
Evolution of Storage Area Networking

• IP SANs
• ? Based on mainstream TCP/IP networks
• ? Enables SAN over WAN
• ? More robust features scalability
• ? Standards-based (IETF, IEEE, ANSI)

Fibre Channel Switched Fabrics ? Data Center /
Departmental SANs many devices ? Dedicated
Bandwidth per port vs. shared bandwidth
Fibre Channel Arbitrated Loop ? Small
departmental SANs small number of devices ?
Shared bandwidth
VAX Clusters / IBM ESCON / SSA ? Proprietary
shared storage solutions ? Superceded by Open
Systems FC
7
Nishan Vision
8
Nishan Vision of SAN Evolution
A dispersed SAN would be key for business
continuance planning and backup, both of which
gained importance after 9/11 -- Mark Taylor,
Wells Fargo
9
Nishan Vision of SAN Evolution
Departmental Servers- iSCSI NICs

• Departmental servers beyond FC reach
• Promotes storage consolidation of servers

Main Data Center
IP Campus/Metro Network
FC Servers
Disaster Recovery
IP WAN
Remote Data Center
Regional Office- iSCSI NICs

• Retail and Banking Applications
• Primary application is remote backup
• Cost Effective
• Converged HBA
• Leverage existing low-speed links

10
Why IP and Nishan?

• Distance Independence
• Allows TRUE Disaster Recovery
• Distributed storage
• Scalability
• Globally distributed Enterprise SANs
• Large Data Center SANs
• Interoperability
• Performance
• Distance-independent wire-speed IP Storage
  solutions
• Nishan enhancement Fast Write, Compression, and
  Jumbo Frames
• Lower Total Cost of Ownership
• Leverage existing infrastructure
• Investment protection by gradual migration to
  iSCSI and newer technologies
• Enhanced Functionality
• Security IPSec encryption and authentication
• Quality of service (QoS)
• Advanced SAN management tools

11
Why IP and Nishan
Distance Independence
12
Infrastructure Alternatives for SAN Extension
Campus
FC SAN
Consolidation
Metro
?
Main Data Center
FC SAN
Consolidation / Disaster Recovery / Mirroring

• Fiber-Channel Extension
• SONET/SDH, Dark Fiber, DWDM
• Tunneling (FC Encapsulation)
• FC-IP
• Native IP
• iFCP, iSCSI

WAN
Remote Backup
13
Fibre Channel SAN Issues - Extensibility
Native FC can not be transported over most
service providers infrastructure. Wide area
networks are based on IP or Ethernet and not FC,
therefore, costly point-to-point Dark Fiber or
DWDM is required to connect FC SAN.
FC SAN
FC SAN

• Fibre Channel Networks are designed for intra
  data center environments
• Not architected with distance or latency in mind
• Enforce strict reliability and performance
  requirements
• Homogeneous FC-2 Transport

Fibre Channel SAN island
14
FC-Extension over Dark Fiber or DWDM
100
Link Utilization
80
Buffer Credits of Single Port
60
Buffer Credits of Four Ports
40
Remote Data Center
Primary Data Center
20
Metro
WAN
0
0
50
100
150
200
250
300
350
400
Fibre Length (Km)

• Limited credits of FC switches causes a dramatic
  decrease in utilization gt 30km
• Can increase distance by robbing FC credits from
  other ports
• Disadvantages
• Lost switch port density
• Doesnt fix link-utilization problem (just delays
  it)
• Still have FC scalability limits if connecting
  FC-switches (few as 8 switches in fabric)
• Service not available in all regions
• Best positioned between limited FC Targets (2)
  that are relatively close, and where availability
  and tariffs not an issue

15
IP Storage Extends Beyond Fibre Channels Reach
Long Distance IP Storage Performance
Results Nishan Systems Lab Testing with Delay
Simulation Equipment 3 NT Servers with FC HBAs,
24 Disk Drives
120
Throughput, in Megabytes per Second
IP Storage Switches Extend Fibre Channels Reach
to Greater Distances and Latencies
100
IP Storage Read and Write Operations
80
60
Fibre Channel Switches can be Used in Low Latency
Environments, such as Corporate Data Centers
Fibre Channel Switches (64 Credits)
40
Fibre Channel Switches (16 Credits)
20
10
20
30
40
50
0
Round Trip Latency, in Milliseconds
Fibre Channel performance based on theoretical
best case, using fully loaded 2112 byte frames
16
Promontory Project Coast-to-Coast IP Storage
IP Storage Traffic
iSCSI and FC
10Gbps
80 ms round-trip latency

• Provides a High-Performance, Distance-Insensitive
  architecture (SAN/MAN/WAN)
• Enables high-speed, cost-effective backup
  operations, using standard high-speed data
  networks
• Geographically distant replication sites TRUE
  disaster recovery
• Distributed Storage

17
Promontory Topology and Performance
18
Why IP and Nishan
Scalability
19
Gluing Together Multi-vendor FC Fabrics
FCIP Each storage fabric limited to a few
switches
iFCP No limit to the size and scale of the
storage network
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
Extreme
FC
Nortel
FC
FC
Avici
Cisco
FC
Nishan
Netgear
3Com
FC
FC
Juniper
Foundry
FC
Riverstone
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC
FC

• FC switch interoperability issues may limit
  connectivity
• No Point-to-Multipoint connectivity

• Multi-vendor interoperability
• Any-to-Any connectivity
• Lower price per port

20
Why IP and Nishan
Interoperability
21
Multi-vendor Interoperability

• Interoperability is a pre-requisite in the IP
  World
• Nishan is a prominent participant in several
  Interoperability forums
• UNH iSCSI Consortium Plugfests
• SNW Interoperability Showcases
• NI IP Storage InteropNet
• Vendor-to-vendor testing efforts
• Partner Certifications
• Hitachi Data Systems
• Xiotech

22
Imation Labs Interoperability Verification
Servers
Operating Systems
Host Bus Adapters
Sun E450 HP N-Class IBM RS/6000 H70 Compaq 6400
Solaris HP-UX IBM RS/6000 AIX Windows NT4 Windows
2000 NetWare 5.1 Red Hat Linux
JNI QLogic Emulex Agilent
Tape Libraries
Disk Arrays
CLARiiON 4500 HP xp512 HDS 7700e Freedom IBM ESS
Shark EMC Symmetrix StorageTek 9176 Compaq
StorageWorks RA8000 XIOtech Magnitude LSI 4400
StorageTek L180 ADIC Scalar 1000 StorageTek 9710
Nishan partners also have done extensive
interoperability testing on these and other
products
23
Why IP and Nishan
Performance
24
Wire-speed iSCSI Performance
Storage Subsystem
219 MBps port throughput
Server With High-Perf. iSCSI HBA
Fibre Channel
IPS3300
iSCSI / Gigabit Ethernet
FC RAID
Low CPU Utilization
25
SCSI Write Transaction
Initiator
Target
Near-End Nishan
Far-End Nishan
DS3 45 Mbps
FC 1G
FC 1G
FCP_CMD_WRT_16MB
FCP_CMD_WRT_16MB
FCP_CMD_WRT_16MB
FCP_XFR_RDY-1K
FCP_XFR_RDY-1K
FCP_XFR_RDY-1K
FCP_ DATA_OUT-1K
FCP_DATA_OUT-1K
FCP_DATA_OUT-1K
FCP_XFR_RDY-512K
FCP_XFR_RDY-512K
Delay 4 x RTT
FCP_ XFR_RDY-512K
FCP_DATA_OUT-512K
FCP_ DATA_OUT-512K
FCP_DATA_OUT-512K
FCP_XFR_RDY-512K
FCP_XFR_RDY-512K
FCP_XFR_RDY-512K
FCP_DATA_OUT-512K
FCP_DATA_OUT-512K
FCP_DATA_OUT-512K
FCP_RSP
FCP_RSP
FCP_RSP
Write Time Aggregation
26
SCSI Write Transaction with Fast Write
96 MB of buffer space per Port
Far-End Nishan
Initiator
Target
Near-End Nishan
FC 1G
DS3 45Mbps
FC 1G
FCP_CMD_WRT_ 16MB
FCP_XFR_RDY-16MB
FCP_CMD_WRT_16MB
FCP_CMD_WRT_16MB
FCP_DATA_OUT-16MB
FCP_DATA_OUT
FCP_XFR_RDY-1KB
Compression
FCP_DATA_OUT-1K
FCP_XFR_RDY-512KB
Minimal local delay
FCP_DATA_OUT-512K
Delay 2 x RTT
FCP_XFR_RDY-512KB
FCP_DATA_OUT-512K
FCP_RSP
FCP_RSP
FCP_RSP
Fast Write can not be done without tracking each
FC sessions state
27
Compression Jumbo Frames

• Compression
• Up to 111 Compression Ratios
• Standard Compression algorithms with extensions
• Smart Compression - compresses just enough to
  match egress data rate to port speed
• Both ends of a connection NEED NOT turn on
  compression to realize gains
• Jumbo Frames
• Support for maximum size FC frames - payloads up
  to 2048 bytes
• Jumbo frame support should be end-to-end in the
  network
• The IPS switches can handle incoming jumbo frames
  even if the support is not turned on.
• Better overall throughput
• Reduced header overhead
• Higher traffic rates higher gains
• Compression works better due to bigger sized
  payload

28
Why IP and Nishan
Lower Total Cost of Ownership
29
Nishan can Leverage existing Fibre Channel
devices
FC Hosts
Leverage existing investment in Fibre Channel
storage devices
Build on well-understood IP networking technology
with existing staff and support
IP Network
Leverage IPs QoS and security features
FC Targets
30
FC SAN Extension with IP Storage Switches (E-Port)

• Existing Fibre Channel Extends Easily to IP
• Use standard E_Port connections on FC switches

Servers
Servers
E_Port
E_Port
OSPF
Fibre Channel Switch
Fibre Channel Switch
IP Network
Multiprotocol IP Storage Switch
Multiprotocol IP Storage Switch
Fibre Channel Gigabit Ethernet
FC RAID
FC RAID
Fibre Channel E_Port Compatibility
31
Nishan can Transition from Fibre Channel to iSCSI
iSCSI Server
FC Server
Built-in iSCSI support
Nishan IP Storage Switch
IP Network
IP/GE
IP/GE
FC
FC
Fibre Channel
IP / Gigabit Ethernetand iSCSI
FC RAID
FC RAID
Heterogeneous iSCSI and Fibre Channel Access
32
Why IP and Nishan
Enhanced Functionality
33
Security of Storage Data
Authentication and Data Encryption for Storage
Design Features
Benefits

• Standard IPSec encryption
• Highly secure 3DES encryption
• Off-the-shelf equipment
• MAN or WAN application
• Unlimited distance

• Protects against traffic intrusion / snooping on
  critical applications (financial, government,
  etc.)
• Additional service-revenues based on secure
  transport (SSP)

Secured Data Transport
IP Network
Cisco CheckPoint CipherOptics SonicWall NetScreen
RapidStream
IPSec / Firewall
IPSec / Firewall
34
Enterprise SAN Deployments with QoS
FC Director
FC RAID
Mission-Critical VLANs/Zones prioritize
mission-critical data over other data flows
35
Management Scheme Scenario 1
ConfigA
ConfigB
SANvergence Manager
ZoneA
ZoneB
ZoneC
WWN-2 Domain 07, Port 5
WWN-1 Alias A
WWN-2
Alias A WWN3
FC Vendor Mgmt App
Zone1
Zone2
WWN-A WWN-B
WWN-C
Read-only SoIP Zones
FC Island 1
Active ZoneSet ConfigA
ConfigA
ConfigB
IPS Fabric
ZoneA
ZoneC
WWN-4 WWN-5
WWN-6
Active ZoneSet Includes all Zones
FC Island 2
FC Vendor Mgmt App
Active ZoneSet ConfigA
Read-only SoIP Zones
36
Management Scheme Scenario 2
ConfigA
ConfigB
ZoneA
ZoneB
ZoneC
WWN-2 Domain 07, Port 5
WWN-1 Alias A
WWN-2
Alias A WWN3
Zone1
Zone2
WWN-A WWN-B
WWN-C
FC Island 1
Active ZoneSet ConfigA
ConfigA
ConfigB
IPS Fabric
ZoneC
ZoneA
WWN-6
WWN-4 WWN-5
Active ZoneSet Includes all Zones
FC Island 2
Active ZoneSet ConfigA

• pwwn01 is the lower 3 bytes of the Principal
  E_Ports WWN for SAN Island 1
• pwwn02 is the lower 3 bytes of the Principal
  E_Ports WWN for SAN Island 2

37
Storage over IPApplications
38
Storage Over IP Applications

• Disaster Recovery/Remote Disk Replication
• EMC, HDS, Xiotech, Compaq
• Data Centers
• Converged
• Distributed
• IP SAN Extension
• Joining SAN islands
• Service Provider Solution
• Managed Storage Hosting Services
• iSCSI integration
• Storage Consolidation of Diskless Server
• Remote Mirroring with iSCSI
• Remote Backup with iSCSI

39
Disaster Recovery/Remote Disk Replication
FC Servers
FC Servers
Nishan switches attached directly to storage
devices
IP Network
HDS/EMC
HDS/EMC
Remote Data Center
Main Data Center

• Disk-to-Disk Replication over distance
• EMC Symmetrix Remote Data Facilities (SRDF) -
  Symmetrix
• HDS TrueCopy and TrueCopy 390 - Lighting 9900V
  Series
• XIOtech REDI SAN Links - Magnitude
• Compaq Data Replication Manager

40
Converged Data Center
Remote Data Center
Main Data Center
FC / iSCSI Servers
SAN and WAN in a Box
FC Disks

• SAN and WAN in a Box
• Combines Fabric Switching and SAN Extension
  platform into a single platform for Remote Data
  Centers
• Very cost effective No need to two sets of
  platforms
• Leverages Customer IP expertise

41
Distributed Data Center
iFCP/iSCSI Attached Departmental Servers
Main Data Center
FC Servers
iFCP/iSCSI Attached Regional Office

• Connect Departmental or Regional Servers Beyond
  FC Reach over IP Network
• Cost Effective Backup, Storage Consolidation
• No need for any tunnel-termination at both ends
  Converged LAN/iSCSI HBAs
• IBM Certification
• iSCSI connectivity to Shark, FAStT Includes
  VERITAS, Tivoli, Legato

Data Center Boundary Expands via iSCSI
42
SAN Extension via E_Port

• Storage Consolidation (converged zoning)

43
Multi-Point SAN Extension
Data Center SAN
IP
IP Network
FC
IP/GigE
IP
Nishan IP Storage Switch
Fibre Channel
IP / Gigabit Ethernet
Data Center SAN
44
Service Provider - Managed Storage Hosting
Services
SP-Managed Backup
Storage Services Revenue Plus Transport Revenue
Subscriber VLANs
IPSec or IP Firewall
IPSec or IP Firewall
Fibre Channel SAN
Service Provider POP
Metro Ring
Switches can be COE or CPE
Subscriber B CPE Fibre Channel Servers and
Storage
Subscriber A CPE Fibre Channel or
iSCSI Servers, Storage, Switches
To Other Subscribers
Gigabit Ethernet / iFCP / iSCSI Fibre Channel
45
Storage Consolidation of Diskless Servers
Solution For cost-sensitive customers who have
not yet storage-networked their diskless servers,
implement an IP Storage Network with Nishan IP
Storage switches for direct access to centralized
data storage
Central Data Center
Campus Site
iSCSI Hosts
High Availability IP Storage Fabric
MAN Site
iSCSI Hosts

• Benefits
• Familiar IP and Ethernet management tools for IP
  network and IP Storage switches
• Lower TCO on hardware components for end users ,
  esp Fibre Channel HBAs
• Migration path to new IP Storage deployments
• Robust IP security and encryption tools protect
  mission critical data

46
IP Storage Remote Mirroring Configuration with
iSCSI
Solution For customers that cant implement
TrueCopy, implement an IP Storage Network with
Nishan IP Storage switches for direct access to
remote disk devices
Central Data Center
Remote Data Center
IP or FC Hosts
Volume Manager/ Replicator
High Availability IP Storage Fabric

• Benefits
• Zero investment in FC fabrics
• Native IP extension for Volume Management across
  any IP network
• Familiar IP and Ethernet management tools for IP
  network and IP Storage switches
• Lower TCO on hardware components for end users
• Robust IP security and encryption tools protect
  mission critical data

47
IP Storage Remote Backup Configuration with iSCSI
Solution For green-field deployments, implement
an IP Storage Network with Nishan IP Storage
switches for direct access to remote tape devices
Central Data Center
Remote Data Center
iSCSI Hosts
NetBackup
High Availability IP Storage Fabric
FC Tape

• Benefits
• Native IP extension for NetBackup across any IP
  network
• Familiar IP and Ethernet management tools for IP
  network and IP Storage switches
• Lower TCO on hardware components for end users
• Migration path to new IP Storage deployments
• Robust IP security and encryption tools protect
  mission critical data

48
Nishan Customer Profiles
49
Carlson Companies - Worlds Largest IP SAN
www.carlson.com

• Carlson Companies implemented a distributed IP
  SAN in order to
• Storage network 10 TB of disk for a data
  warehouse
• Support multi-customer storage service level
  agreements via
• flexible bandwidth management
• Efficiently access remote site for disaster
  recovery
• Implement centralized backup to the central data
  center
• Integrate new native IP devices like iSCSI HBAs

By moving storage data to our IP network, we can
meet both our data center and wide-area storage
requirements and seamlessly integrate new
storage technologies such as iSCSI. Steve
Brown, Carlson CIO / Sr. VP
50
Carlson Companies
51
Wells Fargo ICS Customer

• Des Moines Data Centers
• Project involved
• Data migration from an EMC Symmetrix to a
  Lightning 9960 via VERITAS software locally
• Then Hitachi TrueCopy via a Nishan switch to a
  remote Lightning 9960
• During test, link was broken and re-established
  pairs re-synched
• Results
• Exchange server data was migrated
• Server was relocated to target location
• Data was verified and server activated

Exchange Server Windows NT 4
Lightning 9960 Remote
Lightning 9960 Primary
FC
GE
GE
FC
FC
IP Network
9960
52
Another Financial Institution

• Un-announced reference site
• 6TB in NYC, 5TB in Delaware, T3 link
• DR initiative
• Bi-directional replication with Hitachi TrueCopy
• Solaris Windows NT servers, Exchange, Sybase,
  NFS
• Brocade SilkWorm 2800 fabric switch and Nishan
  IPS 3300 border switch
• Performance
• 60ms of latency on Nishan IPS 3300 _at_ 10Mb
• 80 of link used, 800Kb/sec
• Moving to GigE, 8.5GB/min
• 33 min to copy 280GB

53
Nishan Products
54
IP Storage Switch Design Criteria

• Wire-speed Performance
• Full gigabit bandwidth
• Sustained bandwidth utilization over enormous
  distances
• Wire-speed protocol conversion FC / iFCP / iSCSI
• Flow Control
• Multiprotocol Support
• Compatibility with existing FC devices and
  switches
• FC, iSCSI, iFCP, iSNS, OSPF, STP
• Provide smooth transition path to IP SAN
• Advanced Management
• Embedded SNS server for FC
• Zoning utility for device assignment

55
Nishan Multiprotocol Storage Product Family
FC
FC
16 Configurable MultiService Interfaces
IPS 4000 Series
FC
IPS
IPS
IPS
FC
FC
IPS 3000 Series
8 Configurable MultiService Interfaces
FC
IPS
IPS
IPS
SANvergence Management Software
Configuration, Management

• Device Management
• Zoning

Switch Functionality

Fibre Channel
IP Storage Switches
End Systems (Storage and Servers)
Backbone (IP Storage Network)
Fibre Channel
Gigabit Ethernet
iSCSI
Gigabit Ethernet
iSCSI
InfiniBand
56
IPS 4000 Series
FC
FC
FC
IPS
IPS
IPS
Features
IPS 4000 Series Switch

• IP Storage protocols
• 16 ports, 1 RU
• IPS4300 has 4 ports with wire speed TCP for iSCSI
  and iFCP
• Small Form Pluggable interfaces
• E_Port and auto-sensing F/FL/N ports
• FastWrite Compression on TCP Ports

• FC switch, fully interoperable
• (i.e. no change to existing FC infrastructure)
• Connects FC devices (RAID, JBOD, Tape) to IP
  networks at Gbps wire-speed
• Connects FC devices to iSCSI devices
• IP switching and routing (QoS, OSPF, etc)

57
IPS 3000 Series
FC
FC
FC
IPS
IPS
IPS
Features
IPS 3000 Series Switch

• IP Storage protocols
• 8 ports, 2 RU
• IPS3300 has 2 Ports with wire speed TCP
  termination for iSCSI and iFCP
• SC connectors (SX, LX, ZX GBIC)
• E_Port and auto-sensing F/FL/N ports
• FastWrite Compression on TCP Ports

• FC switch, fully interoperable (i.e. no change
  to existing FC infrastructure)
• Connects FC devices (RAID, JBOD, Tape) to IP
  networks at Gbps wire-speed
• Connects FC devices to iSCSI devices
• IP switching and routing (QoS, OSPF, etc)

58
Element Manager

• Management GUI viewed via a browser
• All ports are Dual-Personality FC or GE

iSCSI and/or iFCP (SAN Interconnect Ports)
FC End Device Port
FC ISL Ports
Data Center GE ISL Ports
59
SANvergence Manager
SANvergence Manager

• Manage mixed environments of Fibre Channel,
  Gigabit Ethernet (iSCSI) and SCSI storage
• Standards based - SNMP compliant
• Multi-platform support
• Enterprise system management integration CA
  Unicenter TNG, HP OpenView,Tivoli NetView

Features

• Web based management
• Device and network-wide scope
• Auto-discovery of IP Storage switches and
  gateways
• Secure in-band and out-of-band management

60
Comparison betweenNishan and CiscoProducts
61
Basic Product Comparison
Cisco 5420 Cisco 5428 Nishan 4300
GigE Int. 1 2 4 to 16
FC Int. 1 8 Up to 12
FC Switch connectivity Yes (single) No Yes (multiple)
FC Port Type N_Port F_Port F/FL/E/Auto
Power Supply Single Single Redundant
Support iFCP NO NO YES
62
IP SAN ReferenceMaterial
63
IP Storage Reference sites

• IEFT IP Storage Group
• http//www.ietf.org/html.charters/ips-charter.html
  
• Julian Stratans IPS Archive
• http//www.haifa.il.ibm.com/satran/ips/

64
Book on IP SANs

• Partnering for Enterprise-class Storage Solutions
• Committed to Open Systems / Customer Choice
• First transcontinental IP SAN
• First wire-speed iSCSI demonstration
• SAN extension for disaster recovery, remote
  vaulting/mirroring
• Proven interoperability
• Migration path for heterogeneous FC and IP SAN
• Best of breed high availability storage and IP
  storage networking
• ISBN 0-201-75277-8

Designing Storage area Networks A practical
Reference for Implementing Fibre Channel
SANs by TOM CLARK
65
Questions and Answers
66
Thank you