ECE 7970: Advanced Computer Architecture Networked Storage Systems

1
ECE 7970 Advanced Computer ArchitectureNetworke
d Storage Systems

• Instructor Dr. Xubin (Ben) He
• Email Hexb_at_tntech.edu
• Tel [phone]
• Course web http//iweb.tntech.edu

2
A Server-to-Storage Bottleneck
Source Brocade
3
How is a server connected to a storage?

• Traditional SCSI bus architecture (DAS)
• Both physical transport and a protocol
• Device drivers under the OS talk with SCSI
  devices
• Parallel bus with 8 bits, 16 bits, 32 bits
• Problems
• Termination of unused ports
• Number of devices connected to a SCSI bus 15
• Length limitation 15-25 meters

4
Problem

• Each new application server requires its own
  dedicated storage solution.

5
Solution

• Simplify storage management by separating the
  data from the application server.

6
Benefits of storage networking

• Consolidation
• Centralized Data Management
• Scalability
• Fault Resiliency

Network delays Network cost
Storage capacity/volume Administrative
cost Network bandwidth
7
Storage as a service
SSP ASP
Storage Service Provider
Application Service Provider
Outsourcing storage and/or applications as a
service. For ASPs (e.g., Web services), storage
is just a component.
8
Current State-of-the-art

• Network Attached Storage(NAS)
• Storage accessed over TCP/IP, using industry
  standard file sharing protocols like NFS, HTTP,
  Windows Networking
• Provide File System Functionality
• Take LAN bandwidth of Servers
• Storage Area Network(SAN)
• Storage accessed over a Fibre Channel switching
  fabric, using encapsulated SCSI.
• Block level storage system
• Fibre-Channel SAN
• IP SAN
• Implementing SAN over well-known TCP/IP
• iSCSI Cost-effective, SCSI and TCP/IP

9
NAS vs SAN?
10
Typical NAS
11
High BW NAS

• Accelerate applications
• Data sharing for NT, UNIX, and Web
• Offload file sharing function

12
Typical SAN

• Backup solutions (tape sharing)
• Disaster tolerance solutions (distance to remote
  location)
• Reliable, maintainable, scalable infrastructure

13
NAS vs. SAN

• In the commercial sector there is a raging debate
  today about NAS vs. SAN.
• Network-Attached Storage has been the dominant
  approach to shared storage since NFS.
• NAS NFS or CIFS named files over
  Ethernet/Internet.
• E.g., Network Appliance filers
• Proponents of FibreChannel SANs market them as a
  fundamentally faster way to access shared
  storage.
• no indirection through a file server (SAD)
• lower overhead on clients
• network is better/faster (if not cheaper) and
  dedicated/trusted
• Brocade, HP, Emulex are some big players.

14
Network File System (NFS)
server
client
syscall layer
user programs
VFS
syscall layer
NFS server
VFS
FS
NFS client
FS
RPC over UDP or TCP
15
NFS Protocol

• NFS is a network protocol layered above IP.
• Original implementations (and most today) use UDP
  datagram transport for low overhead.
• Maximum IP datagram size was increased to match
  FS block size, to allow send/receive of entire
  file blocks.
• Some implementations use TCP as a transport.
• The NFS protocol is a set of message formats and
  types.
• Client issues a request message for a service
  operation.
• Server performs requested operation and returns a
  reply message with status and (perhaps) requested
  data.

16
CFS Cluster File Systems
storage client
storage client
shared block storage service (FC/SAN, Petal, NASD)
xFS Dahlin95 Petal/Frangipani
Lee/Thekkath GFS Veritas EMC Celerra
issues trust compatibility with NAS
protocols sharing, coordination, recovery
17
Sharing and Coordination
block allocation and layout locking/leases,
granularity shared access separate lock
service logging and recovery network
partitions reconfiguration
NAS
SAN
storage service lock manager
18
Storage Architectures
19
Storage Area Networks
20
SAN connection

• FC
• FC-SAN
• LAN (Ethernet)
• IP-SAN
• iSCSI
• Other networks
• Petal (ATM)

21
A real SAN.
22
NAS and SAN shortcomings

• SAN Shortcomings--Data to desktop--Sharing
  between NT and UNIX--Lack of standards for file
  access and locking
• NAS Shortcomings--Shared tape resources--Number
  of drives--Distance to tapes/disks

• NAS--Focuses on applications, users, and the
  files and data that they share
• SAN--Focuses on disks, tapes, and a scalable,
  reliable infrastructure to connect them
• NAS Plus SAN--The complete solution, from
  desktop to data center to storage device

23
NAS plus SAN.

• NAS Plus SAN--The complete solution, from
  desktop to data center to storage device

24
Petal/Frangipani
NFS
NAS
Frangipani
SAN
Petal
25
(No Transcript)
26
The Convergence of Networking and Storage
Host
I/O Application
File System
Networking TCP, UDP, IP GBE,
10GBE
Storage Parallel SCSI Fibre
Channel IDE, ATA
TCP/IP Network
iSCSI
Storage
iSCSI-based Storage Area Network (SAN)(block
level)
27
What is iSCSI

• The protocol for transfer of SCSI commands and
  data over TCP/IP networks is called iSCSI.

28
The layering of SCSI commands and protocols
29
The layering of SCSI architecture
30
Different methods of implementing iSCSI

• Software
• Intel, NIH, UMass, STAR lab
• Hardware
• Adaptec
• IBM
• CISCO

31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
iSCSI Session

• iSCSI works in the session context
• A session comprises one or more TCP connections
  between the initiator and target
• A session has four phases
• Initial login phase
• Security Authentication Phase (optional)
• Operational Negotiation Phase (optional)
• Full Featured Phase
• The first three phases normally occur at the
  system start time (in iSCSI driver).
• iSCSI data and command exchange in the
  full-featured phase.

37
A Normal iSCSI Setup Process

• Discovery Session
• Goal find all available targets
• Establish TCP connection with the gateway
• Initial login between the initiator and gateway
• Parameter negotiation (security phase omitted)
• Obtain all available targets
• Setup Session
• Goal initiator establish a session with each
  target
• Establish TCP connection for each target
• Initial Login with each target
• Parameter negotiation (security phase omitted)
• Full featured phase

38
iSCSI Target Identification

• Static Configuration
• The initiator statically configure the IP address
  or target name
• Used in a small number of initiators
• Service Location Protocol (SLP)
• Use SLP protocol to discover targets
• Adequate for medium sized network
• Internet Service Naming Service (iSNS)
• Use iSNS protocol to discover targets
• Internet

39
References

• E. Lee and C. Thekkath, Petal Distributed
  Virtual Disks, Proceedings of the international
  conference on Architectural support for
  programming languages and operating systems
  (ASPLOS 1996)
• P. Sarkar, S. Uttamchandani, and K. Voruganti,
  Storage Over IP When Does Hardware Support
  Help? Proc. of 2nd USENIX Conference on File And
  Storage Technologies (FAST2003)
• C. Thekkath, T. Mann, and E. Lee, Frangipani A
  scalable distributed file system, Proceedings of
  the 16th ACM Symposium on Operating Systems
  Principles (SOSP), pp. 224-237, October 1997
• iSCSI RFC (IETF), Oct. 2000
• HP Tech. Report, SCSI over TCP/IP, Sept. 2000
• IBM Research Lab, iSCSI-protocol and
  implementation, Sept. 2000
• Cisco Tech. Report, Jan. 2001

40
References

• Storage Netwoking Indurstry Association
  www.snia.org
• Xubin He, Ming Zhang, and Qing Yang, "STICS
  SCSI-To-IP Cache for Storage Area Networks,"
  Journal of Parallel and Distributed Computing,
  vol. 64, no. 9, pp.1069-1085, September 2004.
• G. Gibson, D. Nagle, W. Courtright II, N. Lanza,
  P. Mazaitis, M. Unangst, and J. Zelenka, NASD
  Scalable Storage Systems, USENIX99
• T. Anderson, M. Dahlin, J. Neefe, D. Patterson,
  D. Roselli, and R. Wang, Serverless Network File
  Systems, ACM Transactions on Computer Systems
• R. Hernandez, C. Kion, and G. Cole, IP Storage
  Networking IBM NAS and iSCSI Solutions,
  Redbooks Publications (IBM)
• E. Miller, D. Long, W. Freeman, and B. Reed,
  Strong Security for Network-Attached Storage,
  Proc. of the Conference on Fast and Storage
  Technologies (FAST2002)
• D. Nagle, G. Ganger, J. Butler, G. Goodson, and
  C. Sabol, Network Support for Network-Attached
  Storage, Hot Interconnects1999