Data Sharing in Storage Area Networks

1
(No Transcript)
2
Data Storage Tutorial

• Thomas Feil
• Dir. Marketing Storage Solutions EMEA

3
Agenda

• Introduction to Fibre Channel
• Promises of Storage Area Networks
• Delivering on the Promise Share Data
• Data Management in SANs
• SGIs SAN Status and Vision

4
What is Fibre Channel?

• A definition for a serial I/O bus capable
• to transfer data between two ports
• at speeds of 1.0625 Gbit
• for Storage via the SCSI-3 protocol
• using optical and copper media.

5
What Standards?
Fibre Channel Standards X.3230-1994-Fibre Channel
Physical and Signaling Standard (FC-PH)initial
core standard
6
What Standards?

• Like Ethernet, FC standards are evoloving
• FC-PH, ANSI X3.2301994,
• IPI, SCSI, HIPPI, 1.0625Gbit
• FC-AL, ANSI X3.2721996
• FC-FG, ANSI X3.2891996
• FC-PH-2, ANSI X3.2971997
• Broadcast, Multicast, Alias Address, Rates gt
  1,0625Gbit
• FC-PH-3, ANSI X3.3031998

7
Technology Comparison
8
Fibre Channel Topologies

• Point-to-Point

• 100MByte/s per connection
• Just defines connection between storage
  system and host

9
Fibre Channel Topologies

• FC-AL, Arbitrated Loop

Single Loop Data flows around the loop, passed
from one device to another
Dual Loop Some data flows through one loop while
other data flows through the second loop

• Each port arbitrates for access to the loop
• Ports that lose the arbitration act as repeaters

10
Fibre Channel Topologies

• FC-AL, Arbitrated Loop with Hubs

Hubs make a loop look like a series of point to
point connections.
2
3
1
HUB
4
Addition and deletion of nodes is simple and
non-disruptive to information flow.
11
Fibre Channel Topologies

• FC-Switches

Switches permit multiple devices to communicate
at 100 MB/s, thereby multiplying bandwidth.
2
1
3
SWITCH
4
Fabrics are composed of one or more switches.
They enable Fibre Channel networks to grow in
size.
12
Why Fibre Channel?

• Gigabit Bandwidth Now - 1 Gbs today, soon 2 and
  4Gbs
• High Efficiency - FC has very little overhead
• Multiple Topologies - Point-to-point, Arbitrated
  loop, Fabric
• Scalability - from Point-to-Point FC scales to
  complex Fabrics
• Longer cable lengths and better connectivity than
  existing SCSI technologies
• Fibre Channel is an ISO and ANSI standard

13
Agenda

• Introduction to Fibre Channel
• Promises of Storage Area Networks
• Delivering on the Promise Share Data
• Data Management in SANs
• SGIs SAN Status and Vision

14
Promise of Storage Area Networks

• Centralized Management
• Storage Consolidation/Shared Infrastructure
• High Availability and Disaster Recovery
• High Bandwidth
• Scalability
• Shared Data !?

15
Centralized Management

• Storage today is
• Server attached, therefore at the servers
  location.
• Difficult to manage
• Expensive to manage
• Difficult to maintain

Building A
Building B
Building D
Building C
16
Centralized Management

• Storage is
• Network attached
• independent from location of server
• easy and cost effective to manage and maintain

IT-Department
17
Storage Consolidation

• Storage still
• is split amonst different storage systems for
  different host

IRIX

• Storage now
• shares a common infrastructure

18
High Availability and Desaster Recovery

• Highly redundant network with no single point of
  failure.

19
High Availability and Desaster Recovery
Clients
Clients
Building 2
Building 1
Campus Network
2x Serial, 1x Private Ethernet
Appl. Server 1
Appl. Server 2
Distance 10 km
Fibre Channel
Tape Library
SAN
Fibre Channel Raid 1
Fibre Channel Raid 2
20
High Bandwidth

• Storage Area Networks today provide
• 1.06Gbs today
• 2.12Gbs this year
• 4.24Gbs in the future
• Multiple Channels expand bandwidth

I.e.800 Mbyte/s
21
Scalability

• Can scale to very large configurations quickly

22
Share Data!?

• Currently there is just shared infrastructure

23
Agenda

• Introduction to Fibre Channel
• Promises of Storage Area Networks
• Delivering on the Promise Share Data
• Data Management in SANs
• SGIs SAN Status and Vision

24
Delivering on the Promise Share Data!

• What can we share today?
• How can we share data?
• CXFS Overview
• CXFS Concepts
• CXFS Performance
• CXFS Future Roadmap
• CXFS Summary

25
What can we share today?

• ANSWER Infrastructure
• Zoning
• LUN-Masking
• Partitioning

Linux
NT
IRIX
26
How can we share Data?
Shared file systems provide the advantage of all
other sharing techniques.
Data
Data
Data
FS
Shared File System
Level of Sharing
Zoning, PartitioningLUN Masking
Shared FS
Physically Consolidated SAN-attached Storage
Physically Consolidated SAN-attached Storage
Direct Attached Storage
Direct Attached Storage
27
CXFS Overview- Based on XFS, A World-Class
Filesystem

• Reliable
• Log/Journal
• Field proven
• Fast
• Fast metadata speeds
• High bandwidths
• High transaction rates
• Scalable
• Full 64 bit support
• Dynamic allocation of metadata space
• Scalable structures and algorithms

28
CXFS Overview- XFS Reliability

• Field proven
• Run for years on thousands of IRIX systems.
• Part of IRIX since 1994
• Released as part of IRIX 5.3
• Log/Journal
• XFS designed around log
• No UNIX fsck is needed
• Recovery time is independent of filesystem size
• Depends on system activity levels

Usually, recovery completes in under a second
29
CXFS Overview- XFS Reliability

• Fast metadata speeds
• B-Trees everywhere (Nearly all lists of metadata
  information)
• Directory contents
• Metadata free lists
• Extent lists within file
• High bandwidths on SGI Origin 2000
• 7.32 GB/s on one filesystem (32p O2000, 897 FC
  disks)
• gt 4 GB/s to one file (same Origin, 704 FC disks)
• Large extents (4 KB to 4 GB)
• Request parallelism
• Read ahead/Write behind
• High transaction rates 92,423 IOPS

30
CXFS Overview- XFS Speeds

• B-tree Directory Speed

31
CXFS Overview- XFS Speeds

• Full 64 bit support
• Large Filesystem
• 18,446,744,073,709,551,615 264-1 18 million
  TB
• Large Files
• 9,223,372,036,854,775,807 263-1 9 million TB
• Dynamic allocation of metadata space
• Unlimited number of files
• Scalable structures and algorithms
  (B-Trees)
• Performance is not an issue with large numbers
  offiles and directories

32
CXFS Clustered XFS

• CXFS Attributes
• Shareable high-performance XFS file system
• Shared among multiple IRIX nodes in a cluster
• Near-local file system performance.
• Direct data channels between disks and nodes.
• Resilient File System (highly available)
• Failure of a node in the cluster does not prevent
  access to
  the disks from other nodes
• Convenient Interface
• Users see standard Unix File Systems
• Single System View (SSV)

33
CXFS Concepts- The Metadata Model

• Metadata
• The data about a file, including
• size, inode, create/modify times, and permissions
• Metadata server node (a.k.a. CXFS server)
• One machine in the cluster that is responsible
  for controlling the metadata of files. It also
  plays traffic cop to control access to the
  file.
• Backup metadata servers designated for fail-over
• No single point of failure
• Metadata client node (a.k.a. CXFS client)
• A machine in the cluster that is not the metadata
  server.
• Must obtain permission from metadata server
  before accessing the file.

34
CXFS Concepts- The Metadata Model
CXFS Server Node
CXFS Client Node
Coherent System Data Buffers
Coherent System Data Buffers
MetadataIP-Network Fast RPCs
MetadataPath
Token Protected Shared Data
CXFS Client
Token Protected Shared Data
CXFS Server
XFS
XFS
RAID
DirectChannels
Log
RAID
Shared Disks
35
CXFS Concepts- The Metadata Model

• Fast-asynchronous XFS metadata transactions in
  server
• Customized RPC mechanism
• maximize communication speed among clients and
  the metadata server

Some other shared-file systems use NFS
communication to read and write the metadata.
This slows access to data
36
CXFS Concepts- Full POSIX Filesystem API Support

• Efficient buffering of metadata in clients
• Metadata is buffered in the clients
• Reread metadata if the file size or position
  changes
• The CXFS application programmer interface (API)
  is POSIX compliant
• Fully coherent buffering, as if a single system
• Writes flush cashes on other nodes
• Compliant with POSIX file system calls
• Including advisory record locking
• No special record-locking libraries required
• For example NFS supplies a separate non-POSIX
  record-locking library, which is not needed
  with CXFS.

37
CXFS Concepts- Read Metadata Flow
Metadata Server
Metadata Client
38
CXFS Concepts- Write Metadata Flow
Metadata Client B request write token
Metadata Server
Metadata Client A holds write token
39
CXFS Concepts- Resilience

• Single server manages metadata
• Backup metadata servers designated for fail-over
• No single point of failure
• Available with IRIX 6.5.7 not IRIX 6.5.6

MetaDataBackup-Server
LAN
Up to 64 nodescurrently 8
Private Network
MetaDataServer
SAN fabric
40
CXFS Performance

• CXFS/SAN-attach compare to XFS/direct-attach
• CXFS/SAN-attach XFS/direct-attach
• all tests in lab and customer site havent shown
  any measureable performance degradation
• CXFS/SAN-attach compare to alternative sharing
  technologies on IRIX
• CXFS versus NFS(GigaBit Ethernet)
• NFS raw transfer speeds 5 - 60MB/s avrg. 25MB/s
• CXFS over FC 60 - 95MB/s avrg. 85MB/s

41
CXFS Performance

• Concept based considerations
• Multiple processes on multiple hosts that are
  reading and writing the same file using buffered
  I/O
• direct-access I/O (e.g. databases) are okay
• When there will be many metadata operations such
  as
• Opening and closing files
• Changing file sizes (usually extending a file)
• Creating and deleting files
• Searching directories

42
CXFS Scalability

• Supports up to 64 clients or servers per cluster
• IRIX 6.5.6 supports 8 clients
• Multiple metadata servers can exist in a cluster
• One per file system
• Files accessed exclusively locally on CXFS
  metadata server see local XFS metadata
  performance

43
CXFS Future Roadmap
CXFSIRIX/SNMIPSClient/Server
ongoing Feature/Maintenance Releases
1.0
1.1
CXFSLinux IAClient/Server
CXFSBeta
CXFS1.0
XFS-Beta
XFS
CXFSBeta
CXFS1.0
CXFS NTClient-only
SolarisAIX, HPUX
Other Clients
CY1999
CY2000
CY2001
44
CXFS Summary (1/3)

• Supports guaranteed-rate IO and real-time file
  systems
• For real-time and digital media applications
• NOT on IRIX 6.5.9
• Fast recovery times No fsck
• Avoids unnecessary writes by delaying writes as
  long as possible
• Contiguous allocation of disk space to avoid
  fragmentation
• 9 Peta Byte File System Size
• If historical trends continue, will last 60
  years

45
CXFS Summary (2/3)

• Fast directory searches
• Sparse file support
• Holes allowed in files for large direct-access
  addressing
• DMAPI for Hierarchical File Systems (HFS)
• Interfaces to SGIs Data Migration Facility
  (DMF)and third-party HSMs Veritas, FileServ,
  ADSM
• Available on IRIX 6.5.8

46
CXFS Summary (3/3)
Shared file systems provide the advantage of all
other sharing techniques.
Data
Data
Data
FS
Level of Sharing
Zoning, Partitioning LUN Masking
Physically Consolidated SAN-attached Storage
Physically Consolidated SAN-attached Storage
Direct Attached Storage
Direct Attached Storage
47
Agenda

• Introduction to Fibre Channel
• Promises of Storage Area Networks
• Delivering on the Promise Share Data
• Data Management in SANs
• SGIs SAN Status and Vision

48
Data Management in SANs

• Data Management
• LAN-free Backup
• Server-less Backup
• HSM

49
Data Management

• Amount of information needing protection is
  exploding
• Backup windows are shrinking
• Primary backup solutions on SGI platforms are
• Veritas NetBackup
• Legato Networker
• Need high performance migration to tape
• Veritas Storage Migrator
• DMF

50
LAN-free Backup

• Regular Backup
• Disadvantage
• Backup window depended
• LAN gets flooded by backup data.
• Server who owns the library is single point of
  failure

LAN
SCSI
51
LAN-free Backup

• LAN-free Backup to a SCSI-Library
• Benefits
• Via CXFS all file systems can be mounted to
  create global backup.
• No LAN traffic involved, offloads LAN for
  non-stop applications.

LAN
Private Network
SAN fabric
SCSI
52
LAN-free Backup

• LAN-free Backup with Fibre Channel Tape
• Benefits
• Any member of the SAN can share the Library
• No LAN traffic involved
• Library takes advantage of FC bandwidth.
• Disaster Recovery setups easier to implement

LAN
Private Network
SAN fabric
53
Server-less Backup

• Benefits
• Server just admins meta data and indices.
• Server has minimal load
• LAN is off-loaded
• Ideal for low bandwidth server architectures
• Disadvantages
• No standard, either ANSI/ISO nor de-facto
• Still early adopter phase.

LAN
Private Network
SAN fabric
54
HSM - Data Migration Facility (DMF)

• SGIs Hierarchically Storage Management (HSM)
• Storage Space Management
• Key parameters
• Free space minimum
• Free space target
• Migration target

Free Space
Dual State
Files
Regular
Files
Disk Cache
55
HSM - Data Migration Facility (DMF)

• Functional Overview
• Mission Maintain free space on disk
• Move old/big files to more efficient media
• Migrate files automatically or manually
• Maximize utilization of offline media
• DMF manages native file system data
• Inodes and directories are not migrated
• Administrator orientation is file system
• Administrator functions - Policy selection
• File ranking rules (candidate list)
• Capacity thresholds (file system monitor)
• Number and location of offline copies
• Tape management, journal backup and recovery
• Tape merging (garbage collection)

56
HSM - Data Migration Facility (DMF)

• Capacity and Performance
• Number of files is determined by file system
• Typical site -- 1 - 5M files
• Largest site today -- 5 - 10M files
• Largest tested capacity -- 50M files
• Practical issue -- inode space and database
  processing time
• Offline capacity is unlimited
• Typical site -- 10 to 50TB, and 100s TBs
• Largest site today -- 1PB ( and counting )
• Practical issue -- media density
• Data Rates
• Tunable media transport usage -- read/write
• Full, rated channel speeds -- async i/o
• Parallel writes/parallel reads
• Busiest site -- 7 to 10 TB/day and growing
• Practical issue -- balance between disk channels
  and tape channels

57
HSM - Data Migration Facility (DMF)

• DMF 2.6
• 25-40 dedicated file server machines
• Capacities ranging over 1PB
• Typical sites (10-25TB) (50-100sTB)
• Fully transparent access to offline data
• Maintain POSIX conformance
• Implemented as extension to native XFS file
  system
• Network access via NFS and FTP

58
HSM - Data Migration Facility (DMF)

• DMF Backup-API ships with DMF 2.6
• Open architecture, other vendors to follow
• No technical reason not to Open Source
• Will work with Networker 5.5 and up
• Resolves issues with migrated and dual state
  files
• With Networker will be able to backup
• dirs, inodes, online data special files (dump
  -a)
• Real-time fss, capabilities, ext attributes,
  ACLs
• No additional license fees (NW or DMF)

59
HSM - Data Migration Facility (DMF)

• Migration steps
• 1) User Process issues read
• 2) XFS notices file is offline
• 3) Process sleeps
• 4) DMF sends recall request via DMAPI
• 5) DMF determines residence
• 6) DMF sends request to MSP
• 7) MSP restores data
• 8) MSP responds to DMF daemon
• 9) DMF responds to kernel
• 10) Kernel wakes process
• Considerations
• unscheduled interrupts
• multiple requests
• atomic database references

60
Agenda

• Introduction to Fibre Channel
• Promises of Storage Area Networks
• Delivering on the Promise Share Data
• Data Management in SANs
• SGIs SAN Status and Vision

61
What is a SAN?- Full SGI Support - Today
Origin3000
Onyx2
HDS 7700E
AIT-2
Ciprico
DLT 7000
62
SGI FC Infrastructure
SGI Fibre Channel Hub (10-port single, 5-port
dual), LH5000 Cascade to support up to 126 active
FC-AL devices Maintain loop integrity, remote
administration
Industry leading Storage Area Networks switch SGI
resell Brocade Silkworm 2400 (8-port), Silkworm
2800(16-port) Certified with QLA2200 Full
cascading Fabric support
63
SGI TP 9100 Storage Array

Entry Level full Fibre Raid at SCSI Raid pricing

• 4U Deskside/Rack
• JBOD or RAID
• Simplex or Duplex Controller
• 12 Drives 18,4 GB (10k RPM),
• 36,7 GB (10K RPM),
• 73,4 GB (10K RPM)
  
• Dual Power / Dual Fans
• Deskside Kit and Rack
• JBOD 108 Disks
• RAID 32 (124) Disks / Controller Pair ,
  
  
• IRIX, NT, Linux , Windows 2000
• SAN ready

64
SGI TP9400

• Hot Scale Technology - Drives, Enclosures, Hubs.
• Path Redundancy to Drives
• Immediate LUN Availability
• Major Event Log for 8192 Events
• Dynamic Segment Sizing
• Dynamic Raid Level Migration
• Automatic Code Synchronization
• Non-disruptive Firmware download
• Email Notification

Read 357 MB/s Write 239 MB/s
Measured lab results Two controllers, 512MB
cache, 4 host channels, 4 back end paths,four
81 LUNs, 4 MB transfers
65
Hitachi (HDS) 7700e

• Large Scale 7.4 TB usable Capacity
• 16GB cache, 16 FC Ports
• Non-stop Availability
• Advanced Features
• Shadow Volume Kit
• Enables the Duplication of LUNs for off-line
  Backup, Testing, etc.
• Resource Manager 7000
• an SNMP-compliant Storage Resource Management
  Tool that includes
• Graph Track for graphical Performance
  Monitoring and Tuning
• Flash Access Manager enables data to belocked
  in Cache in Real-time
• LUN Manager allows for defining, configuring,
  adding, deleting.
  Resizing LUNs assigns
  LUNs to specific paths.
• Remote Copy Kit
• Provides synchronous/asynchronous Data
  Duplication to Remote Site

66
StorageTek Tape Libraries

• Timberwolf
• STK 9730
• STK 9714
• STK 9710

• Drive Types Up to 10 9840 Drives, DLT 7000
  Drives or a combination.
• Capacity 252, 420 or 588 (with expansion door)
  Cartr. Slots,
• DLT 7000 20.6 TB (35GB Cartr.)
• 9840 11.8 TB (20 GB Cartr.)
• Throughput
• 9840 360 GB/hr
• DLT 7000 180 GB/hr

Drive Types up to 6 DLT 7000 Drives.
Capacity 40,60,80 or 100 Cartr. Slots DLT 7000
3.5 TB (35GB Cartr.) Throughput DLT 7000 108
GB/hr

• Drive Types Up to four DLT 7000 Drives
• Capacity 20 or 30 Cartr. Slots (28 with four
  Drives)
• DLT 7000 1.05 TB (35GB Cartr.)
• Throughput DLT 7000 72 GB/h

67
STK L Series Libraries
Distributed Leadership Series
L3500 325-600 Slots/Module Up to 120 Drives
L700 270-700 Slots 20 Drives
L180 60-180 Slots 10 Drives
Performance Leadership Series
L80 80 Slots 8 Drives
L40 20-40 Slots 4 Drives
L20 10-20 Slots 1-2 Drives
68
High Bandwidth Servers ...
SGI 2800 64-512 proc
SGI 2400 2-64 proc
I/O Bandwidth 320GB/sec sustained
I/O Bandwidth 40GB/sec sustained
I/O Bandwidth 5GB/sec sustained
Origin 200
I/O Bandwidth 5GB/sec sustained
I/O Bandwidth 1,26GB/sec sustained
69
What is a SAN?- Full SGI Support - Future
Irix Servers
Solaris Servers
Irix Workstations
HPUX Servers
Linux/NT
AIX servers
SAN fabric
Linux/NT/Win2K
HDS 7700E
9840
AIT-2
TP9400
DLT 7000
70
Questions?

• Thomas Feil
• thomasf_at_sgi.com
• THANK YOU !

71
(No Transcript)