PARAID: A Gear-Shifting Power-Aware RAID

1
PARAIDA Gear-ShiftingPower-Aware RAID

• Charles Weddle, Mathew Oldham, Jin Qian, An-I
  Andy Wang Florida St. University
• Peter Reiher University of California, Los
  Angeles
• Geoff Kuenning Harvey Mudd College

2
Motivation

• Energy costs are rising
• An increasing concern for servers
• No longer limited to laptops
• Energy consumption of disk drives
• 24 of the power usage in web servers
• 27 of electricity cost for data centers
• Root to other issues, e.g. server room cooling
• Is it possible to reduce energy consumption in
  RAID devices without degrading performance while
  maintaining reliability?

PARAID A Gear-Shifting Power-Aware RAID
3
Challenges

• Energy
• Not enough opportunities to spin down RAIDs
• Performance
• Essential for peak loads
• Reliability
• Server-class drives are not designed for frequent
  power switching

PARAID A Gear-Shifting Power-Aware RAID
4
Existing Work

• Most trade performance for energy savings
  directly.
• For example, vary the speed of disks
• Most are simulated results

PARAID A Gear-Shifting Power-Aware RAID
5
Observations

• Over provisioning of resources
• RAID is configured for peak performance
• RAID keeps all drives spinning for light loads
• Unused storage capacity
• Over-provision of storage capacity
• Unused storage can be traded for energy savings
• Fluctuating load
• Cyclic fluctuation of loads
• Infrequent on-off power transitions can be
  effective

PARAID A Gear-Shifting Power-Aware RAID
6
Power-Aware RAID

• Skewed striping for energy savings
• Preserving peak performance
• Maintaining reliability
• Evaluation
• Conclusion
• Questions

PARAID A Gear-Shifting Power-Aware RAID
7
Skewed Striping for Energy Saving

• Use over-provisioned spare storage
• Can use fewer disks for light loads

PARAID A Gear-Shifting Power-Aware RAID
8
Skewed Striping for Energy Saving

• Operate in gear 1
• Disks 4 and 5 are powered off

1
2
3
4
5
Soft State
RAID
Gears
1
2
3
PARAID A Gear-Shifting Power-Aware RAID
9
Skewed Striping for Energy Saving

• Approximate the workload
• Gear shift into most appropriate gear
• Minimize the opportunity lost to save power

Conventional RAID
PARAID
Energy ( Powered On Disks )
workload
Workload ( Disk Parallelism )
PARAID A Gear-Shifting Power-Aware RAID
10
Skewed Striping for Energy Saving

• Adapt to cyclic fluctuating workload
• Gear shift when gear utilization threshold is met

PARAID A Gear-Shifting Power-Aware RAID
11
Preserving Peak Performance

• Operate in the highest gear
• When the system demands peak performance
• Uses the same disk layout
• Maximize parallelism within each gear
• Load is balanced on each gear
• Uniform striping pattern within each gear
• Delay block replication until gear shifts
• Track block writes as an optimization

PARAID A Gear-Shifting Power-Aware RAID
12
Maintaining Reliability

• Reuse existing RAID levels (RAID-5)
• Also used in soft state
• Drives have a limited number of power cycles
• Ration number of power cycles

PARAID A Gear-Shifting Power-Aware RAID
13
Logical Component Design
PARAID A Gear-Shifting Power-Aware RAID
14
Data Layout

• Parity for 5 disks does not work for 4 disks
• For example, replicated block 12 on disk 3

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Soft State (RAID-5) (1-4) 8 12 ((1-4),8,12)
Soft State (RAID-5) 16 20 (16,20,_) _
RAID-5 1 2 3 4 (1-4)
RAID-5 5 6 7 (5-8) 8
RAID-5 9 10 (9-12) 11 12
RAID-5 13 (13-16) 14 15 16
RAID-5 (17-20) 17 18 19 20
PARAID A Gear-Shifting Power-Aware RAID
15
Data Layout

• Cascading parity updates
• Must also update parity in soft state

Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Soft State (RAID-5) (1-4) 8 12 ((1-4),8,12)
Soft State (RAID-5) 16 20 (16,20,_) _
RAID-5 1 2 3 4 (1-4)
RAID-5 5 6 7 (5-8) 8
RAID-5 9 10 (9-12) 11 12
RAID-5 13 (13-16) 14 15 16
RAID-5 (17-20) 17 18 19 20
PARAID A Gear-Shifting Power-Aware RAID
16
Update Propagation

• Up-shift propagation
• Full synchronization
• On-demand synchronization
• For example, shifting from 3 to 5 disks
• Downshift propagation
• Full synchronization

PARAID A Gear-Shifting Power-Aware RAID
17
Asymmetric Gear-Shifting Policies

• Up-shift (aggressive)
• Moving average moving standard deviation gt
  threshold
• Downshift (conservative)
• Modified moving average moving standard
  deviation lt threshold
• Moving average modified to account for extra
  parity updates

PARAID A Gear-Shifting Power-Aware RAID
18
Implementation

• Prototyped in Linux 2.6.5
• Open source, software RAID
• Implemented block I/O handler, monitor, disk
  manager
• Implemented user admin tool to configure device
• Updated Raid Tools to recognize PARAID level

PARAID A Gear-Shifting Power-Aware RAID
19
Evaluation

• Challenges
• Prototyping PARAID
• Commercial machines
• Benchmarks are designed to measure peak
  performance
• Trace replay
• Time consuming

PARAID A Gear-Shifting Power-Aware RAID
20
Evaluation

• Measurement framework

PARAID A Gear-Shifting Power-Aware RAID
21
Evaluation

• Measurement framework

  Server Client
Processor Intel Xeon 2.8 Ghz Intel Pentium 4 2.8 Ghz
Memory 512 Mbytes 1 Gbytes
Network Gigabit Ethernet Gigabit Ethernet
Disks 36.7Gbytes 15K RPM SCSI Ultra 320 160 Gbytes 7200 RPM SATA
PARAID A Gear-Shifting Power-Aware RAID
22
Evaluation

• Three different workloads using two different
  RAID settings
• Web trace - RAID level 0 2 gears (2,5)
• Mostly read activity
• Cello99 - RAID level 5 2 gears (3,5)
• I/O intensive workload with writes
• Postmark - RAID level 5 2 gears (3,5)
• Measure peak performance and gear shifting
  overhead
• Speed up trace playback
• To match hardware
• Explore range of speed up factors and power
  savings

PARAID A Gear-Shifting Power-Aware RAID
23
Web Trace

• UCLA CS Dept Web Servers (8/11/2006 8/14/2006)
• File system 32 GB (500k files)
• Trace replay 95k requests with 4 GB data (260
  MB unique)

PARAID A Gear-Shifting Power-Aware RAID
24
Web Trace Energy Savings
64x 60 requests/sec
Energy Savings
64x - 34
128x - 28
256x - 10
128x 120 requests/sec
256x 240 requests/sec
PARAID A Gear-Shifting Power-Aware RAID
25
Web Trace Latency
256x
Overhead
256x - within 2.7
64x - 240 80ms vs. 33ms
128x
64x
PARAID A Gear-Shifting Power-Aware RAID
26
Web Trace Bandwidth
256x
Overhead
256x - within 1.3 in high gear
128x
64x
PARAID A Gear-Shifting Power-Aware RAID
27
Cello99 Trace

• Cello99 Workload
• HP Storage Research Labs
• 50 hours beginning on 9/12/1999
• I/O intensive with 42 writes

PARAID A Gear-Shifting Power-Aware RAID
28
Cello99 Energy Savings
32x 270 requests/sec
Energy Savings
32x - 13
64x - 8.2
128x - 3.5
128x 1000 requests/sec
64x 550 requests/sec
PARAID A Gear-Shifting Power-Aware RAID
29
Cello99 Completion Time
128x
Overhead
32x - 1.8ms, 26 slower due to time spent in
low gear
64x
32x
PARAID A Gear-Shifting Power-Aware RAID
30
Cello99 Bandwidth
128x
Overhead
lt 1 degra- dation during peak hours
64x
32x
PARAID A Gear-Shifting Power-Aware RAID
31
Postmark Benchmark

• Popular synthetic benchmark
• Generates ISP-style workloads
• Stresses peak read/write performance of storage
  device

PARAID A Gear-Shifting Power-Aware RAID
32
Postmark Performance
PARAID A Gear-Shifting Power-Aware RAID
33
Postmark Power Measurements
PARAID A Gear-Shifting Power-Aware RAID
34
Ongoing Work

• Try more workloads
• Optimize PARAID gear configuration
• Explore asynchronous update propagation
• Speed up recovery
• Live testing

PARAID A Gear-Shifting Power-Aware RAID
35
Lessons Learned

• Third version of design, early design not
  portable
• Data alignment problems
• Difficult to measure system under normal load
• Hardware and operating system optimizations
• Matching trace environment

PARAID A Gear-Shifting Power-Aware RAID
36
Conclusion

• PARAID reuses standard RAID-levels without
  special hardware while decreasing their energy
  use by 34.
• Optimized version can save even more energy
• Empirical evaluation important

PARAID A Gear-Shifting Power-Aware RAID
37
Questions
PARAID A Gear-Shifting Power-Aware RAID
38
PARAID Gear-Shifting
Web Trace Gear-Shifting Stats
256x 128x 64x
Number of gear switches 15.2 8.0 2.0
time spent in low gear 52 88 98
extra I/Os for update propagations 0.63 0.37 0.21
Cello99 Gear-Shifting Stats
128x 64x 32x
Number of gear switches 6.0 5.6 5.4
time spent in low gear 47 74 88
extra I/Os for update propagations 8.0 15 21
39
Storage Requirement for PARAID5
Storage consumption Si for the total RAID for the
ith gear

• PARAID uses around (D G1)/(D 1) of the total
  RAID-5 storage to store soft states.
• For RAID-5, D gt 3 disks, M gears with Gi disks
  within the ith gear (1 lt i lt M, 3 lt Gi lt Gi1 lt
  GM D)

40
Storage Requirement for PARAID5
Space needed for extra parity blocks
S1(G1 1)
G2 5
G1 4
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Gear 1 RAID-5 (1-4) 8 12 ((1-4),8,12)
Gear 1 RAID-5 16 20 (16,20,_) _
Gear 2 RAID-5 1 2 3 4 (1-4)
Gear 2 RAID-5 5 6 7 (5-8) 8
Gear 2 RAID-5 9 10 (9-12) 11 12
Gear 2 RAID-5 13 (13-16) 14 15 16
Gear 2 RAID-5 (17-20) 17 18 19 20
S1
S2
S1 S2 1
S1(G1 1) S2(G2 G1)
S2(G2 G1)
41
PARAID
Target percentage energy savings

• Energy savings increase with more disks and
  fewer disks in the lowest gear.
• A higher active/standby ratio.
• D the number of disks in the array
• G1 the disks in gear 1.
• P power in standby/active/idle

42
PARAID
Modified moving utilization

• Accounts for cascading parity writes in lower
  gear.
• Gi the disks in gear i.
• Aread/write the read or write activity
• W the weight to account for additional parity
  writes
• (RAID5 1.5)

43
PARAID
max min configuration empty storage max energy savings formula
4 3 3-4 33.00 25.00 0.333333
5 3 3-5 50.00 40.00 0.5
5 4 4-5 25.00 20.00 0.25
5 3 3-4-5 50.00 40.00 0.5
6 3 3-6 60.00 50.00 0.6
6 4 4-6 40.00 33.00 0.4
6 5 5-6 20.00 17.00 0.2
6 3 3-4-6 60.00 50.00 0.6
6 3 3-5-6 60.00 50.00 0.6
6 4 4-5-6 40.00 33.00 0.4
6 3 3-4-5-6 60.00 50.00 0.6
7 3 3-7 66.67 57.14 0.666667
7 4 4-7 50.00 42.86 0.5
7 5 5-7 33.00 28.57 0.333333
7 6 6-7 16.67 14.29 0.166667