Designing Large Memories Using Hardware Prototyping

1
Designing Large Memories Using Hardware
Prototyping

• Myles Watson
• Kelly Flanagan

2
The Next 15 Minutes

• Research Questions
• DiskRAM
• Search for Hardware
• Gotchas
• Benefits
• Wrap up

3
Starting Point

• Memory needs continue to increase
• Memory bigger is better
• Single-level store
• Disk, memory, intelligence
• Not just swap

4
Dynamically Tracking Page Miss Ratio Curves (Zhou
et al.)

• Adds 7-10 overhead
• Reduces page faults 30-50
• Reduces response time 2x-6x
• Sped up non-interactive programs
• Less disk contention

5
Research Questions

• How does the system change
• With 200 GB of memory?
• To enable persistence?
• With flash, disk, or other storage technology?
• Can we improve RAM Disk interactions?
• Swapping
• File mapping
• Disk caching

6
Why Not Simulate?

• Complexity
• Determinism
• Time
• Benchmark Constraints

7
DiskRAM
Prototype Controller
8
Requirements

• Custom memory controller i.e.
• Maps into large physical address space
• Emulates various storage types
• Large amount of total storage
• Feasible
• Price, complexity, speed, etc.

9
Reality Check

• Memory controllers are proprietary
• AMDs is integrated with processor
• Intel has proprietary bus

10
Opteron Replacement
Remove RAM
11
Cray XD1
12
Cray XD1
DiskRAM
13
Hot-Pluggable Memory Board

• On-board controller
• Standard DIMMS

14
Hot-Pluggable Memory Board

• Proprietary bus
• Not latency tolerant
• enough for disk

15
FPGA Board

• Virtex 4 FPGA
• 8-lane PCI-Express
• DDR2 SODIMM
• SATA, USB, Ethernet, GPIO,

16
Commercial Boards
XtremeData Inc XD1000
DRC Computer Coprocessor
17
Difference
18
Revised Block Diagram
19
Enabling Technologies

• LinuxBIOS
• Opteron
• 40-bit physical addresses
• HyperTransport
• Glueless multiprocessor support

20
Gotchas

• Slowdown (caches help)
• Slower DDR controller
• Slower HyperTransport
• Extra latency
• Relative system performance

21
Benefits

• Abstraction
• Transparency
• Known goods/support
• Focus prototyping effort

22
Wrap Up
23
Questions?
24
Global Optimization Disk and Memory

• OS cant see disk position
• OS cant see hits in memory
• OS expected to optimize both!
• Result Thrashing when memory is low

25
Persistence

• Fast and cheap hibernate
• No data translation for permanence
• Different view of the file system
• Configuration files
• Preferences
• Temporary files
• Backups