Token Coherence: Decoupling Performance and Correctness

1
Token Coherence Decoupling Performance and
Correctness

• Milo M. D. Martin
• Mark D. Hill
• David A. Wood
• University of Wisconsin-Madison
• ISCA-30 (2003)

2
Outline

• Token Coherence Basics
• What Is Token Coherence?
• What Are Its Advantages?
• Review of Snooping and Directory-Based Coherence
  Schemes
• Token Coherence Details
• Evaluation Results and Conclusions

3
Token Coherence Basics

• Decouple Interconnect Performance and Protocol
  Correctness
• Remember Amdahls Law Go Faster In Most Cases At
  Expense Of Occasional, More Expensive Corner Case
  Handling
• Fast, Unordered Bus For Protocol Traffic
• Races Allowed, Handling Built-In, But Deferred
• Cache Lines Have Tokens To Track State
• Token Ownership Correlates With Coherence State
  (MOSI)

4
Token Coherence Benefits

• Fast Bus Makes Common Case Fast
• Some Additional Bus Traffic (Not Too Much)
• Common Is Very Common (95)
• Benefits Of Snooping And Directories
• Unordered Bus Messages (Directories)
• Cache-Cache Transfers Without Indirection
  (Snooping)

5
Outline

• Token Coherence Basics
• What Is Token Coherence?
• What Are Its Advantages?
• Review of Snooping and Directory-Based Coherence
  Schemes
• Token Coherence Details
• Evaluation Results and Conclusions

6
Coherence Review Snooping

• Requests Broadcast, All Agents Snoop
• One-Hop Communication (Low Latency)
• Every Agent Processes Every Message
• Bus Is Point Of Synchronization (Total Order)
• Hard To Scale To Higher Speeds/Larger Systems

Total Order of Events 1 Before 2 Before 3 To All
Bus Agents
P0
P1
P2
P3
1 2 3
1 2 3
1 3
2 3
1
2
Mem
2 1
3
7
Coherence Review Directories

• Directory Manages Ordering Of Requests
• Bus Can Be Faster (Unordered)
• Adds Level Of Indirection To Cache-Cache
  Transactions (Extra Latency)
• Messages Addressed To Specific Agents

Bus Order Irrelevant, Ordering At Directory 1
Before 2 Before 3
P0
P1
P2
P3
3
3
1
2
3
3
Dir
2 1
3
8
Outline

• Token Coherence Basics
• What Is Token Coherence?
• What Are Its Advantages?
• Review of Snooping and Directory-Based Coherence
  Schemes
• Token Coherence Details
• Evaluation Results and Conclusions

9
Token Coherence Goals

• Achieve Efficiency Of Snooping
• Low-Latency Inter-Cache Transfers
• Keep Advantage Of Directories
• Unordered, Fast Interconnects Allowed
• How?
• Use Of Tokens To Implement Correctness
  Substrate And Protocol(s)
• Tokens Associated With Each Cache Line
• At Least N Tokens Per Line (N Processors)

10
Correctness Substrate

• Enforce Safety Count Tokens
• Tokens Preserved (Always T Tokens Per Line)
• Must Hold All Tokens To Write
• Must Hold One Token To Read
• If A Message Has A Token, It Must Have Data
• Optimization Add Special Owner Token
• Tokens Preserved With One Owner Token Per Line
• Must Hold All Tokens To Write
• Must Hold One Token And Valid Data To Read
• If A Message Has Owner Token, It Must Have Data

11
A Token Coherence Protocol

• TokenB Token Coherence Using Broadcast
• Processors Broadcast Transient Token Requests
• Snooping Mayhem Ensues Without Total Ordering Of
  Transactions On Bus
• Races Allowed To Occur Unanswered Messages
  Retried
• Last Resort (After Several Retries) Is
  Aptly-Named Persistent Request

12
Outline

• Token Coherence Basics
• What Is Token Coherence?
• What Are Its Advantages?
• Review of Snooping and Directory-Based Coherence
  Schemes
• Token Coherence Details
• Evaluation Results and Conclusions

13
Performance Evaluation

• Compared To Snooping, Directories, And Glueless
  Estimated Protocol
• Simulation Results Show
• Token Coherence Works, Performs Well, Even
• For 16-Processor System, gt95 Of First Transient
  Requests Succeed With TokenB
• Potentially Significant Additional Traffic
  Generated Over Directory
• Not Ultimately As Scalable As Directories

14
Conclusions And Discussion

• Token Coherence Provides Novel Solution To
  Snooping/Directory Hybrid Optimization Problem
• Is Additional Space For Token Storage An Issue?
• Why Didnt Hammer/21364 Use This?
• With Glueless A Trend, Is This Doomed To
  Academia?
• How Does This Compare To Multicast Snooping?