ThreeDimensional Layout of OnChip TreeBased Networks

1
Three-Dimensional Layout of On-Chip Tree-Based
Networks

• Hiroki Matsutani (Keio Univ, Japan)
• Michihiro Koibuchi (NII, Japan)
• D. Frank Hsu (Fordham Univ, USA)
• Hideharu Amano (Keio Univ, Japan)

2
Outline

• Introduction
• Network-on-Chip (NoC)
• 2-D vs. 3-D
• Fat Tree
• 2-D layout
• 3-D layout
• Fat H-Tree
• 2-D layout
• 3-D layout
• Evaluations
• Area, Wire length, Energy

Matsutani, IPDPS07
3
Network-on-Chip (NoC)

• Tile architectures
• MIT RAW
• Texas U. TRIPS
• Intel 80-tile NoC
• Various topologies
• Mesh, Torus
• Fat Trees
• Fat H-Tree (FHT)

Tile (core router)
Taylor, Micro02
Buger, Computer04
Vangal, ISSCC07
16-core Tile architecture
Packet switched network on a chip
We proposed FHT as an alternative to Fat Trees
Matsutani, IPDPS07
4
2D Topologies Mesh Torus

• 2-D Mesh

• 2-D Torus
• 2x bandwidth of mesh

RAW Taylor, IEEE Micro02
Router
Core
5
2D Topologies Fat Tree
p of upward links q of downward links c
of core ports

• Fat Tree (p, q, c)

Rank-1
Rank-2
In this talk, we focus on 3-D layout scheme of
tree-based topologies
Fat Tree (2,4,2)
Fat Tree (2,4,1)
Router
Core
6
2D NoC vs. 3D NoC

• 2D NoCs
• Long wires (esp. trees)
• Wire delay
• Packets consume power at links according to their
  wire length

• 3D NoCs
• Several small wafers or dices are stacked
• Vertical link
• Micro bump
• Through-wafer via
• Very short (10-50um)

Long horizontal wires in 2D NoCs can be replaced
by very short vertical links in 3D NoCs
Ezaki, ISSCC04
Burns, ISSCC01
Next slides show the 3D layout scheme of Fat Tree
and FHT
7
Outline

• Introduction
• Network-on-Chip (NoC)
• 2-D vs. 3-D
• Fat Tree
• 2-D layout
• 3-D layout
• Fat H-Tree
• 2-D layout
• 3-D layout
• Evaluations
• Area, Wire length, Energy

Matsutani, IPDPS07
8
Fat Tree 2-D layout
p of upward links q of downward links c
of core ports

• Fat Tree (p, q, c)

Fat Tree (2,4,2)
Fat Tree (2,4,1)
Router
Core
We preliminarily show the 3D layout scheme of Fat
Trees
9
Fat Tree 3-D layout (4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Top-rank routers are distributed to each layer
Dividing into 4 layers
Original 2-D layout
10
Fat Tree 3-D layout (4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Top-rank links are replaced with vertical
interconnects (10-50um)
Layer-0
This 3-D layout is evaluated in terms of area,
wire, energy
3-D layout (4-stacked)
Original 2-D layout
11
Outline

• Introduction
• Network-on-Chip (NoC)
• 2-D vs. 3-D
• Fat Tree
• 2-D layout
• 3-D layout
• Fat H-Tree
• 2-D layout
• 3-D layout
• Evaluations
• Area, Wire length, Power

Matsutani, IPDPS07
12
Fat H-Tree Structure

• Fat H-Tree
• Red Tree (H-Tree)
• Black Tree (H-Tree)

Matsutani, IPDPS07
Combining two H-Trees (red black)
By shifting the location of black tree, the
connection pattern of trees is different from the
original Fat Trees
Router
Core
Router
Core
13
Fat H-Tree Structure

• Fat H-Tree
• Red Tree (H-Tree)
• Black Tree (H-Tree)

Matsutani, IPDPS07
Combining two H-Trees (red black)
Fat H-Tree is formed on red black trees
Router
Core
Router
Core
14
Fat H-Tree Structure

• Fat H-Tree
• Red Tree (H-Tree)
• Black Tree (H-Tree)

Matsutani, IPDPS07
Combining two H-Trees (red black)
Fat H-Tree is formed on red black trees
Router
Core
Router
Core
15
Fat H-Tree Structure

• Fat H-Tree
• Red Tree (H-Tree)
• Black Tree (H-Tree)

Matsutani, IPDPS07
Combining two H-Trees (red black)
Fat H-Tree is formed on red black trees
Router
Core
Router
Core
16
Fat H-Tree Structure

• Fat H-Tree
• Red Tree (H-Tree)
• Black Tree (H-Tree)

Matsutani, IPDPS07
Combining two H-Trees (red black)
Each core is connected to both red black trees
Rank-2 or upper routers are omitted in this figure
Router
Core
Router
Core
17
Fat H-Tree 2-D layout on VLSI

• Fat H-Tree
• Torus structure
• ? Folded as well as the folded layout of 2-D Torus

Matsutani, IPDPS07
(Long feedback links across the chip)
Topologically equivalent
Fat H-Trees 2-D layout
The next slides propose the 3D layout scheme of
Fat H-Tree
Router
Core
18
Fat H-Tree 3-D layout (overview)

• Fat H-Tree
• (Problem) Fat H-Tree has a torus structure
• Folding so as to keep the torus structure

consisting of red black trees
(step 1) fold it horizontally (step 2) fold it
vertically
Until the of folded pieces meets the of
layers the 3-D IC has
E.g., four layers ? fold twice
19
Fat H-Tree 3-D layout (overview)

• Fat H-Tree
• (Problem) Fat H-Tree has a torus structure
• Folding so as to keep the torus structure

consisting of red black trees
(step 1) fold it horizontally (step 2) fold it
vertically
Until the of folded pieces meets the of
layers the 3-D IC has
E.g., four layers ? fold twice
20
Fat H-Tree 3-D layout (overview)

• Fat H-Tree
• (Problem) Fat H-Tree has a torus structure
• Folding so as to keep the torus structure

consisting of red black trees
(step 1) fold it horizontally (step 2) fold it
vertically
Until the of folded pieces meets the of
layers the 3-D IC has
E.g., four layers ? fold twice
Here we show the 3D layouts of red black trees
separately
21
Fat H-Tree 3-D (Red tree 4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Original 2-D layout
22
Fat H-Tree 3-D (Red tree 4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Top-rank links are replaced with vertical
interconnects (10-50um)
Layer-0
3-D layout (4-stacked)
Original 2-D layout
23
Fat H-Tree 3-D (Black tree4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Original 2-D layout
24
Fat H-Tree 3-D (Black tree4-split)

• 2-D coordinates

• 3-D coordinates

transformation
The periphery cores are connected to different
layers
Original 2-D layout
25
Fat H-Tree 3-D (Black tree4-split)

• 2-D coordinates

• 3-D coordinates

transformation
Top-rank links are replaced with vertical
interconnects (10-50um)
Layer-0
The periphery cores are connected to different
layers
3-D layout (4-stacked)
Original 2-D layout
26
Fat H-Tree 3-D layout (4-split)
Layer-0
Layer-0
Layer-0
Red tree (3-D)
Black tree (3-D)
Fat H-Tree (3-D)
The 3-D layout of Fat H-Tree can be formed
by superimposing 3-D layouts of red black
trees
27
Outline

• Introduction
• Network-on-Chip (NoC)
• 2-D vs. 3-D
• Fat Tree
• 2-D layout
• 3-D layout
• Fat H-Tree
• 2-D layout
• 3-D layout
• Evaluations
• Area, Wire length, Energy

Matsutani, IPDPS07
28
Evaluations 2-D vs. 3-D

• 2-D layout
• 64-core

• 3-D layout
• 16-core x 4-layer
• Vertical interconnects

L mm
L/2 mm
29
Network logic area of routers
FT1 Fat tree(2,4,1) FT2 Fat tree(2,4,2) FHT
Fat H-Tree

• 3-D mesh/torus node degree 7
• Fat H-Tree node degree 5
• Fat Tree (2,4,2) node degree 6

of routers their ports in trees are less than
mesh/torus
30
Network logic area 2-D vs. 3-D

• Wormhole router
• 1-flit 64-bit
• 3-stage pipeline
• Network interface
• FIFO buffer
• Packet forwarding (Fat H-Tree only)
• Inter-wafer via
• 1-10um square
• 100um per layer per 1-bit signal

• Network logic area
• Routers, NIs
• Inter-wafer vias

Arbiter
FIFO
Davis, DToC05
5x5 XBAR
FIFO
2
Typical wormhole router
Matsutani, ASPDAC08
Inter-wafer via area is calculated according to
of vertical links
Synthesized with a 90nm CMOS
31
Network logic area Overhead of 3D
Synthesis result of 64-core (16-core x 4)
3D layout of trees ? area overheat is modest (at
most 7.8)
FT1 Fat Tree(2,4,1) FT2 Fat Tree(2,4,2)
FHT Fat H-Tree
32
Total wire length of all links

• Total unit-length of links
• Core router
• Router router

How many unit-links is required ?
1-unit distance between neighboring cores
1-unit link
1-unit link
33
Total wire length of all links
1-unit
FT1 Fat Tree(2,4,1) FT2 Fat Tree(2,4,2)
FHT Fat H-Tree
34
Total wire length of all links
1-unit
1-unit
4-stacked
Wire length of trees is reduced by 25-50 (close
to torus)
FT1 Fat Tree(2,4,1) FT2 Fat Tree(2,4,2)
FHT Fat H-Tree
35
Energy NoCs energy model

• Ave. flit energy
• Send 1-flit to dest.
• How much energyJ ?

• Parameters
• 8mm square chip
• 64-core (16-core x 4)
• 90nm CMOS
• Switching energy
• 1-bit switching _at_ Router
• Gate-level sim
• 0.183 pJ / hop
• Link energy
• 1-bit transfer _at_ Link
• 0.150 pJ / mm
• Via energy
• 4.34 fF / via

8mm
Davis, DToC05
36
Energy Reduction by going 3D
2-D layout
Frequent use of longest links
Short hop count ? less energy
FT1 Fat tree(2,4,1) FT2 Fat tree(2,4,2) FHT
Fat H-Tree
37
Energy Reduction by going 3D
2-D layout
3-D layout
Moving distance of packets is reduced
FT1 Fat tree(2,4,1) FT2 Fat tree(2,4,2) FHT
Fat H-Tree
The 3D layout of trees reduces the energy by
30.8-42.9
38
Summary 3-D layout of trees

• Drawbacks of on-chip tree-based topologies
• Long links around the root of tree
• Wire delay problem
• Repeater insertion ? additional energy
  consumption
• 3-D layout schemes of Fat Trees Fat H-Tree
• Wire length is reduced by 25-50
• Area overhead is at most 7.8
• Flit transmission energy is reduced by
  30.8-42.9

In addition, energy-hungry repeater buffers can
be removed
Need to consider negative impacts of 3-D
(cost,heat,yield)
39
Thank you for your attention
40
Backup slides
41
Energy Reduction by going 3D
2-D layout (w/o repeaters)
2-D layout (with repeaters)
()
Energy is increased
FT1 Fat tree(2,4,1) FT2 Fat tree(2,4,2) FHT
Fat H-Tree
() Repeater insertion model N. Weste et.al,
CMOS VLSI Design (3rd ed), 2005.