Robust Design of Air Cooled Server Cabinets

1
Robust Design of Air Cooled Server Cabinets

• Nathan Rolander, Jeff Rambo, Yogendra Joshi,
  Farrokh Mistree
• ASME InterPACK Conference
• 19 July 2005

Systems Realization Laboratory
Support for this work provided by the members of
CEETHERM
Microelectronics Emerging Technologies Thermal
Laboratory
METTL
2
Background What is a data center?

• 10,000-500,000 sq. ft. facilities filled with
  cabinets which house data processing equipment,
  servers, switches, etc.
• Tens to hundreds of MW power consumption for
  computing equipment and associated cooling
  hardware
• Trend towards very high power density servers (30
  kW/cabinet) requiring stringent thermal management

1
Image B. Tschudi, Lawrence Berkeley
Laboratories
3
Introduction Motivation

• Up to 40 of data center operating costs can be
  cooling related
• Cooling challenges are compounded by a lifecycle
  mismatch
• New computer equipment introduced 2 years
• Center infrastructure overhauled 25 years

How do we efficiently integrate high powered
equipment into an existing cabinet infrastructure
while maximizing operational stability?
2
Source W. Tschudi, Lawrence Berkeley
Laboratories
4
Cabinet Design Challenges

• Flow complexity
• The turbulent CFD models required to analyze the
  air flow distribution in cabinets are impractical
  to use iterative optimization algorithms
• Operational stability
• Variations in data center operating conditions,
  coupled with model inaccuracies mean computed
  optimal solutions do not translate to efficient
  or feasible physical solutions
• Multiple design objectives
• Objectives of efficient thermal management,
  cooling cost minimization, operational
  stability are conflicting goals

3
5
Approach Overview

• Integration of three constructs to tackle cabinet
  design challenges

4
6
Introduction to the POD

• Modal expansion of basis functions,
• Fit optimal linear subspace through
• a series of system observations, .
• Maximize the projection of the basis functions
  onto the observations

f
u
5
7
Introduction to the POD

• Modal expansion of basis functions,
• Fit optimal linear subspace through
• a series of system observations, .
• Maximize the projection of the basis functions,
  onto the observations

f
u
Constrained variational calculus problem
lt , gt denotes ensemble averaging
( , ) denotes L2 inner product
5
8
Introduction to the POD

• Modal expansion of basis functions,
• Fit optimal linear subspace through
• a series of system observations, .
• Maximize the projection of the basis functions
  onto the observations

f
u
Assemble observations covariance matrix
lt , gt denotes ensemble averaging
( , ) denotes L2 inner product
5
9
Introduction to the POD

• Modal expansion of basis functions,
• Fit optimal linear subspace through
• a series of system observations, .
• Maximize the projection of the basis functions
  onto the observations

f
u
lt , gt denotes ensemble averaging
Take cross correlation tensor of covariance matrix
( , ) denotes L2 inner product
5
10
Introduction to the POD

• Modal expansion of basis functions,
• Fit optimal linear subspace through
• a series of system observations, .
• Maximize the projection of the basis functions
  onto the observations

f
u
lt , gt denotes ensemble averaging
Take eigen-decomposition of the cross-correlation
tensor
( , ) denotes L2 inner product
5
11
POD Based Turbulent Flow Modeling

• Vector-valued eigenvectors form empirical basis
  of m-dimensional subspace, called POD modes
• Superposition of modes used to reconstruct any
  solution within the range of observations 10
  error
• Flux matching procedure applied at boundaries gtgt
  areas of known flow conditions, resulting in the
  minimization problem
• Values of found using method of least squares
• Resulting model has O(105) reduction in DoF

G is the flux goal
F(.) is contribution to boundary flux from the
POD modes
a is the POD mode weight coefficient
ai
6
see Rambo HT2005-72143 paper for complete
analysis
12
Robust Design Principles

• Determine superior solutions through minimizing
  the effects of variation, without eliminating
  their causes.
• Type I minimizing variations in performance
  caused by variations noise factors
  (uncontrollable parameters)
• Type II minimizing variations in performance
  caused by variation in control factors (design
  variables)
• A common implementation of Type I robust design
  is Taguchi Parameter Design

7
13
Robust Design Application

• Goals

Y
Objective Function
X
Design Variable
8
14
Robust Design Application

• Constraints

X2
Feasible Design Space
Design Variable
Constraint Boundary
X1
Design Variable
9
15
The Compromise DSP Mathematics

• Hybrid of Mathematical Programming and Goal
  Programming optimization routines

 
10
16
Problem Geometry

• Enclosed Cabinet containing 10 servers
• Cooling air supplied from under floor plenum

Cabinet Profile
Server Profile
11
17
Cabinet Modeling

• 9 Observations of Vin 00.252 m/s for POD
• k-e turbulence model for RANS implemented in
  commercial CFD software
• Finite difference energy equation solver used for
  thermal solution, using POD computed flow field
• 1 iteration 12 sec

Vin 0.95 m/s
12
18
Design Variables Objectives
Server Cabinet Model
13
19
Design Variables Objectives
Server Cabinet Model
13
20
Design Variables Objectives
iterate
Server Cabinet Model
13
21
Results

• Baseline vs. Maximum efficient power dissipation

• Without server power re-distribution, increasing
  flow of cooling air alone is ineffective

14
22
Results

• Inlet air velocity vs. Total cabinet power level

• Cooling air is re-distributed to different
  cabinet sections depending upon supply rate gtgt
  server cooling efficiency

15
23
Results

• Maximum chip temperature and bounds

• Maximum chip temperature constraint met as
  variation in response changes with varying power
  flow rates

16
24
Conclusions
How do we efficiently integrate high powered
equipment into an existing cabinet infrastructure
while maximizing operational stability?
17
25
Conclusions

• For the typical enclosed cabinet modeled, over
  50 more power than baseline can be reliably
  dissipated through efficient configuration
• Robust solutions account for variability in
  internal external operating conditions, as well
  as a degree of modeling assumptions inaccuracies
• Server cabinet configuration design can be
  accomplished without center level re-design

17
26
Questions?

• Thank you for attending!

18
27
Final Validation

• Comparison of results obtained using robust
  design and compact model to FLUENT

28
Robust vs. Optimal Configuration

• Pareto Frontier

29
Effects of Robust Solution

• Optimal gtgt Robust Temperature Variation

30
Effects of Robust Solution

• Optimal gtgt Robust Temperature Variation

31
System Model
Control Factors, x Inlet air velocity, Vin ?0,
1 m/s Section a chip power, Qa ?0, 200
W Section b chip power, Qb ?0, 200 W Section c
chip power, Qc ?0, 200 W
Response, y Inlet Air Velocity (m/s) Chip
Temperatures (oC) Total cabinet power (W)
Signal Factors, M Inlet air velocity
(minimize) Chip Temperatures (minimize) Cabinet
Power (nominalize)
Server Cabinet System
Noise Factors, Z Inlet air temperature, Tin 25
oC
32
Design Objective Specification

• System Design Objectives gtgt Goals
• Minimize flow rate of cooling air supplied to
  cabinet
• Minimize server chip temperatures
• Minimize sensitivity of configuration to changes
  in cabinet operating conditions
• System Design Specifications gtgt Constraints
• All server chips must operate at under 85oC
• Total cabinet power must meet target value