Scalable and Adaptive Cooling Solutions For Data Centers

1
Scalable and Adaptive Cooling Solutions For Data
Centers
Presented by Izuh Obinelo, Ph.D. Director,
Center for Airflow and Thermal Technologies
2
About DegreeC

• Engineering Thermal Solutions for Several Target
  Markets

Thermal Technologies
3
About DegreeC

• Cooling Architecture
• Airflow/thermal design
• Air intake, exhaust
• Air mover/filter selection
• Fan controller design
• Fan assembly design

4
Agenda

• Review of Data Center Heat Problem
• Review of Typical Cooling Designs
• Design for Optimal Cooling
• Degree Controls Cooling Solutions
• Data Center Thermal Simulation

5
Data Center Overview

• Agglomeration of computing, storage, and network
  resources under one roof
• Backbone of internet and distributed network
  computing grids
• Internet server applications execute across a
  horizontally scalable server topology
• Distributed systems for communication or for
  solving large scale problems involve aggregation
  and scheduling of many resources
• Data center is a larger fractal model of an
  electronics product
• Dedicated infrastructure for mission critical
  applications requiring high availability -
  99.999 uptime

6
Major Components of a Data Center

• IT hardware (equipment racks)
• Power supply and distribution hardware
• Cooling hardware and cooling fluid distribution
  hardware
• Network infrastructure
• IT Personnel and office equipment

7
Power Heat Loads in Data Centers
Source The Uptime Institute
Updated chart expected from ASHRAE TC9.9 comittee
8
Power Heat Loads in Data Centers

• Blade heat dissipation levels of 250-400W in a 1U
  form factor
• Higher density of compaction to save real estate
• Today typically at 100W/ft2
• EIA projections of 300-500W/ft2

9
Scope of Heat Problem

• Server farms in Seattle, WA dissipate as much
  power as the rest of the Seattle metro area,
  including the Boeing plants in Everett.
• 25 of the total cost of their operation is for
  power and air conditioning.
• Electronics equipment may be one of the drivers
  for the rolling West Coast blackouts.

10
Who Should be Concerned?

• IT Professionals
• Electronics Cooling Professionals
• Data Center Owners Operators
• Facility Planners, Architects Engineers
• Public Policy Planners
• Network Data Consumers
• General Public

11
Agenda

• Review of Data Center Heat Problem
• Review of Typical Cooling Designs
• Design for Optimal Cooling
• Degree Controls Cooling Solutions
• Data Center Thermal Simulation

12
Typical Cooling Designs

• Flooded Room Supply, Flooded Room Return (most
  common)

• Room supply from computer room air-conditioning
  (CRAC) units

• Room return to CRAC units

• Suitable for low power density installations

13
Typical Cooling Designs

• Alternate Hot and Cold Aisles

14
Typical Cooling Designs

• Raised Floor

• Under floor plenum for cold air distribution (and
  cables)

• Perforated tiles in front of racks

15
Typical Cooling Designs

• Ducted Ceiling Return

• Hot air returned thru ducts in the ceiling

16
Typical Cooling Designs

• Some Inefficiencies in Air Circulation

• Mixing of hot and cold streams

• Cold airflow bypass

• Uneven tile flow distribution

17
Uneven Tile Flow Distribution
18
Uneven Tile Flow Distribution
19
Flow Distribution Plenum Blockage
20
Agenda

• Review of Data Center Heat Problem
• Review of Typical Cooling Designs
• Design for Optimal Cooling
• Degree Controls Cooling Solutions
• Data Center Thermal Simulation

21
Power Density Metrics

• Typical Space Allocation

• Typical Heat Dissipation

22
Power Density Metrics

• Uneven heat dissipation hot and cold areas
• Cooling capacity usually defined in terms of
  Power per Room Area
• Power per Room Area leads to over-sizing of
  cooling equipment
• A better metric would be Power per Equipment
  Footprint
• Power per Equipment Volume more representative
  for server stacks

23
Cost of Over-Sizing

• Capital costs
• Maintenance costs
• Operating costs

24
Typical Electricity Consumption
25
Hot Spots

• With current design metric, local hot spots
  often exist even with over-sizing

Design capacity
Row N
100
Row 2
Design capacity
Row 1
10
26
Design Considerations for Optimal Cooling

• Data centers are designed for 15-20 year service
• IT equipment refreshes every 1.5 years
• Power dissipation per equipment increases with
  each refresh
• Computing load is dynamic, therefore heat
  dissipation varies continuously
• Sensible cooling capacity degrades over time
• Reliable (CFD) Simulation is critical

27
Design for Minimal Cooling Costs

• Mantra Reduce costs of over-sizing Avoid hot
  spots

• Comprehensive simulation reduce capital costs
• Adaptive systems reduce running costs
• Scalable systems reduce capital and
  maintenance costs

28
Cost of Over-Sizing
29
Feature Requirements for Optimal Cooling

• Standardization
• Rack and server platforms
• Cooling systems for server rooms
• Modularity
• Reduced installation costs
• Modular systems allow for easy scalability
• Adaptability
• Adapt to changing heat loads
• Environmental sensing
• Automatic controls of CRAC and air distribution
  systems

30
Agenda

• Review of Data Center Heat Problem
• Review of Typical Cooling Designs
• Design for Optimal Cooling
• Degree Controls Cooling Solutions
• Data Center Thermal Simulation

31
DegreeCs Cooling Solutions

• Features

• Modular
• Scalable
• Adaptive
• Designed for existing systems
• Relatively cheap solutions

32
DegreeCs Cooling Solutions

• Floor tiles with adjustable dampers

• Calibrated airflow achieved with adjustable
  dampers
• Balance airflow among tiles
• Flow and pressure feedback

33
DegreeCs Cooling Solutions

• Flow Tiles

• Bring cooling air directly to areas of need
• Variable airflow achieved with fan speed control
• Balance tile flow
• Assist CRAC air movers to overcome crowded
  plenums
• Network Ready
• AC/DC input
• Field programmable thru RS232 interface

34
DegreeCs Cooling Solutions

• Flow Trays

• Boost airflow thru open racks
• Variable airflow achieved with fan speed control
• Temperature or Flow control
• Rack mount ready
• Field programmable thru RS232 interface
• Network Ready
• AC/DC input

35
DegreeCs Cooling Solutions

• ColdFRONT

• Bring cooling air directly to server intakes
• Variable airflow achieved with fan speed control
• Up to 1000cfm per module
• Field programmable thru RS232 interface
• Network Ready
• AC/DC input

36
DegreeCs Cooling Solutions

• FlowRACK

• Modular refrigerated racks easily scalable
• 10 ton (34 kW) to 50 ton (170 kW)
• Upflow systems for non-raised floor installations
• Downflow systems for raised floor installations
• Tightly controlled environment (1F, 3 R.H.)
• High airflow up to 650cfm/ton
• Field programmable
• Network ready

37
DegreeCs Cooling Solutions

• Sensors and Sensor Hubs

• Modular easily scalable
• Up to 16 sensors per hub
• RJ45 network cables
• RS485 backbone
• Temperature and Flow Sensors
• Wireless sensor network

38
DegreeCs Cooling Solutions

• Adaptive Thermal Management Through Intelligent
  Network

39
Agenda

• Review of Data Center Heat Problem
• Review of Typical Cooling Designs
• Design for Optimal Cooling
• Degree Controls Cooling Solutions
• Data Center Thermal Simulation

40
Thermal Simulation

• Design Simulation Reduced Capital Costs
• A priori investigation of different equipment
  populations and arrangements
• A priori investigation of different operating
  scenarios
• Simulation of data centers is tedious, but the
  physics is relatively simple
• Full-scale CFD model of entire room and
  under-floor plenum Characteristic models of rack
  equipment Characteristic models of cooling
  equipment
• Multiple scale problem

41
A Simulation Example
42
A Simulation Example
43
A Simulation Example

• Floor and roof tiles were incorporated into the
  model. Tiles can be passive or fan-assisted

44
A Simulation Example

• Volume heat dissipation for each rack equipment
  1.67 MW of power at full loading
• Volume heat absorption for each CRAC unit 100
  ton refrigeration per unit
• Fan curves for each CRAC, rack equipment, and
  flow tile 98 individual fan curves
• Volume resistance for each rack equipment, CRAC,
  and under-floor plenum 135 volume resistances
• Planar Resistance for floor tiles
• All-CFD model ? k-e turbulence model, 500000
  tetrahedral elements

45
A Simulation Example

• Case 1 Flow distribution obtained without fan
  tiles.
• Case 2 Fan tiles were used in the floor of the
  cold aisles to assist the circulation of cold air
  from the raised floor to the aisles.
• Case 3 Fan tiles were employed on the ceiling
  above the hot aisles to draw hot air from the
  aisles and cut down on the hot air recirculation.
• Case 4 Fan tiles were employed both on the
  floor of cold aisles and ceiling of hot aisles.

46
A Simulation Example

• Case 1 Air temperature distribution in the data
  center obtained without fan-assisted tiles.
• Case 2 Air temperature distribution obtained
  with floor fan tiles.
• Case 3 Air temperature distribution obtained
  with ceiling fan tiles.
• Case 4 Air temperature distribution obtained
  with both floor and ceiling fan tiles.

47
About DegreeC
THANK YOU!