Data Centre Best Practises Workshop

1
Data Centre Best Practises Workshop
Using Computational Fluid Dynamics (CFD) for
improving cooling system efficiency for Data
centers
Shishir Gupta
17th March 2009
2
You are Here ?
Introduction to CFD
Data Centre Case Study Geometrical Details
CFD while designing of HVAC system
CFD during installation of Data Centre
CFD for maintenance of Data Centre Feedforward
System
3
Introduction to CFD

• Computational (having to do with mathematics
  computation)
• Fluid Dynamics (the dynamics of things that
  flow)
• CFD is built upon fundamental physics equations
  equations of motion and conservation. CFD
  applications range from numerical weather
  prediction to vehicular aerodynamics design.
• CFD applications are linked with advances in
  computing software and hardware. CFD software is
  characterized by the physical models in the
  software.
• Fine-scale CFD applications closely match the
  true geometry of the physical objects and
  processes being modeled.

4
What is CFD?
Fluid Problem
C F D
5
Why use CFD?
6
Where use CFD?
Chemical Processing

• Chemical Processing
• HVAC(Heat Ventilation Air Condition)
• Hydraulics
• Aerospace
• Automotive
• Biomedical
• Power Generation
• Sports
• Marine

reactor vessel - prediction of flow separation
and residence time effects.
Hydraulics
HVAC
7
Where use CFD?
Aerospace

• Chemical Processing
• HVAC
• Hydraulics
• Aerospace
• Automotive
• Biomedical
• Power Generation
• Sports
• Marine

Biomedicine
Automotive
Temperature and natural convection currents in
the eye following laser heating.
8
Where use CFD?
Sports
Power Generation

• Chemical Processing
• HVAC
• Hydraulics
• Aerospace
• Automotive
• Biomedical
• Power Generation
• Sports
• Marine

Flow around cooling towers
Marine
9
You are Here ?
Introduction to CFD
Data Centre Case Study Geometrical Details
CFD while designing of HVAC system
CFD while installation of Data Centre
CFD for maintenance of Data Centre Feedforward
System
10
CFD Case Study for Data Centre
11
Introduction to the Case Study

• Case Study is taken from one of the project that
  we did for a Data Centre in India
• The case study includes what we did for the
  client also extends it for what could have been
  done for the same project using CFD
• There were two software applications used for the
  project OpenSource CFD platform of OpenFoam and
  commercial CFD package of Fluent
• Both packages produced about the same results, in
  this presentation the results from OpenFoam are
  being shown

12
Case Description

• The analyzed Data Centre is rectangular with of
  area 516m2 and height 3.35mt
• Cooling is to be provided using raised flooring
  layout and demarcation is done for Cold Aisle and
  Hot Aisle
• The sources of heat gain inside the data centre
  are listed below
• Heat gain through exterior walls accounting for
  thermal resistance of the wall
• Heat gain from Server Racks, 154 Server racks
  each providing about 8 KW combine to about 1.26
  MW
• Three fans of about 500CMH were assumed to
  transport air from cool aisle to hot aisle in
  each rack unit (Since detailed blade
  specification is not known)

13
HVAC System Specification

• 10 CRAC units, 1 Standby Specification
• Each CRAC unit of 30,585 CMH
• Cooling capacity of Each Rack is 150 KW
• Temperature of supply air is 9.4 oC
• Return Air opening area (On top surface) 2.23 m2

• Supply Air Diffuser (Cold Aisle) Specifications
• Dimension of 600mm X 600mm
• 70 open area
• 1 supply diffuser per rack (Total 154)
• Supply air velocity can be controlled using under
  floor fan

• Return Air Diffuser (Hot Aisle) Specification
• Dimension of 600mm X 600mm
• 50 open area
• Total no. of diffusers 242

14
Objective of the Study

• To maintain recommended temperature by ASHRAE for
  Class 1data centre
• The recommended atmosphere is defined as
• Temperature of 20oC - 25oC
• Relative humidity of 40 - 55
• The allowed change in temperature should be less
  than 5oC/hr

15
Recommended Operating Conditions
16
Design Parameters

• Number of CRACs
• Location of CRACs
• Velocity of supply air

17
You are Here ?
Introduction to CFD
Data Centre Case Study Geometrical Details
CFD while designing of HVAC system
CFD while installation of Data Centre
CFD for maintenance of Data Centre Feedforward
System
18
Base Case Design
CRAC Units (11 Nos.)
Return Diffusers
Supply Diffusers
False Ceiling
False Flooring
Server Racks
Isometric View of the Designed Data Centre
19
Case Study Cont
COLD AISLE Diffusers
HOT AISLE Diffusers
CRAC Units (11 Nos.)
Server Racks
Top View of the Designed Data Centre
20
CFD Simulation of Base Case
Temperatures across Y-Z plane
21
Temperature Contour
Temperature Profile at vertical planes along the
racks and cold aisle.
22
CFD Simulation of Base Case
Temperatures across X-Y plane
23
Temperature Contour
Temperature Profile at Horizontal planes along
the racks and cold aisle. Lets look at the
mid-plane contour in more detail..
24
Temperature Contour in Middle Plane
The temperature contour at the Horizontal plane
at the middle portion of the rack
25
CFD Simulation of Base Case
Temperatures across X-Z plane
26
Temperature Contour
Temperature Profile at the middle plane is
showing most uneven distribution. Lets analyse
the middle plane in detail
27
Temperature Contour in Middle Plane
The temperature contour at the vertical plane at
the middle portion of the rack
28
Velocity Vectors in Middle Plane
The Velocity Vectors at the vertical plane at the
middle portion of the rack
29
Conclusion from the base case CFD

1. The Average temperature on the rack surface at
   the cold Aisle side is 15
2. The temperature at Cold Aisle is varying from 12
   to 17
3. The Average temperature on the rack surface at
   the Hot Aisle side is 27
4. The temperature at Hot Aisle is varying from 18
   to 32
5. The simulation shows that a good number of
   servers are experiencing temperature well above
   and below the ASHRAE recommended temperature
   levels
6. Short circuiting of cold air is clearly visible
   in the simulation

30
Optimizing number of CRAC units Supply Air
Velocity

1. Maximum heat load 154 X 8 1264 KW (1.26 MW)
2. Heat capacity of each CRAC 150 KW
3. Minimum number of CRAC required 8.4 9
4. The system was designed with 9 CRAC units and
   velocity of supply air was adjusted to avoid
   short circuiting and temperature stratification
5. In this case the velocity of 2.2 m/s is coming
   out to be higher
6. The simulation was conducted with velocity of
   1.6, 1.7, 1.8, 1.9, 2.0 2.1 m/s
7. The results with 1.8 m/s showed best results

31
Temperature Distribution with 9 CRACs 1.8 m/s
The temperature contour at the vertical plane at
the middle portion of the rack
32
Velocity Vectors with 9 CRACs 1.8 m/s
The Velocity Vector at the vertical plane at the
middle portion of the rack
33
Results of improved design CFD

1. The Average temperature on the rack surface at
   the cold Aisle side is 16
2. The temperature at Cold Aisle is varying from 13
   to 17
3. The Average temperature on the rack surface at
   the Hot Aisle side is 23
4. The temperature at Hot Aisle is varying from 19
   to 29
5. Short circuiting of cold air is reduced to a
   substantial level, however still prevalent
6. The simulation shows that a most of the servers
   are experiencing temperature as recommended by
   ASHRAE

34
Conclusion

• Using Computational Fluid Dynamics the system was
  designed to reduce to 90 of original design,
  thus bringing about first cost saving of 10 in
  the capital cost.
• The new system uses less energy and produces
  better result than the initial design based on
  thumb rules

35
You are Here ?
Introduction to CFD
Data Centre Case Study Geometrical Details
CFD while designing of HVAC system
CFD during installation of Data Centre
CFD for maintenance of Data Centre Feedforward
System
36
Case Description

• The capacity of this data centre of of 42 X 154
  6,468 Server Blades
• 4,000 server blades are to be installed
• 1,000 servers are by Dell, 2,000 by IBM 1000 by
  Sun
• The design variables are
• Number of CRAC units
• Which CRAC unit should be operational
• Location of Server Blades in the racking system
• Velocity of supply air inlet

37
CFD Simulation Setup

• The power requirement of 3000 Server is minimum
  713 KW 5 CRAC (750KW) are minimum number of
  units which can provide the required tonnage
• The CFD simulation were conducted with various
  locations of Servers, CRACs and Supply air
  velocity
• The best result was found with following
  parameters
• Top Racks are empty
• Alternative CRACs are operating
• Velocity of Supply air is 1.2 m/s

38
CFD Simulation Results
Server Positions in the Racks
39
CFD Simulation Results
Operational CRACs
40
Temperature Distribution with 5 CRACs 1.2 m/s
The temperature contour at the vertical plane at
the middle portion of the rack
41
Velocity Vectors in Middle Plane
The Velocity Vectors at the vertical plane at the
middle portion of the rack
42
Calibration during Installation
Temperature Sensors

• The Result from CFD shall be compared with
  average reading shown by temperature and velocity
  sensors
• If there is any difference, the modeling shall be
  improved to arrive at the actual values.

43
You are Here ?
Introduction to CFD
Data Centre Case Study Geometrical Details
CFD while designing of HVAC system
CFD during installation of Data Centre
CFD for maintenance of Data Centre Feedforward
System
44
Feedforward System

• Whenever capacity of the data centre is to be
  increased, the design parameters like number of
  CRACs and supply air velocity should be
  determined using CFD
• If the capacity ramp up is not that frequent than
  CFD simulation can be conducted at that stage to
  arrive at design parameters
• If ramp-up/ramp-down is very frequent then a
  custom made CFD code should be developed using
  OpenSource Libraries. This would enable data
  centre administrator to conduct CFDs for his
  data centre and analyze various design options

45
Conclusion

• CFD can help design and operate the data centre
  HVAC system with optimum efficiency

Thank You