Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central

DOI: 10.5098/hmt.v1.2.3006 ISSN: 2151-8629

Frontiers in Heat and Mass Transfer

Available at www.ThermalFluidsCentral.org

HOT WATER COOLED HEAT SINKS FOR EFFICIENT DATA CENTER

COOLING: TOWARDS ELECTRONIC COOLING WITH HIGH
EXERGETIC UTILITY
Peter Kastena, Severin Zimmermanna, Manish K. Tiwaria, Bruno Michelb, Dimos Poulikakosa,*
a
Laboratory of Thermodynamics in Emerging Technologies, ETH Zurich, 8092 Zurich, Switzerland
b
Advanced Thermal Packaging, IBM Zurich Research Laboratory, 8803 Rueschlikon, Switzerland

ABSTRACT
Electronic data center cooling using hot water is proposed for high system exergetic utility. The proof-of-principle is provided by numerically
modeling a manifold micro-channel heat sink for cooling microprocessors of a data center. An easily achievable 0.5l/min per chip water flow, with
60C inlet water temperature, is found sufficient to address the typical data center thermal loads. A maximum temperature difference of ~8C was
found between the solid and liquid, confirming small exergetic destruction due to heat transport across a temperature differential. The high water
outlet temperature from the heat sink opens the possibility of waste heat recovery applications.

Keywords: Hot water cooling, Exergy, data center, micro-channel manifold, heat sink

studied thoroughly the temperature and the heat flux distribution of a

1. INTRODUCTION micro-channel unit cell. Ryu et al. (Ryu et al. 2003) also developed a
three dimensional model of a MMC heat sink applying a finite-volume
State-of-the-art cooling systems cool the electronic components of data method and optimized the geometrical design parameters with the
centers by forced air convection, wherein the cooling components can steepest descent technique. Their final design reduced the thermal
contribute up to the half of the total power consumption of data centers resistance by 50% and improved the temperature variation compared to
in order to chill and distribute the air (Brunschwiler et al. 2009, a micro-channel heat sink investigated in an earlier study. An interesting
Koomey 2008). Further advances and miniaturization of electronic concept of self-contained oscillating flow based heat sink was explored
components have led and will lead to increased heat generation rates by Waelchli et al. (Waelchli et al. 2010a, b). Escher and coworkers
per unit area in electronic equipment, making the traditional cooling (Escher et al. 2010a, Escher et al. 2010b) were first to consider both the
strategies insufficient (Brunschwiler and Michel 2008, Meijer 2010). manifold and the micro-channel design in their investigations of an
Liquid cooling systems have the potential to overcome these obstacles ultra-thin MMC heat sink. In order to minimize the computational cost,
due to the better thermophysical properties liquid compared to air and they approximated the micro-channels as a porous medium in a three
lower thermal resistance (by a factor of 5 or higher) of liquids cooling dimensional hydrodynamic model of a manifold unit cell. They
systems (Brunschwiler et al. 2006, Colgan et al. 2007). Electronic chip validated the pressure drop obtained through porous medium approach
cooling using liquid has been explored extensively in the literature. against their single micro-channel model. Based on the manifold model,
Tuckerman and Pease(Tuckerman and Pease 1981) first investigated a they proposed and demonstrated a new manifold design providing
liquid cooled micro-channel heat sinks and determined the internal uniform flow distribution over the heat transfer structure for improved
thermal resistance of 0.09 cmK/W for their best heat sink. The heat transfer performance. Their optimized heat sink provides a thermal
limitations of their approach are the high pressure drop and the non- resistance of 0.087 cmK/W and an overall pressure drop < 0.1 bar.
uniform temperature distribution along the long parallel channels. As The reduced thermal resistance in liquid cooling on one hand
improvement, several studies proposed manifold micro-channel (MMC) allows increased processor performance with the resulting high heat
heat sinks. In MMC heat sinks, the liquid coolant is supplied to and fluxes. On the other hand, use of liquid coolant can also reduce the
collected from the micro-channels (i.e. the heat transfer structure) in a thermal gradient needed for heat removal at low heat flux levels: For
direction normal to the base plate, supporting the channels, using example, water cooling to remove the heat of todays products with
supply manifolds. Copeland et al. (Copeland et al. 1997) developed a power densities of 60 W/cm2 only requires temperature gradients of
three dimensional and an analytical one dimensional model of a single 10C as opposed to 60C for air cooling solutions. Therefore, with
micro-channel as a unit cell of the MMC heat sink. Their three- water as coolant, the input coolant temperature can be raised above the
dimensional model showed better agreement with their experimental hot seasons free-cooling limit, thereby eliminating the need for coolant
data than their analytical one, which lacked accuracy for low flow rates. chillers in all climates, even on the hottest days. In addition, a high
Fedorov and Viskanta (Fedorov and Viskanta 2000) used a finite- coolant temperature can also enable energy re-use, thereby improving
difference solving scheme for their three dimensional model and the exergetic utilization of the system. For example, the hot liquid

*
Corresponding author. Email: [email protected], Ph.: +41 44 632 2738, Fax: +41 44 632 11 76
1
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
coming out of data center cooled by a hot coolant can be used for ISSN: 2151-8629
building heating. Such a holistic approach with energy re-use can
minimize the overall carbon footprint at system level by reducing or
eliminating the requirement of fossil fuels for building heating
(Brunschwiler et al. 2009).
In this paper, we explore the concept of cooling electronics in a
way that will enable the cooling of the data centers with hot water as
a means to reduce carbon footprint of data centers and also enhance the
exergetic utility of the cooling unit. We analyze a water cooled MMC
heat sink, which is designed to cool the processors of a blade server and
also explore the benefits of connecting the hot water at the outlet of
blade server to a water heating system of the building for waste-heat
recovery. First, a three dimensional conjugate heat transfer model of a
micro-channel unit cell is developed to investigate the flow and the heat
transfer characteristics of a chip simulator being cooled by hot water at Fig 1: Schematic of micro-channel unit cell domain
60C inlet temperature. Furthermore, a three dimensional hydro-
dynamic model for the complete heat sink is also developed to study the The investigated simplified design with only one inlet and two
flow distribution in the heat transfer structure as well as determining the outlet nozzles is expected to cause a higher pressure drop, less heat
overall pressure drop. The influence of flow distribution on thermal removal and less uniform spatial temperature distribution within the
performance of heat sink is explained through a thermal resistance cooled processor in comparison to MMC heat sinks with more
model. We follow a different approach than Escher at al. (Escher et al. sophisticated manifold designs (Escher et al. 2010b). The design is a
2010b), and model the hydrodynamics of both the inlet and the outlet compromise between fabrication cost and hydrodynamic and thermal
manifolds and developed a turbulence model to capture the high shear performance which makes it well-suited for large-scale implementation
stresses originating in heat sink, as explained further in the results and in a prototype liquid cooled datacenter. The heat sink uses copper fins
discussion section. (micro channel walls) that are cut into the solid copper base plate and
lifted using a patented Micro Deformation process (Wolverine Tube,
2. GEOMETRY AND FABRICATION Inc., USA), The inlet and outlet manifolds are machined into a second
copper plate which is connected to the heat transfer structure by
The investigated heat sink consists of the parallel micro-channel heat diffusion bonding. The dimensions of the entire heat sink are 47.5 mm
transfer structure and water distribution and collection manifolds as x 47.5 mm in length and width. The dimensions of the heat sink fit the
sketched in Figures 1 and 2. The heat sink design is similar to the one size of the copper cap mounted on top of processor packages in blade
proposed by Escher et al. (Escher et al. 2010b), which was proven to be servers. This cap protects the processor from the surroundings and
effective in removing high heat flux from processors in state-of-the-art causes the necessary spatial temperature spreading for conventional air-
datacenter blades. The water flow through the heat sink is marked with cooling. The aspect ratio of the fin is equal to 10. Since the boundary
blue arrows in Figure 2. The coolant water is fed centrally to the inlet layer is thinnest at the fin surface, the heat transfer through the fin is a
manifold with an inlet port connected to the hot water supply. The major component of the heat removal from the entire heat sink.
water from the inlet manifold flows through a slot nozzle in manifolds
bottom wall on to the micro-channels and branches symmetrically into
two parts to flow through the entire length of the channels. The nozzle
slot design ensures that water impinges on the micro-channels
enhancing heat transfer coefficient. The water leaves the micro-
channels at both ends via slot nozzles in the walls of collection
manifolds and the two coolant streams recombine after a 90 turn
before leaving the manifold centrally. Geometrical parameters of the
heat sink are given in Table 1.
Table 1: Geometric parameters of MMC heat sink
Outlet nozzle width, wn,out
Channel width, wch [m] 170 2
[mm]
Inlet manifold width, 5
Fin thickness, wfin [m] 170
wM,in [mm]
Inlet manifold height, 4.25
Channel height, hch [mm] 1.7
hM,in [mm]
Outlet channel width (1st 5
Channel length, Lch [mm] 8.5
leg), wM,out,1 [mm]
Outlet channel width (2nd 3
Number of channels, N [-] 62
leg), wM,out,2 [mm]
Outlet channel height, 4.5
Base thickness, hbase [mm] 1.2
hM,out [mm]
Inlet nozzle height, hn,in 2.8
1 Inlet diameter, dM,in [mm]
[mm]
Inlet nozzle width, wn,in Outlet diameter, dM,out 3
2 Fig 2: Schematic of manifold geometry (only half of the dissected
[mm] [mm]
Outlet nozzle height, hn,out Inlet manifold length, along the symmetry plane is shown).
0.75 21.25
[mm] LM,in [mm]

2
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
3. NUMERICAL MODELING trough the vertical side walls near the channel outlet andISSN: 2151-8629
assume it to be
adiabatic.
3.1 Micro-channel unit cell model The second order high resolution finite volumes scheme of the
The liquid is assumed as steady and incompressible and the thermo- commercial Ansys CFX 12.1 package was used to solve Eqs. (1) (4)
physical properties are all evaluated at the liquid inlet temperature and applying the described boundary conditions while the convergence
assumed to be constant. The continuity, the momentum and the energy criterion was set to 10-6 for the root mean square values for all
equations for the liquid and energy equation for the solid parts are variables. An adequate block-structured, hexagonal, non-uniform grid
of 901320 elements was used to obtain the results. Increasing the
ui element number to 1761120 resulted in relative changes of 0.43 %,
=0 0.050 % and 0.106 %, respectively, in the pressure drop, the fluid
xi (1) temperature change from the inlet to the outlet, and the temperature
2 difference between the maximal base plate and the fluid inlet
ui p ui
u j = + temperatures.
x j xi x 2 (2)
j 3.2 Thermal resistance model
T f k f 2T f
ui u j u j In addition to the detailed three dimensional-model of a single micro-
ui = + +
xi 2
c p xi c p x j xi xi

(3) channel, a one dimensional thermal resistance model for a single micro-
channel is developed to estimate the thermal performance of the heat
2Ts sink. The total thermal resistance separating the maximal heat sink
=0
xi2 (4) temperature from the water inlet temperature can be divided in three
parts. Firstly, since the heat is transferred by conduction through the
base plate of the heat transfer structure, we define its thermal resistance
where the symbols , c p , , k , T , p and ui respectively denote as
the density, specific heat at constant pressure, dynamic viscosity,
thermal conductivity, the temperature, the pressure and the velocity h
Rbase = base . (7)
components. Table 2 lists the numerical values of the properties used. s

Density water, [kg/m3] 983.2 Secondly, convective heat transport from the solid to the water occurs
through the fin surface and through the bottom and top wall of the
Specific heat capacity water, c p [J/kg K] 4185
micro-channels. Neglecting the heat transfer from the top wall to the
Dynamic viscosity water, [kg/m s] 4.6603*10-4 liquid, we assume the thermal resistance of the fin and the bottom wall
Thermal conductivity water, k f [W/m K] 0.65439 to act in parallel. Therefore, we defined the convective, thermal
resistance as
Thermal conductivity copper, k s [W/m K] 401
Table 2: Thermo-physical properties used in the simulation
wch + w fin
The solid and fluid domains in the model are coupled through the Rconv = . (8)
wch hD + 2hD fin h fin
continuity of heat flux and temperature. Due to the periodicity of the
parallel flow channel structure, we assume symmetry in the center of
the channel and consider only half of the channel and half of the fin Spiga and Morini (Spiga and Morini 1996) analytically determined the
(the channel side wall) in the model. Symmetry is also imposed at the Nusselt (Nu) number for fully developed flow in rectangular ducts.
vertical plane where the flow enters and divides in two parts to fill the Their value for a duct aspect ratio of 10 and constant heat flux from
channel length. Assuming uniform flow distribution into all micro- both long sides of the duct is Nu D = 6.096. We used this to calculate
channels, the mass flow at the inlet to modeled channel-half and the
the heat transfer coefficient for fully developed flow
channel Reynolds-number can be calculated by

V
tot Nu D f

=
m , (5) hD = . (9)
4N Dh,ch
u D V
tot
Rech = ch h,ch = , (6)
N ( wch + hch ) The analytical value of Nu from Spiga and Morini (Spiga and Morini
1996) can be used to estimate the heat transferred from the fin surfaces
where N , V
tot , uch , Dh,ch , wch and hch , respectively denote the (side walls) and also to evaluate efficacy of fin surface in terms of heat
transfer. First due to the high aspect ratio of the channels being used, it
number of micro-channels, the total volumetric flow rate, the density of is important to ensure that the fin height is not too large, which could
water, the mean velocity in the micro-channel, the hydraulic diameter of result in inefficient heat transport ( i.e. the so called infinite fin length
the micro-channel, the micro-channel width and height, respectively. limit, see ref. (Incropera and DeWitt 2002)). The height of a fin in the
The no-slip condition is imposed at all walls and an opening condition infinite length limit can be evaluated as
of zero average static pressure is set at the outlet boundary allowing
backflow into the domain. The later was validated by comparisons with
separate simulation with an extended outlet channel. The top wall of 2hD
mh fin, =4.6 , with m = . (10)
the micro-channel is considered to be adiabatic as a big part of it is s w fin
exposed to air while the dissipated heat of the processor is
approximated as a uniform heat flux at the bottom wall. Due to large
overhang of the micro-channels beyond the heat generating processor, Equation (10) yields hfin, value of 7.5 mm, which is well above the
the heat flux in the solid domain is directed primarily to near the inlet selected 1.7 mm. Furthermore, assuming an insulated fin tip, the fin
region of the micro-channel. Therefore, we neglect the heat transfer efficiency for can be computed as(Incropera and DeWitt 2002)
3
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006 ISSN: 2151-8629
k
tanh( mh fin )
(
U j k = ) + t
x + Pk k ,
17)
x j x j k j
fin = . (11)
mh fin

(U j = ) + t
x + k Pk ,
2
(18)
x j x j j
Using Eq. (11), the fin efficiency is determined to be 75.0 % for the
considered thermo-physical properties and geometry. Note that Eq. (11) U i U j U i
Pk = t + , (19)
is derived under the assumption of a constant heat transfer coefficient x j xi x j

from fin to fluid and constant fluid temperature, both of which are not
at all true here (as will be clear from the Results and Discussion
section). Therefore, efficiency values calculated here should just be where the model constants are given as k = 2, = 2, = 0.09, =
used as an indication that we are not severely compromising the 0.0075 and = 5/9. An automatic wall treatment is used to capture the
effectiveness of heat transport using too high pins (i.e. high channel turbulence characteristics in the boundary layer, wherein a low
aspect ratio). Reynolds k and formulation is blended into logarithmic wall
Thirdly, the bulk resistance accounts for the heat transfer to the bulk function for y+ < 11, while pure logarithmic wall function approach is
flow and is visible in the temperature rise of the water. We defined it as applied for y+ > 11. The wall boundary conditions used for k and
are:

Rbulk =
(
Lch wch + w fin ).
V
(12) k =0, (20)
c p tot 6
( 2N ) = ,
2 (21)
( y )
By combining these three parts, the total thermal resistance can be
expressed as where y denotes the distance between the first two mesh points
normal to the wall. Heyerichs and Pollard (Heyerichs and Pollard 1996)
Rtot,1D = Rbase + Rconv + Rbulk , (13) and Chen and Modi (Chen and Modi 1999) reported superior
performance of the k - model over k - models with enhanced wall
since the three parts of the total resistance are connected in series. treatment in recirculation flow situations. This is attributed to the more
realistic treatment of the turbulence characteristics in the boundary
3.3 Hydrodynamic heat sink model
layer and therefore was also used here.
The overall pressure drop of the heat sink and the flow distribution The turbulent equations are also solved in the heat transfer structure
within the heat transfer structure are evaluated using a three approximated as porous medium although the flow in the micro-
dimensional model of the complete heat sink in which the parallel channels is expected to be laminar. To test the influence of this on
micro-channels are approximated as a porous medium in order to model performance, the section from the inlet to the outlet slot nozzles
reduce computational load. Water enters centrally through a circular was simulated separately using the turbulent as well as laminar models.
hole, connected to a hot water supply pipe, into the inlet manifold. Due The effect of additional flow resistance due to the porous medium is
to high flow rates (up to 1 l/min), water travels rapidly to the back wall found to be stronger than the turbulent effect (at least for the turbulence
of the inlet manifold, which reverses it causing a backflow. The high intensity levels observed in the entire heat sink simulation). The
shear stress at the interface between the forward and the backflow in the maximum relative difference between the turbulent and laminar models
inlet manifold made it impossible to model the flow inside the manifold was 2.3% for uniform mass flow and turbulent intensity I = 0.4 at the
at steady state using the laminar model. Therefore, a turbulent flow inlet boundary.
model was used instead. A similar flow characteristic occurs in the It is known from the literature that the k - model is strongly
second leg of the outlet manifold, where the two flow streams run into dependent on the inlet free stream boundary conditions of the
each other before leaving the manifold through outlet port connected to turbulence equations (Wilcox 1998). Therefore, we varied the
a pipe. turbulence intensity and turbulence length scale at the manifold inlet in
The Reynolds Averaged Navier-Stokes equations (RANS) were the range 0.005 < I < 0.06 and 0.01* d M ,in < l < 0.5* d M ,in ,
used to model the turbulent flow in the manifolds
respectively. The overall pressure drop across the heat sink changed by
1.5 % for the turbulence intensity variations and was even smaller for
U i the variation of the turbulence length scale.
=0 , (14)
xi The additional momentum term S p ,i in the Navier-Stokes equation
U j p U U j accounts for the flow resistance of the porous medium and is set to zero
U i = + ( + t ) i + + S p,i , (15) in the inlet and the outlet manifolds. Since ReCh > 30, the quadratic
xi x j xi xi x j
Forchheimer term is added to Darcys law.
k
t = .
(16) S p,i =- U i -closs i-0.5 f U U i (22)
i
The symbols U i , t , k , and S p ,i designate the mean velocity,
eddy viscosity, turbulent kinetic energy, specific dissipation of kinetic The constant loss coefficient closs was evaluated by Beavers et al.
energy and additional body force, respectively. The Wilcox k - (Beavers et al. 1973)as
model (Wilcox 1998) equations, used for closure in the eddy viscosity
approach, can be expressed as
D
closs = 0.55 1-5.5 Ch (23)
DHT

4
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
where we define the characteristic length scales of the pores, DCh and Tmax,bottom T f ,in ISSN: 2151-8629
Rtot ,3 D = (28)
the entire heat transfer structure, DHT , as
qchip
and the pressure drop across the complete heat sink for varying
0.5 volumetric flow rates (see Figure 3). The total thermal resistance of the
DCh = ( wCh hCh ) (24) heat sink decreases with increasing Reynolds-number in a power-law
0.5 fashion as the convective heat transport to the fluid is enhanced by
(
DHT = LM ,in hCh ) . (25) higher velocity gradients at the channel walls.

Contrary to previous attempts to model the heat sink heat transfer

structure as porous medium (Escher et al. 2010b, Jang and Kim 2005,
Kim and Kim 1999), the permeability in the stream wise directions is
assumed to be flow rate dependent. Equation (22) accounts for a
straight flow through porous medium and does not take the two bends
in the micro-channel, or the impingement on the surface or the interface
between the fluid and the porous medium into account. Therefore, the
stream wise permeability is determined by fitting the pressure drop
from the inlet to the outlet nozzle to the pressure drop from the micro-
channel unit simulation while assuming the same boundary conditions
as in the micro-channel model. A second order polynomial is found
suitable to describe this flow dependent stream wise permeability as
Fig 3: Overall performance of heat sink
y ,z *10 9 = 0.7715+5.101*10 -4 Rein + 2.545*10 -8 Rein
2 ,
(26)
4.1 Micro-channel unit cell model
2 The performance of the micro-channel model for different flow rates is
where all coefficients have a unit of m . Since the micro-channel wall
are impermeable, permeability in the x direction is taken to be up to first obtained in order to gain insight into the feasibility of momentum
425 times smaller than the flow dependent values in Eq. (26). A and energy transport inside the hot water cooled heat sink. To analyze
progressively smaller reduction factors were used for lower flow rates, the heat transfer in the micro-channel structure, the local Nusselt
as a high value of reduction factor led to non-convergence of the model. number, defined as
This can be understood as follows. Equation (26) predicts that the
permeability values in y and z directions increase monotonically with h y Dh ,ch
flow rate (i.e. Rein). Therefore, for small flow rates keeping the Nu y = , (29)
kf
permeability reduction factor as high as 425 could lead to a very small
value of permeability in x direction. Since permeability appears in the
denominators of the Forchheimer terms in Eq. (22), an ultra low is determined at 20 equally spaced locations along the channel. In Eq.
permeability will cause a strong spike in that term, which could lead to (29), the local heat transfer coefficient is calculated using
non-convergence.
A uniform velocity is imposed at the inlet port of the manifold and q y,w
hy =
the average static pressure is set to zero at the outlet of an extended (T y ,w Ty ,bulk ), (30)
outlet pipe. The turbulence intensity I and the turbulence length scale
l are assumed to be 0.02 and 0.07 * d M ,in at the inlet. Symmetry is used where q y,w , Ty ,w and Ty ,bulk respectively denote the average local wall
at the centerline of the heat sink and the no-slip condition is imposed at
heat flux density, the average local wall temperature and the local fluid
all walls.
bulk temperature defined, respectively as
The same solver and convergence criterion as in the micro-channel
model were used to solve Eqs. (14) (15) and a first order solver was
used to solve the turbulence model Eqs. (17) (18). We employed a 1
qy,w = *
block-structured, hexagonal, non-uniform grid of 3345328 elements in ch + hch
w
all simulations of the heat sink model as further grid refinement
qy,ch,bottom ( x,0 ) dx+
changed the overall pressure drop only by 0.058 %. wch 2 (31)
,

wch 2 hch

4. RESULTS AND DISCUSSION 0

q
y,ch,top ( x,hCh ) dx+ q


y,ch,fin ( 0,z ) dz

In order to provide proof of principle for using the heat sink in a waste- 0 0
heat recovery fluid loop, the inlet temperature was set to T f ,in = 60 C in 1
Ty,w = *
all simulations. The dissipated power of the chip is kept constant at wch + hch
Q
chip = 90 W and the total volumetric flow rate is varied in the range 0.1 T y,ch,bottom ( x,0 ) dx +
l/min 1 l/min. The inlet Reynolds-number defined as wch 2 (32)
,

wch 2 hch

uin d M ,in V
tot 0 T y,ch,top ( x,hCh ) dx + T

y,ch, fin ( 0,z )
dz
Rein = = , (27)
d M ,in 4 0 0

varies from 1599 < Rein < 15989.

We evaluated the overall thermal and hydrodynamic performance
of the heat sink considering the total thermal resistance
5
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
wHT ,Ch / 2 hHT ,Ch ISSN:
(side) walls shortly after entering the channel section of 2151-8629
the heat

transfer structure. Near the outlet, a recirculation zone and a stagnation
u yTdzdx
zone are being formed near the inner wall of the outlet nozzle and
Ty,bulk = 0 0 . opposite to the outlet nozzle, respectively.
wHT ,Ch / 2 hHT ,Ch
(33)
The differences in the flow field for varying flow rates create
u y dzdx different temperature and heat transfer patterns along the micro-
channel. The average temperatures at the micro-channel outlet and at
0 0
the heat sink bottom can be calculated as
For analyzing the heat flux through the different micro-channel walls
1
into water, a normalized heat flux density in the form Tn,out = *
qy,ch,i ( wCh )
2+w fin 2 wn,out
,
q*y,i = (34) wCh 2 Lch (35)

q0.225mm,ch, fiin
( )
T z = h fin + hn,out dydx
-w fin 2 Lch -wn,out
is introduced. wCh 2 Lch
1
Tbottom=
( wCh 2+ w fin 2 Lch) T ( z=-hbase )dydx . (36)
-w fin 2 0

The temperature difference between the liquid inlet and the highest
micro-channel temperatures is a measure of the gradients developing in
the heat sink. These temperature differences are plotted in Figure 7
against ReCh corresponding to the different flow rates considered in this
work (up to 1 l/min). High flow rates reduce the thermal resistance of
the heat sink and cause a small fluid temperature change from the

Fig 4: Vectors and velocity contours at center-plane of micro-channel

model for ReCh = 303 (volume flow rate=1 l/min)

Fig 7: Difference of average heat sink bottom temperature and average

Fig 5: Vectors and velocity contours at center-plane of micro-channel
fluid outlet temperature to fluid inlet temperature
model for ReCh = 152 (volume flow rate=0.5 l/min)

Fig 6: Vectors and velocity contours at center-plane of micro-channel Fig 8: Temperature contours at center-plane of micro-channel model
model for ReCh = 30 (volume flow rate=0.1 l/min) for ReCh=303 (volume flow rate=1 l/min)
The flow vectors and the velocity distribution at the centerline of micro-channel inlet to the outlet due to the short residence time within
the micro-channel for three different flow rates are presented in Figures the channel. This effect is also visible in the temperature contours at the
4 6. The water enters the micro-channel by impinging onto the surface micro-channel center-plane which are shown for three different flow
opposite to the inlet nozzle creating a small stagnation zone close to the rates in Figures 8 10. Areas of small temperature change coincide
vertical symmetry axis (Webb and Ma 1995). A recirculation zone is fairly well with the high flow velocity areas in Figures 4 6. The
formed at the top wall near the inlet nozzle decreasing the effective highest temperatures are found in the downstream part of the heat sink
flow cross section in the micro-channel and accelerating the flow. The bottom. The variations of the normalized heat flux density, through all
size of this recirculation zone decreases with smaller flow rates and the different solid-liquid interfaces in the micro-channel, along channel
almost vanishes at the lowest investigated flow rate. Away from channel length are plotted in Figure 11. The trends in the heat flux density
inlet, the liquid fills up the entire channel cross-section and the flow variation can be understood as follows. At high and medium flow rates,
starts developing with respect to the top and bottom wall. In contrast, the water falls on to the channel bottom as a strong impinging jet. This
the flow is hydrodynamically fully developed with respect to the fin jet impingement impedes the build-up of a hydrodynamic and a thermal
6
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
the different solid-liquid interfaces in the micro-channel, along channel ISSN:
fin surface is thermally developed stopping any further 2151-8629
reduction in
length are plotted in Figure 11. The trends in the heat flux density heat transfer. At low flow rates, contrary to high flow rates, the
variation can be understood as follows. At high and medium flow rates, recirculation zone close to the inlet nozzle vanishes and the entire fin
the different solid-liquid interfaces in the micro-channel, along channel surface is effectively used for heat transfer. Therefore, the heat flux
density through the fin decreases monotonically downstream due to the
decreasing temperature gradient between fin surface and water bulk
temperature. The variation of the recirculation zone near the top for
different flow rates also affects the heat flux density at the top wall of
the channel. At high and medium flow rates, the recirculation zone
inhibits effective heat transfer trough the top wall close to the inlet
nozzle. After the recirculation zone, the channel fills up resulting in
increasing velocity gradient with respect to the top wall and therefore
increasing heat flux density along the channel length. The vanishing
recirculation zone allows for thermal boundary layer growth
downstream causing the heat flux density to decrease. The stagnation
zone opposite to the outlet nozzle causes a large diffusion length for
Fig 9: Temperature contours at center-plane of micro-channel model heat transfer and a reduced heat flux density in this region. Since, in
for ReCh=152 (volume flow rate=0.5 l/min) this region, the thermal resistance to the fin is smaller than to the fluid
bulk, the heat is transported from the channel bottom through the liquid
to the fin for small flow rates inducing the average heat flux to even
turn negative and the bulk temperature to be higher than the average
wall temperature.

Fig 10: Temperature contours at center-plane of micro-channel model

for ReCh=30 (volume flow rate=0.1 l/min)
length are plotted in Figure 11. The trends in the heat flux density
variation can be understood as follows. At high and medium flow rates,
the water falls on to the channel bottom as a strong impinging jet. This
jet impingement impedes the build-up of a hydrodynamic and a thermal
boundary layer at the channel bottom reducing the thermal diffusion Fig 12: Axial development of Nu-number
length to the convective fluid. Away from the inlet region, however, the
thermal boundary layer growth results in a smaller heat transfer rate at The effect of progressively stronger impingement with increasing
the bottom channel (see Figure 11). In contrast, the inlet flow flow rate is evident in the axial development of the Nusselt number
momentum is not high for low flow rate. Therefore, water does not plotted in Figure 12, where it is higher for high volume flows in the
impinge on the micro-channel bottom with sufficient strength and the inlet region of the micro-channel. Away from the inlet region and before
heat flux is almost constant along the channel. The slight increase of the the outlet, the flow becomes similar to a simple channel flow. The heat
heat transfer rate in the first part of the channel is attributed to the transfer through the fin surface is dominant in the investigated high
increased velocity gradient in this section. For the fin (side) wall, the aspect ratio micro-channel due to the large heat transfer area compared
heat flux density is highest in the inlet region and decreases sharply to the bottom and top wall of the channel. Moreover, the heat flux
downstream along the channel length. Right after the inlet region, the density of the fin surface is higher than for the other liquid-solid
recirculation zone near the channel top reduces effective heat transfer surfaces as the hydrodynamic and thermal boundary layers are thinnest
area causing the heat transfer to decrease. Moreover, a thermal for the fin surface. For all investigated flow rates, the flow is
boundary layer should also start to grow on the fin surface and should hydrodynamically and thermally fully developed for the fin surface
shortly after entering the channel section causing the Nu-number to
approach a value of 6.25 close to end of the micro-channel, before the
outlet section. Spiga and Morini (Spiga and Morini 1996) analytically
determined Nusselt numbers for hydrodynamically and thermally fully
developed laminar flow in rectangular ducts imposing constant heat
flux to both long sides of the channel and considering the short sides to
be adiabatic . Their value for channels of the aspect ratio 10 is NuD =
6.096, which serves as a good validation for the accuracy of the model
developed here.
4.2 Thermal resistance model
The variation of the total thermal resistance of single micro-channel
based on the thermal resistance model is presented Figure 3. The one
Fig 11: Axial development of relative heat flux density at channel dimensional resistance model overestimates the thermal resistance by
bottom, channel top and fin wall less than 10% for all applied volumetric flow rates. The bulk resistance
is the dominant thermal resistance and decreases with increasing flow
partially account for the heat flux reduction. Further downstream the rate. Therefore, the fluid temperature gradient within the micro-channel
liquid fills up the entire channel and the thermal boundary layer on the decreases with increasing flow rate. Since we assume an adiabatic tip of
7
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI:
the 10.5098/hmt.v1.2.3006
fin in our resistance model, we neglect the heat transport from the ISSN:
Vortices and secondary flows disturb the main flow 2151-8629
in both legs
top wall to the fluid. Furthermore, we applied a constant heat transfer of the outlet manifold. This should lead to higher pressure loss
coefficient in our convective resistance term neglecting the enhanced especially at high flow rates. The flow streams from two parts of the
heat transfer in the inlet region due to the impinging jet. These two first leg of the outlet manifold run into each other in the second leg.
assumptions may cause the thermal resistance model to under predict This generates turbulence kinetic energy and high shear stresses
the heat transfer within the single micro-channel. contributing further to the pressure loss.
Distribution of coolant in the micro-channels is crucial to ensure
4.3 Complete heat sink model improved performance of heat sink. In order to extract the mass flux
distribution into the channels, we divided the cross section of the heat
transfer structure at y = LCh/2 in equally spaced sections of width x .
The relative mass flow rate was calculated using

xi +x/2 hHT,Ch

i dzdx
m
xi -x/2 0

i* =
m . (37)
N i x +x/2 hHT,Ch
Fig 13: Turbulent kinetic energy contours and velocity vectors at

1
center-plane of the inlet manifold at Rein = 7995 (volume flow rate=0.5
l/min)
N
i=1

i dzdx
m
xi -x/2 0
In order to gain insight into the heat sink performance, it is also
important to analyze the flow pattern in the manifold and the flow The resulting relative mass fluxes are plotted in Figure 15 for the three
distribution within the heat transfer structure in addition to computing different flow rates designated by three different Rein. The mass flux
the overall pressure drop (Figure 3). The flow structure in the manifolds distribution into the micro-channels seems to vary quite strongly
did not vary much at different flow rates considered here. Therefore, for depending on the channel position in x direction. This is expected since
the sake of brevity we only show the velocity vectors and the turbulence the water enters as a jet into the inlet manifold (c.f. Figure 13). The
kinetic energy distributions for Rein = 7995 in Figure 13 and Figure 14. three different flow rates in Figure 15 correspond to ReCh values of 30,
As seen from Figure 13, the coolant is laterally supplied through a pipe 152 and 303 respectively, if the flow distribution into all channels were
to the inlet manifold creating a flow core in the inlet manifold, which to be uniform. Due to non-uniform flow distribution, the lowest relative
spreads slightly when approaching towards the heat transfer structure. mass fractions into channels, as seen from Figure 15, are 0.35, 0.4 and
A small vortex is generated close to the inlet due to the backward- 0.55 respectively, for the low (0.1 l/min), medium (0.5 l/min) and high
facing step type geometry. Additionally, the back wall of the inlet (1 l/min) flow rates considered. These fractions correspond to lowest
manifold, at the far end in the flow direction in Figure 13, induces a ReCh values of 10.5, 61 and 167, respectively. A low value of ReCh can
large recirculation zone above the inlet flow core. The flow core drags lead to an unacceptably high chip temperature as seen in Figure 7.
the surrounding fluid in the flow direction generating high shear Moreover, the minimum ReCh studied, i.e. ReCh =30 (with
stresses and turbulence which is shown by the high turbulence kinetic corresponding volumetric flow rate of 0.1 l/min), leads to a 15C
energy in this area. Figure 14 shows the velocity vectors and turbulent difference between maximum heat sink and water inlet (60C)
kinetic energy contours for the mid yz-plane of the heat sink. It clarifies temperatures. Therefore, the minimum ReCh of 10.5 observed for the
the flow patterns in the heat transfer structure modeled as porous lowest flow rates will clearly lead to a much higher chip temperature,
medium. Upon entering the porous medium, the liquid momentum in which makes the lowest flow rate unacceptable. However, for the
flow direction changes sharply, creating a second zone of high medium and high flow rates, the minimum ReCh values (61 and 167) are
turbulence. However, the turbulence kinetic energy dissipates quickly high enough for a very good thermal performance of heat sink with
while flowing through the porous medium and remains small in the first inlet water temperature of 60C. The maximal temperature difference of
leg of the outlet section of the manifold (see Figure 14). The quick ~8C (see Figure 7) for these flow rates should ensure a minimal
turbulence dissipation in the first part of the porous medium (heat entropy generation due to a heat transport across a temperature
transfer structure) shows that turbulent modeling of the hydrodynamics differential. The higher mass flows through the micro-channels lying in
of entire heat sink is not compromised by low turbulence in heat the middle of the heat sink (see Figure 15) should only improve the
transfer structure (micro-channels).Therefore, use of a laminar model cooling effectiveness and not degrade it. Moreover, the outlet
(as was done in Section 4.1) is adequate to evaluate the heat transfer in temperature of at least 62C, makes it useful for secondary applications
single micro-channel unit cell. A detailed thermal model for entire heat such as heating the building hosting such as hot water cooled
sink including the inlet and outlet manifolds, however, should certainly datacenter. Experimental validation of this principle and investigation
use turbulent approach. This will be considered in our future work. into a prototype water cooled datacenter is currently under investigation
by group and would be reported in near future.

5. CONCLUSION AND OUTLOOK

Using numerical modeling, feasibility of electronic datacenter cooling
using hot water is explored for high system exergetic utility. The model
considers a manifold micro-channel (MMC) heat sink, with high aspect
ratio (10) micro-channels, suitable for large scale fabrication and
integration. A three dimensional model of conjugate heat transport in a
micro-channel unit cell, a thermal resistance and a three dimensional
hydrodynamic model of the complete heat sink were developed to
evaluate the thermal performance of the heat sink and also to explore
Fig 14: Turbulent kinetic energy contours and velocity vectors of yz- the possibility of waste-heat recovery. In the heat sink hydrodynamic
plane at x=LM,in/2 for Rein =7995 (volume flow rate=0.5 l/min) model, a k - type turbulent model was used to capture the high shear
stresses in the supply and collection manifolds, whereas the flow
8
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
distribution in the micro-channels (heat transfer structures) was specific dissipation rate of turbulent kinetic ISSN:
energy2151-8629
(m/s)
obtained by modeling the heat transfer structure as porous medium, Subscripts
which saved computation time. base heat sink base plate
As Expected, the thermal resistance of the heat sink decreased with bottom bottom of micro-channel or heat sink
increasing flow rate while the pressure drop across the heat sink bulk bulk
increased. The local Nu-number close to the outlet of the high aspect ch micro-channel
ratio micro-channels approached 6.25, a value comparing well with the conv convection
literature value of 6.096 for channels heated through two walls. The D developed
manifold was found to distribute the flow non-uniformly to the micro- f fluid
channels. The repercussion of this non-uniform distribution was fin fin
analyzed. Indeed, an easily achievable minimum flow rate of 0.5l/min HT heat transfer structure
for water at 60C inlet temperature was found sufficient to address the i index
thermal loads typical in a datacenter chip. The maximum temperature in inlet
difference of ~8C was found between the solid and liquid, confirming M manifold
small exergetic destruction due to heat transport across a temperature n nozzle
differential. Furthermore, the high water outlet temperature from the out outlet
heat sink opens the possibility for secondary use of the waste heat. s solid material
A conjugate heat transport model of the entire heat sink and tot total
experimental validation of the proof-of-principle of hot water cooled w wall
datacenters will be investigated in our future work.
REFERENCES
ACKNOWLEDGEMENTS Beavers, G. S., Sparrow, E. M., and D.E., R., 1973, "Influence of Bed
We acknowledge G. I. Meijer and S. Paredes for sharing the mechanical Size on the Flow Characteristics and Porosity of Randomly Packed
design parameters (Table 1) and thermal simulations of these cold Beds of Spheres," Transactions ASME, Series E, Journal of Applied
plates, which are used in the IBM BladeCenter Cluster Aquasar. Mechanics, 40 (3), 655-660.
Brunschwiler, T., and Michel, B., 2008, "Thermal Management of
NOMENCLATURE Vertically Integrated Packages," Handbook of 3D Integration, WILEY-
VCH Verlag GmbH & Co, KGaA, Weinheim.
cp specific heat capacity (J/kgK) doi: 10.1002/9783527623051.ch33
D characteristic length scale (m)
Dh Hydraulic diameter (m) Brunschwiler, T., Rothuizen, H., Fabbri, M., Kloter, U., Michel, B.,
d diameter (m) Bezama, R. J., and Natarajan, G., 2006, "Direct Liquid Jet-Impingement
h height (m) Cooling with Micron-Sized Nozzle Array and Distributed Return
h heat transfer coefficient (W/mK) Architecture," IEEE Proceedings 10th Intersociety Conference on
k turbulent kinetic energy (m/s) Thermal and Thermomechanical Phenomena in Electronics Systems.
k thermal conductivity (W/mK) doi: 10.1109/ITHERM.2006.1645343
L length (m) Brunschwiler, T., Smith, B., Ruetsche, E., and Michel, B., 2009,
m
mass flow rate (kg/s) "Toward Zero-Emission Data Centers through Direct Reuse of Thermal
m
* relative mass flow rate (-) Energy," IBM Journal of Research & Devlopement, 53 (3), Paper 11.
N number (-) doi: 10.1147/JRD.2009.5429024
Nu Nusselt-number (-) Chen, Q., and Modi, V., 1999, "Mass Transfer in Turbulent Impinging
p pressure (N/m) Slot Jets," International Journal of Heat and Mass Transfer, 42 (5),
q heat flux density (W/m) 873-887.
q* relative heat flux density (-) doi: 10.1016/S0017-9310(98)00215-4
R thermal resistance (mK/W)
Re Reynolds-number (-) Colgan, E. G., Furman, B., Gaynes, M., Graham, W. S., Labianca, N.
Sp body force due to porous medium (kg/ms) C., Magerlein, J. H., Polastre, R. J., Rothwell, M. B., Bezama, R. J.,
T temperature (K) Choudhary, R., Marston, K. C., Toy, H., Wakil, J., Zitz, J. A., and
U mean velocity (m/s) Schmidt, R. R., 2007, "A Practical Implementation of Silicon
u velocity (m/s) Microchannel Coolers for High Power Chips," IEEE Transactions on
Components and Packaging Technologies, 30 (2), 218-225.
V
volumetric flow rate (m/s)
doi: 10.1109/tcapt.2007.897977
w width (m)
xi coordinates (m) Copeland, D., Behnia, M., and Nakayama, W., 1997, "Manifold
y+ dimensionless wall distance (-) Microchannel Heat Sinks: Isothermal Analysis," IEEE Transactions on
y distance between first mesh points normal to the wall (m) Components, Packaging, and Manufacturing Technology Part A, 20
Greek Symbols (2), 96 -102.
closure coefficient of k- model (-) doi: 10.1109/95.588554
closure coefficient of k- model (-)
closure coefficient of k- model (-) Escher, W., Brunschwiler, T., Michel, B., and Poulikakos, D., 2010a,
fin efficiency (-) "Experimental Investigation of an Ultra-Thin Manifold Microchannel
thermal conductivity (W/mK) Heat Sink for Liquid-Cooled Chips," ASME Journal of Heat Transfer,
dynamic viscosity (kg/ms) 132 (8), 081402.
t eddy viscosity (kg/ms) doi: 10.1115/1.4001306
density (kg/m)
k closure coefficient of k- model (-)
closure coefficient of k- model (-)
9
Frontiers in Heat and Mass Transfer (FHMT), 1, 023006 (2010) Global Digital Central
DOI: 10.5098/hmt.v1.2.3006
Escher, W., Michel, B., and Poulikakos, D., 2010b, "A Novel High ISSN: 2151-8629
Ryu, J. H., Choi, D. H., and Kim, S. J., 2003, "Three-Dimensional
Performance, Ultra Thin Heat Sink for Electronics," International Numerical Optimization of a Manifold Microchannel Heat Sink,"
Journal of Heat and Fluid Flow, 31 (4), 586-598. International Journal of Heat and Mass Transfer, 46 (9), 1553 - 1562.
doi: 10.1016/j.ijheatfluidflow.2010.03.001 doi: 10.1016/S0017-9310(02)00443-X
Fedorov, A. G., and Viskanta, R., 2000, "Three-Dimensional Conjugate Spiga, M., and Morini, G. L., 1996, "Nusselt numbers in laminar flow
Heat Transfer in the Microchannel Heat Sink for Electronic Packaging," for H2 boundary conditions," International Journal of Heat and Mass
International Journal of Heat and Mass Transfer, 43 (3), 399 - 415. Transfer, 39 (6), 1165-1174.
doi: 10.1016/S0017-9310(99)00151-9 doi: 10.1016/0017-9310(95)00205-7
Heyerichs, K., and Pollard, A., 1996, "Heat Transfer in Separated and Tuckerman, D. B., and Pease, R. F. W., 1981, "High-performance heat
Impinging Turbulent Flows," International Journal of Heat and Mass sinking for VLSI," Electron Device Letters, 2 (5), 126-129.
Transfer, 39 (12), 2385-2400.
Waelchli, R., Brunschwiler, T., Michel, B., and Poulikakos, D., 2010a,
doi: 10.1016/0017-9310(95)00347-9
"Combined local microchannel-scale CFD modeling and global chip
Incropera, F. P., and Dewitt, D. P., 2002, Fundamentals of Heat and scale network modeling for electronics cooling design," International
Mass Transfer, 5th ed., John Wiley & Sons, Inc. Journal of Heat and Mass Transfer, 53 (5-6), 1004-1014.
Jang, S. P., and Kim, S. J., 2005, "Fluid Flow and Thermal doi: 10.1016/j.ijheatmasstransfer.2009.11.012
Characteristics of a Microchannel Heat Sink Subject to an Impinging
Waelchli, R., Brunschwiler, T., Michel, B., and Poulikakos, D., 2010b,
Air Jet," Journal of Heat Transfer-Transactions of the ASME, 127 (7),
"Self-Contained, Oscillating Flow Liquid Cooling System for Thin
770-779.
Form Factor High Performance Electronics," Journal of Heat Transfer-
doi: 10.1115/1.1924628
Transactions of the Asme, 132 (5).
Kim, S. J., and Kim, D., 1999, "Forced Convection in Microstructures doi: 051401 10.1115/1.4000456
for Electronic Equipment Cooling," Journal of Heat Transfer-
Webb, B. W., and Ma, C. F., 1995, "Single-phase liquid jet
Transactions of the ASME, 121 (3), 639-645.
doi: 10.1115/1.2826027 impingement heat transfer," Advances in Heat Transfer, 26, 105-217.
doi: 10.1016/S0065-2717(08)70296-X
Koomey, J. G., 2008, "Worldwide Electricity Used in Data Centers,"
Wilcox, D. C., 1998, Turbulence Modeling for CFD, 2nd ed., DWC
Environmental Research Letters, 3 (3).
Industries, La Canada, California.
doi: 10.1088/1748-9326/3/3/034008
Meijer, G. I., 2010, "Cooling Energy-Hungry Data Centers," Science,
328 (5976), 318-319.
doi: 10.1126/science.1182769

10