Applied Thermal Engineering: Jackson Braz Marcinichen, Jonathan Albert Olivier, John Richard Thome
Applied Thermal Engineering: Jackson Braz Marcinichen, Jonathan Albert Olivier, John Richard Thome
Applied Thermal Engineering: Jackson Braz Marcinichen, Jonathan Albert Olivier, John Richard Thome
a r t i c l e i n f o a b s t r a c t
Article history: Cooling of datacenters is estimated to have an annual electricity cost of 1.4 billion dollars in the United
Received 5 May 2011 States and 3.6 billion dollars worldwide. Currently, refrigerated air is the most widely used means of
Accepted 5 December 2011 cooling datacenter’s servers, which typically represents 40e45% of the total energy consumed in
Available online xxx
a datacenter. Based on the above issues, thermal designers of datacenters and server manufacturers now
seem to agree that there is an immediate need to improve the server cooling process. The goal of the
Keywords:
present study is to propose and simulate the performance of a novel hybrid two-phase cooling cycle with
Datacenter
micro-evaporator elements (multi-microchannel evaporators) for direct cooling of the chips and auxil-
Microprocessor
Hybrid two-phase cooling cycle
iary electronics on blade server boards (savings in energy consumption of over 60% are expected).
Micro-evaporator Different working fluids were considered, namely water, HFC134a and a new, more environmentally
Power plant friendly, refrigerant HFO1234ze. The results so far demonstrated that the pumping power consumption is
Energy recovery on the order of 5 times higher for the water-cooled cycle. Additionally, a case study considering the
hybrid cooling cycle applied on a datacenter and exploring the application of energy recovered in the
condenser on a feedwater heater of a coal power plant was also investigated (modern datacenters require
the dissipation of 5e15 MW of heat). Aspects such as minimization of energy consumption and CO2
footprint and maximization of energy recovery (exergetic efficiency) and power plant efficiency are
investigated.
Ó 2011 Elsevier Ltd. All rights reserved.
1359-4311/$ e see front matter Ó 2011 Elsevier Ltd. All rights reserved.
doi:10.1016/j.applthermaleng.2011.12.008
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
2 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
Nomenclature Acronyms
AE auxiliary electronics
CHF critical heat flux [Wcm2]
Roman COP coefficient of performance [-]
E_ d rate of exergy destruction due to irreversibilities CPU central processing unit
within the control volume, W EEV electric expansion valve
e_ fi, e_ fe inlet and outlet flow exergies, J/kg iHEx internal heat exchanger
m _ i; m
_e inlet and outlet mass flow rate, kg/s LA liquid accumulator
Q AE heat load associated with the auxiliary electronics, W LP liquid pump
Q cond heat transfer rate at the condenser, W LPR low pressure receiver
Q_ j heat transfer rate, W ME micro-evaporator
Q ME heat load associated with the microprocessor, W MPAE microchannel cold plate applied on auxiliary
Q sub heat transfer rate at the subcooler, W electronics
Tj instantaneous temperature,C PCV pressure control valve
T0 dead state temperature,C SP_W single-phase liquid water in the ME
W _ energy transfer rate by work, W TCV temperature control valve
CV
WLP_or_VC pumping power consumption of the driver, which can TP two-phase
be the liquid pump or the vapor compressor, W TP_HFC134a two-phase HFC134a in the ME
TP_HFO1234ze two-phase HFO1234ze in the ME
Subscripts VC vapor compressor
cond condenser VSC variable speed compressor
sub subcooler
single-phase flow, porous media flow, jet impingement cooling and heat also implies that chip temperatures are much more uniform. It
microchannel two-phase flow [1]. [25] showed that the use of such has been shown that heat fluxes as high as 300 W/cm2 [40] can be
new technology for cooling of chips could produce savings in achieved using a two-phase refrigerant while maintaining chip
energy consumption of over 60% [1] highlighted that the most temperatures below their critical value. Heat fluxes of 1000 W/cm2
promising of the four technologies was microchannel two-phase have been reached using refrigerants [24], although this was under
cooling. subcooled boiling with mass fluxes and pressure drops being an
Single-phase microchannel chip cooling has its apparent order of magnitude higher than microchannel two-phase boiling. It
advantage in that it is relatively easy to use. This technology is was further shown that heat fluxes of 180 W/cm2 can be removed
currently being used in the recently started Aquasar [16], which is with a saturation temperature as high as 60 C, while maintaining
an IBM blade center converted to make use of a water-cooled cycle. the chip temperature below 85 C [27]. A refrigerant has the
However, water has many disadvantages in that it requires a high advantage of being a dielectric fluid with a long successful history
pumping power to keep temperature gradients on the micropro- in industrial applications, is inert to most engineering materials,
cessor to within acceptable limits. Furthermore, water presents readily available and relatively inexpensive. New refrigerants also
a problem with its high freezing point, microbe formation, corro- have a negligible impact on the environment.
sive properties, electrical conductivity and erosive nature due to The main advantage of making use of on-chip cooling for the
high fluid velocities. cooling of datacenters is that the heat gained from cooling the chips
Two-phase microchannel flow takes advantage of the latent can be easily reused elsewhere. This is because the heat removal
heat of the fluid, which is much more effective in removing heat process is local to the chip and the fact that the heat carrying
than when using the sensible heat of a single-phase fluid. The latent capacity of the refrigerant is much higher than that of air, thus
minimizing any losses to the environment. The opportunity thus
exists to reuse the datacenter’s waste heat in secondary applica-
tions instead of being disposed of into the atmosphere. This not
only has the potential to reduce datacenter energy costs, but also to
reduce its carbon footprint and hence, its environmental impact.
A problem with recovering the heat of the datacenter is not in
the quantity of heat available, but rather in the quality of heat.
Currently, heat is being ejected into the atmosphere at tempera-
tures of about 40 C when using traditional air cooling methods.
This is because chip temperatures are being cooled at 15e20 C.
Due to the effective cooling of chips when using on-chip cooling,
fluid approach temperatures of about 60 C can be realized, while
removing high heat fluxes and keeping chip temperatures below
85 C [27]. Although the temperature, and hence the quality of the
heat is higher, the applications for using this heat are still limited.
Making use of a vapor compression cooling cycle, where
temperatures in excess of 90 C can be realized, can increase the
quality of the heat, i.e., the exergy, even further. Exergy of a system
is the maximum useful energy that brings the system into the
equilibrium with its surroundings. After the system and
Fig. 1. Heat flux trend of commercial processors. Red markers from [49]. surroundings reach equilibrium, the exergy is zero. The potential
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 3
for this kind of heat not only increases the number of applications small charge (a few milliliters). For a baseline operating condition,
that can use it, but also increases its value. Vapor compression when the evaporator and condenser temperatures and the heat
cooling cycles for computer chips have been demonstrated load were 50 C, 90 C and 50 W, the coefficient of performance
successfully [34] and [55], although it was focused on a small (COP) obtained was 2.25. The COP reached 3.70 when the
system where the recovery of heat was not of concern. evaporator and condenser temperatures increased and decreased
In this context, the objective of the present study is primarily to by 10 C from the baseline conditions and the heat load was
propose and analyze potential two-phase cooling cycles able to reduced to 44 W. The small-scale refrigeration system achieved
maintain the temperature of the chip below its upper operating 25e30% of the Carnot efficiency (ideal COP for a Carnot cycle),
limit (about 85 C) and to recover energy from the cycle’s values comparable with those obtained in today’s household
condenser for reuse, such as for heating a building, residence, refrigerators.
hospital, district heating, etc. To do this, an in-house integrated [55] designed, built and evaluated the performance of a miniature-
simulation code was developed which has the capability to design scale refrigeration system (MSRS) suitable for electronics cooling
the components and determine the performance of a hybrid cool- applications. Their MSRS had the following components: a commer-
ing cycle (liquid pump or vapor compressor as the driver of the cial small-scale compressor, a microchannel condenser, a manual
working fluid) under steady state conditions and for different needle valve as the expansion device, a cold plate microchannel
working fluids. The code is also able to evaluate the performance of evaporator, a heat spreader and two compressor cooling fans. A
the cooling cycles for single-phase and two-phase flow in the suction accumulator to avoid liquid flow to the compressor, an oil
micro-evaporators. filter to return oil to the compressor and guarantee good lubrication,
The new refrigerant HFO1234ze of Honeywell Inc. is considered and heat sources to simulate the chips were also installed. HFC134a
here as a potential substitute of HFC134a. This fluid has a “Global was the working fluid. System performance measurements were
Warning Potential” of only 6 against 1410 of HFC134a, i.e., it is conducted at evaporator temperatures from 10 C to 20 C and
considered as an immediate/future replacement for HFC134a. Both condenser temperatures from 40 C to 60 C. The cooling capacity of
HFC134a and HFO1234ze are dielectric fluids and thus compatible the system varied from 121 W to 268 W with a COP of 1.9e3.2 at
with electronics. HFC134a is currently the most widely used pressure ratios of 1.9e3.2. Their MSRS was able to dissipate CPU heat
refrigerant for refrigeration and air conditioning systems. fluxes of approximately 40e75 W/cm2 and keep the junction
Secondly, a case study is developed to demonstrate the overall temperature below 85 C for a chip size of 1.9 cm2. It was concluded
potential savings that can be made by implementing two-phase that a new compressor design for electronics cooling applications was
on-chip cooling within a datacenter. The investigation will take needed to achieve better performance of the system (the most
into account the benefits over the datacenter and a secondary significant losses occurred in the compressor, which was not designed
application of the waste heat. For this paper a coal fired thermal for the operating conditions of electronics cooling). It was also rec-
power plant will be analysed since more than 41% of the world’s ommended to study the development of an automatic expansion
energy is produced from these types of plants [57]. This plant was device and a suitable control strategy for the MSRS.
also chosen as it produces most of the greenhouse gases, with any [52] surveyed the advances in thermal modeling for flow boiling
potential improvements in efficiency translating to savings for both of low pressure refrigerants in multi-microchannel evaporators for
the power plant and the datacenter. cooling of microprocessors. According to them, multi-microchannel
evaporators hold promise to replace the actual air cooling systems
2. Literature review and can compete with water cooling to remove high heat fluxes,
higher than 300 W/cm2, while maintaining the chip safely below its
[18] have proposed a pumped liquid multiphase cooling system maximum working temperature, providing a nearly uniform chip
(PLMC) to cool microprocessors and microcontrollers of high-end base temperature [2] and minimizing energy consumption.
devices such as computers, telecommunications switches, high- Variables such as critical heat fluxes, flow boiling heat transfer
energy laser arrays and high-power radars. According to them, coefficients and two-phase friction factors were evaluated and
their system could handle applications with 100 W heat loads characterized as important design parameters to the micro-
(single computer chip) as well as applications with short time evaporator for high heat flux applications.
periods of kW heat loads (radar). Their PLMC consists of a liquid [53] simulated two-phase cooling elements for microprocessors
pump, a high performance cold plate (evaporator) and a condenser with micro-evaporation. Heat fluxes of 50 W/cm2 and 150 W/cm2
with a low acoustic noise air mover to dissipate the heat in the in a micro-evaporator with channels 75 mm wide, 680 mm high and
ambient air. A comparison between a single-phase liquid loop 6 mm long with 100 mm thick fins were simulated for flow boiling.
(water) and the system proposed with HFC134a was made for The size of the chip was assumed to be 12 mm by 18 mm and the
a 200 W heat load. The HFC134a system had a mass flow rate, micro-evaporator was considered with the fluid inlet at the
a pumping power and a condenser size that were 4.6, 10 and 2 centerline of the chip and outlets at both sides, i.e., a split flow
times smaller than the water-cooled system. The coolant temper- design to reduce the pressure drop and increase the critical heat
ature rise was 10 C for the water but negligible for HFC134a. They flux. Results of pumping power, critical heat flux, and junction and
emphasized the significant benefits from efficiency, size and weight fluid temperatures were generated for HFC134a at an inlet satura-
that were provided with the PLMC solution. tion temperature of 55 C (chosen to allow for heat recovery). The
[34] designed and built a small-scale refrigeration system following conclusions were reached: i) the influence of mass flux on
applicable for a notebook computer. The system included a mini- the fluid, chip and wall temperatures was small, ii) for the heat flux
compressor, a microchannel condenser, a microchannel evaporator of 150 W/cm2, the chip temperature was 70 C or less, i.e., well
and a capillary tube as the throttling device and is considered to be below its operational limit of 85 C, iii) the junction-to-fluid
the first refrigeration system developed that can fit within the tight temperature difference was only 15 K for the heat flux of
confines of a notebook and operate with high refrigeration effi- 150 W/cm2, which is lower than that with liquid cooling systems,
ciencies. HC600a (isobutane) was the working fluid, chosen from an iv) the fluid working temperature could still be raised by 10 K to
evaluation of 40 candidate refrigerants. According to them, HC600a a junction temperature of 80 C while rejecting heat at 65 C for
presented the best efficiency at a low pressure ratio and was readily reuse, and v) the critical heat flux increased with the mass flux and
available. Although it is flammable the system only required a very the lower limit was about 150 W/cm2 for 250 kg/m2s. The channel
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
4 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
width had a significant effect on the wall and junction tempera- heat fluxes are achieved with a smaller EEV opening and higher
tures, and there was a turning point at about 100 mm when heat input supplied to the accumulator and iii) a trade-off between
considering 1000 kg/m2s of mass flux and 150 W/cm2 of base heat the system COP and CHF is necessary to prevent the device burnout,
flux, at which these temperatures reached a minimum. For the i.e., imposed heat flux must be lower than the CHF considering
same mass flux and base heat flux, the reduction of channel width a safety margin. Finally, they presented a preliminary validation of
also reduced the energy consumption to drive the flow (pumping the model with initial experimental data showing a satisfactory
power). prediction ability of the model. The authors do not mention
From a system viewpoint [53], showed an approximate anything about the geometry assumed for the evaporators.
comparison of performances of liquid water cooling versus two- The present study is a continuation of the earlier study devel-
phase cooling. For the same pumping power consumption to oped by [28]. They proposed a two-phase cooling cycle considering
drive the fluids, two-phase cooling allowed the chip to operate a liquid pump as the driver of the fluids, a micro-evaporator for
about 13 K cooler than water cooling or it could operate at the same cooling the chip and its auxiliary electronics, and a microfin tube-
junction temperature but consume less pumping power using in-tube condenser for heat recovery. A standard length (30 cm)
a lower refrigerant flow rate. The two-phase cooling system and internal diameter (3 mm) was defined for the pipes joining the
appeared to be more energy-efficient than classical air cooling or components, which were assumed to be only straight and hori-
direct liquid cooling systems while also exhausting the heat at zontal, i.e., the effects of bends and static height difference between
higher reusable temperatures. Regarding the choice between components and pipes were ignored. The performance of the
a pump and a compressor as the driver for a micro-evaporation cooling cycle considered 3 different working fluids; HFC134a,
heat sink system, they emphasized that the choice depends on HFO1234ze and water (in an analogous single-phase cooling cycle).
the economic value of the reused energy. The system with The results showed that for a design of the cooling cycle so that the
a compressor is ideal for energy reuse because of the higher heat total pressure drop is about 1 bar, the liquid water cooling cycle had
rejection temperature; however the additional energy consumed a pumping power consumption 16.5 times that obtained for the
by the compressor compared to the pump has to be justified by the two-phase HFC134a cooling cycle. When comparing with the
reuse application. HFO1234ze cooling cycle, which showed a total pressure drop of
[31] evaluated the performance of a multi-microchannel copper 1.2 bar, the difference drops to 13.2 times. These results can be
heat sink with respect to critical heat flux (CHF) and two-phase considered a differential when compared with demonstration
pressure drop. A heat sink with 29 parallel channels (199 mm projects, such as that for the new supercomputer called AQUASAR
wide and 756 mm deep) was tested experimentally with a split flow [16], which considers the implementation of a liquid water cooling
system with one central inlet at the middle of the channels and two cycle on a rack cabinet with power consumption of around 10 kW. It
outlets at either end. Three working fluids were tested (HFC134a, is important to mention that the simulations presented were
HFC236fa and HFC245fa) and also the parametric effects of mass considered as a benchmark and that the energetic comparison
velocity, saturation temperature and inlet temperature. The anal- should be applied to an actual server’s specifications, which is the
ysis of their results showed that a significantly higher CHF was subject that is considered here. Additionally, a case study consid-
obtainable with the split flow system compared to the single inlet- ering a vapor compression and a liquid pumping cooling systems
single outlet system [41], providing also a much lower pressure applied on a datacenter and exploring the application of energy
drop. For the same mass velocity, the increase in CHF exceeded 80% recovered in the condenser on a feedwater heater of a coal power
for all working fluids evaluated due to the shorter heated length of plant will be presented. Aspects such as the minimization of energy
a split system design. For the same total refrigerant mass flow rate, consumption and CO2 footprint and the maximization of energy
an increase of 24% for HFC134a and 43% for HFC236fa were ob- recovery and power plant efficiency are investigated.
tained (no comparable data were available for HFC245fa). They
concluded that the split flow system had the benefit of much larger 3. Hybrid two-phase cooling cycle
CHF values with reduced pressure drops and further developments
in the design of split flow system could yield an interesting ener- Fig. 2 depicts the proposed hybrid two-phase cooling cycle, i.e.,
getic solution for cooling of computer chips. a multi-purpose cooling cycle able to interchange the cycle driven
[58] developed a steady state model of a refrigeration system for by a liquid pump or a vapor compressor. The change of cycle would
high heat flux electronics cooling. The refrigeration system be accomplished by means of shut-off valves 1 to 7 (SOV). The
proposed consists of multiple evaporators (microchannel tech- choice of the cooling cycle would depend on the demand for heat
nology), liquid accumulator with an integrated heater, variable recovery, or whether cycle maintenance is required (repair of the
speed compressor, condenser and electric expansion valves (EEV). compressor or pump with one mode as a backup to the other
To obtain more efficient heat transfer and higher critical heat flux, mode). The microprocessors cannot operate without cooling and
the evaporators were considered to operate only with two-phase thus the interchangeability of the cycles represents a safety
flow. To guarantee the safe operation of the refrigeration system mechanism in case of failure of the pump or compressor. The cons
the authors considered the presence of an integrated heater- of the hybrid cycle would mainly be the higher initial cost but
accumulator to fully evaporate the two-phase flow coming out of certainly the advantages gained (system online reliability,
the evaporator, which naturally represents a decrease of the cycle controllability, cycle interchangeability and flexibility in heat
COP. A parametric study to evaluate the effects of external inputs on recovery) may prove to justify this. Furthermore, this hybrid cycle
the system performance (secondary fluid temperature in the represents a plug-and-play option where any one of the three
condenser, evaporator heat load, compressor speed, EEV cycles can be installed based on the particular application to
percentage opening and heat supplied to the accumulator) and standardize the design, thus minimizing engineering costs.
a Pareto optimization to find the optimal system operating condi- The goal is to control the chip temperature to a pre-established
tions were also developed. A heat load of 1500 W and 2500 W, level by controlling the inlet conditions of the micro-evaporator
which represent respectively a heat flux of 94 kW/m2 and (pressure, subcooling and mass flow rate). It is also imperative to
156.6 kW/m2 were considered. The main points observed were: keep the micro-evaporator (ME) outlet vapor quality below that of
i) the system COP can be improved without compromising the the critical vapor quality, which is associated with the critical heat
critical heat flux when handling higher heat flux, ii) higher critical flux. The critical vapor quality is determined from the critical heat
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 5
flux [39] and an energy balance on the ME. Due to this limitation, avoids potential problems of mal-distribution in each ME line
additional latent heat is available for further evaporation, which due to non-uniform heat load on the microprocessors and bad
can be used by other low heat flux generating components in the pipeline designs. Preliminary experimental evaluation can be
blade server. found in [30]; where one SMV controlling the mixture (junc-
Another parameter that must be controlled is the condensing tion of piping) outlet vapor quality of two MEs proved to be
pressure (condensing temperature). The aim, during the winter, is feasible and efficient.
to recover the energy dissipated by the refrigerant in the condenser c) Micro-evaporator (ME): transfers the heat generated by the
to heat buildings, residences, district heating, etc. In order to microprocessor to the refrigerant.
accomplish this, the idea is to use a variable speed compressor d) Microchannel cold plate for auxiliary electronics (MPAE):
(VSC) and an electric expansion valve (EEV), as will be discussed additional component used to cool the auxiliary electronics
below. using the remaining latent heat, which is available due to the
Fig. 3 depicts the two-phase cooling cycle where the flow rate is limitations enforced on the micro-evaporator.
controlled by a liquid pump. The components considered and their e) Pressure control valve (PCV): controls the condensing pressure.
main functions are presented below: f) Condenser: counter-flow tube-in-tube exchanger.
g) Liquid accumulator (LA): guarantees that there is only satu-
a) Variable speed liquid pump: controls the mass flow rate rated liquid at the subcooler inlet, independent of changes in
circulating in the system. thermal load.
b) Stepper motor valve: controls the liquid flow rate to control the h) Temperature control valve (TCV): controls the subcooling at the
outlet vapor quality in each micro-evaporator (0%e100%). It inlet of liquid pump.
Fig. 3. Schematic of the liquid pump cooling cycle. Fig. 4. Schematic of the vapor compression cooling cycle.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
6 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
efficiency. This is a good operating option when the energy dissi- load on the blade, but will have a larger surface area compared to
pated in the condenser is not recovered, typically during the the CPU and thus a lower heat flux.
summer season. However, the heat can still be recovered if there is Finally, when considering an entire rack, a very sizable heat load
an appropriate demand for low quality heat (low exergy). is generated, which represents a good opportunity to recover the
Fig. 4 shows a two-phase cooling cycle where a vapor heat rejected. In this case, reuse of the heat removed from the
compressor is the driver of the working fluid. The components blades for a secondary application will greatly reduce the CO2
considered and their main functions are: footprint of the system. For example, if we consider a datacenter
with 50 vertical racks, where each rack has 64 blades and each
a) Variable speed compressor: controls the ME inlet pressure and blade dissipates 300 W, the total potential heat to be recovered will
consequently the level of inlet subcooling. be 0.96 MW. Such a heat recovery system requires a secondary heat
b) Pressure control valve (PCV): controls the condensing pressure. transfer fluid to pass through all the condensers (either water or
c) Condenser: counter-flow tube-in-tube exchanger. a refrigerant) and then transport the heat to its destination. Addi-
d) Liquid accumulator: guarantees that there is only saturated tional information on such cycles can be found in the previous work
liquid at the internal heat exchanger (iHEx1) inlet. of the [29].
e) Internal heat exchanger liquid line/suction line (iHEx1):
increases the performance of the cooling system. 4. Simulation code
f) Electric expansion valve (EEV): controls the low pressure
receiver level. An in-house two-phase cooling cycle simulation code was
g) Low pressure receiver (LPR): this component can be seen as developed to design and evaluate the performance of the liquid
a second internal heat exchanger liquid line/suction line, which pump and vapor compressor cooling cycles under steady state
increases the EEV inlet subcooling and allows an overfeed to conditions. The simulation code is able to design the condenser and
the ME since the ME outlet returns to this receiver. subcooler, to evaluate the performance of the ME and various
h) Stepper motor valve: controls the liquid flow rate to control the component coolers for a given heat load, and to calculate the
outlet vapor quality in each micro-evaporator (0%e100%). pumping power consumption to drive the cooling cycle. The pres-
i) Micro-evaporator (ME): transfers the heat away from the sure drop of each component and the piping are also calculated.
microprocessor. Table 1 shows the principal methods implemented in the code.
j) Microchannel cold plate for auxiliary electronics (MPAE): cools In summary the following assumptions are considered:
the auxiliary electronics.
steady state simulation;
This cycle is characterized by a high condensing temperature adiabatic piping;
(high heat recovery potential), a high range of controllability of the isentropic pumping and compression;
ME inlet subcooling (characteristic of systems with VSC and EEV), isenthalpic expansion (adiabatic);
and potentially a medium overall efficiency when compared with uniform heat load on the microprocessors (no hotspots);
the liquid pumping cooling cycle. This is a good operating option uniform flow distribution on the MEs;
when the energy dissipated in the condenser is recovered for other one-dimensional heat transfer in the ME
use, typically during the winter season when considering a district
heating application or the entire year for a feedwater heater of The input data required to run the simulation code are: (i) the
a coal power plant (high exergy). geometrical parameters of ME’s and heat exchangers, (ii) the heat
It is worth mentioning that the applicability of these cooling load on the ME’s and MPAE’s, (iii) the evaporating temperature and
cycles is not restricted to only one microprocessor but can be subcooling at the ME inlet, the condensing temperature at the
applied to blade servers and clusters, which may have up to 64 condenser inlet (only for the vapor compression cooling cycle), the
blades per rack cabinet. Each blade, such as that shown in Fig. 5, can water temperature at the condenser and subcooler inlet and outlet
have two (or more) microprocessors with a heat generation (secondary fluid flowing in the annulus), and (iv) the length and
capacity higher than 150 W. If the auxiliary electronics (memories, direction of the pipes and elbows joining the components.
DC/DC, etc.) on the blade are included, the total heat generation per The analysis took into account the thermal performance, the
blade can be higher than 300 W. Thus, the microchannel cold plate pumping power consumption and the total pressure drop of the
(MPAE) described in the cooling cycles has the function to cool the cooling cycle. A comparison of 5 simulated cases considering
auxiliary electronics that can represent about 60% of the total heat different working fluids and cooling cycles was made. For simulated
cases 1 and 4, two-phase (TP) HFC134a and single-phase liquid
water (SP_W) in the ME, the design considered for the liquid pump
cooling system was such that the total pressure drop was about
1.5 bar. The design constraint imposed on the condenser and the
subcooler was that the pressure drops in the working and
secondary fluids were, respectively, 0.05 bar and 1 bar. Cases 2, 3
and 5 considered the same heat exchanger geometries
(ME, condenser and subcooler) as defined for case 1. The difference
in cases 2, 3 and 5 with respect to case 1 are that for case 2 the
internal diameter of piping on the blade was reduced from 3 mm to
2 mm, for case 3 the working fluid was changed from HFC134a to
HFO1234ze and for case 5 the vapor compression cooling cycle was
considered.
A blade server with 16 blades was taken into consideration for
the total heat load. Each blade, for example as that showed in Fig. 5,
Fig. 5. Typical blade with two microprocessors and a heat generation capacity higher presents two electronic systems in parallel and with each system
than 300 W. being composed of one microprocessor (60 W of heat load) and the
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 7
Table 1 Table 2
Methods in the code. Input data.
QME and QAE are the heat loads associated with the microprocessor
auxiliary electronics (55.6 W of heat load). Fig. 6 shows the rack and the auxiliary electronics, which are transferred respectively by
cabinet designed, with the piping configurations and components the ME and MPAE. WLP_or_VC is the pumping power consumption of
for the liquid pump cooling cycles with water and volatile working the driver, which can be the liquid pump or the vapor compressor.
fluid (a, b) and (c) the vapor compression cooling cycle. Finally, Qcond and Qsub are the heat transfer rate at the condenser
In summary the following comparisons were made: i) SP_W and subcooler.
versus TP_HFC134a, ii) TP_HFO1234ze versus TP_HFC134a, both It is worth pointing out that the condenser is a tube-in-tube heat
comparisons for the liquid pump cooling cycle and iii) liquid pump exchanger with spiral microfins on the internal surface of the inner
cooling cycle versus vapor compression cooling cycle, both cycles tube with a smooth external surface. For the tube-in-tube sub-
with TP_HFC134a. cooler, the internal surface of the inner tube is again microfinned
Table 2 shows the input data considered for each working fluid and smooth on the external surface. For both heat exchangers the
evaluated. The other thermodynamic parameters required to objective was to find its length, while the other geometrical
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
8 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 9
Table 6
Cooling cycle piping designs and pressure drops on the blade and overall system.
CASES 1 2 3 4 5
Cooling Cycle Liquid pump Vapor compression
Working Fluid HFC134a HFC134a HFO1234ze Water HFC134a
Internal diameter of piping on the blade [mm] 3.0 2.0 3.0 3.0 3.0
Internal diameter of piping for single-phase flow [mm] 4.1 4.1 4.1 7.7 4.1
Internal diameter of piping for two-phase flow [mm] 7.0 7.0 7.0 e 7.0
Pressure drop in the piping on the blade [bar] 0.021 0.134 0.028 0.127 0.021
Total pressure drop [bar] 1.54 1.65 1.79 1.50 1.87
mass flow rate and heat exchanger volume and the higher outlet and typical efficiency values (20% for the pump and 60% for the
temperature of secondary fluid (viz. Table 7 and Fig. 9), when compressor) a power consumption of about 32 W and 1250 W
compared with all the cases evaluated. This implies that a lower are obtained for the pump and compressor, respectively.
pumping power for the secondary fluid is required while material Therefore, for the expected efficiency values, the power
costs of the condenser are lower due to the lower volume and that consumption of the compressor is 39 times higher than the
there exists a higher potential for energy recovery (high exergy). As liquid pump, compared to 116 times higher if 100% efficiency
mentioned beforehand, this cycle would be a good operating option were assumed (Fig. 7).
when the energy dissipated in the condenser is recovered for other
uses, for example into the feedwater heaters of thermal power
plants. It is also worth mentioning that the higher heat transfer 5. Datacenter and coal fired power plant synergy: case study
duty in the condenser (viz. Table 7) is associated with the additional
work imparted by the compressor on the fluid. This subject will be The case study is performed where it is considered that the
explored in the following section. datacenter uses on-chip cooling to cool the servers, with the heat
Finally, Fig. 10 shows the effect the driver’s isentropic efficiency captured being redistributed to a power plant. The datacenter will
has on the driver’s power consumption for cases 1 and 5. The be modeled as a cooling cycle consisting of an evaporator (on-chip
simulations assumed that the extra power associated with the cooling elements), a condenser, a fluid driver (compressor or liquid
inefficiency of the drivers was lost to the environment, i.e., it does pump), and an expansion valve in the case of a vapor compression
not have an effect on the cooling cycle’s thermodynamic conditions system. The power utility will be a thermal Rankine cycle consisting
and design declared beforehand. Due to the low viscosity of of a boiler, a high and low pressure turbine, a condenser, a low
HFC134a, 0.12 mm2/s at 60 C, the current pumps are not expected pressure and high pressure feedwater pump and a feedwater
to have high isentropic efficiency (slip and leakage reasons/lower heater. The feedwater heater receives heat from steam tapped after
than 20%), on the other hand conventional vapor compressors are the high pressure turbine. The optimal pressure for tapping the
expected to have 50%e70% isentropic efficiency [19] and [17]. It is steam will be calculated to obtain maximum thermal efficiency. The
also worth mentioning that such efficiency is highly dependent on datacenter waste heat will be injected into the Rankine cycle after
the manufacturer of the driver, economic aspects and thermody- the condenser and prior to the feedwater heater. This is shown
namic conditions, reasons which will not be explored in this paper. schematically in Fig. 11.
The results show that the driver’s isentropic efficiency has The operating conditions for both cycles are listed in Table 8,
a considerable impact on its power consumption. For expected where the values of a typical coal fired steam cycle power plant
reported by [56] are considered. A first-law analysis is performed
on the two cycles, showing their overall performances. The
Table 7 following assumptions are made:
Heat exchangers simulations.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
10 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
1600 80
1400 Case 5 70
(Case 5)
(Case 1)
800 40
600 30
400 20
200 10
0 0
0 20 40 60 80 100 120
Isentropic efficiency [%]
Heat exchanger volume savings in CO2 would be in the order of 195 000 tons of CO2 per
Heat exchanger volume [cm 3 ]
Secondary fluid mass flow rate year. This could potentially save about $ 3 000 000 per year if
Mass flow rate [kg/h]
850 100
a carbon tax of $15 per ton of CO2 [23] was considered.
700 70
729.9
Table 8
705.2
704.0
Operating conditions for the power utility and datacenter cooling cycle.
Boiler Evaporator
Pressure [MPa] 16.55 1.681
400 10 Turbine Compressor
HFC134a HFC134a HFO1234ze Water HFC134a Inlet Pressure [MPa] 16.55 1.681
Case1 Case2 Case3 Case4 Case5 Inlet Temperature
[C] 538 70
LP LP LP LP VC
Condenser
Condensing Temperature
[C] 46 60e100
Fig. 9. Heat exchanger volume and secondary fluid mass flow rate.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 11
5.2. Datacenter
0.479
0.48
Due to datacenter growth, it is more than likely to see data-
Thermal Efficiency, -
centers in excess of 100 000 servers (viz. Fig. 14). Therefore, the
simulations following will be based on a datacenter containing
0.47
100 000 servers, with each server generating 325 W, which
0.462 includes the main processor and auxiliary electronics (memories,
DCeDC converters, hard drives, etc). Fig. 15 shows a graph of the
0.46
total power supply required by the datacenter to operate the IT
equipment and the cooling system. Included in the graph is on-chip
two-phase cooling methods using a vapor compression cycle and
0.45 liquid pumping cycle. As a comparison, traditional air cooling units
40 50 60 70 80 90 100 are also simulated, where it is assumed that their power
Condensing Temperature, oC consumption is the same as that required to operate the IT equip-
ment ([22] and [21]). The results are plotted as a function of the
Fig. 12. Thermal efficiency of power plant.
compressor or pump overall efficiencies (defined as the ratio
between isentropic compression or pumping power and the elec-
trical power for the drivers). All simulations were developed for an
0.97 0.05 evaporating temperature of 60 C (on the chip), with condensing
temperatures being 60 C and 90 C for the liquid pumping and
kg CO2 savings / kWh
0.96 0.04
vapor compression cycles, respectively.
0.95 0.03 It is seen that the cycle using a vapor compressor has a strong
kg CO2 / kWh
Fig. 14. Datacenter size and information technology (IT) power requirements.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
12 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
Air cooling + IT the case for a liquid pumping cycle due to its energy consumption
Vapor compressor + IT being very low. The vapor compression cycle always has some
80
unrecoverable heat due to inefficiencies, with this heat being lost to
Data Center Power Supply, MW
Liquid pump + IT
70 the environment. At this point in stage it would appear as if the use
IT load
of a liquid pumping cycle far outweighs the vapor compression
60 cycle. However, what the graphs do not show is the quality of the
heat being sold. The former can sell heat at a temperature of 60 C,
50 while the latter can sell it at 90 C. The quality of heat is therefore
important, not only due to the monetary value it adds, but also to
40 the application to which it is sold. A limited number of applications
can use 60 C of heat, with the limits becoming less as the
30 temperature is increased.
20
0 20 40 60 80 100 5.3. Exergy analysis
Compressor and liquid pump overall efficiency, %
To better explore and understand the difference between the
Fig. 15. Datacenter power supply - IT and cooling system.
two possibilities of cooling cycles (vapor compressor and liquid
pump) regarding energy recovery, i.e., exergy available at the
the order of 50% when using a liquid pumping cycle and 41% for condenser for a secondary application, the concept of exergy is
a vapor compression cycle with a compressor having an overall introduced. The cooling cycles defined in the previous section were
efficiency of 60%. considered (viz. Figs. 3 and 4). For this analysis the heat load of an
Further savings can be made when energy recovery is consid- entire blade center composed of 14 blades, which has the micro-
ered. Since on-chip cooling is used, recovering this energy would be processors and auxiliary electronics to be cooled, is defined as
a simple process by just incorporating a condenser, where the a heat source of 4900 W at 85 C, while a counter-flow heat
energy absorbed by the fluid from the server is transferred to exchanger is used as the condenser. The secondary fluid is water at
a secondary fluid, like water, in the condenser. Fig. 16 shows a graph an inlet and outlet temperature of 46 C and 5 C lower than the
of the datacenter power supply for the three types of cooling inlet temperature of working fluid, respectively. [38] showed for
technologies as a function of the efficiency with which energy is energy recovery applications, that 46 C is about the condenser
recovered. 100% efficiency implies that all the heat generated by the outlet temperature of a coal power plant, being the potential point
servers is recovered, while 0% means that none of the heat is where the recovered heat of a datacenter can be injected. An overall
recovered. Note that there is no change in power consumption for efficiency of 60% was considered for the drivers, i.e., pump and
air cooling as it was assumed that the heat was not recovered, compressor. HFC134a is used as working fluid and a MEþMPAE
although this technology does exist, albeit not as effective as for evaporating temperature of 60 C is defined. An outlet vapor quality
liquid cooling. The plots for the liquid pumping and vapor of about 80% at the MPAE is obtained from the simulations of both
compression cycles assume that the pump and compressor has an cycles. The mass flow rate of the working fluid and secondary fluid
overall efficiency of 100% and 60%, respectively, although the choice were adjusted so that the energy balance is respected.
of efficiency for the pump is negligible (viz. Fig. 15). It should be The steady state exergy rate balance is defined by Eq. (2) [35].
noted that 0 MW datacenter supply does not mean the datacenter The first and second terms in the right side of equality represent the
requires no power, but rather that all the power received as elec- exergy transfer accompanying heat and work, the third and fourth
tricity is sold as heat. The financial implications would show this are the time rate of exergy transfer accompanying mass flow and
since the value of the heat sold would be different from the elec- flow work and, finally, the last term is the rate of exergy destroyed.
tricity purchased and will be a function on the application to which With some mathematical manipulation it is possible to prove that
the heat is sold. the last term in Eq. (2), i.e., E_ d is the entropy generation multiplied
Fig. 16 shows that practically all the power purchased in the by the dead state temperature T0.
form of electricity can potentially be sold as heat. This is especially !
X T0 _ X X
0 ¼ 1 Qj W _ cv þ m _ i efi m_ e efe
Tj e
j i
70 |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}
rate of exergy transfer
Data Center Power Supply, MW
60
Air cooling + IT E_ d (2)
Vapor compressor + IT
|{z}
50 rate of exergy destruction
Liquid pump + IT
40 IT load It can be observed that an exergy reference environment is
necessary to be defined. Such an environment represents the state
30
of equilibrium or dead state. This equilibrium state defines the
20 exergy as the maximum theoretical work obtainable when another
system in a non-equilibrium state interacts with the environment
10 to the equilibrium. For the present work, the reference is defined as
295 K, 100 kPa for water and 295 K, 603.28 kPa and 50% of vapor
0 quality for HFC134a.
0 20 40 60 80 100
The goal of the analysis is to determine, for each cycle, the
Energy Recovery Efficiency, %
exergy supplied, recovered and destroyed for a control volume in
Fig. 16. Datacenter power supply - IT and cooling system - potential of energy the cooling cycle. With this the overall exergetic efficiency, defined
recovery. as the ratio between recovered and supplied exergies, can be
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 13
Air cooling + IT
LP cycle VC cycle LP cycle VC cycle
Exergy supplied [W] 900.8 1991 Exergy 428.1 890 400 Compressor + IT
recovered [W] Liquid pump + IT
300
Exergy destroyed or Exergetic
irreversibility [W] efficiency [%]
200
Pump or compressor 14.2 450.4 Pump or 60.0 60.0
compressor 100
Condenser 137.6 209.2 Condenser 75.7 81.0
ME þ MPAE 320.9 320.8 ME þ MPAE 62.3 62.3 0
iHEx e 39.12 iHEx e 82.8 0 0.2 0.4 0.6 0.8 1
LPR e 25.86 LPR e 82.3
Energy recovery efficiency, -
EEV e 55.79 EEV e 0
Total 472.7 1101 Overall 47.5 44.7
Fig. 18. Reduction in CO2 for three datacenter cooling technologies.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
14 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
Datacenter - Compressor
350 70
Datacenter - Liquid pump
Mass CO2 consumed (datacenter),
50 4.5
Mass CO2 savings (power plant),
M$ per year
200 40 40 2.7
M$ per year
150 30
35 1.8
100 20 Datacenter - Compressor
30 0.9
50 10 Datacenter - Liquid pump
Power plant
0 0 25 0
40 60 80 100 40 50 60 70 80 90 100
Condensing temperature, oC Condensing temperature, oC
Fig. 19. Carbon footprint of datacenter and CO2 savings of power plant. Fig. 21. Datacenter and power plant savings.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16 15
40
and the MEþMPAE are the components with the lowest exergetic
90o C efficiency and would be the main components to be improved in
35 terms of thermodynamic design. It was also shown that the overall
exergetic efficiency of the vapor compression cooling cycle is
30 strongly influenced by the compressor overall efficiency, which
showed to be more exergeticaly efficient than the liquid pumping
cooling cycle for an overall efficiency higher than 67%.
25
0 0.2 0.4 0.6 0.8 1 A case study was developed to investigate the potential savings
in energy a datacenter can make by implementing on-chip cooling
Energy recovery efficiency, -
with waste heat recovery. As an application for the waste heat,
Fig. 22. Total savings for liquid pumping and vapor compression cooling technologies. a coal fired power plant was analyzed. The results showed that,
when compared with traditional air cooling systems, the energy
consumption of the datacenter could be reduced by as much as 50%
also not exempt from these taxes, which will be introduced in the when using a liquid pumping cycle and 41% when using a vapor
following years to come [33]. Fig. 21 shows the potential savings compression cycle. The overall consumption can be reduced even
made by a datacenter with a size of 100 000 servers and a coal further if the recovered energy is sold to a secondary application,
power plant with a size of 175 MW if heat were captured from the such as a thermal power plant. Power plant thermal efficiency
datacenter and sold to the power plant. The savings not only improvements in the order of 2.2% are possible if datacenter waste
includes that saved in energy costs by implementing a liquid heat is incorporated in the power plant’s feedwater. This could
pumping or vapor compression cycle instead of a traditional air imply huge savings in terms of fuel as well as carbon tax due to
cooling cycle, but also that saved in carbon tax. The savings of the a reduced carbon footprint.
power plant is in terms of fuel saved and the savings made in
carbon tax. For fuel costs, a value of $90/ton of coal was used.
Acknowledgements
For the datacenter, the graph shows that most savings are made
if a liquid pumping cycle was used, with the potential savings being
The Commission for Technology and Innovation (CTI) contract
in the order of $45 000 000 per year, while a vapor compression
number 6862.2 DCS-NM entitled “Micro-Evaporation Cooling
cycle would save in the order of $40 000 000 per year. The power
System for High Performance Micro-Processors: Development of
plant, when recovering heat from a datacenter using a liquid
Prototype Units and Performance Testing” directed by the LTCM
pumping cycle, will only save about $1 000 000 per year, with
laboratory sponsored this work along with the project’s industrial
savings reaching almost $4 500 000 a year if a vapor compression
partners: IBM Zürich Research Laboratory (Switzerland) and
cycle were used.
Embraco (Brazil). J.B. Marcinichen wishes to thank CAPES (“Coor-
The overall savings are given in Fig. 22. This graph shows that
denação de Aperfeiçoamento de Pessoal de Nível Superior”) for
the total savings, if a datacenter was to sell waste heat at 60 C with
a one year fellowship to work at the LTCM laboratory. The authors
a liquid pumping cycle, could be about $46 000 000 a year, while
also wish to thank Honeywell Inc. for providing the thermophysical
selling heat at 90 C with a vapor compression cycle could save
properties of HFO1234ze for the calculations.
a total of $43 000 000 per year. These are savings that a customer
would potentially not have to pay. Even though it seems as if the
clear solution is to use a liquid pumping cycle within a datacenter, References
the incentive for a power plant to cooperate with a datacenter
[1] B. Agostini, M. Fabbri, J.E. Park, L. Wojtan, J.R. Thome, B. Michel, State of the art
would be greater if a vapor compression cycle was used. of high heat flux cooling technologies, Heat Transfer Engineering 28 (2007)
258e281.
6. Conclusion [2] B. Agostini, M. Fabbri, J.R. Thome, B. Michel, High heat flux two-phase cooling
in silicon multimicrochannels, IEEE Transactions on Components and Pack-
aging Technologies 31 (3) (2008) 691e701.
A hybrid two-phase cooling cycle has been proposed and [3] A. Azzi, U.S.T.H.B. Alger, L. Friedel, Two-phase upward flow 90o bend pressure
simulated to cool microprocessors and auxiliary electronics of loss model, Forschung im Ingenieurwesen 69 (2005) 120e130.
[4] A. Azzi, L. Friedel, S. Belaadi, Two-phase gas/liquid flow pressure loss in bends,
blade server boards with two-phase evaporating flow in the micro- Forschung im Ingenieurwesen 65 (2000) 309e318.
evaporator cooling elements. A simulation code was developed and [5] A. Barnea, Transition from annular flow and from dispersed bubble flow e
5 cases were simulated considering 3 different working fluids, unified models for the whole range of pipe inclinations, International Journal
of Multiphase Flow 12 (5) (1986) 733e744.
HFC134a, HFO1234ze and water (in an analogous single-phase [6] D. Barnea, O. Shoham, Y. Taitel, Flow pattern transition for vertical downward
cooling cycle) and different internal diameters of the pipes and two phase flow, Chemical Engineering Science 37 (5) (1982) 741e744.
elbows joining the components. The results showed that the liquid [7] H. Blasius, Das Ähnlichkeitsgesetz bei Reibungsvorgängen in Flussigkeiten
(1913) Fors-chg. Arb. Ing. -Wes., 131 Berlin.
water cooling cycle has a pumping power consumption 5.5 times [8] Carbon Tax, In Wikipedia, the Free Encyclopedia Retrieved 08:12, July 26,
that obtained for the two-phase HFC134a cooling cycle, both 2010, (2010) Available from: http://en.wikipedia.org/w/index.php?
considering a liquid pump as the driver of the fluid. When title¼Carbon_tax&oldid¼374709823 :.
[9] A. Cavallini, D. Del Col, L. Doretti, G.A. Longo, L. Rossetto, Heat transfer and
compared with the HFO1234ze cooling cycle the difference drops to
pressure drop during condensation of refrigerants inside horizontal enhanced
4.4 times. The simulation of the vapor compression cooling cycle tubes, International Journal of Refrigeration 23 (2000) 4e25.
showed higher pumping power consumption when compared with [10] A. Cicchitti, C. Lombardi, M. Silvestri, G. Soldaini, R. Zavattarelli, Two-phase
the other cycles simulated. However, this cycle can be justified cooling experiments-pressure drop, heat transfer and burnout measurements,
Energia Nuclear 7 (1960) 407e425.
when the waste heat at the condenser is recovered for applications [13] F.W. Dittus, L.M.K. Boelter, Publications on Engineering, vol. 2, University of
such as district heating and preheating of boiler feedwater. California, Berkeley, 1930, 443.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008
16 J.B. Marcinichen et al. / Applied Thermal Engineering xxx (2012) 1e16
[14] V. Dupont, J.R. Thome, A.M. Jacobi, Heat transfer model for evaporation in [37] W. Nordhaus, A Question of Balance e Weighing the Options of Global
microchannels. Part II: comparison with the database, International Journal of Warming Policies, Yale University Press, New Haven, 2008.
Heat and Mass Transfer 47 (2004) 3387e3401. [38] Olivier, J.A., Marcinichen, J.B and Thome, J.R., 2010, “Two-phase cooling of
[15] EPA, Report to Congress on Server and Data Center Energy Efficiency Public datacenters: reduction in energy costs and improved efficiencies”, in
Law pp. 109e431, U.S. Environmental Protection Agency, 2007. Proceedings of the 13th Brazilian congress of thermal sciences and engi-
[16] P. Ganapati, Water-Cooled Supercomputer Doubles as Dorm Space Heater neering e ENCIT2010, Uberlandia, MG, Brazil.
Retrieved 12:12, February 18, 2011, (2009) Available from: http://www. [39] C.L. Ong, J.R. Thome, Macro-to-microchannel transition in two-phase flow:
wired.com/gadgetlab/2009/06/ibm-supercomputer/. part 2 - flow boiling heat transfer and critical heat flux, Experimental Thermal
[17] J.M. Gonçalves, C. Melo, C.J.L. Hermes, J.R. Barbosa, Experimental mapping of the and Fluid Science (2011). doi:10.1016/j.expthermflusci.2010.12.003.
thermodynamic losses in vapor compression refrigeration systems, Journal of the [40] Park, J.E., 2008. “Critical heat flux in multi-microchannel copper elements
Brazilian Society of Mechanical Sciences and Engineering 33 (2) (2011) 159e165. with low pressure refrigerants”, PhD Thesis, École Polytechnique Fédérale de
[18] R. Hannemann, J. Marsala, M. Pitasi, Pumped liquid multiphase cooling, in: Lausanne, Switzerland.
Proceedings IMECE - International Mechanical Engineering Congress and [41] J.E. Park, J.R. Thome, Critical heat flux in multi-microchannel copper elements
Exposition, Anaheim, CA, USA, Paper 60669 (2004). with low pressure refrigerants, International Journal of Heat and Mass
[19] C.J.L. Hermes, C. Melo, F.T. Knabben, J.M. Gonçalves, Prediction of the energy Transfer 53 (2010) 110e122.
consumption of household refrigerators and freezers via steady-state simu- [42] Perez-Tellez, C., 2003, “Improved bottomhole pressure control for
lation, International Journal of Applied Energy 86 (2009) 1311e1319. underbalanced drilling operations”, Ph.D. Dissertation, Louisiana State
[20] IBM, IBM BladeCenter: Build Smarter IT Retrieved 09:30, May 20, 2011, (2010) University.
Available from: ftp://public.dhe.ibm.com/common/ssi/ecm/en/blb03002usen/ [43] T.S. Ravigururajan, A.E. Bergles, General correlations for pressure
BLB03002USEN.PDF. drop and heat transfer for single-phase turbulent flow in internally
[21] J. Ishimine, Y. Ohba, S. Ikeda, M. Suzuki, Improving IDC cooling and air condi- ribbed tubes, Augmentation of Heat Transfer in Energy Systems 52
tioning efficiency, FUJITSU Scientific & Technical Journal 45 (2009) 123e133. (1985) 9e20.
[22] J.G. Koomey, Estimating Total Power Consumption by Servers in the U.S. and [44] R. Revellin, J.R. Thome, A theoretical model for the prediction of the critical
the World February 15,, Analytics Press, Oakland, CA, 2007, Available from: heat flux in heated microchannels, International Journal of Heat and Mass
http://enterprise.amd.com/us-en/AMD-Business/Technology-Home/Power- Transfer 51 (2008) 1216e1225.
Management.aspx. [45] G. Ribatski, L. Wojtan, J.R. Thome, An analysis of experimental data and
[23] J.B. Larson, America’s Energy Security Trust Fund Act of 2009 (2009) H.R. 1337. prediction methods for two-phase frictional pressure drop and flow boiling
[24] J. Lee, I. Mudawar, Low-Temperature two-phase microchannel cooling for heat transfer in micro-scale channels, Experimental Thermal and Fluid
high-Heat-Flux thermal Management of Defense electronics, IEEE Trans- Science 31 (2006) 1e19.
actions on Components and Packaging Technologies 32 (2) (2009) 453e465. [46] M.A. Rosen, I. Dincer, M. Kanoglu, Role of exergy in increasing efficiency and
[25] P.L. Leonard, A.L. Phillips, The Thermal Bus Opportunity e A Quantum Leap in sustainability and reducing environmental impact, Energy Policy 36 (2008)
Data Center Cooling Potential, ASHRAE Transactions, Denver, CO, 2005. 128e137.
[26] D. Liu, S. Wang, Flow pattern and pressure drop of upward two-phase flow in [47] M. Saini, R.L. Webb, Heat rejection limits of air cooled plane fin heat sinks for
vertical capillaries, Industrial and Engineering Chemistry Research 47 (2008) computer cooling, IEEE Transactions on Components and Packaging Tech-
243e255. nologies 26 (1) (2003) 71e79.
[27] Y. Madhour, J.A. Olivier, E. Costa-Patry, S. Paredes, B. Michel, J.R. Thome, Flow [48] E. Samadiani, S. Joshi, F. Mistree, The thermal design of a next generation data
Boiling of R134a in a Multi-Microchannel Heat Sink with Hotspot Heaters for center: a conceptual exposition, Journal of Electronic Packing 130 (2008)
Energy-Efficient Microelectronic CPU Cooling Applications, IEEE Transactions 041104e41111 041104e041108.
on Components, Packaging and Manufacturing Technology 1 (6) (2011) [49] R.R. Schmidt, B.D. Notohardjono, High-end server low-temperature cooling,
873e883. IBM Journal of Research and Development 46 (2002) 739e751.
[28] Marcinichen, J.B., Thome, J.R., 2010, “New Novel Green Computer Two-Phase [50] P.L. Spedding, E. Benard, G.M. McNally, Fluid flow through 90 degree bends,
Cooling Cycle: a Model for Its Steady-State Simulation”, in Proceedings of the Developments in Chemical Engineering and Mineral Processing 12 (2004)
23rd International Conference on Efficiency, Cost, Optimization, Simulation and 107e128.
Environmental Impact of Energy Systems - ECOS2010, Lausanne, Switzerland. [51] Y. Taitel, D. Bornea, A.E. Dukler, Modelling flow pattern transitions for steady
[29] J.B. Marcinichen, J.R. Thome, B. Michel, Cooling of microprocessors with upward gas-liquid flow in vertical tubes, AIChE Journal 26 (3) (1980)
micro-evaporation: a novel two-phase cooling cycle, International Journal of 345e355.
Refrigeration (2010). [52] Thome, J.R., Agostini, B., Revellin, R. and Park, J.E., 2007, “Recent advances in
[30] J.B. Marcinichen, J.A. Olivier, V. Oliveira, J.R. Thome, A Review of On-Chip thermal modeling of micro-evaporators for cooling of microprocessors”, in
Micro-Evaporation: Experimental Evaluation of Liquid Pumping and Vapor Proceedings of the ASME International Mechanical engineering Congress and
Compression Driven Cooling Systems and Control, International Journal of exposition (IMECE), Seattle, Washington.
Applied Energy 92 (2011) 147e161. [53] Thome, J.R. and Bruch, A., 2008, “Refrigerated cooling of microprocessors with
[31] A.W. Mauro, J.R. Thome, D. Toto, G.P. Vanoli, Saturated critical heat flux in micro-evaporation heat sinks: new development and energy Conservation
a multi-microchannel heat sink Fed by a split flow system, Experimental Prospects for green datacenters”, in Proceedings Institute of refrigeration
Thermal and Fluid Science 34 (2010) 81e92. (IOR).
[32] J.P. Meyer, J.A. Olivier, Transitional flow inside enhanced tubes for fully devel- [54] J. Thome, V. Dupont, A. Jacobi, Heat transfer model for evaporation in
oped and developing flow with different types of inlet disturbances: part IIeheat microchannels. Part I: presentation of the model, International Journal of Heat
transfer, International Journal of Heat and Mass Transfer 54 (2011) 1598e1607. and Mass Transfer 47 (2004) 3375e3385.
[33] R.L. Mitchel, Carbon tax could whack data centers, Computerworld (2010) [55] S. Trutassanawin, E.A. Groll, S.V. Garimella, L. Cremaschi, Experimental
Available from: http://blogs.computerworld.com/16128/carbon_tax_could_ investigation of a miniature-scale refrigeration system for electronics cooling,
whack_data_centers\. IEEE Transactions on Components and Packaging Technologies 29 (3) (2006)
[34] Mongia, R., Masahiro, K., DiStefano, E., Barry, J., Chen, W., Izenson, M., Pos- 678e687.
samai, F., Zimmermann, A. and Mochizuki, M., 2006, “Small scale refrigeration [56] S. Voss, G. Gould, The Rankine Cycle: Workhorse of the Coal-fired Utility
system for electronics cooling within a notebook computer”, in Proceedings Industry, TechBriefs, Burns & McDonnell, 2001, No 3, pp. 4e6.
ITHERM, San Diego, CA. [57] World_Coal_Association, Coal & Electricity (2011) Available from: http://
[35] M.J. Moran, I. Howard, N. Shapiro, Fundamentals of Engineering Thermody- www.worldcoal.org /coal/uses-of-coal/coal-electricity/.
namics, sixth ed. John Wiley & Sons, 2010, 725 pages. [58] R. Zhou, T. Zhang, J. Catano, J.T. Wen, G.J. Michna, Y. Peles, M.K. Jensen, The
[36] H. Muller-Steinhagen, K. Heck, A simple pressure drop correlation for two- steady-state modeling and optimization of a refrigeration system for high
phase flow in pipes, Chemical Engineering and Processing: Process Intensifi- heat flux removal, Applied Thermal Engineering (2010). doi:10.1016/
cation 20 (1986) 297e308. j.applthermaleng.2010.05.023.
Please cite this article in press as: J.B. Marcinichen, et al., On-chip two-phase cooling of datacenters: Cooling system and energy recovery
evaluation, Applied Thermal Engineering (2012), doi:10.1016/j.applthermaleng.2011.12.008