Printer Friendly

Cooling System with Nearly Zero Cooling Power for Server Rooms.


In order to drastically reduce the power for cooling information and communication technology (ICT) equipment such as servers, research is ongoing for the development of cooling elements and cooling systems for Internet data centers (IDCs) (Ministry of the Environment, 2016). The current project is to realize a power usage effectiveness (PUE = total power of the IDC / power of ICT equipment) of 1.0x. Today's IDCs use liquid immersion to cool servers with high heat density and air cooling to cool hard disk storage and other such components with relatively low heat density. These different systems become intermingled in a server room. To save energy, this paper proposes a server room configuration where ICT equipment is zoned according to the heat load density (high, medium, or low), and a cooling system is employed that utilizes the energy of outside air.

A real-scale cooling system was constructed by using simulated load devices, including a real server machine and cooling auxiliary devices such as a cooling tower and pump. Stable operation was confirmed by measurements taken in the winter, and the feasibility of an annual average PUE of less than 1.04 was confirmed by energy trial calculations based on the actual measurement results.

An IDC is a facility with a cooling load density that is several times to ten times higher than that of general offices. In the future, the cooling load density of such facilities is expected to increase because of improvements in communication infrastructure performance, cloud computing of ICT equipment, increased data volume, and high integration of ICT equipment into racks. Cooling the ICT equipment can require as much as 20%-30% of the total power consumption of an IDC. Reducing the power consumption for cooling is important for reducing the total power consumption of the IDC and improving the PUE. The latest IDCs have over 1000 racks; energy conservation through the use of outside air and natural energy has progressed, and a PUE of around 1.1 has been realized (Aizawa et al. 2013-2015); however, further energy conservation for large-scale IDCs and energy-saving technologies that are compatible with small- and medium-scale server rooms (less than several hundred racks) are desired. The aim of the current project is to develop an approach that is applicable to not only large-scale IDCs and server rooms but also small- and medium-scale ones.

Figure 1 shows the development frame of this project. The goal was to develop cooling technologies corresponding to high, medium, and low heat densities and an entire cooling system with a PUE of 1.0x (the level of PUE of around 1.0) to realize zero auxiliary power for cooling.

The development concept of the project is presented next. As shown in Figure 2, the server room can be zoned according to the heat load density: high, medium, and low. When a server room is cooled with mixed heat load densities, the cooling efficiency is lowered because the cooling is dictated by the high heat load density. A high heat load density is 16 kW or more per rack; this is difficult to manage with air cooling. A medium heat load density is 9-16 kW. At low heat load densities of 8 kW or less, liquid immersion is ineffective. Hard disk storage has a low heat load and can only be air-cooled. In this project, a low PUE was realized by using different types of cooling for different heat load densities: liquid immersion cooling by the natural convection of a refrigerant in a liquid tank for high heat loads (Matsuoka et al. 2017), drop coolant for medium heat loads, and air cooling for racks with a low heat load (Impress 2017).

As shown in Figure 3, the basic concept of the cooling system is the utilization of outside air energy. Based on the production of cooling water in the cooling tower, relaxing the temperature conditions of the server maximizes the cooling tower usage period and minimizes the chiller operation time. During summer peaks when the outside air temperature is high and sufficient cooling water cannot be obtained, cooling water is produced in the cooling tower at night when the outside air temperature is relatively low, and the stored cooling water is used. In the 2016 fiscal year, a general-purpose pump and cooling tower were adopted for cooling auxiliary equipment, and the operating characteristics of each element and the stability of the water supply temperature were confirmed during operation with general-purpose energy-saving methods (variable flow rate/air volume control). The power consumption and its breakdown were clarified.


Figure 4 shows the system diagram of the cooling system and items related to the PUE calculation. The cooling system consists of a cooling tower (fan, pump), a heat exchanger, a heat storage tank, and primary and secondary pumps. It allows indirect outside air cooling (free cooling) and heat storage. In this cooling system, cooling water produced by the cooling tower is supplied to each load, and the ICT equipment is cooled by exchanging heat with the heat medium (refrigerant or air) after the header. Cooling water is produced by the cooling tower and carried without air circulation of the entire server room for each load. Natural convection is utilized for each load. Therefore, the PUE is calculated from the power consumption of the cooling tower fans and pumps, primary pump for the cooling water, secondary pump, and chiller. Compared with an IDC that uses chillers throughout the year and circulates air in the server room, the proposed system can reduce the power of heat sources and fan power of indoor circulating air.

Figure 5 shows the installation conditions of the cooling tower and heat storage tank on the roof of the building in the city where the experiment was conducted. The pump unit of the experimental apparatus was installed downstairs of the rooftop, and the header supplied cooling water to each load. For the cooling tower, a device with a cooling capacity of about 70 kW and sufficient cooling capacity for cooling the demonstration device was selected. The thermal storage tank had a water storage volume of 10 [m.sup.3], which allowed for cold heat radiation operation of about 100 min against the demonstration load. For high-precision testing of the thermal environment on the load side, cold water was produced with an air-cooled chiller. The cooling water was sent to the pump/heat exchanger unit in the vicinity of the load device, and a heat exchange was performed with the necessary amount of coolant for each load.

Figure 6 shows measurement points for data such as the temperature, flow rate, and pressure. The temperature, temperature difference, flow rate, differential pressure, etc., of each load, around the heat exchanger, and between the headers were measured to calculate the calories and system coefficient of performance (COP). The power consumption was measured with a power meter or ammeter on the power panel. The PUE was calculated from the average power consumption for a 30 min steady-state operating period. Power consumption for lighting and power losses were not included in the PUE calculation. For the operation of the IT equipment in the experiment, the input power was set to a constant condition with a high load of around 6 kW, medium load of around 2.5 kW, and low load of around 1.5 kW for a total load of around 10 kW.


Experiments were carried out to characterize the auxiliary machines and measure the power consumption and operating status of the whole system connected to each load.

Basic characteristics of pump/cooling tower. In order to confirm the operating characteristics of each pump, the differential pressure between the supply header and return header of the cooling water and power consumption were measured, and the correlation with the flow rate was confirmed individually. Figure 7 shows an example of the measured power consumption of a pump. Figure 8 shows the outdoor wet-bulb temperature and cooling water temperature when cooling water was produced in the cooling tower at a fan frequency of 50 Hz. The approach (i.e., cooling water outlet temperature--outside air wet-bulb temperature) was confirmed to be around 2[degrees]C.

Heat exchanger. Figure 9 shows an example of the measured temperature exchange efficiency of the heat exchanger. The temperature exchange efficiency was 0.45-0.75, and the average was around 0.6. For other heat exchangers installed this year, the basic characteristics were also determined in the same manner.

System COP. Figure 10 shows the system COP based on the operating patterns of the cooling tower, variable flow rate, and heat storage tank. The variation in the system COP depending on the operating pattern was determined. The cooling load reached up to 16 kW, which left a sufficient margin for the load processing capacity.

Operation result of cooling system. Figure 11 shows the measured power consumption during the trial operation of the constructed cooling system with high, medium, and low loads as a reference. The ratio of the cooling tower fan and primary pump was greater than that of the other auxiliary machines, and the total power consumption was 1.17 kW. This state was before various measures were performed, which will be described later.

Figure 12 shows the transition of the total power consumption of the cooling system and the driving situation during the experiment. Variable flow rate control was applied to the cooling tower fan/pump, primary pump, and secondary pump for each load state. The cooling tower pump, primary pump, and secondary pump reached the lowest frequency (15 Hz) with variable flow rate control before 20:00. This was the lower limit of the variable flow rate control for the cooling tower fan (40 Hz [right arrow] 15 Hz). The flow was controlled on the primary side of each load heat exchanger according to the load-side water supply temperature. The power of the cooling system was reduced to around 1 kW. Stable cooling operation was also confirmed based on the transition of the cooling water temperature.
Figure 11 Power consumption during the trial operation with high,
medium, and low operating loads after construction of the cooling
system as a reference.


tower fan,  0.41
Pump,       0.29
pump,       0.40
Pump for
high load,  0.19
Pump for
load,       0.18
Pump for
low load,   0.25

Note: Table made from pie chart.

The above experimental results confirmed the power consumption of the cooling tower fan and primary pump. The power consumption of each secondary pump was reduced by controlling the variable flow rate of the cooling water. Figure 13 shows the calculated power consumption of the cooling system, together with the assumed performance in the rated load case (36 kW). The resulting power consumption of the cooling system was around 1.3 kW, which is equivalent to a PUE of 1.034.
Figure 13 Results for the power consumption of the cooling system and
the assumed performance for the rated load case of 36 kW.


tower fan,   0.30
tower Pump,  0.29
pump,        0.30
Pump for
high load,   0.15
Pump for
load,        0.14
Pump for
low load,    0.13

Note: Table made from pie chart.

Based on the above results, the annual average PUE of the cooling system was estimated based on the assumed performance of the chiller (COP = 4.5). The water supply temperature (upper limit temperature) of the cooling water was used as a parameter. An average annual PUE of less than 1.04 can be realized when the upper limit of the water temperature is 30 [degrees]C. The individual experiments confirmed that the ICT equipment can be stably operated under a high heat load.


A cooling system is proposed that utilizes the zoning concept to manage different heat densities (liquid immersion cooling for high loads, sprinkle-type cooling for medium loads, and air cooling for low loads) and outside air energy to drastically reduce the carbon emissions of an IDC.

An experiment was conducted in the winter using a real-scale cooling system and simulated load devices, including a real server machine and cooling auxiliary devices. The annual average PUE was then estimated. The results confirmed stable operation of the cooling system and the feasibility of an annual average PUE of less than 1.04. Future work will involve confirming the estimated PUE by actual measurements during the summer and intermediate seasons and lowering the PUE with additional measures to reduce the power consumption.


This study was supported by the development and demonstration projects for C[O.sub.2] emission reduction of the Ministry of the Environment in Japan. Liquid immersion type and refrigerant sprinkle-type ICT equipment cooling elements were developed by Osaka University, EEC Research Institute, and Fujitsu Limited. Sincere thanks are given to the members of these institutions for their helpful comments and discussions. Thanks are also given to Mr. Matsuoka and Mr. Matsuda of Osaka University and EEC Research Institute; Mr. Fujimaki, Mr. Yamamoto, and Mr. Kubo of Fujitsu Limited; and Mr. Shibata and Mr. Ikeda of Takasago Thermal Engineering Co., Ltd. for the fruitful discussions.


Ministry of the Environment. 2016. The development and demonstration projects for C[O.sub.2] emission reduction of the Ministry of the Environment in Japan. Tokyo: Ministry of the Environment of Japan. (

Aizawa, N., Shibata, K., Ikeda, M., Matsuoka, M. et al. 2013-2015. Development of air conditioning system for data center toward radical low-carbon (1st-3rd). Tokyo: The Society of Heating, Air-conditioning and Sanitary Engineers of Japan.

Matsuoka, M., Matsuda, K., and Kubo, H. 2017. Liquid immersion cooling technology with natural convection in data center. IEEE 6th International Conference on Cloud Networking. Plague: IEEE.

Impress. 2017. Demonstration project of next-generation data center supporting IoT society starts.
COPYRIGHT 2018 American Society of Heating, Refrigerating, and Air-Conditioning Engineers, Inc. (ASHRAE)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Aizawa, Naoki
Publication:ASHRAE Conference Papers
Article Type:Report
Date:Jan 1, 2018
Previous Article:Proposal of Cooling System for High Performance Computing by Drip-Feeding Cooling.
Next Article:Data Center Sustainability Index.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters