Printer Friendly

High density cooling solutions--taking IT to the next level: the Cold Aisle Containment alternative.

INTRODUCTION

A key focus in the IT market continues to be on cooling strategies and heat removal from medium to high-density heat loads. Whether at the component, enclosure or room level, end users are struggling to keep up with constantly escalating heat loads. And a condition being addressed almost as rapidly as the heat buildup, with a wide variety of solutions, configurations and components, either available now or in the near future, that will help, at the least, reduce the impact of having too much heat to get rid of.

Every facility operator--small to large, end user to hosting site, private or public sector, is aware of this situation and trying to cope as best as possible. Many strategies have already been successfully implemented: Use of rack mount blank panels, vertical air baffles, improved floor tile management, hole grommets, CRAC/CRAH unit placement, etc. are all in current use to maximize facility climate control capacity. Improved cable management and control and use of under floor and overhead ceiling spaces can also stretch these resources. Even the revised ASHRAE specifications for operating conditions can increase heat removal capacity. But only up to a point.

And at many locations, that point has been reached.

With these greater loads and higher density installations becoming more commonplace, two newer methods are being deployed--aisle containment solutions and close-coupled heat transfer systems. Both product sets are designed to increase heat removal capacity in the IT space. However, the IT community--from the design engineer, through component vendor and ultimately the end user, are tasked to not only design, install and operate a solution but to also provide the data--operational performance results, to justify the expense and verify the claims. This has led to a new awareness of best practices for data centers, with the clear goal to improve facility operations and performance. This situation has also expended a great deal of planning resources to insure IT facilities will be able to handle future growth and new product deployments.

This paper will review some of the design considerations for a Cold Aisle Containment (CAC) installation and operational capacities that can be achieved with the system.

System Review

Separate the air. This has emerged as a key strategy in improving cooling and heat removal capacities in IT facilities. In addition to enclosure (blank panels, vertical baffles) and floor solutions (tile management, hole grommets), a next step to be considered would be to enclose the actual hot or cold aisle spaces within a facility. This type of installation would significantly improve airflow management to active components, providing maximum separation between the hot and cold airflow paths.

Any aisle containment system will consist of just a few components that should be designed to provide easy retrofit and deployment into existing spaces and onto existing enclosures. The two key elements will be row end doors to close off aisles and ceiling panels to cap aisle spaces. Whether choosing to seal off the hot aisle or cold aisle, these systems should be able to be placed in either raised or solid floor installations and work in conjunction with existing facility climate control components. And if being considered for new construction or for use in a facility that has reached maximum heat removal capacity, an aisle containment solution should work with in row cooling systems to provide additional heat removal capacities (see Figures 1 and 2).

[FIGURE 1 OMITTED]

[FIGURE 2 OMITTED]

Test Review

Testing of the Cold Aisle Containment (CAC) system was conducted in the Dallas-Metro region of Texas in an operational data center with a 24 in./610 mm raised plenum floor. A separate test area, approximately 1100 [ft.sup.2]/102.2 [m.sup.2], was isolated from the main data center by using insulated walls placed from the subfloor, through the raised floor and up to the drop ceiling--preventing any air from mixing with main data center spaces. The test setup consisted of 10 server enclosures placed in two facing rows of 5 enclosures each with a four-foot cold aisle between them. The individual enclosures were 24 in./ 600 mm wide and 48 in./1200 mm deep, providing 42U of mounting space, and each one was isolated from the adjacent enclosures with an internal partition wall. Row ends were secured with sidewalls. The enclosures were configured with a standard 64% perforated single door on the front, split perforated rear doors, and a perforated roof. Vertical air baffles were installed on the front 19 in. component mounting rails to ensure proper airflow and prevent any internal cabinet air recirculation and mixing.

The cold aisle was fully contained on the aisle ends with 180[degrees]-hinged double doors. The top of the aisle was covered and sealed with clear polycarbonate panels. Ten perforated floor tiles were installed in the contained cold aisle to provide cold air supply from the raised floor plenum. Numerous tests were performed utilizing perforated tiles with 25, 65, and 80% open surface area. Nine (9) 2.4 kW load banks were installed in each test enclosure, each occupying 4U of space, which allowed for a test range up to 20 kW per enclosure, or 200 kW for the entire contained area. The load banks were designed to mimic server airflow patterns with a nominal [DELTA]T (temperature change) of 45[degrees]F/7.2[degrees]C (as expected with newer, high density servers) and with fans that could deliver 300 cfm per load bank.

Three (3) 20-ton nominal, direct expansion CRAC units provided space cooling. Power was provided from a 225 KVA Power Distribution Unit (PDU). All equipment and ambient room spaces were instrumented, providing a complete array of temperature and environmental sensors to ensure accurate data collection.

Test Scenarios

During testing, steady state comparisons for the system were measured to determine operational capacity. Tests were completed with 25 and 65% perforated floor tiles. 80% perforated floor tiles were also tested. cfm data for each tile configuration was recorded to determine the total airflow in the CAC area. Average inlet air temperatures, outlet air temperatures, room ambient temperatures and hot aisle temperatures were recorded to capture steady state operation for all three scenarios.

During the test, a maximum allowable [DELTA]T of 45[degrees]F/7.2[degrees]C was established. Since newer, high-density servers have much higher outlet air temperatures than older servers, testing with the higher [DELTA] is desirable. This can be illustrated simply by comparing a variety of new 1U and blade server chassis, which can have [DELTA]T's as high as 52[degrees]F/11.1[degrees]C (Dell 2008, Capacity Planner).

65% Perforated tiles were chosen as an operational standard. The majority of testing conducted utilized 65% tiles. Each enclosure was set to a 20 kW load with all 3 CRAC units operated at full capacity. Each CRAC unit brought all of its compressors online to handle the maximum load, with steady state operation achieved after approximately one hour. Peak temperature increases were not noted until the full thermal capacities of the CRAC units were reached, at which time the load was reduced. The average [DELTA]T was approximately 30[degrees]F (-1.1[degrees]C), not the worst-case scenario of 45.0[degrees]F (7.2[degrees]C) as described earlier.

Additional Operational Scenarios

As any IT professional can attest, a data center is a very dynamic and ever changing environment. And while the initial test conditions were valuable, it was decided to view system performance under a variety of operational scenarios that occur in the IT space. And even under adverse or abnormal conditions that can occur due to component or resource failure.

Tests started with a "typical" installation, simulating a space not built with any means to separate and/or contain the cold or hot air streams in data center spaces. Recirculation and mixing occur, reducing the overall effectiveness of the mechanical system to reject the heat from IT loads. Comparing data with containment to data without containment was one goal of the test and considered necessary to determine the true load capacity of the mechanical cooling system. This comparative data will help illustrate the gains achieved by using a cold aisle containment system.

To demonstrate this, the doors and roof panels of the cold aisle containment system were removed and the aisle was fitted with standard 65% perforated tiles. The system was started with a single CRAC unit operating, with additional CRAC's brought online to prevent server temperature runaway as additional load was added.

As it turned out, for all testing above 6.9 kW per enclosure, the [DELTA]T observed was found to be unacceptable. Even assuming high [DELTA]T servers, the maximum load achieved during testing without containment was 11.5 kW per enclosure. The maximum load was determined when the load banks went into thermal overload, much like a typical server would. It is clear from the data that cold aisle containment resulted in significant performance improvements over the non-contained aisle testing. This is assumed to be a result of the complete elimination of air mixing and the slight pressurization of the cold aisle when the containment system was utilized.

And then what happens when something goes wrong? Typical data centers operate with some level of facility, system and component redundancy. The ability to overcome unforeseen failure events is critical for any data center. During testing, a number of failure scenarios were simulated to determine the impact of the cold aisle containment on different fault condition scenarios.

Generator Startup / UPS Fail Over

To replicate this failure, a load of 20 kW in each of 10 enclosures was established with all CRAC units running. At exactly the same instant, all of the CRAC units were de-energized at their disconnects, resulting in the total loss of air flow to the cold aisle containment system. After approximately 30 seconds (to simulate emergency generator start up) the CRAC units were restarted and began to supply air to the equipment. Power requirements of the load were monitored at all times to ensure that no single load bank was lost on a thermal overload. The generator restart resulted in a temporary increase in the hot aisle temperature, as was expected due to the loss of airflow. The shutdown also impacted the cold air temperature; undoubtedly due to the extra thermal energy that the CRAC units were required to dissipate after restart.

Opening of Aisle End Doors

Personnel activity is a day-to-day reality in a data center. Unfortunately, these movements can disturb the air paths that are required to maintain a sealed cold aisle containment system. These impacts are usually momentary, but what if a row end door of the cold aisle containment space was opened for over 30 minutes when running at 20 kW per enclosure? Room ambient temperature, as well as hot aisle temperature increased due to the short cycling of air from the cold aisle back to the CRAC units. This did reduce cold aisle temperature due to the lower temperature of air being cycled directly to the CRAC units. Even in this state, however, the air temperature to the servers was maintained within the normal operating range, and although the [DELTA]T rose due to lower air volumes being provided to the servers, the [DELTA]T was still within established guidelines.

Loss of Single CRAC Unit

The system that was tested could only be considered an "N" redundant solution since all CRAC units were required to be operational to support the 20 kW/enclosure IT test load. Though many data centers operate in at least an N+1 redundant configuration, what if this weren't the case? During testing, the system was operated at a steady state of 20 kW/ enclosure, and then a single CRAC unit was shut down. [DELTA]T Increased quite rapidly, and inlet temperature to the load banks was affected over time. The two remaining CRAC units were rated for a theoretical 70 kW capacity each, but with the loss of one of the CRAC units, the remaining two units were able to support 100 kW each. By the end of the test, it was determined that the load was indeed stable at these conditions, allowing sufficient time for repair of the "failed" CRAC.

Benefits Review: CapEx and OpEx

The data gathered is valuable in understanding the operation of a cold aisle containment system. However, it is critical to understand potential operational and installation {CapEx (Capital Expenditure) and OpEx (Operational Expenditure)} cost savings that can be realized from a relatively small cold aisle containment system.

The operational efficiencies data seen in Table 1 show that, on average, when considering a load of 10 kW/enclosure, a single 20-ton CRAC unit has the effective capacity of 50 kW when used without containment and a capacity of 70 kW per unit with containment. This improvement results in a reduction of the total CRAC units required Day 1, thereby reducing CapEx costs and reducing overall costs throughout the year by virtue of lower OpEx.
Table 1. Costs Analysis

            Cold Aisle Containment Analysis of DX Air Conditioners

Number   kW per   Total kW  W/O Containment  With Containment  Savings
of Cabs  Cabinet               50 kW per        70 kW per      Less ACs
                              20 Ton Unit      20 Ton Unit     Required
                               Number of        Number of
                              CRAC Units       CRAC Units

           4.8       48           1.0              0.7             0
   10      7.1       71           1.4              1.0             1
           9.4       94           1.9              1.3             0

           4.0       80           1.6              1.1             0
   20      6.0      120           2.4              1.7             1
           8.0      160           3.2              2.3             1
          10.0      200           4.0              2.9             1

           4.0      160           3.2              2.3             1
   40      6.0      240           4.8              3.4             1
           8.0      320           6.4              4.6             2
          10.0      400           8.0              5.7             2

           4.0      240           4.8              3.4             1
   60      6.0      360           7.2              5.1             2
           8.0      480           9.6              6.9             3
          10.0      600          12.0              8.6             3

           4.0      320           6.4              4.6             2
   80      6.0      480           9.6              6.9             3
           8.0      640          12.8              9.1             3
          10.0      800          16.0             11.4             4

           Cold Aisle Containment Analysis of DX Air Conditioners

Number of  kW per   Total kW  Annual Energy    One-Time Purchase
   Cabs    Cabinet              Benefits     and Install Benefits

             4.8       48           --                --
    10       7.1       71       $28,611.00        $49,500.00
             9.4       94           --                --

             4.0       80           --                --
    20       6.0      120       $28,611.00        $49,500.00
             8.0      160       $28,611.00        $49,500.00
            10.0      200       $28,611.00        $49,500.00

             4.0      160       $28,611.00        $49,500.00
    40       6.0      240       $28,611.00        $49,500.00
             8.0      320       $57,222.00        $99,000.00
            10.0      400       $57,222.00        $99,000.00

             4.0      240       $28,611.00        $49,500.00
    60       6.0      360       $57,222.00        $99,000.00
             8.0      480       $85,833.00       $148,500.00
            10.0      600       $85,833.00       $148,500.00

             4.0      320       $57,222.00        $99,000.00
    80       6.0      480       $85,833.00       $148,500.00
             8.0      640       $85,833.00       $148,500.00
            10.0      800      $114,444.00       $198,000.00

Air Conditioner

Actual (kW)  $28,611.00
Purchase     $30,000.00
Install      $19,500.00


CONCLUSION

A key strategy to increase cooling efficiency that is currently being deployed in operational data centers and discussed by the IT market at large, is to maximize the separation of cold and hot air flow paths. Aisle containment has emerged as a relatively low cost and easy to implement solution. The testing of the Cold Aisle Containment system validated the assumption that utilizing a cold aisle containment system enables the data center environment to support higher density loads while providing a more efficient use of the environmental equipment. Increased efficiencies of up to 40% were achieved utilizing cold aisle containment in comparison to environments without containment. The CRAC units were able to support [DELTA]T's (supply/return) ranging from 75 to 100[degrees]F (23.89 to 37.78[degrees]C), depending on load conditions.

Environments can be built within existing data centers, utilizing cold aisle containment, to support 20 kW/enclosure loads while most of today's data centers (without CAC) only support load densities of approximately 3 to 5 kW per individual enclosure. These significant improvements can help extend the lifespan of many operational data centers. As a result, a reduction of the capital expenditure needed to maintain a company's computing environments is made possible by postponing the build-out of new data center space.

Although highly effective in existing data center spaces, when employed in the design of new data centers, Cold Aisle Containment can also help to maximize energy efficiency potentials and reduce both capital and operational expenses. In either case, Cold Aisle Containment provides a viable option to meet increasing computing demands in an efficient and cost-effective manner.

SOURCES

Hills, R. and S. Iyer. 2009. Improved data center efficiency--Incorporating air stream containment. EDS (An HP Company), Whitepaper.

Kennedy, D. and H. Villa. 2009. Cold aisle containment for improved data center cooling efficiency. Rittal Corp. Whitepaper, 506.

ASHRAE. 2008. Best practices for datacom facility energy efficiency. Atlanta: American Society of Heating, Refrigeration and Air-Conditioning Engineers, Inc.

DISCUSSION

H. Ezzat Khalifa, Professor, Syracuse University, Syracuse, New York: Why did the two 9.4 kW racks on slide 12 have different temperature rises? Were the CRACs operated with constant speed fans and constant air exit temperatures?

Herb Villa: Thank you for your question. The two data points for the 9.4 kW test scenarios are presented to document the temperature change for two different test scenarios. The first 9.4 kW result was with one CRAC unit operating, while the second test point was with two units operating, thus the lower temperatures. All fans operated at constant speed.

Herb Villa is a customer solutions engineer with Switch and Data, Tampa, FL.
COPYRIGHT 2010 American Society of Heating, Refrigerating, and Air-Conditioning Engineers, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2010 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Villa, Herb
Publication:ASHRAE Transactions
Article Type:Report
Geographic Code:1USA
Date:Jan 1, 2010
Words:3010
Previous Article:System architectures and fluids for high heat density cooling solutions.
Next Article:The Virtual Cybernetic Building Testbed--a building emulator.
Topics:

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters