# Real-time prediction of rack-cooling performance.

ABSTRACTA software tool has been developed that estimates the cooling performance of racks bounding a common cold aisle in a raised-floor data center. The tool predicts, in real time, the effect of row length, rack power and airflow distribution, and perforated tile flow rate on cooling performance. For any scenario of interest, airflow patterns within the cold aisle are computed based on the superposition of elemental airflow patterns that have been determined in advance ("offline") from computational fluid dynamics (CFD) simulations and stored as empirical curve fits. Rack-cooling performance is based on the recirculation index (RI), which is defined as the percentage of airflow ingested by a rack, which originated outside the cold aisle.

This paper discusses the algorithm upon which the software tool is based and provides examples of its application. Novel aspects of this work include the real-time prediction of rack-cooling performance, the definition and use of the recirculation index as a cooling performance metric, and the computation of cold-aisle airflow patterns based on superposition and empirical correlations.

INTRODUCTION

A cold aisle in a data center is typically formed between two rows of racks that face one another. All of the racks share the cool air supplied through perforated floor tiles located between the two rows. Typically, some of the cooling airflow leaves the cold aisle without passing through the racks, while some warm air enters the cold aisle from other parts of the room. As a result, some rack inlet locations may become too hot even when there is a net excess of cool supply airflow relative to the total rack airflow. This situation is particularly prevalent and difficult to predict when, as is often the case in real data centers, the distribution of rack power and airflow are highly nonuniform.

Although a more uniform distribution of rack power would significantly simplify cooling-performance prediction, other design considerations make a highly nonuniform distribution a necessity. First, a single, scalable cluster of racks may include everything from patch panels with negligible power and airflow to UPS racks with modest power and airflow to racks of blade servers with extreme power and airflow. Second, the refresh rate of IT equipment may be every one to three years, while that of the cooling infrastructure may be every 10 to 25 years (ASHRAE 2005). A given rack's IT loading is generally not known prior to initial build out, and it is virtually impossible to plan rack-by-rack populations even further into the future. And yet, the cooling infrastructure, which is expected to adequately cool whatever equipment is installed, has to be built to some specification (e.g., 100 W/[ft.sup.2] or a peak and average rack power over a specified zone). One approach would be to provision for sufficient cooling and power for the peak anticipated loads at all rack locations. As very few racks would actually reach the peak power and airflow, this approach is grossly inefficient (Rasmussen 2005b). Consequently, it is practical to provision for a much lower average cooling capacity and allow neighboring racks to share cooling airflow. Intuitively, there are some practical limits on the ability of an overpowered rack to capture unused cooling airflow from underpowered neighbors. Predicting cooling performance for a general distribution of nonuniform-power racks is further complicated by the fact that many such interactions typically occur simultaneously, creating complex airflow patterns within the cold aisle.

This paper describes the methodology behind a new design tool that estimates the cooling performance of all racks sharing cooling resources from a common cold aisle. Calculations are performed in "real time" and apply to all practical (nonuniform) values of rack power and airflow. The scope of the tool, as discussed here, is restricted to specific rack topologies and room environments; however, the general methodology may be readily extended to broaden the range of applications and improve accuracy. In this paper, two example applications are discussed with results compared to corresponding computational fluid dynamics (CFD) simulations. Airflow patterns and hot-spot location predictions agree well with CFD simulations. Useful cooling performance predictions are made even when the actual data center room environment is outside the intended scope of the tool.

The method involves partially decoupling the cold aisle from the room environment and computing only the airflow within the cold aisle. The effect of the room environment enters the calculation primarily through prescribed airflow boundary conditions at the two ends of the cold aisle. (The appropriate end airflow boundary conditions are determined separately from empirical correlations derived from CFD simulations.) The airflow patterns in the horizontal direction are computed based on the superposition of simpler elemental airflow patterns derived empirically from CFD simulations. The elemental airflows are associated with only one activated "driving force boundary condition" (e.g., a rack drawing air from the cold aisle). After the horizontal airflow patterns are computed from superposition, the flow out of or into the top of each control volume in the cold aisle is computed based on the conservation of mass. With the complete airflow solution known, the migration of warm, recirculated air throughout the cold aisle is tracked by computing the concentration of recirculated air in each control volume. The latter is numerically equal to the recirculation index (RI)--the key rack-cooling performance metric--for the rack, which shares a face with the control volume.

The tool offers substantial improvement in accuracy and speed over hand calculations and can be used without special skills to quickly perform "what-if" analyses of potential layouts. It may also serve as the engine for a broader cooling optimization tool or be used to narrow design options prior to performing a more detailed and expensive CFD simulation.

Experimental validation of the CFD simulations used to develop the tool was not performed as part of the present study. However, experimental validation data are available in the literature (Patel et al. 2001; VanGilder and Schmidt 2005; Shrivastava et al. 2006b) for similar data center applications modeled using the same commercially available CFD software and using modeling practices similar to those employed here.

SCOPE OF APPLICATIONS

The tool estimates rack-cooling performance for a two-row cluster of racks bounding a common cold aisle as shown in Figure 1. There are an equal number of contiguous racks in each row, and the two tile (4 ft) wide cold aisle is completely covered by perforated floor tiles that all deliver an identical airflow rate. IT equipment is assumed to be uniformly distributed throughout the racks, and there are no leakage paths through the racks (e.g., blanking panels are installed in all unoccupied IT slots). The method discussed here is applicable to any room environment; however, the room environment considered for discussion purposes has large aisle spacings and a 14 ft ceiling height, and the room return airflow is distributed uniformly over the ceiling.

Input to the algorithm includes row length (i.e., the number of racks), the power and airflow of each rack, and the perforated tile airflow rate, plus details of the surrounding room environment. Though not discussed here, the method could be readily extended to include alternative geometric layouts (e.g., a three-tile cold aisle), nonuniform rack loading, and nonuniform perforated tile airflow rates.

RACK-COOLING PERFORMANCE METRIC: THE RECIRCULATION INDEX (RI)

The ultimate cooling objective is to precisely control the rack inlet air temperatures, which depend on the airflow patterns and temperatures within and around the cold aisle. Air drawn in from outside the cold aisle is generally heated to some degree by the rack exhaust and will be hereafter referred to as recirculated air. While the temperature of the recirculated air depends heavily on the application and measurement location (and is therefore difficult to predict), air that travels directly from a perforated tile to a rack inlet will remain close to the supply temperature. Thus, good cooling performance can be ensured when the great majority of airflow ingested by a rack comes directly from the perforated tiles located within its own cold aisle. Furthermore, a cluster of racks that has all its cooling needs met by airflow originating from its own cold aisle represents a scalable unit from which a larger facility with predictable cooling performance may be constructed.

[FIGURE 1 OMITTED]

With the above in mind, the recirculation index (RI) is defined as the percentage of recirculated air ingested by the rack. An RI of 0% implies that all of the rack inlet air is drawn directly from the perforated tiles, while an RI of 100% implies that all of the rack inlet air is drawn from outside the cold aisle. Note that a low RI is sufficient to guarantee cool inlet temperatures; however, because the RI is not based on temperature, a high RI does not guarantee excessively high inlet temperatures (e.g., the room environment outside the cold aisle might be sufficiently cool).

Like the supply heat index (SHI) of Sharma et al. (2002) and Bash et al. (2003), the RI is a dimensionless measure of the infiltration of warm (recirculated) air into the cooling stream. The RI and the SHI differ in that the RI quantifies the infiltration of warm air based primarily upon airflow patterns in the cold aisle without regard to temperature. The SHI quantifies the infiltration of warm air based on the energy content (enthalpy) of the relevant airflow streams. As such, relevant temperatures must be known before SHI can be computed. In general, RI and SHI values differ; however, both are zero when there is no infiltration of warm air into the cooling stream.

REFERENCE CONTROL VOLUMES AND AIRFLOW NAMING CONVENTION

For each layout of interest, the airflow patterns in the cold aisle must be determined before the RIs may be computed. A reference grid is required to establish the mapping of airflows to locations within the cold aisle. Such a grid is established by dividing the cold aisle into a number of control volumes, each with a one-tile footprint and rack height. Figure 2 shows one typical control volume and neighboring rack with all other control volumes and racks hidden for clarity. Computed airflows are assumed uniform over each of the six faces of the control volume; a fairly coarse representation of the cold aisle airflow pattern is sought in the interest of speed and simplicity. Figure 2 also shows the vertical airflows: the perforated tile airflow [Q.sub.T] and the top airflow QAto[p.sub.1]. The latter is directed outward in the figure but in practice may also be directed inward.

[FIGURE 2 OMITTED]

Figure 3 shows the horizontal airflow naming convention for a general n-length cluster of racks where n is the row length in rack or tile widths (2 ft increments). Though not explicitly shown, there are 2n total control volumes. We assume that the tile and rack airflows are known; the latter may be known directly or deduced from the total rack power and an estimated temperature rise across the rack. The end airflows (QA[x.sub.0], QB[x.sub.0], QA[x.sub.n], and QB[x.sub.n]) are determined separately (though also based on empirical correlations from CFD simulations) and may be regarded as known at this stage. So, for any layout, determining the airflow pattern means computing a total of 5n - 2 airflows located within the interior and along the top of the cold aisle.

PARTIAL DECOUPLING OF THE COLD AISLE AIRFLOW FROM THE ROOM ENVIRONMENT

The airflow within the cold aisle depends on the details of the cluster itself and also on the room environment (including thermal effects, airflow from neighboring clusters of racks, etc.). It is possible to perform a CFD simulation of the coupled cold aisle and room environment; however, this is expensive and slow, particularly if a large number of design scenarios are to be considered.

[FIGURE 3 OMITTED]

It is possible, under many scenarios of practical significance, to determine the airflow within the cold aisle with only weak coupling to the room environment. Airflow in the cold aisle is fairly "ideal," being largely free of strong jets and substantial buoyancy forces. As such, computing airflow patterns within the cold aisle is a relatively simple task compared to computing airflow patterns elsewhere in the data center environment, and there are viable alternatives to CFD.

In the partial decoupling of the cold aisle, the airflow at the ends of the cold aisle is assumed known. The perforated tile and rack airflows are also known. Along the top of the cold aisle, the airflow is allowed to "float" subject to a prescribed constant ambient pressure. Provided the end airflows are known accurately, differences between the isolated-cold-aisle solution and the completely coupled solution arise only from real deviations from the assumed constant pressure boundary along the top of the cold aisle.

The partial decoupling approach inherently assumes that the end airflow can be adequately determined by some means. The end airflows are strongly influenced by the cluster details and the room environment, so predictions must generally be based on CFD simulations. In order to predict cold aisle airflow patterns on a real-time basis, (many) CFD simulations may be run "offline" with the results reduced to empirical correlations through statistical analysis.

COLD AISLE AIRFLOW DYNAMICS, DIMENSIONLESS ANALYSIS, AND SUPERPOSITION

Airflow patterns associated with "blowing" and "sucking" sources are markedly different. The former are typically referred to as jets, which are usually turbulent and characterized by a core region of high velocity surrounded by a region of large velocity gradients. With the latter, airflow may be characterized as laminar or having only a low level of turbulence as air is drawn gently from all directions; velocity gradients are very small except, perhaps, in the immediate vicinity of the source. (Anyone who has tried to "suck out" a candle can appreciate the difference in airflow patterns.)

For the present purposes, the racks represent sucking sources. And, while the perforated tiles technically represent blowing sources, the airflow is spread uniformly across the cold aisle so that there are no large velocity gradients (except at the ends of the aisle). In this case, the tile airflow is better described as plug flow rather than jet flow. Furthermore, in practical scenarios, the bulk of the cold aisle is filled with cool supply air of nearly uniform temperature. Consequently, buoyancy forces are negligible relative to pressure and momentum forces. Under these conditions, the airflow within the cold aisle will be inviscid, incompressible, and irrotational. (See, for example, White [1986]). For such an "ideal" flow, the conservation of mass equation may be expressed as Laplace's equation,

[[nabla].sup.2][phi] = 0, (1)

where the velocity components in the x, y, and z directions are related to the velocity potential [phi] by u = [partial derivative][phi]/[partial derivative]x, v = [partial derivative][phi]/[partial derivative]y, and w = [partial derivative][phi]/[partial derivative]z, respectively. Here, Equation 1 is used only as a vehicle for discussing a couple of important simplifications that can be made in analyzing the cold-aisle airflow. First, Equation 1 (or explicit versions of the mass and momentum equations), along with appropriate boundary conditions, can be nondimensionalized with the tile width L and tile airflow velocity [V.sub.T] selected as representative length and velocity scales. The details are not presented here; however, the results indicate that it is not necessary to consider all combinations of tile and rack airflows explicitly. In dimensionless terms, computed airflows depend only on the ratio of rack velocity to tile velocity (or the ratio of airflows, since tile and rack airflow areas are fixed). This provides an enormous savings in effort and storage when correlating airflow solutions. In determining cold-aisle airflow patterns, only one (dimensional) perforated tile airflow rate needs to be used in the CFD simulations, although the dimensionless airflow results are valid for all perforated tile airflow rates.

The second and most important simplification suggested by Equation 1 is the use of superposition to construct a general airflow solution out of several simpler solutions. If [[phi].sub.1] and [[phi].sub.2] are solutions to (the linear partial differential) Equation 1, [[phi].sub.1] + [[phi].sub.2] is also a solution to Equation 1. So, velocity potentials (or actual velocity components or total airflows over a consistent area) of simpler, elemental flow solutions may be added together to obtain a new, physically valid flow solution. For example, assume the airflow patterns associated with racks A1 and B3, each individually activated, are known. The airflow pattern associated with both racks A1 and B3, simultaneously activated, may be computed by simply summing the two individual airflow solutions.

It is worth emphasizing that the use of dimensionless airflows and superposition makes it possible for airflows to be computed for any general layout using empirical correlations derived from CFD simulations. Otherwise, there would be far too many combinations of rack and tile airflows to compute and store empirically.

COMPUTING HORIZONTAL AIRFLOWS WITH SUPERPOSITION

Horizontal airflows are computed based on superposition of elemental airflow solutions associated with each rack and each of the four end airflows. Figure 4 illustrates this approach for a two-rack (n = 1) cluster. Note that Figure 4 indicates which airflow boundary conditions are activated and deactivated in each of the elemental airflow solutions; however, it is the horizontal airflow components internal to the cold aisle (not shown in Figure 4) that are actually added to one another. In general, a total of 2n + 4 elemental solutions are required for each layout. Fewer elemental solutions are required if one or more racks have zero airflow and if one or both of the ends of the cold aisle are sealed (e.g., with doors or are coincident with walls [VanGilder 2005]).

Only the horizontal airflows are determined through superposition because the tile airflow is activated ("on") in each of the elemental airflow solutions. If the vertical airflows at the top of the cold aisle were computed using superposition of the elemental airflow solutions, the QATo[p.sub.i] and QBTo[p.sub.i] values would be overpredicted by a factor of 2n + 4. The solution is to simply compute the top airflows as a separate step based on conservation of mass.

[FIGURE 4 OMITTED]

DETERMINING EMPIRICAL CORRELATIONS FROM CFD MODELING FOR THE ELEMENTAL AIRFLOWS

The elemental airflow solutions could be obtained, in theory, in any manner, including physical testing. In practice, it is economical to obtain them through CFD modeling, and this is the approach taken here.

As stated above, there are, in general, 2n + 4 elemental solutions for each layout of interest--2n elemental solutions associated with racks plus four end-airflow elemental solutions. Because of the geometric symmetry of the cluster, only about one quarter of the elemental airflow solutions are "unique" and need be determined from CFD. There are n/2 + 1 unique elemental solutions if n is even; (n + 1)/2 + 1 if n is odd. The remaining (non-unique) elemental airflow solutions may be determined from an appropriate reinterpretation of the unique solutions by changing variable indices and signs as appropriate. In addition to being efficient, this use of symmetry forces the final output from the tool to be perfectly symmetric. Figure 5 shows the boundary conditions associated with the five unique elemental solutions that must be determined and stored for an n = 7 cluster of racks.

As discussed above, the elemental airflow solutions need only be determined at a single perforated tile airflow rate because dimensionless airflows depend only on dimensionless rack or end airflows. (In dimensionless terms, the results are the same whether the perforated tile airflow is modeled as, for example, 100 or 750 cfm). Consequently, the tile airflow rate in the CFD simulations was fixed at 300 cfm, while rack airflow rates were varied from 0 to 1800 cfm, corresponding to dimensionless rack airflow rates in the range of 0 to 6. End airflows were varied from 0 to 900 cfm or 0 to 3 in dimensionless terms. Figure 6 shows the dimensionless airflows induced in the cold aisle by rack A1. There are four curves in Figure 6 because there are four internal airflows associated with an n = 2 cluster of racks. In general, 3n - 2 internal airflows are computed for each elemental airflow solution. The dimensionless internal airflows can be related to the dimensionless rack airflows conveniently with a least-squares fit to a cubic polynomial of the generic form

[FIGURE 5 OMITTED]

Q* = [c.sub.1](QR[A*.sub.1]) + [c.sub.2](QR[A*.sub.1])[.sup.2] + [c.sub.3](QR[A*.sub.1])[.sup.3] (2)

so that only the coefficients [c.sub.1], [c.sub.2], and [c.sub.3] need be stored. Using a curve-fit rather than a look-up table offers the benefit that results outside the original domain are automatically interpolated.

Several hundred CFD runs are required to produce the several thousand empirical airflow correlations required to cover a practical range of row lengths. It is therefore necessary to automate the process of creating the CFD runs and converting the raw CFD data into the curve-fit constants of Equation 2.

To obtain the elemental airflow solutions, the cold aisle is modeled with a commercially available CFD program. The k-[epsilon] turbulence model is used, although laminar-only simulations provide essentially the same results for the few selected scenarios checked. A structured Cartesian grid with uniform 6 x 6 x 6 in. grid cells provides a good match to the geometry and is fairly grid independent. The cold aisle was subject to the following boundary conditions:

* Airflow enters the computational domain uniformly in the vertical direction over the entire perforated tile area.

* Airflow exits the computational domain uniformly in the horizontal direction over the area of the rack face for any activated rack.

[FIGURE 6 OMITTED]

* Airflow enters or exits the computational domain uniformly in the horizontal direction over the area of the end of the rows for any activated end airflow.

* The gauge pressure at the top of the cold aisle is fixed at zero so that airflow may either enter or exit in the vertical direction.

* Vertical surfaces corresponding to deactivated racks or end flows are modeled as "symmetry" surfaces.

DETERMINING END AIRFLOW: EFFECT OF THE ROOM ENVIRONMENT

Unlike the airflow within the cold aisle, the end airflow is strongly coupled to the airflow in the surrounding room environment. As viscous and buoyancy forces may be significant, the flow is not ideal. End airflows do not depend simply on the dimensionless rack airflow rates, and they cannot be determined by simple superposition of the elemental rack airflow solutions. The end airflows can still be derived empirically from CFD data; however, a relatively large number of simulations must be performed to characterize a practical range of rack populations and room environments. The focus here is restricted to predicting the end airflow as a function of rack power and airflow distribution for any row length and perforated tile flow rate for a single fixed room environment. A more general end airflow model is discussed in Shrivastava et al. (2006a).

The environment considered here is large and free of other racks and objects. Air is supplied at 60[degrees]F while the temperature of the room far away from the racks is 68[degrees]F. Air is exhausted uniformly over a 14 ft high ceiling. Recalling the airflow naming convention shown in Figure 3, it is reasonable to assume that (for a fixed room environment and tile airflow rate) the QA[x*.sub.0] dimensionless end airflow can be predicted by the multiple linear regression model:

QA[x*.sub.0] = [a.sub.0] + [a.sub.A1]QR[A*.sub.1] + [a.sub.A2]QR[A*.sub.2] + ... + [a.sub.An]QR[A*.sub.n] + [a.sub.B1]QR[B*.sub.1] + [a.sub.B2]QR[B*.sub.2] + ... + [a.sub.Bn]QR[B*.sub.n] (3)

where the regression coefficients [a.sub.Ai] and [a.sub.Bi] effectively weight the relative effect of each rack on the end airflow. In general, the regression coefficients associated with racks located near the end of the row will be larger than those of interior racks. For the fixed environment considered, the constant [a.sub.0] is negative, implying that the flow is "out" when there is zero rack airflow. To determine the values of the coefficients in Equation 3, 100 CFD simulations were performed at each of several perforated tile flow rates considered. Each CFD simulation contained a unique distribution of rack powers drawn randomly from a large pool of values. The pool of rack power values was in turn based on actual data center rack populations as reported in a survey of data center designers and operators (Rasmussen 2005a) and reproduced as Figure 7. The airflow for each rack was set based upon a 20[degrees]F average temperature rise across the rack.

Since four end airflows (QA[x.sub.0], QB[x.sub.0], QA[x.sub.n], and QA[x.sub.n]) are computed in each CFD simulation, a total of 400 data sets were generated from each series of simulations at a fixed perforated tile airflow rate. A commercially available statistical tool was used to determine the least-squares fit of the coefficients in Equation 3. In general, the value of the coefficient of determination ([R.sup.2]) is a measure of the fit of the regression model to the data. The calculated values of [R.sup.2] (above 95%) ensure that the above end airflow model successfully captures the effects of a general distribution of rack power and airflow (Montgomery 2001).

Finally, the value of an accurate end airflow prediction should be kept in the proper context. With only one control volume in the vertical direction, some end airflow details will be lost no matter how good the estimate of the total end airflow. Furthermore, since the effects of end airflows are localized, much of the airflow in longer clusters may be unaffected by the end airflow.

COMPUTING AIRFLOW INTO OR OUT OF THE TOP OF THE COLD AISLE

The airflows into or out of the top of each control volume have been left "floating" as necessary degrees of freedom in the model. Now, with all of the horizontal airflows computed as discussed, the airflow at the top of each control volume is computed based on the conservation of mass. Using dimensional quantities, the equations for A-row and B-row control volumes are:

QAto[p.sub.i] = [Q.sub.T] - QR[A.sub.i] + QA[x.sub.i-1] - Q[z.sub.i] - QA[x.sub.i] (4a)

QBto[p.sub.i] = [Q.sub.T] - QR[B.sub.i] + QB[x.sub.i-1] + Q[z.sub.i] - QB[x.sub.i] (4b)

Applied to all control volumes, Equations 4a and 4b represent a total of 2n equations. At this stage, there is only one unknown per equation (QAto[p.sub.i] and QBto[p.sub.i]) so they may be solved sequentially.

[FIGURE 7 OMITTED]

COMPUTING THE RECIRCULATION INDEX

At this point, all airflows within the cold aisle are known for the problem at hand. What remains is to track the airflow into each rack so that its origin may be identified and the recirculation index (RI) can be calculated. Recall that the RI is the percentage of recirculated air ingested by a rack. The recirculated air can enter the cold aisle at any point where there is inflow at the ends of the rows or along the top of the cold aisle. Further, the warm recirculated air need not directly enter the cold aisle via the control volume immediately adjacent to a rack of interest; it may enter anywhere, travel anywhere the airflow patterns take it, and end up at the inlet of any rack.

To compute the RI for each rack, the relative amounts of cool supply air and warm recirculated air must be determined at all points in the cold aisle. This may be expressed mathematically by defining the concentration of recirculated air at any point in the cold aisle as

[C.sub.recirc] = (mass of recirculated air)/(total mass of air). (5)

Strictly speaking, it follows from Equation 5 that the supply airflow emerging from the tiles has a [C.sub.recirc] = 0 and that anywhere there is inflow along the top or sides of the cold aisle [C.sub.recirc] = 1. In practice, it is found that setting [C.sub.recirc] = 1 at the ends of the cold aisle is unnecessarily conservative; in actual data centers, a substantial fraction of the air around the ends of the cold aisle (particularly closer to the floor) may be fairly cool. It is found empirically that setting [C.sub.recirc] = 0.5 at the ends of the cold aisle is a reasonable approximation for many scenarios. Note that, strictly speaking, the RI implicitly depends on temperature to some degree whenever [C.sub.recirc] is set to a value other than "1" over the sides or top of the cold aisle.

Now consider a thin volume just covering a rack inlet. Equation 5 applied to this volume represents the average [C.sub.recirc] over this volume. By dividing the numerator and denominator by a small time increment [DELTA]t and taking the limit as [DELTA]t [right arrow] 0, it becomes apparent that the average [C.sub.recirc] over a rack inlet is numerically equivalent to the RI (when expressed as a percentage) for the rack of interest. Thus, to quantitatively determine the RIs for each rack, the average [C.sub.recirc] over each rack inlet must be computed. Referring back to Figure 2, the average [C.sub.recirc] over each rack inlet can be estimated as the average [C.sub.recirc] within the control volume immediately adjacent to the rack of interest. [C.sub.recirc] for each control volume may be computed based on the conservation of mass of the recirculated air:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], (6)

where Q, a known value at this stage of the calculation, is the total airflow rate through a control volume face.

Next Equation 6 is applied to the two control volumes associated with the generic section of cold aisle shown in Figure 8. For convenience, [C.sub.recirc] crossing each control volume surface is labeled following the same convention used for airflows while dropping the "recirc" subscript. The result is:

[C.sub.T][Q.sub.T] + (CA[x.sub.i-1])(QA[x.sub.i-1]) = (CR[A.sub.i])(QR[A.sub.i]) + (CA[x.sub.i])(QA[x.sub.i]) + (C[z.sub.i])(Q[z.sub.i]) + (CAto[p.sub.i])(QAto[p.sub.i]) (7a)

[C.sub.T][Q.sub.T] + (CB[x.sub.i-1])(QB[x.sub.i-1]) + (C[z.sub.i])(Q[z.sub.i]) = (CR[B.sub.i])(QR[B.sub.i]) + (CB[x.sub.i])(QB[x.sub.i]) + (CBto[p.sub.i])(QBto[p.sub.i]) (7b)

[FIGURE 8 OMITTED]

Equations 7a and 7b cannot be solved directly as written because the number of unknown [C.sub.recirc] values exceeds the number of equations. Estimating the [C.sub.recirc] crossing each control volume face as the average [C.sub.recirc] from the "upwind" control volume results in the proper balance of 2n unknown [C.sub.recirc] values and 2n equations. With this upwind approach, the appropriate [C.sub.recirc] values cannot be inserted into Equations 7a and 7b until after the airflow patterns in the cold aisle have been computed, thereby establishing the upwind directions.

Table 1 shows the appropriate upwind values of [C.sub.recirc] to be used in Equations 7a and 7b where the C[A.sub.i] and C[B.sub.i] are the average [C.sub.recirc] over the relevant A and B control volumes, respectively. Not included in the table are the settings for the end airflows (QA[x.sub.0], QB[x.sub.0], QA[x.sub.n], and QB[x.sub.n]). In this case, [C.sub.recirc] = 0.5 for any "inflow," as discussed above.

With the values of [C.sub.recirc] taken from Table 1, the 2n equations represented by Equations 7a and 7b are solved simultaneously for the 2n C[A.sub.i] and C[B.sub.i] values. These simple linear equations can be solved using a variety of well-known solution techniques. Calculation time with common computing hardware is negligible for any practical scenario. Finally, as previously discussed, the computed C[A.sub.i] and C[B.sub.i] values may be directly interpreted as the RIs for the A and B racks, respectively.

Note that because of the similarity of the energy and concentration equations, if desired, the average temperature over each control volume could be computed following an analogous procedure.

SUMMARY OF THE SOLUTION STEPS

Figure 9 provides a step-by-step summary of the algorithm employed in the software tool. The details of all steps have been discussed above with the exception of the need for a tuning constant in step 5. It is found empirically that the horizontal airflows induced in the cold aisle by any inward-directed end airflow should be reduced by approximately 60% before being summed with the other elemental airflow patterns in step 6. Inward or jet flow is much less "ideal" than outflow, as discussed previously. Furthermore, the inflow can transport warm air into the cold aisle, increasing the importance of buoyancy and making the flow even less ideal. The empirical tuning constant partially compensates for the loss of accuracy under these conditions.

[FIGURE 9 OMITTED]

APPLICATION EXAMPLES

In this section, two example applications are discussed in which n = 7 and n = 12 scenarios are compared to CFD data. Figure 10 shows the n = 7 scenario, in which the CFD model closely matches the scope of the tool. In other words, the environment is large and free of other racks and objects. Air is supplied at 60[degrees]F, while the temperature of the room far from the racks is 68[degrees]F. Air is exhausted uniformly over a 14 ft high ceiling. As such, this scenario represents a benchmark comparison, showing the typical level of accuracy that can be expected from the tool when the end airflow model precisely applies to the actual room environment of interest. With supply airflow of 400 cfm/tile and an assumed 20[degrees]F average temperature rise across each rack, 50% total excess cooling air is supplied to the cold aisle.

Figure 10 shows good agreement of computed RIs and airflows with the benchmark CFD model. The tool somewhat underpredicts the RI values, mainly due to its limited resolution in the vertical direction at the ends of the cold aisle. While the CFD simulation shows outflow near the floor and inflow near the top of the racks at the ends of the cold aisle, with only one control volume in the vertical direction, the tool predicts only outflow. As a result, the CFD simulation predicts that some warm recirculated air enters the ends of the cold aisle while the tool predicts that warm recirculated air enters only through the top of the cold aisle.

[FIGURE 10 OMITTED]

Figure 11 shows the n = 12 scenario, in which the CFD model is based on a more realistic data center layout (although the tool is based upon the same large, open room environment as discussed above). One goal is to assess the accuracy of the predictions when applied to a realistic data center layout, even though this is outside the scope of the end airflow model in the tool. A second goal is to compare the RIs predicted by the tool with maximum rack inlet temperatures predicted by CFD. As the maximum rack inlet temperature is the ultimate cooling performance metric, the RIs must follow the same trend in order to serve as a useful design metric. Furthermore, this comparison provides one example of how the magnitude of the RIs compare to realistic maximum inlet temperatures.

The results of the CFD simulation in this case have been extracted from a simulation of a larger room environment, which is not shown. As with the previous example, air is supplied at 60[degrees]F, while the temperature of the room far from the ends of the rows is 68[degrees]F. Air is exhausted uniformly over a 14 ft high ceiling. However, the rows of racks repeat on a standard seven-tile pitch (ASHRAE 2004) using a mirror symmetry layout of racks. In other words, there is a 3 ft hot aisle on each side of the cluster, and racks immediately across the hot aisle from one another are of the same power. The room environment at the ends of the rows is large and open. Again, with a supply airflow of 400 cfm/tile and an assumed 20[degrees]F average temperature rise across each rack, 20% less total cooling air is supplied to the cold aisle than consumed by the racks.

[FIGURE 11 OMITTED]

Figure 11 shows reasonable agreement between airflow predictions and shows that the tool correctly locates the hot spot. Overall, the trend of the predicted RIs reasonably tracks the maximum inlet temperatures predicted by CFD. Most airflows are somewhat underpredicted; some of this is due to the fact that the end airflows are underpredicted (as this example is outside the scope of the tool's end airflow model), and some of this is due to the deviation from ideal flow in the cold aisle. As discussed previously, substantial inflow at the ends of the rows generally leads to less ideal airflow. This example provides a good range of maximum inlet temperatures and RIs for example purposes; however, with insufficient cooling air supplied to the racks, it would likely not be an acceptable design. More acceptable designs would have less end inflow, improving the predictions from the tool.

Relative to the maximum inlet temperatures from CFD, the tool also somewhat underpredicts the RIs of the extreme outboard racks. In this case, the assumed [C.sub.recirc] = 0.5 is too conservative, as the CFD simulation shows that the ends of the cold aisle are relatively cool.

Finally, the scenario considered in Figure 11 provides one example of how RI values may be related to maximum inlet temperatures. ASHRAE (2004) recommends inlet temperatures in the range of 68[degrees]F-77[degrees]F for Class 1 equipment. A maximum inlet temperature of 77[degrees]F corresponds to a maximum RI of about 20% in this scenario.

CONCLUDING REMARKS

The software tool discussed here may be used to perform what-if studies of alternative equipment layouts and perforated tile flow rates or may serve as part of a broader optimization engine. The tool is much faster, cheaper, and easier to use than CFD yet handles complex, nonuniform rack power and airflow distributions that cannot be analyzed adequately by hand calculations. Still, predictions are approximate by nature and reasonable accuracy can be expected only when the scenario of interest is somewhat consistent with the tool's scope of applications. The basic methodology outlined here can be readily extended, however, to cover alternative cluster geometries and room environments as well as nonuniform perforated tile airflow rates. Finally, one obvious extension to the methodology would be to divide the cold aisle into a greater number of control volumes, particularly in the vertical direction. This would improve overall accuracy, allow nonuniform distributions of IT equipment within a single rack to be modeled explicitly, and allow the application of more sophisticated end airflow boundary conditions.

NOMENCLATURE

[phi] = velocity potential

[a.sub.i] = regression coefficient for end airflow model

CAto[p.sub.i] = value of [C.sub.recirc] crossing the top of A-row control volumes

CA[x.sub.i] = value of [C.sub.recirc] crossing A-row control volume faces in x-direction

CBto[p.sub.i] = value of [C.sub.recirc] crossing the top of B-row control volumes

CB[x.sub.i] = value of [C.sub.recirc] crossing B-row control volume faces in x-direction

[c.sub.i] = empirical curve-fit coefficient for airflow within the cold aisle

CR[A.sub.i] = value of [C.sub.recirc] ingested by Rack [A.sub.i]

CR[B.sub.i] = value of [C.sub.recirc] ingested by Rack [B.sub.i]

[C.sub.recirc] = concentration of recirculated air

[C.sub.T] = value of [C.sub.recirc] crossing perforated tiles

C[z.sub.i] = value of [C.sub.recirc] crossing control volume faces in z-direction

L = characteristic length scale (2 ft)

n = row length in rack-width (2 ft) increments

Q = generic airflow

QAto[p.sub.i] = airflow at the top of A-row control volumes

QA[x.sub.i], = airflow in the x-direction for A-row control volumes

QBto[p.sub.i] = airflow at the top of B-row control volumes

QB[x.sub.i] = airflow in the x-direction for B-row control volumes

QR[A.sub.i] = airflow rate of Rack [A.sub.i]

QR[B.sub.i] = airflow rate of Rack [B.sub.i]

[Q.sub.T] = perforated tile airflow rate

Q[z.sub.i] = airflow in the z-direction

[R.sup.2] = coefficient of determination

RI = recirculation index

SHI = supply heat index

u = velocity in the x-direction

v = velocity in the y-direction

w = velocity in the z-direction

* = denotes dimensionless quantities

REFERENCES

ASHRAE. 2004. Thermal guidelines for data processing environments. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.

ASHRAE. 2005. Datacom equipment power trends and cooling applications. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.

Bash, C.E., C.D. Patel, and R.K. Sharma. 2003. Efficient thermal management of data centers--Immediate and long-term research needs. HVAC & R Research 9(2):137-52.

Montgomery, D.C. 2001. Design and Analysis of Experiments, 5th ed., pp. 392-426. New York: John Wiley & Sons, Inc.

Patel, C.D., C.E. Bash, and C. Belady. 2001. Computational fluid dynamics modeling of high compute density data centers to assure system inlet air specifications. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2001), Kauai, Hawaii, July 8-13.

Rasmussen, N. 2005a. Cooling strategies for ultra-high density racks and blade servers, White Paper #46. American Power Conversion. http://www.apcc.com/prod_docs/results.cfm?class=wp&allpapers=1.

Rasmussen, N. 2005b. Guidelines for specification of data center power density, White Paper #120. American Power Conversion. http://www.apcc.com/prod_docs/results.cfm?class=wp&allpapers=1.

Sharma, R.K, C.E. Bash, and C.D. Patel. 2002. Dimensionless parameters for evaluation of thermal design and performance of large-scale data centers. 8th ASME/AIAA Joint Thermophysics and Heat Transfer Conference, St. Louis, Missouri. June 24-26.

Shrivastava, S.K., J.W. VanGilder, and B.G. Sammakia. 2006a. A statistical prediction of cold aisle end airflow boundary conditions. Intersociety Conference on Ther mal and Thermomechanical Phenomena in Electronic Systems (ITHERM 2006), San Diego, California, May 30-June 2.

Shrivastava, S.K., M. Iyengar, B.G. Sammakia, R.R. Schmidt, and J.W. VanGilder. 2006b. Experimental-numerical comparison for a high-density data center: Hot spot heat fluxes in excess of 500 w/[ft.sup.2]. Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITHERM 2006), San Diego, California, May 30-June 2.

VanGilder, J.W. 2005. Partial cold-aisle isolation for raised-floor data centers. International Microelectronics and Packaging Society (IMAPS) Advanced Technology Workshop on Thermal Management for High Performance Computing and Telecom/Wireless Applications 2005, Palo Alto, California, October 23-26.

VanGilder, J.W., and R.R. Schmidt. 2005. Airflow uniformity through perforated tiles in a raised-floor data center. Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2005), San Francisco, California, July 17-22.

White, F.M. 1986. Fluid Mechanics, 2d ed., pp. 441-42. New York: McGraw-Hill.

DISCUSSION

Hamid Rahai, Professor, CSULB/CEERS, Long Beach, CA: (1) Needs validation with experimental results. (2) What kind of CFD package was used for the CFD calculations?

James W. VanGilder: Direct experimental validation of the real-time tool was not performed. Instead, CFD predictions were used as benchmark solutions for the development of the tool. The CFD package and type of models used were compared to experimental data in several other studies, which are referenced in the paper. The CFD software used is a commercially available tool targeted specifically at building airflow applications.

Kishor Kmankari, Lead Consulting Engineer, Fluent, Inc.: This reduced order model is a CFD model with very coarse grid with approximate boundary conditions.

VanGilder: CFD was used "off line" to calculate and store fundamental airflow patterns associated with individual racks drawing air at a specified flow rate subject to a specified perforated-tile flow rate. These runs were conducted with uniform 6 x 6 x 6 in. grid cells--not very refined by CFD standards, but probably more refined than one would consider "coarse grid." Once all of this data was reduced to curve-fits describing the airflow patterns, the complete airflow pattern for any problem at hand was calculated in real time by superposition.

James W. VanGilder, PE

Associate Member ASHRAE

Saurabh K. Shrivastava

Student Member ASHRAE

James W. VanGilder is staff mechanical engineer at American Power Conversion Corporation, Billerica, MA. Saurabh K. Shrivastava is a PhD student in the Department of Mechanical Engineering, State University of New York at Binghamton, Binghamton, NY, and a CFD intern at American Power Conversion Corporation.

Table 1. [C.sub.recirc] Settings Based on Airflow Direction Upwind Value of [C.sub.recirc] Airflow Airflow [greater than or equal to] 0 Airflow < 0 [Q.sub.t] 0 0 QA[x.sub.i] C[A.sub.i] C[A.sub.i+1] QB[x.sub.i] C[B.sub.i] C[B.sub.i+1] Q[z.sub.i] C[A.sub.i] C[B.sub.i] QAto[p.sub.i] C[A.sub.i] 1 QBto[p.sub.i] C[B.sub.i] 1

Printer friendly Cite/link Email Feedback | |

Author: | VanGilder, James W.; Shrivastava, Saurabh K. |
---|---|

Publication: | ASHRAE Transactions |

Geographic Code: | 1USA |

Date: | Jul 1, 2006 |

Words: | 7612 |

Previous Article: | Solar heat gain coefficients for high-mass glazing blocks. |

Next Article: | A new generation of zonal models. |

Topics: |