Safely Transitioning the Data Center to a new Operating Environment

The Misunderstood Risk
Adding hot or cold aisle containment to an existing site offers the promise of energy savings, rescued capacity, and elimination of hot spots. Containment almost always offers strong return on investment with seemingly little downside risk. The problem is that “little downside” can be significantly misunderstood due to a common view of aisle containment as “put up some walls and turn off some CRACs.”

This misguided view of containment leads operators to plow ahead with airflow isolation projects, yet with no clear transition path to the new operating environment. These narrowly-defined containment projects result in money left on the table and huge risks associated with crippled post-install airflow.

To mitigate these risks and assure a successful outcome, aisle containment retrofits should be viewed more broadly as a Monitor, Contain, and Control project driven by a proper Commissioning Plan. This broader view of containment is based on three important factors:

• Data center airflow is dynamic, not static.
• Airflow must be managed continuously, not just once or even sporadically.
• Post-install, the cooling plant operates closer to its limits, which demands better monitoring.

Considering these factors, it’s easy to see that a containment project involves more than just putting up walls and turning off CRACs. It requires discussion and planning around monitoring, balancing airflow, controlling cooling and setting parameters and thresholds to which the site will be managed. All of these specifications and parameters are captured in a proper Containment Commissioning Plan.

Retrofit Trends
Increasing computing densities, increasing power costs, and demand for ever more capacity from existing infrastructure are key drivers of containment adoption. While these drivers apply to both new construction and containment retrofits, our focus here is on the latter.

The typical scenario in containment retrofits is cold aisle airflow management with roof panels and a raised floor site with perimeter cooling. Generally, and depending on the fire marshal, drop-away roof panels are more popular because they can be used under sprinklers with no special accommodations. This option is not available for vertical panels more often associated with hot aisle containment. Also in legacy sites, there are often overhead obstructions that make vertical panels untenable, which leaves the roof as the only practical option.

With this roof-based trend in retrofits, cold aisle containment essentially creates little rooms within the data center, each of which requires a balance of supply and demand airflow. These separate balanced airflows make the project more challenging than merely installing doors and panels. Furthermore, the constantly changing airflow supply and demand, combined with operating the cooling plant closer to minimum required capacity, forces a careful approach to the airflow equation. These factors make a suitable Containment Commissioning Plan critical to risk management and overall project success.

Herding Cats
A proper Retrofit Containment Commissioning Plan aligns stakeholders to cooling changes and project goals, and outlines various monitoring, balancing, and set-point adjustments steps. More simply, a proper Containment Commissioning Plan answers two key operating questions:

• What is our new cold aisle operating temperature?
• How will we know when we get there?

Persuading the Facilities, Operations and IT groups within the enterprise to agree on the new cold aisle temperature is typically the hardest part. Facilities advocates for ASHRAE’s upper limit recommendation of 80.6°F (27°C), Operations is more comfortable with cold aisle temperatures around the mid- to upper-70s and IT prefers even cooler targets, often in the low 70s.

The second question is simply answered by “monitoring,” though more often than not, legacy sites have little or no temperature monitoring at the rack level. Once stakeholders agree on an answer to the first question it becomes obvious there is a monitoring gap, so planning typically turns to closing that gap. At this point in the process, people working through this planning phase often experience an “ah-ha” moment, realizing that the project scope is broader than their narrow, initial view.

What’s in the Plan?
After answering the two key questions (above), the rest of the Commissioning Plan is fairly straightforward. Again, we assume the project is cold aisle containment with a roof. The Plan outline that follows may be modified for other containment topologies.

• Determine maximum cold aisle temperature threshold
• Establish a monitoring plan
• Conduct an initial airflow balancing
• Determine and set a schedule for turning off CRAC/CRAHs
• Determine and set a schedule for raising set points
• Rebalance and continue adjustments until limits are reached

Temperature threshold and monitoring
Temperature threshold is a standard that drives all subsequent commissioning activities, including monitoring, which assures thermal safety through the project. To highlight the importance of monitoring, understand that at the start of a project a site often does have cold aisles out of balance, with one feeding another. Without containment, this common overfed/starving situation goes unnoticed, but after installing containment, underfed aisles immediately starve.

Since most legacy sites lack rack-level monitoring, a decision must be taken to invest in a monitoring/DCIM system, such as an automated, wireless mesh network. Short of that, monitoring may be relegated to a form of a manual ‘Sneaker Net’ process with temperature strips and/or laser temperature guns. A manual approach can work and many projects use it. Now at this point, an operator could also consider under-floor or differential air pressure, but we find that most retrofit projects lack the appetite for this step.

Initial Balancing
The objective of initial balancing is to eliminate gross imbalances prior to the adjustment phase. The current cooling lineup is maintained as floor tiles are juggled to achieve a common inlet temperature. That initial temperature target is loosely set by sampling the room to determine the current average inlet temperature at the front of the rack. A stock of some solid 25% and 60% perforated floor tiles will be needed for this and subsequent phases. It is common for a site to have too many perforated tiles installed.

CRAC/CRAHs and Set Points
The target number of CRACs to be turned off is usually set during the analysis phase. The Plan should identify specific CRACs, as well as a sequence and schedule for turning them off. The schedule simply spaces out shutdowns so that the room has enough time to stabilize, which could require as little as a few hours to as much as a couple days. Importantly, monitoring and rebalancing occurs during the time between shutting down CRAC units.
New set point targets are a function of the CRAC controls. Most legacy sites are controlling by Return Air Temperature (RAT), and a typical scenario is RAT set to 75°F. Clearly, if the new cold aisle temperature target is 78°F then RAT must be elevated well into the 90’s. New high temperature alarm set points must be set as well. The process of raising the RAT mirrors the process of turning off CRAC units. A proper Containment Commissioning Plan includes a schedule for gradually raising set points, allowing adequate time for the environment to stabilize. The Commissioning Plan should include multiple cycles of adjustment, monitoring and rebalancing.

Adjustment Process/Dynamic Airflow
As new containment is installed, monitoring assures that aisle starvation does not result. After containment is installed, the shutting down of CRACs, the set point increases and the rebalancing steps are taken. The Commissioning Plan is followed until the new target cold aisle temperature threshold is reached and is stable. A likely outcome is that the threshold will be reached for some zones while other areas will remain cooler than the target. At this point, the containment commissioning project is complete, though monitoring should be sustained, on the expectation that airflow supply-demand balance will change over time, necessitating additional adjustments.

The Containment Commissioning Plan outlined here does not address automation of monitoring and control functions, which we have left as a manual process. Short of deploying a full-fledged automated monitor and control system, active fan tiles are an option for achieving partial automation without the cost and complexity of procuring and integrating a full-fledged system. Fan tiles can help with localized balancing, which is especially important in the cold aisle containment scenario typical of retrofits. Fan tiles perform a local monitor and control function by adjusting fan speed to increase or decrease airflow into the aisle as temperature rises or falls. The fan tile option should be considered during initial discussion and planning of monitoring.

Conclusion
A proper Commissioning Plan for new aisle containment assures a seamless transition to the new operating environment and a successful outcome from both the financial and operational risk management perspectives. Approaching a new aisle containment project with a holistic view that includes monitoring and controls will help align stakeholders and facilitate a Plan that adequately addresses thermal safety. Lastly, any new implementation of airflow containment creates a new cooling environment that requires ongoing monitoring and continuous adjustment. The implementation of a well-thought out Plan will rise to the challenges of refining such a changing environment.

Originally published on Thermal News.