Industry Perspective: Who Owns Containment?

Who owns containment? No one. This is the problem.

Written by Cary Frame, President and founder of Polargy, a provider of hot- and cold-aisle containment solutions.

Hot- and cold-aisle containment is a data center best practice experiencing hyper-growth in adoption because of its large impact on energy efficiency and operating cost savings. Interestingly, there is still no clear ownership of containment within the enterprise, among industry trades or between manufacturers.

Polargy works on the leading edge of growth in data center containment by focusing on product innovation and enabling fast and precise implementation. We offer this perspective on containment ownership based on our observations over more than five years in the containment market.

In our experience, what drives ambiguity around containment ownership is that it exists along the boundaries of job scope for multiple traditional players within data center whitespace. It also represents a more customized solution set than much of the industry is accustomed to.

On the user side, containment physically touches data center server racks, which are the responsibility of IT or IT Ops management within the enterprise, but it significantly impacts air conditioning performance, which is typically under the purview of facilities management. In addition, some enterprises have corporate energy managers who want or need to participate in the discussion. On the supply side, no single manufacturer type has claimed the category and no trade (mechanical, electrical, etc.) has taken a lead role. Because no one has stepped into full ownership of containment, up to five separate groups inside and outside the data center currently get involved.

Within the enterprise, Polargy has seen retrofit projects managed by data center operations as often as by facilities. However, we rarely see IT responsible for driving decisions, and though we find energy managers at the table, they almost never drive a project, but rather consult on ROI. When it comes to commissioning containment, all three constituents have strong stakes in the upgraded operating environment.

As part of Polargy’s standard engagement process we request that all three groups participate in outcome targets and commissioning planning. The key question these groups must agree on is what the new cold aisle temperature will be. Typically, IT people seek cold aisle temperatures in the mid-60s, data center operations people tend to favor temperatures in the low-70s, and facilities people prefer to run near the ASHRAE limit of 80.7°F. Besides these three operational groups, trades and manufacturers also suffer containment ownership ambiguity.

As a lead containment contactor, Polargy routinely trains and subcontracts a variety of firms from other trades to install containment. Polargy turnkey solutions have been installed by low voltage, flooring, interior, mechanical, and electrical contractors. Scholes Electrical and Mechanical in New Jersey has both electrical and low voltage groups, and Polargy has done projects with both groups for the same client. No particular contractor type has emerged as the one best suited to initiate and own containment projects.

“At CRB, we’ve seen a growing number of owners procure containment from containment companies like Polargy, but also from rack makers like Chatsworth,” reports Daniel Bodenski, Director of Mission Critical Services at CRB. “Likewise, in our mission critical project work, we’ve seen a variety of subcontractors install containment, including electricians, flooring contractors, and again the containment vendors themselves. No single group appears to be claiming full ownership yet.”

To the contrary, we’ve even seen contractors avoiding containment opportunities. In Chicago, the engineering design house Environmental Systems Design attempted to bid a containment project to four low voltage contractors, but none of them responded because they didn’t know what they were getting into. In Phoenix, a prominent mechanical contractor walked away from containment opportunities because they felt containment wasn’t “in their wheelhouse.”

The case of mechanical contractors is particularly curious because three things should give the mechanical trade an advantage in containment:

  • They’re already responsible for air flow supply in the data center.
  • They’re already doing routine maintenance on CRAC units, so they have regular access to customers they could sell containment to.
  • They already have the mechanical skills necessary to install containment.

Yet, we haven’t even seen mechanical get traction with containment. Lack of clear ownership for containment means mechanical and other trades lose out on significant opportunities due to nothing more than lack of familiarity.

Among manufacturers, containment is dispersed among different types, from pure-play companies solely focused on containment to resellers and divisions within large diversified corporations. Containment is largely custom-designed to unique site conditions and varying rack sizes and layouts. This is why pure-play companies like Polargy with deep knowledge of requirements for both retrofit and new construction currently enjoy an advantage. As rack, low voltage and flooring manufacturers encroach on the containment market they are being forced to overcome the customization barrier.

Lack of clear and consistent ownership for containment among facilities management, trades, and manufacturers is clearly hamstringing containment-related decisions and implementations today. Containment is moving up the adoption curve, but the market will continue to see a variety of players at the table until market norms are established.

While these different constituent groups remain involved in containment to some degree, communication about business outcomes and implementation is paramount. Until there is better clarity vis-a-vis ownership of containment, decision authority will remain dispersed and responsibility will be shared, necessitating communication and coordination among more parties than necessary.

According to Rich Garrison, Senior Principal at Alfa Tech, “Because containment is intended to control airflow by separating hot and cold air, containment solutions are a fundamental part of the Mechanical solution. On the other hand, because it can be considered a wall or partition and often has an aesthetic component, the Architects claim ownership.  To further complicate it, containment solutions are often positioned as accessories to rack solutions and can be considered part of the IT infrastructure. Ownership ambiguity extends to the trades as well. We see General Contractors, Mechanical, Electrical and Low Voltage sub-contractors all doing containment installation.”

In the current environment, while ownership remains ambiguous, Polargy sees MEP consulting engineers as the ideal containment owner from a design perspective due to their responsibility for air flow controls and monitoring. On the implementation side, pure-play containment manufacturers like Polargy, who double as containment contractors, will remain best suited to manage installation. Their advantage comes from superior product knowledge and deep and varied experience as the “go-to guy”during the formative period of the containment industry.

We anticipate this will remain the situation in this market for the next 2-4 years while data center containment ownership gets sorted out. Clear ownership of containment will facilitate even faster adoption of cutting edge containment solutions and lead to even greater data center efficiency.

Originally published on Data Center Knowledge.

Industry Perspective: Commissioning Containment Retrofits

Safely Transitioning the Data Center to a new Operating Environment

The Misunderstood Risk
Adding hot or cold aisle containment to an existing site offers the promise of energy savings, rescued capacity, and elimination of hot spots. Containment almost always offers strong return on investment with seemingly little downside risk. The problem is that “little downside” can be significantly misunderstood due to a common view of aisle containment as “put up some walls and turn off some CRACs.”

For Electronics Protection piece

This misguided view of containment leads operators to plow ahead with airflow isolation projects, yet with no clear transition path to the new operating environment. These narrowly-defined containment projects result in money left on the table and huge risks associated with crippled post-install airflow.

To mitigate these risks and assure a successful outcome, aisle containment retrofits should be viewed more broadly as a Monitor, Contain, and Control project driven by a proper Commissioning Plan. This broader view of containment is based on three important factors:

  • Data center airflow is dynamic, not static.
  • Airflow must be managed continuously, not just once or even sporadically.
  • Post-install, the cooling plant operates closer to its limits, which demands better monitoring.

Considering these factors, it’s easy to see that a containment project involves more than just putting up walls and turning off CRACs. It requires discussion and planning around monitoring, balancing airflow, controlling cooling and setting parameters and thresholds to which the site will be managed. All of these specifications and parameters are captured in a proper Containment Commissioning Plan.

Retrofit Trends
Increasing computing densities, increasing power costs, and demand for ever more capacity from existing infrastructure are key drivers of containment adoption. While these drivers apply to both new construction and containment retrofits, our focus here is on the latter.

The typical scenario in containment retrofits is cold aisle airflow management with roof panels and a raised floor site with perimeter cooling. Generally, and depending on the fire marshal, drop-away roof panels are more popular because they can be used under sprinklers with no special accommodations. This option is not available for vertical panels more often associated with hot aisle containment. Also in legacy sites, there are often overhead obstructions that make vertical panels untenable, which leaves the roof as the only practical option.

With this roof-based trend in retrofits, cold aisle containment essentially creates little rooms within the data center, each of which requires a balance of supply and demand airflow. These separate balanced airflows make the project more challenging than merely installing doors and panels. Furthermore, the constantly changing airflow supply and demand, combined with operating the cooling plant closer to minimum required capacity, forces a careful approach to the airflow equation. These factors make a suitable Containment Commissioning Plan critical to risk management and overall project success.

Herding Cats
A proper Retrofit Containment Commissioning Plan aligns stakeholders to cooling changes and project goals, and outlines various monitoring, balancing, and set-point adjustments steps. More simply, a proper Containment Commissioning Plan answers two key operating questions:

  • What is our new cold aisle operating temperature?
  • How will we know when we get there?

Persuading the Facilities, Operations and IT groups within the enterprise to agree on the new cold aisle temperature is typically the hardest part. Facilities advocates for ASHRAE’s upper limit recommendation of 80.6°F (27°C), Operations is more comfortable with cold aisle temperatures around the mid- to upper-70s and IT prefers even cooler targets, often in the low 70s.

The second question is simply answered by “monitoring,” though more often than not, legacy sites have little or no temperature monitoring at the rack level. Once stakeholders agree on an answer to the first question it becomes obvious there is a monitoring gap, so planning typically turns to closing that gap. At this point in the process, people working through this planning phase often experience an “ah-ha” moment, realizing that the project scope is broader than their narrow, initial view.

What’s in the Plan?
After answering the two key questions (above), the rest of the Commissioning Plan is fairly straightforward. Again, we assume the project is cold aisle containment with a roof. The Plan outline that follows may be modified for other containment topologies.

  • Determine maximum cold aisle temperature threshold
  • Establish a monitoring plan
  • Conduct an initial airflow balancing
  • Determine and set a schedule for turning off CRAC/CRAHs
  • Determine and set a schedule for raising set points
  • Rebalance and continue adjustments until limits are reached

Temperature threshold and monitoring
Temperature threshold is a standard that drives all subsequent commissioning activities, including monitoring, which assures thermal safety through the project. To highlight the importance of monitoring, understand that at the start of a project a site often does have cold aisles out of balance, with one feeding another. Without containment, this common overfed/starving situation goes unnoticed, but after installing containment, underfed aisles immediately starve.

Since most legacy sites lack rack-level monitoring, a decision must be taken to invest in a monitoring/DCIM system, such as an automated, wireless mesh network. Short of that, monitoring may be relegated to a form of a manual ‘Sneaker Net’ process with temperature strips and/or laser temperature guns. A manual approach can work and many projects use it. Now at this point, an operator could also consider under-floor or differential air pressure, but we find that most retrofit projects lack the appetite for this step.

Initial Balancing
The objective of initial balancing is to eliminate gross imbalances prior to the adjustment phase. The current cooling lineup is maintained as floor tiles are juggled to achieve a common inlet temperature. That initial temperature target is loosely set by sampling the room to determine the current average inlet temperature at the front of the rack. A stock of some solid 25% and 60% perforated floor tiles will be needed for this and subsequent phases. It is common for a site to have too many perforated tiles installed.

CRAC/CRAHs and Set Points
The target number of CRACs to be turned off is usually set during the analysis phase. The Plan should identify specific CRACs, as well as a sequence and schedule for turning them off. The schedule simply spaces out shutdowns so that the room has enough time to stabilize, which could require as little as a few hours to as much as a couple days. Importantly, monitoring and rebalancing occurs during the time between shutting down CRAC units.
New set point targets are a function of the CRAC controls. Most legacy sites are controlling by Return Air Temperature (RAT), and a typical scenario is RAT set to 75°F. Clearly, if the new cold aisle temperature target is 78°F then RAT must be elevated well into the 90’s. New high temperature alarm set points must be set as well. The process of raising the RAT mirrors the process of turning off CRAC units. A proper Containment Commissioning Plan includes a schedule for gradually raising set points, allowing adequate time for the environment to stabilize. The Commissioning Plan should include multiple cycles of adjustment, monitoring and rebalancing.

Adjustment Process/Dynamic Airflow
As new containment is installed, monitoring assures that aisle starvation does not result. After containment is installed, the shutting down of CRACs, the set point increases and the rebalancing steps are taken. The Commissioning Plan is followed until the new target cold aisle temperature threshold is reached and is stable. A likely outcome is that the threshold will be reached for some zones while other areas will remain cooler than the target. At this point, the containment commissioning project is complete, though monitoring should be sustained, on the expectation that airflow supply-demand balance will change over time, necessitating additional adjustments.

The Containment Commissioning Plan outlined here does not address automation of monitoring and control functions, which we have left as a manual process. Short of deploying a full-fledged automated monitor and control system, active fan tiles are an option for achieving partial automation without the cost and complexity of procuring and integrating a full-fledged system. Fan tiles can help with localized balancing, which is especially important in the cold aisle containment scenario typical of retrofits. Fan tiles perform a local monitor and control function by adjusting fan speed to increase or decrease airflow into the aisle as temperature rises or falls. The fan tile option should be considered during initial discussion and planning of monitoring.

Conclusion
A proper Commissioning Plan for new aisle containment assures a seamless transition to the new operating environment and a successful outcome from both the financial and operational risk management perspectives. Approaching a new aisle containment project with a holistic view that includes monitoring and controls will help align stakeholders and facilitate a Plan that adequately addresses thermal safety. Lastly, any new implementation of airflow containment creates a new cooling environment that requires ongoing monitoring and continuous adjustment. The implementation of a well-thought out Plan will rise to the challenges of refining such a changing environment.

Originally published on Thermal News.