They’re the engines fueling the phenomenal growth of the Internet economy and the foundation upon which information is fast becoming the new utility of the 21st century. Today’s data centers are highly secure, reliable, and expensive information fortresses built to move information with maximum efficiency—and keep it moving no matter what.
Running the gamut from Internet hotels that host computer services for Internet service providers to telecom carrier hubs, co-location centers, server farms, and private enterprise data centers, these world-class facilities are custom-designed with raised floors, sophisticated environmental control systems, seismically braced racks, and redundant power systems to ensure failsafe site performance 24/7.
In a world where minutes of downtime can translate to million dollar losses, when it comes to delivering power, reliability and capacity are leading priorities. As a result, data centers are among the most power-hungry of today’s critical power environments, consuming many times the energy required to power commercial office space. This is especially true of the massive data servers that provide the infrastructure for Web and Internet hosting services, leasing rack space for some of the world’s most powerful computer servers. Depending on the nature of the IT equipment and its packing density, power demand in these raised floor environments can range from 100W to a stunning 300W per sq. ft., with an average load factor of 200W/sq ft—a lot of concentrated power. In this environment, the appetite for power is rivaled only by an insatiable need for reliability.
According to Henry Hansen, Jr., senior corporate engineer with Shooshanian Engineering of Boston, the demand for capacity shows no signs of diminishing as high-powered servers and storage devices become even more powerful and memory-intensive. “Just from 1998 to now, the number of disk drives in the same physical size data storage unit has increased from 128 to 384,” he says. “Today’s storage devices are equipped with exponentially more memory and storage space, drawing many times the electrical load. Because they’re smaller, where you once had four to seven servers per rack, you’re now supporting 21 servers per rack. Even though the new devices are more energy-efficient, when you go from four servers per rack at 300W to 21 at 100W, more components in the same footprint equal increased power load density.”
As a result, the onus is on contractors, builders, and owners of these facilities to assure tenants and prospective tenants that their IT equipment, servers, and processes are safe, that the power won’t go out, the servers won’t go down, and data will keep moving. So they do whatever it takes to assure customers that the facility will support future load requirements, no matter how quickly the data center fills out, or how many servers the data hotel ultimately sleeps.
Many data centers have built their businesses based on the assurance of plentiful power capacity and ironclad reliability, incorporating guarantees of minimum levels of both into their contracts—at the expense of efficiency and cost savings. Of course, for every upside there is a downside, and in the data center world, the downside of promising and delivering power reliability in the “high nines”—99.9999% uptime—doesn’t come cheap.
Costly propositions—promising redundancy, reliability, and capacity. Capacity planning is one of the trickiest challenges design engineers and builders face. The data center must be supplied with a reliable and redundant source of power because IT components subjected to frequent power interruptions fail at a much higher rate. This explains why many data centers go to extremes to assure reliability, installing several layers of costly backup and redundancy with heavy investments in equipment like switchgear, switchboards, UPS battery backups, and parallel generators.
Assuring the reliability of the incoming power source means installing multiple sources of incoming power with dual utility feeds, usually from different substations or power utility grids that are interconnected via breakers to the facility’s main load. Dual-unit substations (transformers) are then stepped down to deliver power to the server cages and cabinets. So, if one power source or high-voltage line goes down, the system can switch to the redundant feed in a fraction of a cycle—a matter of milliseconds—without losing any electrical load. Likewise, if a transformer fails, the backup system picks up the load instantaneously.
As a result, the cost of energy can account for a large portion of a company’s cost of doing business. And one of the most significant costs of delivering redundant power to the electrical distribution system is the installation of oversized transformers rated to accommodate high-capacity loads, redundancy, and harmonic current. Transformers, which step down voltage from transmission voltages to distribution voltages that can be used by the data center’s electrical distribution system, are available in sizes ranging from pole mounted 10kVA units at the low end to 1,000MVA generator step-up transformers at the high end.
To support future expansion, most data centers install transformers rated between 1,000kVA and 3,750kVA, based on calculations that factor in the current load and the potential maximum load anticipated for five to seven years down the road. The downside of super-sizing is that when transformers carry light loads relative to their nameplate sizes, the considerable losses that result from energizing the magnetic core drive efficiency down.
Hansen says he has found considerable disparity between the data center equipment nameplate load and actual demand load on some of the transformers. “Some of the devices have measured continuous demand load values of 40% to 50% of nameplate rated load with short duration peaks of up to 80% or more for servers and storage units,” he says. Clearly, there are reasons to optimize load efficiency. Fig. 1 illustrates the advantages of correctly sizing transformers to optimize efficiency.
Compounding the issue, transformers are often sized 30% to 50% bigger than the anticipated load to compensate for the negative effects of harmonic loads—the electrical trash generated and fed back into the electrical system when AC power is converted to DC to run the servers. So the high purchase and installation costs are further inflated by the lifecycle costs of running at lower efficiency levels, wasted electricity, and load losses that occur when power passes through the transformer. Clearly, there is a need for a better way to balance the cost of serving the load and losses from transforming energy.
Rightsizing transformers for more efficient and cost-effective power delivery. All of this begs the larger question, Can you assure capacity and reliability while managing losses and controlling costs? The answer is yes. And the key is optimizing the costs of energy, reliability, maintenance, and installation. To achieve a better total cost of ownership, many experts advocate sizing transformers to step-down voltages as close to load as possible by optimizing rather than oversizing the transformer.
Smaller, optimized transformers compensate for harmonics and minimize the losses incurred by energizing the transformers (core losses). So they can be sized more efficiently and can be implemented as part of a modular data center design that assures scalability, enabling highly complex systems to be built from smaller, more manageable building blocks.
Marcus Hassen, an electrical engineer with Clark, Richardson & Biskup Consulting Engineers, Inc. (CRB), a Kansas City, Mo.-based consulting engineering firm confirms this design approach.
“There is definitely a tendency on the part of owners to want to over-design power distribution solutions,” Hassen says. “While reliability is still the number one consideration, where we formerly saw specs with transformers rated at K20 to compensate for harmonics, we’re seeing more specs using different solutions to optimize the transformer and reduce harmonic current. Likewise, we’re designing scalable power distribution plans that let us add transformers in the future to support higher power densities.”
Clearly, in the transformer equation, optimization equals efficiency, and the most efficient way to deliver energy to critical power facilities is to buy what you need today and add capacity incrementally as the data center fills out. This extensible approach contradicts the conventional method of determining power capacity based on watts per square foot. However, it makes more economic sense on several counts. First, the initial costs of purchasing and installing smaller transformers are lower than that of larger ones. Second, lowering the initial cost enables you to recoup the time value of money. Third, you can eventually buy the capacity you need by purchasing a second transformer when you need it—four, five, or six years down the road. So instead of running a large transformer at 20% or 30% capacity or less, running a right-sized transformer at between 50% and 75% capacity can result in dramatic savings (Fig. 2).
So optimized transformers and modular designs save money, but what about capacity and reliability? When you shift the focus to efficiency and cost savings, are you not reversing the previous equation at the expense of reliability? Not if you specify transformers optimized to operate at low temperatures rising to prevent overheating, to support higher loads, to carry and dissipate harmonic current, and to manage the magnetic core current, minimizing the losses that contribute most to low efficiency for lower loads. Because optimized transformers are sized more appropriately for the loads they carry, incur smaller core losses, and use less energy, they perform more efficiently than larger transformers. As for redundancy, smaller transformers can be used in redundant configurations to deliver the reliability that critical power applications require just as easily as larger transformers.
Hansen says factoring in low power load support spaces and the diversity of demand between building and mechanical equipment power loads lets you lower the service system’s design capacity from a total based on “connected” plus growth and safety factors to a demand total that includes reasonable growth factors. “Combine this demand load philosophy with an algorithm for calculating power consumption for the mechanical systems, lights, and miscellaneous loads and a scalable, modular design becomes possible,” he says. “The result is lower overall design power load calculations and manageable service transformer sizes that allow for growth and reliability. You also must designate future space to accommodate additional transformers and equipment for growth, which enables selection of smaller equipment ratings, by eliminating the need to oversize now for potential load growth.”
Know what you’re using...and what you’re losing. To remain competitive in the data center business, the current trend is to do and pay whatever it takes to promise and deliver reliability. However, as the data center industry matures, transforming a competitive foothold into a competitive edge will mean delivering reliability in the most cost-effective manner. Today, competitive advantage goes to those who can deliver reliability. Tomorrow, the next level of competition will reward those data centers that excel at providing reliability at the lowest possible cost. This trend will take the concept of rightsizing transformers a step further, maximizing both reliability and efficiency by incorporating a power monitoring solution that enables intelligent monitoring and management of power consumption down to the rack level.
With an intelligent, integrated power solution, usage and costs can be tracked and measured throughout the data center by tenant, simplifying the process of identifying losses and their causes. Access to real-time performance data allows data center owners and managers to balance all of the components of total cost of ownership, making whatever modifications are required to return efficiency to peak levels whenever necessary.
In the end, balancing reliability and redundancy with cost savings comes down to numbers. With the proper calculations, a modular approach to right-sizing transformers, and the ability to keep a finger on the electrical pulse of the power system adds up to a better bottom line.
Hellman is the marketing manager with Siemens Critical Power Team, in Andover, Mass. Arsenovic is the business development manager for Power Distribution and Control, in Gloucester, Va.