Power systems prove to be critical element in data center design
Power is an essential component of any mission-critical facility, and data centers are no exception. Not only does the computer equipment need to be continuously powered to ensure an uninterrupted flow of data, but continuous cooling also must be provided to prevent thermal runaways. Computer room air-conditioner units, pumps, and chillers must be able to function through a power outage, with the ride-through matching the runtime of uninterruptible power supply (UPS) power for the IT equipment load. These needs increase the construction cost of a data center project because of rotating equipment on a supplemental backup system as well as onsite generators.
In addition to holding project costs down, electrical engineers must weigh the pros and cons of various electrical system designs and equipment options when it comes to data centers, considering the choices between open or closed transition systems, rotary or static UPSs, and AC or DC power systems to ensure the best fit for each project.
Open transition vs. closed transition systems
When designing mission-critical spaces, you should always weigh the benefits and drawbacks of open transition and closed transition operations of electrical systems.
An open transition system provides transfer of power between two separately derived sources, such as the transfer of power between onsite generators and an electric utility feeder. The open transition follows a “break-before-make” method of transfer between power sources, which creates a momentary outage that can last for a few seconds. To maintain power to critical equipment during this brief outage, the critical loads should be connected to onsite energy storage devices. Keep in mind, however, that the mechanical systems will shut down and begin a start sequence. For high-density data centers, this can be an issue unless continuous cooling — pumps and computer room air conditioners (CRACs) tied to a UPS that includes a chilled water storage system — is also provided.
Open transition systems are simpler in design and operation than closed transition systems; therefore, they are inherently more reliable. The open transition system allows for reduced fault current, which directly relates to the rating of electrical infrastructure. With higher fault current design requirements, the cost of the electrical infrastructure can increase significantly. A facility with high fault currents, for example, may consider an open transition design to minimize the fault duty requirements for the data center.
As a “make-before-break” system, the closed transition transfer is the opposite of open transition. In this situation, there is a sharing of power between two sources, and the closed transfer may occur in less than 100 milliseconds — or longer if soft load transfer is required. This fast transfer is advantageous, particularly because most loads will not experience a noticeable power loss. However, there can be the potential for large block loading and back electromotive force (EMF) between the sources of power being transferred. Using a soft load transfer extends the length of time with two power sources, allowing for gradual ramping and load sharing between alternate sources of power. With this method, the transfer will not include large block loads that may affect the performance of power sources or damage the equipment with voltage dips, spikes, and transients.
A closed transition transfer system is advantageous because it allows facilities to operate and maintain the infrastructure without wear on batteries, as it does not “bump” the UPS system (meaning it doesn't discharge because the UPS always stays within tolerance, which is typically -15% to +10%). In addition, CRAC units and associated mechanical systems continue to operate during maintenance or test mode without disruption. A major benefit of a closed transition system is to allow the facility to pre-transfer power to an alternate source for routine maintenance or if a foreseeable outage may occur on the utility grid. Closed transition is typically more costly than open transition, however, and has more components that may fail.
Rotary vs. static UPS systems
Rotary UPS systems work well because of their superior overload and fault clearing capabilities. Direct coupled to an engine generator or electrically coupled, the system typically uses a motor generator or inductive choke to provide conditioned power to the critical loads. However, certain facilities may find the use of rotary UPS systems (particularly those with flywheels) difficult because of their enormous space and weight requirements. Rotary systems have a long life expectancy, so the large units will not have to be replaced as often as the more compact static UPSs.
Used as an “insurance policy” for a mission-critical facility, UPSs provide continuous power during events such as transients, electromagnetic interference, and power failures.
The static UPS may provide conditioned power via double-conversion technology, where AC power is converted to DC power and then back to AC power. Or, by using a line-interactive UPS, power can be provided through an inline inductor (choke) with either a battery or flywheel to provide ride-through in the event of power loss or out of tolerance voltage. The disadvantage is that the line-interactive UPS cannot provide frequency regulation as compared to double-conversion UPS technology. The line-interactive UPSs, however, are typically more energy efficient compared to similarly rated double-conversion UPSs. Static UPS systems are used in most modern data centers and other facilities with equipment that requires continuous power. Typically cheaper than rotary units, static systems work best when smaller, modular systems are required for backup power.
When choosing a UPS, electrical engineers must also consider the system's storage device. Rotary UPSs may use flywheels or batteries, while static systems typically use batteries. Depending on the size of the UPS and de-rating of the UPS module, the flywheel can provide backup time anywhere from 12 seconds to several minutes. Battery-based systems should be used for projects that require an extended run time of conditioned power beyond the few minutes that a flywheel can provide. The battery can provide backup power to the critical load for hours, if necessary, but with a longer runtime for a battery reserve, the cost of the UPS is increased. Careful consideration should be given if the backup battery is used for an extended length of time, especially when used with high-density IT equipment. Without continuous cooling flow, the power delivered to the IT equipment via the UPS may cause thermal runaway and shut down the IT equipment.
Although they provide larger amounts of power in order to cover an extended outage, batteries have several disadvantages to consider. They typically require equal, if not more, maintenance than a flywheel system. Without this maintenance, batteries will fail prematurely. In addition, they typically require more space for installation. Another consideration is battery storage, disposal, and maintenance. Because large quantities of batteries are necessary for facilities with continuous power requirements, the disposal of spent acids could also be in the hundreds of gallons.
Flywheels, which can be designed either external or internal to the UPS unit, rely on kinetic energy to generate electricity — rotational energy is converted to electrical power. Flywheels are appropriate for use with a UPS unit because they provide sufficient run time to allow generators to start. However, they cannot provide cost-effective ride-through backup power at the same level as batteries. One consideration for flywheel usage is to ensure the ability of the unit to regulate power during discharge and recharge of the flywheel. The flywheel provides power to the UPS during an outage, but needs power to recharge itself when utility or generator power is available.
AC vs. DC power
In traditional data center design, AC power is provided at medium or low voltages, depending on the facility load from the electric utility. Multiple transformations to step down the service voltage to utilization voltage for the IT equipment are required. The service transformation, conditioning in the UPS, and transformation at the servers (AC to DC to AC) provide significant losses in the system.
An alternate configuration gaining in popularity is the use of DC power connected directly to the servers. The number of transformations are reduced by converting AC power to DC power, providing this power directly to the DC to DC converter in the server.
DC power does have its limitations, though. Consider the following: The economics of the distribution system for a large-scale facility may or may not work for a particular project. Because DC power is not as widely used in data centers as AC power, there can be a lack of qualified trade labor to install a DC power system. On the same token, parts for a complete infrastructure system at a large facility may not be readily available. Current codes and regulations for safe installation of a DC system may also adversely affect the design and construction of the overall project. Furthermore, the owner may not be willing to make these sacrifices for the installation of a DC power system.
Medium- or low-voltage distribution
Once the AC or DC decision has been made, engineers must select a medium-voltage or a low-voltage distribution system. This choice is typically based on the facility's projected load — a load greater than 4 to 5 megawatts tends to be best served by a medium-voltage system.
Medium-voltage systems typically have a lower mean-time-between-failures rate, but a higher mean-time-to-repair rate. The medium-voltage system is also slightly more costly to install than a low-voltage system. This cost difference is due to the increased insulation and protection required to operate safely because, with a higher voltage, the electrical equipment operates under duress. In addition, power distribution beyond 5 megawatts begins to increase costs. Some of the cost difference can be offset, however, because medium-voltage systems often provide better utility rates. One item to consider in this initial design decision is there is a larger pool of facility management resources that can maintain the operation of a low-voltage system.
The final decision
Electrical engineers have a myriad of choices when designing the infrastructure of a mission-critical data center.
An open transition system works best in a facility with higher fault current, because it can reduce electrical infrastructure costs. However, the system requires cooling tied to a UPS. Although more costly than an open transition system, a closed transition system allows for fast transfers and continuous operation.
In facilities with available space, bulky rotary UPS systems are the best choice because of their exceptional capabilities and extended lives. But when space and costs are concerns, static UPS systems can provide compact backup power. UPS systems with batteries work best in facilities where hours of conditioned power might be required. Flywheels are more reliable and easier to maintain than batteries, but they cannot provide large amounts of power to cover long outages.
AC power is the standard for data centers, but multiple transformations in stepping down the voltage can cause losses. Slowly gaining in popularity in data centers, DC power can alleviate this problem, but the system is unproven in large facilities.
The ultimate use of a data center — as well as equipment, installation, and maintenance costs — should be carefully examined in order to determine power needs and choose between an open or closed transition system, rotary or static UPS, and AC or DC power distribution systems.
Battish is principal in the Applied Technology Group at RTKL in Baltimore.