A new breed of data center is turning power monitoring upside down. If you haven't already worked in an Internet data center, chances are you will soon.
Credit card processing, check clearing, e-commerce, point-of-sale inventory management -- all of these applications depend on data centers. Because of the economic value involved in these operations, loss of even one minute of uptime in a data center can cause multimillion dollar losses for its customers.
When companies like State Street, DST, Value Web, and AOL operate a data center, they plan against everything that might cause downtime. Their staffs not only actively participate in reliability-focused trade groups and conventions, but their attention to electrical infrastructure is also a model for total reliability. How do they do it? These companies install and maintain redundant switchgear, backup generators, 3-phase UPSs, and battery rooms. They distribute power via 480V (or higher) buses going to distribution transformers and panels and watch it all over a power monitoring network. Qualified experts review this information and are ready to handle whatever anomalies come up, before they turn into power interruptions.
But not all data centers fall into this category. Today, a special type of data center, an Internet Data Center (IDC), hosts Web sites, Internet Service Providers (ISPs), and other operations closely associated with the World Wide Web. However, the image of reliability and security of traditional data center fits very few IDCs.
Power chaos. Typically, an IDC has hundreds of customers. The customers simply rent space in which they can locate their own equipment (such as servers for e-commerce). The typical IDC is full of cages, small or large house-like enclosures, rows upon rows of rack cabinets, and other types of enclosures, plus hundreds or thousands of telephone lines and associated telecommunications equipment.
At the lower end of the scale are the small Web host/ISPs. While unable to land accounts with banks and other big customers, these operations handle sensitive data, such as credit card information involved in e-commerce, because they offer ultra-cheap Web hosting and bare bones e-commerce packages. It's easy to set up a facility for such a purpose, so barriers to entry are low.
Such a facility is not hardened, has no control room, and has no generators. Power distribution often consists of surge strips daisy-chained across the floor. Backup power and peak load handling usually consist of several point-of-use 120V UPSs. Although this type of operation is growing in market share, each is a catastrophe waiting to happen.
A larger IDC is often a hardened facility that contains elaborate power distribution schemes, redundant UPS, and emergency generators. Like traditional data centers, it may contain a large control room or network control center called the nerve center. One key difference between an expert at a traditional data center and one at a larger IDC is their nerves are usually frazzled. You'll find "ad hoc planning" routine, and more so as the facility grows its customer base.
Lack of planning. When laying out a new building for an IDC, an engineer sizes the UPS systems, power distribution panels, and backup generators for planned power consumption. The engineer factors in harmonics of nonlinear computer loads and maybe some future expansion. Then, a contractor builds the IDC. This whole process can take as little as a month from beginning to end. Usually, the IDC rents out all available space before the builders complete their work.
A few months later, the IDC is fully operational and totally out of control. And it just gets worse. How do you manage 400 customers who simply rent space and consume power? It's next to impossible, especially when power distribution may include hundreds of surge strips. The host cannot manage the load unless the host can measure the load. That requires power monitoring with complex capabilities. Power monitors miss nothing. They collect everything, alarm when conditions pass a threshold, and archive into an ever-growing database.
You must analyze and interpret that information, and look for trends that can help you predict failures. Top-level power monitors have enormous capabilities, but no matter how sophisticated the device is, it still takes the expert to make the daily and strategic decisions on how to react to what the monitor says.
Who needs to know? Traditionally, the person who responds to the power monitor's information is responsible for the control of the data center and its environment. While this has not changed, there are many others who need to know the quality and quantity of power available to them.
In a typical IDC, there may be hundreds of clients responsible for their individual power. Maintenance alarms and other predictive information are available to these people, but they often don't know what to do with it. To compound the problem, many individuals responsible for their cage in the IDC don't care about their neighbors, some of whom may be competitors!
Do these people need trends and reports, or just alarms? First, any data center's facilities group needs to know all of the things regarding power quality, infrastructure items, events, and consumption information. This allows for proper planning to handle growth of the client operations. It also allows them to predict when they need to add new capacity or correct problems using trended data. Most of these activities require expert analysis best left to outsourced experts who know how to interpret monitored data and trends.
Second, but perhaps more importantly, the clients need to know what their capacity is, where it's going, and when it's time to expand their requirements from the host. For example, if a hosted client is starting to overload a circuit or the trend is developing, they need a recommendation for correction. If you install or upgrade power monitoring in one of these facilities, find a power monitoring expert to help you, by referral or collaboration.
No matter what kind of facility you work with, many of these concepts apply. But as you do electrical work in data centers, make sure you keep this new breed (and its particular problems) in mind.
Cotton is President of Data Power Monitoring Corp., Rocklin, Calif.
Sidebar: Enlisting An Expert
Think of your power monitoring expert as your electrical accountant. You need this person to audit your procedures, review your "books," and generally make sure you are on the right track. You basically have two ways to find such a person, at least for now. You can contact the power monitor manufacturer for a list of people who have been through their training, or you can ask around for personal recommendations. The best places to ask are conferences and seminars whose audience includes the data center crowd.
Specifically, a power-monitoring expert will look for:
• Gross problems (which you should be able to spot on your own)
• Trends or patterns that disagree with those at similar facilities (for example, a small but routine voltage dip at 9:00 each day)
• Excessive numbers of "pre-alarm" or "near threshold" conditions
• Improper thresholds
• How power variations relate to specific operations in the facility
• Inadequate response procedures
• Efficacy of alarm annunciators, paging devices, and remote controls
• Problems in the communication network
• The need for extra capacity, harmonics correction, power factor correction, or improvements in load management.
• System changes that require reconfiguration of the monitor, reassignment of monitoring points, or changes in response procedure.