Before disaster strikes, will your power monitor predict it or tell you what happened?
Power engineers love power monitors. Electrical forensics engineers love power monitors. Plant engineers love power monitors-all with good reason. When applied properly, power monitors can help your power quality program and troubleshooting efforts excel. But that's not what always happens. Too often, a power monitor is an expensive wall ornament, rather than the powerful tool it should be. People also expect these devices to be the remedy for all power problems. This same attitude prevails against UPSs, as evidenced in the "Letters to the Editor" column in the June '98 issue of EC&M. As a result, end-users don't solve their problems. This situation extends to monitoring in general, including monitors of battery systems, security systems, fire and safety systems, and the environment.
Where are we dropping the ball? Someone must watch the monitoring system and react to its outputs. Someone must interpret the data. But most of all, someone must do both in a timely, expert manner.
Before the days of so many nonlinear loads, deregulation, and infrastructure overloading, you could efficiently handle most power quality problems without being a specialist. Today, that's not the case. Many companies are successfully outsourcing for that expertise. Some use the services of power monitoring system manufacturers-some use companies of power quality specialists who do not manufacture. Monitoring systems and surveillance are expensive, and require planning and organization. The sidebar, on page 50, gives you tips on how to make the most of your monitoring efforts.
Before deciding whether to use continual monitoring, tally the costs of automation and compare them to the costs of periodic manual maintenance and surveillance. Don't forget: Periodic testing is less reliable (doesn't always get done) than continual, automated monitoring. If you look at the statistics on preventive maintenance, you'll find manual inspections lead to a large percentage of failures and industrial accidents. Thus, the trend in maintenance today is to automate tests you can automate, and save manual testing for those who have the qualifications to do it correctly and safely. Power monitoring falls into the category of "tests you can automate."
The trick then becomes knowing what to do with those automated tests, but this requires training. When you look at the costs of training inhouse staff versus outsourcing, don't forget to account for time; you don't become an expert overnight or after a little classroom training. Be sure you factor in the costs of downtime caused by system failures, power outages, battery failures, and general disasters.
Real-life examples. A company purchased three battery monitors, so it could predict battery failure in UPS applications. It made this purchase because of catastrophic battery failures, which seriously disrupted operations. It made sure its staff got the proper training in remote access and data interpretation techniques.
Three years later, El Nino caused a power failure, and the UPS failed to provide power to the computer operation. This cost the company nearly six figures. Why, with good monitors installed and a well-trained staff, did this happen? No one was downloading data from the monitors, and nobody was interpreting the results.
A forensic investigation revealed the failure stemmed from corrosion on battery intercell connections. The condition began several months before the failure. What was really interesting about this was the monitor had stored the data that showed this condition. However, no one was watching the monitor, so nobody found out about the condition.
Another company purchased a battery monitor for its UPS system. It outsourced remote monitoring and data interpretation. This allowed the staff to "nip" small problems "in the bud." In this case, the cost was less than that of the old manual monitoring method.
Two years into the life of this arrangement, a large and long power outage occurred. The battery system, sized to run the UPS for 20 min at full load, held up the UPS for 41 min. This allowed the company to perform an orderly shutdown.
An attorney will tell you every contract has three essential ingredients: offer, acceptance, and consideration. Leave out any one of these, and the contract does not work. So it is with power monitoring systems.
You must have all the physical, technical, and administrative pieces in place. That means you need the power monitoring equipment, analysis/expertise, and management/response. (See the Figure, on page 48, original article): Keep each leg of this triangle strong, and you will be monitoring your reliable power long after the lights have gone out on those who don't.
Sidebar: How Can You Make Sure Your Monitoring System Protects You?
Make sure your monitoring system protects your electrical system, rather than informing you of a disaster you already know about. Here are some guidelines to help you have a successful monitoring system.
1. Determine what plant or facility systems you want to monitor. Your list should always include utility power. If you have a generator, you should monitor that, the transfer switch, UPS, and batteries. Your system might also monitor HVAC, fire protection, security, and communications systems.
2. Determine what quantities you want to monitor within each system. Your list should always include power quality, consumption, disturbances, and outages in main power and standby/emergency power systems. For each quantity, determine what alarms you need. Determine which quantities you need data trending and predictive analysis for.
Finally, decide which security points you need to include. If your security system doesn't monitor or control access to electrical equipment, your facility is vulnerable to an extended shutdown. Your facility is also open to punitive and compensatory damages, should underage trespassers have easy access to this equipment.
3. Determine who is responsible for receiving and responding to alarms. If you outsource, you need to understand the monitoring company's response and contingency plans. If you confine your first line responses to inhouse staff, you will need communications and the ability to resolve problems 24 hours a day, seven days a week. You'll also need emergency spending authority, no matter which way you do this.
Many companies find a hybrid system works well. They have a third party take care of the monitoring and notifying. The third party has a list of contacts and phone numbers, and they go down the list until they reach someone (on the client side) who can take action on the problem.
One plant that had such an arrangement had a substation and switchgear fire at its main manufacturing building. The outsourced service contacted the utility, fire department, and plant engineer. Then it took care of contacting the plant manager, corporate engineering director, division vice president, key personnel in the maintenance department, two contractors, and two key suppliers. The plant had new equipment in place and energized the next day.
4. Decide if you will do your monitoring locally or remotely. Look at total cost, resources available each way, and analysis capabilities.
Editor's note: I visited a remote monitoring center last year, and saw how this works. This particular service is a spin-off of a power monitor manufacturer. It communicates with clients via T1 lines, and it can see events in real time-even from hundreds of miles away. Further, the company's experienced engineers can compare anomalies and cures at one plant with anomalies and potential cures at another. They "work with this stuff all day long." You can draw your own conclusions as to how effective they are.
5. Determine how many separate specialized monitoring systems you may need for all monitoring tasks. Look for a monitoring standard, which would allow all specialized monitoring systems to operate as one. As of this writing, there is not a monitoring standard that allows different monitoring system manufacturers to integrate their systems seamlessly with one another. Microsoft Windows monitoring applications come close. Because several systems can operate at the same time in a Windows environment, Windows is the most popular platform.
Take care, though-just because Windows is popular does not mean it is the best system. Meanwhile, the IEEE is working on Standard 1159-1995 "Monitoring Electric Power Quality" and PAR 1491 "Guide to Selection and Use of Battery Monitoring Equipment in Stationary Battery Applications."
6. Prepare a specification of monitoring and outsourcing tasks to conduct for your company. Do this before you contact sales engineers who deal with monitoring systems and services. This ensures consistency of bids and makes it easier to compare solutions. It also gives you the background to understand new technology or services and make design adjustments prudently.
7. After the bids are in, determine the costs of equipment, installation services, planning, project management, and data management over the next few years. Calculate costs for inhouse and outsource, no matter which way you intend to go. Doing so covers your bases when upper management asks if you explored all reasonable options, or if they ask you to justify costs.
8. Write procedures for the selected system for continuing operation. Without written procedures, the system will deteriorate into a "management by entropy" or "management by panic"situation. The procedures need not be elaborate. In fact, it's best if they are not. The procedures must address what the major steps and responsibilities are, and who will carry them out.
9. Measure progress and quality. You'll need to include metrics for determining if your system is working well-administratively as well as technically.
If you outsource your monitoring, is the company really staying on top of things? Does it provide expert advice that solves your problems, or give you solutions that remind you of the mechanic who eventually rebuilds your whole engine to solve a starting problem? Build the metrics and operational reviews into the procedures. Also build in a way to modify the procedures as needed.
10. Implement the plan. Don't allow casual deviations from the plan. Communicate, train, and solicit feedback. Change whatever is not working, so the system can continue to do its job.