Anatomy of a Meltdown

Sep 1, 2002 12:00 PM, By Bob Ross, Electro-Test, Inc.

A catastrophic failure in a switchboard can teach us important lessons.

A switchboard failure can cause immense destruction, loss of life, and extended loss of facility usage. As you conduct your predictive and preventive maintenance, this is one of the problems you should be most eager to prevent. Yet are you sure you’re doing all the right things as often as you should? Let’s look at a case history of such a failure, and see what lessons we can learn from it.

This switchboard failure occurred in a community infrastructure-related facility—one required for safe human habitation of a metropolitan area. When the Electro-Test team arrived on the scene, it found a badly damaged switchboard with a supply bus rating of 2,000A, a section bus rating of 2,000A, and a neutral bus rating of 1,000A. Its short-circuit rating was 50,000A. The system was 3-phase, 4-wire, 480/277V.

The first step in the investigation involved a physical inspection to determine where the short circuit originated. This inspection covered the switchboard, the MB1 main breaker, the bus, the metering compartment, and another bus nearby. The inspection further revealed corrosion within the failed breakers and their connections, as well as corrosion on nearby breakers.

The team then developed the fault tree to identify the possible causes of this type of failure. It constructed this table by developing scenarios that may have caused each kind of damage, and then testing each scenario to determine which was the most likely cause of failure.

Examination summary. The arcing damage occurred primarily in the MB1 breaker cubicle, vertical bus, and the metering compartment above the breaker. The examination revealed evidence of arcing (vaporized and missing metal) on the control wiring, on the load side vertical bus, on the line side vertical bus and cabinet metal, and on the MB1 finger clusters and stabs.

The feeder cables and horizontal bus pieces showed no evidence of arcing. Both the damaged and undamaged bus bar showed considerable corrosion and flaking—this was true of all the bus bar near this switchboard. Bolted-together bus bar surfaces showed no corrosion where they had no exposure to the air.

Chain of events. Based on information from the facility’s personnel, examination results, and conclusions derived from the fault tree table process, the team was able to develop the following likely scenario that set this facility up for switchboard failure.

The failed MB1 main breaker was normally in a “racked out” configuration. This meant it was in a de-energized state, with its line-side and load-side bus connection contacts (finger clusters) not connected to the bus.

Hydrogen sulfide was present in the air of the breaker’s electrical room. Over time, the hydrogen sulfide caused corrosion and flaking of the finger cluster surfaces. This gas also attacked the mating surfaces of the breaker cubicle bus connections (stabs)—so they too corroded and flaked.

About twenty days prior to the failure, electricians racked in the MB1 breaker. Because the contact surfaces of its finger clusters and stabs were irregular and corroded, they produced an electrical connection with higher-than-normal electrical resistance. Load current passing through these connections produced heating, which produced further oxidation and deterioration of the connection surfaces. That, in turn, exacerbated the heating problem. The heating may have been less intense on the outer phases (A Phase and C Phase) since they have more cooling due to the physical geometry. The heating would likely be most intense on the center phase (B phase).

The heat was enough to melt the B phase connection surfaces. The normal load current produced an arc that bridged across the small gap where the copper surfaces had melted. Since the load current continued to flow through the arc, it melted and vaporized more of the contact surface material, producing extreme heat and ionized (conductive) gases. Intense heat and ionized gases entered the upper compartment and produced a short circuit of the 480V (potential transformer) control wiring.

Melted wiring insulation may have allowed separate phases to make contact, or the melted fuse block may have similarly allowed separate phases to make contact. The control wiring short circuit was on the line side of the breaker and the line side of the control wiring fuses, and consequently didn’t cause the main breaker to open or the fuses to blow.

The tremendous heat of the control wiring short circuit arc and the load current arc caused flammable material in the immediate vicinity, such as wire insulation and plastic materials in the meters, to ignite and produce smoke. To make matters worse, the control wiring short circuit arc caused loss of correct sensing functions and caused the tie breaker to trip by ground fault.

The cloud of ionized (conductive) gas produced by the load current arc and the control wiring arc caused a short circuit on the load side bus at the MB1 main breaker. This short circuit current caused the MB1 main breaker to trip and extinguish the load current arc. This gas cloud also caused a short circuit on the line side bus at the main breaker. Magnetic forces caused the arc to move to the upper end of the vertical bus where the short circuit arc continued, vaporizing the bus and cabinet metal.

The 12kV/480V 65T transformer failed due to the continued short-circuit current. The magnetic forces produced by this current probably caused winding movement, which would have resulted in a turn-to-turn short circuit. The 480V arcing and failure of the transformer caused upstream fusing to blow, interrupting the fault current and de-energizing the circuit. The flammable material continued to burn until no more oxygen or fuel remained.

Prevention steps. Following this diagnosis of the failure, the team recommended that the facility take the following steps to prevent such devastation from occurring again:

Annually remove breakers, operate them, and clean their contacts.
If leaving breakers and other equipment in the “racked out” or open position, don’t energize them as though they are ready for service. Instead, remove them to clean and inspect their contacts—then put them in service.
Perform regular infrared inspection of the electrical equipment.
Confirm that protective device settings and sizes are appropriate for the existing equipment and plant configuration. This may require new studies of short circuit protection and breaker coordination
Establish a program that involves measuring the resistance of bolted connections. This would augment infrared scanning and help overcome some limitations of applying that technology. Those limitations include accessibility and the lag between a resistance increase and a thermal increase.

Failures such as this are all too common. Yet, nearly any firm that supplies third-party testing can name facilities where such failures are just waiting to happen. Saving money by deferring or eliminating testing/maintenance is usually very costly, and occasionally results in lost lives.

Ross is area manager for Electro-Test, Inc., in Lee’s Summit, Mo.