Electrical service technicians today must understand the communications problems that crop up in multilayered building control systems
The advent of electromechanical controls in the 1970s, analog controls in the 1980s, direct digital control (DDC) in the 1990s, and complete building control integration in the 2000s has progressively offered building owners and facility managers powerful new options for reducing energy costs and improving occupant comfort. However, as these multiple generations of pneumatic, electromechanical, digital, and integrated controls are layered on top of each other, communications problems become much more difficult to identify and troubleshoot.
Many HVAC service and control technicians are familiar with troubleshooting direct digital control (DDC) input and output (I/O) 4mA to 20mA and 0VDC to 10VDC or 1V to 5V analog signals. But as new systems become integrated over digital networks, many other problems arise between field controllers, between DDCs, from field controllers to DDCs, and from sensors to DDCs.
The reality is that many service technicians must now improve and broaden their skills to cover network and signal troubleshooting. This article outlines communications problems that crop up in today's multilayered building control systems and provides a high-level guide to identifying their source. By moving beyond electrical/mechanical solutions to consider communications, an in-house maintenance staff as well as third-party specialty contractors can greatly improve their troubleshooting efforts.
Following is a short list of typical problems that pop up in building control systems.
At the lowest level, a sensor or actuator may fail due to aging, stress, or inadvertent physical damage. Some setups fail as a result of communication cabling or downstream devices. These failures are typically signaled by a flashing LED.
Yellow or red alarms frequently flash on workstations in the management control room, indicating that a field controller is not receiving an acceptable value from a sensor.
Individual DDCs may generate alarms, if a certain panel or building is not communicating.
Similarly, the management level workstation may generate error messages if it becomes disconnected from one or many buildings.
In other cases, problems may be randomly called in by occupants — such as a room is too hot or too cold.
With all of these different alarm indicators and potential problems arising, where do you start in your attempt to rectify the situation?
These problems may stem from a wide range of sources. Starting at the lowest level, there may be a failure in the equipment, sensor, or actuator. There could be a problem with the cabling running from the sensor to the field controller. Something may have gone wrong in the control loop. For example, the process feedback may be going to the wrong panel, because it was never wired properly and commissioned on startup — or the field controller panel or DDC controller may have failed.
There could also be problems in the network communications path between the field controller and the Ethernet/IP backbone. Communications problems usually fall into one of three categories: cabling infrastructure, signal transmission, and networking protocols. Communications problems also occur in the Ethernet/IP backbone.
In addition to the three basic network problems just mentioned, backbones are also prone to configuration errors that usually fall into the information technology (IT) domain. Finally, the management level controller may experience configuration or other problems.
Modern building control systems are often so complex the first challenge a troubleshooter faces is determining specifically where the problem is located. If applicable, a good starting point is to log into the central management controller to get a bird's-eye view of the problem. Here are some examples of where to start your search for a culprit.
Controller checks — The controller might show you that communications are cut off from a section of the building, the whole building, or the complete campus. If it's isolated to a section of the building, the most likely source is the field controller panel, analog circuitry, sensor, or actuator. If the entire building is dark, then the network communications between the management level controller and the DDC, network routers, or switches should be the first suspects on your investigation list. On the other hand, if you can't see the entire campus, you should immediately suspect a problem on the Ethernet or IP backbone.
Ethernet checks — If it looks like a building-level or campus-level communications problem, take a look at the Ethernet network. A good first step is to verify operation by checking the network connectivity icons or Ethernet port link status LEDs. Next, use a network troubleshooting tool to quickly give you a broad overview of critical network performance parameters, including protocol alarms and errors, visibility of all switches and VLANs, routers, access points, and IP devices. The alarms, errors, and protocol information can help localize the problem down to a subnet, port, controller, or particular device.
Cabling checks — Once the problem is isolated, use a cable analyzer to check the cabling connecting the problematic devices. These tools will find common problems such as opens or shorts and will verify the cable's electrical performance. If the network is built on standard AWG cabling instead of twisted-pair cabling (UTP), then use a precision low-ohms digital multimeter to check for basic cable opens, shorts, and impedance problems.
Signal checks — If the cabling checks out OK, then take a look at the quality of the communication signals traveling on the network by using an oscilloscope to capture the waveforms. The scope will help you determine whether signals are getting through at all, and (if they are) whether they are attenuated or too distorted to be read clearly, in which case, something is disturbing the line (Figure on page 12).
Analog checks — If the problem is occurring in only one segment of the building, it's far more likely to have an analog component, because that's the most local part of the building control architecture.
Sensors communicate by converting their output signal to a 4mA to 20mA DC current, with 4mA representing the sensor's zero-level output and 20mA representing the sensor's full scale output. Other systems use 1VDC to 5VDC or 0VDC to 10VDC circuits in a similar fashion. Occasionally, the 4mA to 20mA signal is converted to a 2VDC to 10VDC signal by adding a 500-ohm resistor across the load.
The fastest troubleshooting route for these systems is to use a milliamp clamp meter to measure the current or voltage between the sensor/actuator and the field controller and a voltage-/current-sourcing tool to simulate control signals back into the controller. However, you'll need to know what the circuit should read, based on the desired control operations.
If the problem lies within the circuit, use a milliamp or voltage sourcing tool to further test the line. If the problem traces back to the sensor, use humidity, temperature, pressure, or other measurement devices to verify the accuracy of the sensor reading. Other possible problems include bad I/O cards, isolators, power supplies, and transmitters.
Issues that crop up sporadically are usually the most difficult to troubleshoot, because service technicians don't have the luxury to just stand around hoping to catch the error when it randomly occurs. The best way to identify these problems is to use a logging multimeter or oscilloscope. Simply connect the equipment to the circuit in question, and leave it there to monitor the situation. Once the problem surfaces, the logger will catch it and lead you to the faulty component. This approach can save you many hours of nonproductive work and reduce your frustration level.
Problems like these are sometimes caused by degradation of equipment, controls, or cabling due to vibration or environmental changes. Another possible cause is installation errors, most commonly made during tenant improvements. Such errors include running unshielded network cabling near a high-voltage power source or an inductive load, using incorrect cable type, or installing an excessively long cable run.
Power issues can also cause devices to intermittently trip or fail altogether. If there has been a building or system upgrade recently, the old power supply may be overloaded or unbalanced. There's also a possibility that the new controls may either require cleaner power than the old electrical distribution system can deliver, or new electronic equipment may be introducing harmonic disturbances into the power supply.
In some cases, retrofits will take some time to cause failures, making the cause less obvious. If you suspect power supply problems, use a power quality analyzer to measure and log voltage, current, dips, swells, interruptions, harmonics, inter-harmonics, flicker, power, energy, transients, frequency, unbalance, and inrush.
Shields is a product manager, Field Calibration, Fluke Corp., Everett, Wash.; Hammond is a product manager for Fluke Precision Measurement, Everett, Wash.; and Jourdan is an HVAC college professor and program director at Wenatchee Valley College in Omak, Wash. They can be reached at jim.shields@fluke.com, hilton.hammond@fluke.com, and gjourdan@wvc.edu.
Let's quickly review the four-level architecture found in most modern building control systems (click here to see Figure). Starting from the bottom, at the sensor/actuator level, present control systems commonly use fully electronic, 2-way, 3-way, and pressure independently characterized control valves (PICCVs). Variable-frequency drives (VFDs) are used as standard equipment on cooling towers, variable air volume (VAV) fans, pumps and chillers. Typical sensors include humidity and temperature transmitters, CO2 sensors for indoor air quality, power meters, branch circuit monitors, and energy meters.
Field level controllers talk to sensors and actuators, often using 4mA to 20mA or 0VDC to 10VDC analog signals. Field level controllers are typically connected together using a BACnet master/remote over local area networks (LAN) or a proprietary network. The BACnet MS/TP LAN may be linked through a BACtalk Integrator (BTI) to a BACnet/IP or BACnet/Ethernet wide area network (WAN) that links multiple DDCs to the management level controller.