Commissioning Mission Critical Facilities

Facilities can prevent costly downtime by developing a commissioning plan during the design phase of a project

If you asked 10 different people to define commissioning, you would most likely get 10 different answers. Some may view it as a simple process that requires some documentation to verify that the specified pieces were delivered and properly installed. Others may define it as a detailed verification of equipment specification, proper installation, and integrated system testing. In reality, commissioning is a systematic process for ensuring, through documented verification, that all building systems perform according to the design intent and the owner’s operational needs. To understand why commissioning is defined in so many ways, you must understand the risk factors involved in the failure of an electrical or mechanical system.

In a critical facility, the loss of the cooling system will generate some angry phone calls to the building office and some loss of productivity from overheated employees. But even in non-critical facilities, the equipment in the server rooms must remain cool. High temperatures generated in small server rooms can inflict damage to the batteries that the UPS units use these batteries to supply power in the event of a loss of utility power. When temperatures rise above 77°F, the batteries may be damaged and lose their ability to support the servers when needed. In both cases, the monetary losses are relatively small and therefore the cost of detailed commissioning is hard to justify.

An operator of this type of facility may use commissioning to answer questions like “Did we get what we paid for?” and “Are the life safety systems functioning properly?” However, if the risk of loss is great, then those responsible for the operation of a mission critical facility can justify the need for a much more detailed commissioning effort. They can then verify that all the critical systems are installed correctly, function properly, the entire building is fault tolerant, and any interactions between critical systems that could cause a failure have been addressed.

To quantify the costs involved in an electrical or mechanical failure in different types of businesses refer to the Table below.

After establishing that the commissioning effort varies in accordance with the risk involved, it’s possible to set some guidelines for beginning the commissioning effort, who should be involved, and how detailed the testing should be.

The commissioning process should start in the design phase so the commissioning provider is able to review the drawings prior to ordering the equipment. Greater consideration can then be given to concurrent maintainability, ease of operation, and the reduction of operating costs in some cases. Due to the focus of the commissioning provider, these advantages are realized when reviewing the design drawings. The following topics should be investigated as part of the commissioning design review:

Commissioning facilitation.
Verify accessibility to equipment to be commissioned is generally adequate, including provision for things like test plugs, isolation valves, and dampers.
Commissioning specifications.
The specification should give the contractor detailed information to accurately estimate the cost of supporting commissioning and avoid change orders.
Control system and control strategies.
Offer alternate sequences of operation and more reliable or more efficient configurations.
Operations and maintenance (O&M).
Make suggestions to make the site more maintainable.
Functionality for occupants.
Make suggestions that would help the site engineers make timely decisions for the addition of features that make the system more user friendly.
O&M documentation.
Verify proper access to equipment.
Training.
Make sure that the specified training is of sufficient duration.
Component energy efficiency.
Recommend items like lighting control systems to reduce energy costs.
Owner’s design standards.
Verify that the owner has properly documented design standards.

The owner, equipment vendors, engineer of record, general contractor, facility operating engineers, and third-party commissioning provider should all be involved in the commissioning process.

Systems commissioning. The environment of a data center must be closely controlled. Data center equipment can be adversely affected by temperature and humidity extremes and, even more importantly, by the rate of change of temperature. Building owners and computer equipment manufacturers have established standards for temperature and humidity ranges, as well as for allowable rate of change (typically 3°F in 12 min). Therefore, commissioning a mission critical facility must include verification of these criteria. Rate of change can only be evaluated under load conditions.

The building management system (BMS), which typically controls the sequencing of all HVAC equipment, must be tested for correct operation of every control point and sequence in all possible operating conditions. Proper alarming and seamless transfer to redundant equipment must be verified.

A failure of a mechanical/HVAC system usually allows sufficient time for facility engineers to take the necessary corrective actions due to the relatively slow rate of change in environmental conditions. However, an interruption of electrical power greater than 8.83 msec to the critical load can result in the shutdown of, or damage to, sensitive computer loads. Consequently, mission critical facilities commissioning focuses heavily on the electrical power protection systems.

Most mission critical applications require high-quality uninterrupted power to the critical load(s) during all operating conditions. Typical utility reliability is about 99%, which is far too low for a mission critical facility. To increase the overall system reliability, additional power protection equipment is required. The components of a typical mission critical facility’s power protection system include a UPS, the engine generator system, automatic transfer switches, and distribution systems. Each of these systems must be fully commissioned as individual units, and again as a fully integrated critical power system.

Full-load systems testing must be performed on the entire facility to simulate the full load operating and failure mode conditions to which the facility will be subjected. This shouldn’t be confused with HVAC capacity testing.

Electrical power system commissioning for a mission critical facility requires specialized monitoring and measurement equipment. Since the flow of electrons is far more discrete than the flow of air or water, it’s necessary to use high-end electrical test instruments to ensure that major assemblies, and even subassemblies, of each piece of the critical power system are operating properly. Typically the instruments used by the equipment vendor are inadequate for the measuring and recording of high-speed electrical transients. The use of high-end instrumentation and metering equipment by an independent, third party commissioning provider gives an unbiased assurance that all systems are operating properly. Proper verification of equipment operation often involves duplicating many portions of the vendor’s equipment start-up/site acceptance testing activities, which results in additional site time on the project.

Considering the high cost of failure shown in the Table, it’s clear that the benefits of duplicating some vendor activities outweigh the additional cost of commissioning testing.

Testing. The commissioning agent, as part of the site acceptance testing phase of commissioning, must verify the major functions of every piece of critical equipment, such as the electrical, HVAC, and fuel oil. Commissioning of any facility includes acceptance testing in all normal modes of operation. A design intent that achieves five nines of reliability incorporates multiple levels of component and system level redundancy. Failure mode testing is a necessity when attempting to prove that this design intent is met.

Since a mission critical facility is often a 24/7 operation, it’s imperative that all possible operational scenarios are covered during the commissioning process. Once a 24/7 facility is operational, substantial risks are associated with failure mode testing. Every potential point of failure must be examined for all pieces of equipment and systems in the facility. Then a detailed test procedure must be written for each scenario to ensure that the design intent is truly met.

All problems identified during the site acceptance testing phase of commissioning must be resolved/repaired before the commencement of integrated testing. Integrated system testing of all failure modes must be performed while the facility is operating at the full electrical design load, which can be extremely costly. However, the quantifiable prevention of failure, as well as the peace of mind achieved by physical verification of all operating modes, justifies the additional costs involved. Testing at full design load shouldn’t be confused with HVAC capacity testing. It’s not possible to achieve “design day” conditions—worst case outside temperature and humidity at full electrical loading—during the commissioning process.

The desired environmental trends for the integrated testing should be reviewed with the BMS vendor prior to performing the test. This will verify that the BMS is capable of recording the necessary information, displaying it in charts, and providing an electronic copy of the results in a spreadsheet-compatible format for further analysis. Typically the BMS is capable of charting trends in space temperature and humidity. It can also chart supply and return water temperatures for chiller plants and supply and return air temperatures for air handling equipment. Data loggers should be used if the BMS can’t provide all of the desired information.

Integrated testing is a cumulative exercise to prove the reliability of the overall design and compatibility between all critical systems, including the electrical, mechanical, and environmental components. The intent of such testing is to simulate the real-life conditions that the facility could be subjected to on any given day during operations. The best method to verify total facility integration is to simulate a full building design load, remove the building utility source, and monitor all systems as they respond. Resistive load banks, which convert electrical energy into heat, are used to produce this load. Integrated testing is the only way to demonstrate that all critical systems function collectively.

Commissioning mission critical facilities can help ensure that building systems perform as designed. The entire team must meet during the design phase to evaluate the mechanical and electrical systems, test the full system loads, and monitor the equipment. While it may be costly to resolve issues during the commissioning process, it will prevent much bigger problems from arising down the road.

Guzzardo is a Managing Principal at EYP Mission Critical Facilities, Inc. in Middletown, N.Y.