Data Center Maintenance Window Software: Who Is Checking That Both Feeds Are Not Down at Once?

Why change control in a data center is a different animal

In most organisations change management is about software releases, and the worst case is a rollback. In a critical facility the change is a person putting a screwdriver into live electrical infrastructure that is currently carrying customer load, and there is no rollback. The building is designed so that any single component can be taken out of service without dropping load, which is exactly the property that makes the risk invisible. Redundancy means nothing goes wrong when you do the first dangerous thing. It goes wrong when somebody else, in good faith, does the second one at the same time.

That is the actual failure mode, and it is boringly consistent. One team has an uninterruptible power supply on the A path in bypass for battery replacement. Another team, three days later in a different email thread, gets approval to service the static transfer switch on the B path. Both activities were reviewed. Both were approved by people who knew what they were doing. Neither reviewer knew about the other, because the review happened in email against a Word document and there was no place where the building's current redundancy state was written down.

The other half of the problem is the method of procedure itself. It is a Word file, written by whoever is available, reviewed by whoever replies, and executed on site by a vendor technician who may or may not be reading the version that was approved. Version control is a filename. The step by step execution record, if it exists, is a signed printout that gets scanned and filed somewhere nobody will look until an incident review.

The scenario every critical facilities manager recognises

A colocation building, concurrently maintainable design. The generator vendor is on site for annual load bank testing, which requires the automatic transfer switch to be exercised. Approved two weeks ago. Meanwhile a computer room air handler on the same mechanical loop has been offline since Monday for a coil leak, which was an unplanned event that nobody reflected back into the change record. A third activity, breaker maintenance on a remote power panel, was approved this morning by a manager who read the method of procedure carefully and had no way to know either of the first two things.

Nothing in that sequence involves incompetence. Every individual decision was defensible. What is missing is a single authoritative answer to the question of what redundancy is currently degraded in this building, right now, including the unplanned degradations. Without that, every approval is made with partial information, and the operator's protection is that most of the time the second dangerous thing does not happen on the same day.

Then there is the customer side. Colocation contracts carry notification obligations with specific notice periods for planned maintenance affecting customer environments. Those notices are drafted by hand from the method of procedure, sent by email, and tracked in a spreadsheet. When a customer says they were not notified, the operator has to prove they were, against a contract, from an inbox.

What ServiceNow Change Management and Nlyte actually fail at

Both are real and both are widely deployed in this space. ServiceNow Change Management is a mature change process engine with approval routing, a change advisory board model and a full audit trail. Nlyte is a serious data center infrastructure management product with strong asset, capacity and power chain modelling.

ServiceNow's limitation here is that it does not know your building. It will route an approval flawlessly and it has no opinion about whether the change conflicts with another change, because conflict in its world means overlapping configuration items, not overlapping electrical redundancy. Two changes on physically paired but logically unrelated assets look independent. Expressing the rule that no two activities may concurrently degrade both sides of a redundant pair requires the topology, and the topology is not in the change system.

Nlyte and its peers do hold the power chain, which is closer. What they generally do not hold is the live state of that chain including unplanned degradations, or the workflow that gates a person walking on site with a torque wrench. Infrastructure management products are built around capacity planning and asset lifecycle, which is a design and planning job. A maintenance window is an operations job on a different clock.

The deeper issue is that the conflict rules are specific to the building. Whether two activities can safely run together depends on the actual topology of that facility, its transfer schemes, its mechanical loops, its fuel and water dependencies, and the operator's own risk appetite. No product ships with your building in it, and configuring a generic product to know your building is most of the work of building something anyway, with less control over the outcome.

What a custom maintenance window build has to include

The foundation is a topology model with live state. Represent the power chain and the mechanical systems as a graph: utility feeds, generators, transfer switches, uninterruptible power supplies, distribution, and on the mechanical side chillers, pumps, loops and air handlers. Every node has a state of normal, degraded, in maintenance or failed. Unplanned failures update the same model as planned work, because a coil leak and a scheduled outage remove the same redundancy.

On top of that sits conflict detection. When an activity is proposed, the system computes what the topology looks like with that activity in effect plus everything else already approved or in progress in the same window. If the result leaves any load path without redundancy, the request is flagged with the specific conflicting activity named. This is the feature that justifies the project. It has to run at proposal time, at approval time and again immediately before execution, because the picture changes.

The method of procedure becomes a controlled document with a real lifecycle: authored from a template library, versioned, reviewed by named roles in sequence, approved with a risk level, and executed step by step on a device with each step timestamped and initialled. If a step fails, the abort criteria and the escalation path are in the document, and the abort is recorded rather than discussed. When the same procedure runs quarterly, it comes from the library rather than from someone copying last quarter's file and forgetting to update a breaker number.

Vendor and access handling belongs in the same record. Who is coming, which company, what escort is required, what areas they may enter, what tools and what test equipment. The security desk should be able to see that a technician's arrival is expected against an approved activity, and an activity that has not been approved should not produce a badge.

Customer notification should generate from the activity rather than being drafted separately. The system knows which customers are on the affected path, knows the notice period each contract requires, and produces the notice with an auditable send and acknowledgement record. This is one of the highest value pieces for a colocation operator and it is usually the least well handled today.

Finally, integrate with your electrical power monitoring and building management systems for read only state. Automatic detection that a feed is actually de energised, and that it came back, closes the loop between what the paperwork says and what the building is doing. Operators who have this stop relying on somebody remembering to update a status.

What it costs and how long it takes

In Digital Heroes delivery experience, a first release runs $70,000 to $140,000 and ships in 12 to 16 weeks. That covers the topology model for one building with live state, the method of procedure library with staged review and mobile step execution, and conflict detection across concurrent activities. It is used on the next real maintenance window, which is the only meaningful acceptance test.

A full platform at $170,000 to $380,000 phased over 6 to 12 months adds multi site support, vendor and access workflow, contract driven customer notification, power monitoring and building management integration, risk level policy with escalating approval requirements, and post activity reporting for internal audit or customer review.

What raises the cost in this category: the number of buildings, because each topology is modelled separately and no two are identical even in the same portfolio. Depth of the topology model, since modelling to breaker level is far more work than modelling to distribution level and you should decide deliberately how far down you go. Building management and power monitoring integration, which varies enormously by vendor and vintage. And customer notification, if contract terms differ per customer and have to be read out of agreements rather than configured once.

What holds it down: one building, power chain only, mechanical in phase two. Power is where the load lives and where the conflict rules bite hardest.

When you should not build this

If you run a single computer room with one uninterruptible power supply, no concurrent maintainability and no customer notice obligations, do not build. A calendar, a checklist and a competent manager are proportionate to the risk.

Build when your design is concurrently maintainable, because that is precisely the design in which two approved activities can quietly remove the same protection. Build when you have more than one team or vendor working in the facility in any given week. Build when you carry contractual notice obligations to customers. And build immediately if you have ever had a near miss where two approvals turned out to overlap, since that near miss is the system telling you what it is going to do next.

How to choose a developer for critical facilities software

Ask them to model your power chain on a whiteboard from a single line diagram you hand them. A developer who can do this asks about transfer schemes, which loads are dual corded, and what the mechanical dependencies of the electrical equipment are. A developer who draws a list of assets with a status column is going to build a ticketing system with your building's name on it.

Ask specifically how conflict detection handles an unplanned failure. If unplanned degradation does not enter the same state model as planned work, the conflict engine will approve activities against a picture of the building that is out of date, which is worse than no engine at all because people will trust it.

Ask what they have integrated on the facilities side. Building management and electrical power monitoring systems speak protocols that are not web APIs, and someone on the team needs to have worked with them. Ask for the specific vendor and protocol, not a general claim about integrations.

Ask who owns the code and settle it before kickoff. You should hold the repository, the infrastructure accounts and the right to hire another firm. At Digital Heroes the client owns the code from the first commit. The execution records this system holds are what you will hand to a customer or an insurer after an incident, and there is no acceptable scenario where a vendor relationship stands between you and them.