Frac Stage Data Software Problems: The 5 That Cost Real Money, and How to Avoid Them

Why does the stage data project keep growing into a real time platform?

Almost every completions data build starts with a clear, correct scope: take the van files, normalize them, segment them into stages, and produce designed versus pumped across the program. Then someone in the room asks whether the engineers could also see the job live from the office. The answer is yes, of course, and the project changes shape that afternoon.

The reason this is so common in completions specifically is that batch file processing and live streaming look adjacent and are not. Batch ingestion is a file arriving after the job, on a network you control, where a failure means you reprocess. A real time feed from the van is a connectivity problem on a pad that may have poor coverage, a buffering and reconnection problem, a partial data problem where the last five minutes are missing and the summary has to say so, and an operational support problem because somebody has to be on call when the feed dies at 2am mid stage.

The fix is a hard sequencing rule. The batch pipeline ships and is trusted before any live work starts, and the live feed is written as a separate phase with its own budget, its own support model and its own definition of done. Operators who do this get a working designed versus pumped report in the first release. Operators who merge the two phases usually get neither, because the live work absorbs the schedule and the engineers are still in Excel eighteen weeks later.

What goes wrong when you load ten years of archived pads?

Historical backfill is where the timeline actually breaks, and it breaks for a reason that is specific to this industry. Your archive is not one dataset. It is a record of every pumping contractor you have used and every acquisition software version those contractors ran, going back years. Each combination is a new channel mapping and a new test set, and the contractors that produced your 2018 files may not exist under that name any more, so there is nobody to ask what PPA meant in their export.

Two failures follow. The first is scope: the team quotes ingestion for your current contractor, then discovers eleven historic signatures in the archive and the mapping work multiplies. The second is worse and quieter. A mapping that is approximately right for an old contractor produces summaries that look plausible, get loaded alongside your current data, and then contaminate every cross program comparison you run. A stage from 2019 that reports 88 percent of design because a channel was misread will drag a fluid system comparison in the wrong direction, and nobody will ever question it.

The fix is to treat backfill as a separate, explicitly funded workstream that runs after the current pipeline is proven, batch by contractor, with a labelled validation set for each signature before any batch is published. Start with your last two years, prove the mapping registry is mature, then go back. Historic pads should also carry a provenance marker so a doubtful comparison can be filtered to verified data in one click.

Why do the integrations that matter break after launch?

Three joins carry the value of a completions data build, and all three fail after go live rather than during it: field ticket and authority for expenditure data from accounting, offset well pressure gauges from a service company, and production results.

They break for a common reason: none of them is under the control of the team that built the pipeline. The accounting system gets upgraded and the export changes. The gauge vendor changes their file naming and the time alignment quietly shifts by an hour because a daylight saving assumption was baked in. The production accounting group re-allocates a well and the identifiers move. In every case the completions system keeps producing output, because there is no natural error, and the join simply stops matching.

The fix is a monitored contract on each integration rather than a script. Every feed needs a freshness check, a row count expectation and an identifier match rate that is measured on every run, with an alert when any of them drifts. Well and stage identifiers deserve particular attention: agree one canonical identity for a wellbore across accounting, completions and production, and store the alternates as aliases rather than hoping four systems spell an API number the same way. Time zone handling should be explicit on every timestamp, stored in a single reference frame, and converted only for display.

What happens when units, segmentation and disclosure are not covered?

These three gaps are where a project that looked finished becomes a liability. Units first, because it is the cheapest to prevent and the most damaging to miss. If a channel is stored without an explicit unit, a contractor sending metric data will silently produce a pressure record an order of magnitude off, and averaged across a program that error is invisible in every chart.

Segmentation is the second. Automatic stage detection tuned on conventional single well operations will get zipper and simul frac wrong, because one file covers interleaved treatment of two wells and the naive rules split it by rate and pressure behaviour alone. The consequence is stages attributed to the wrong wellbore, which corrupts both the well level summary and any comparison downstream.

Chemical disclosure is the third. Filing obligations exist under state rules and are commonly satisfied through FracFocus, and the volumes you report should derive from the same measured concentration and rate channels that produced your stage summaries. When disclosure is assembled separately from contractor paperwork weeks later, you have two versions of what went downhole and no way to reconcile them.

The fixes are concrete. Store a unit on every channel and convert only at read time. Validate segmentation against a labelled set of stages your own engineers have already reviewed, including zipper and simul examples, and re-run that validation on every build. Derive disclosure from the measured record with supplier composition sheets, and confirm the specific filing requirements with your regulatory group rather than with your developer.

Should you build custom or configure what you already own?

Be honest about this before you spend anything. Well Data Labs is built for exactly this problem. It takes messy van files, normalizes and segments them, and gives completions engineers stage level analytics. If your requirement is stage data processing and analysis and nothing else, configure that and stop. Corva is genuinely strong while a job is pumping and while a rig is turning to the right, and if your pain is live operations rather than archived analysis, that is where to look first.

Under roughly 200 stages a year with one contractor and one fleet, do not build anything. The engineering cost will exceed the engineer hours you recover, and the money belongs in the completion itself. The same applies if your real complaint is that reporting is slow rather than that the underlying data is unusable, which is a dashboard engagement of a couple of weeks, not a platform.

The build case appears at the joins. When the question you want answered needs your accounting data, your offset gauge data, your own design of record and your production results alongside stage data, you are already exporting from a vendor platform into a warehouse. At that point you are paying for a pipeline you do not own. The second trigger is a design of record problem: if your stage sheets are PDFs with handwritten changes, fix that first, because designed versus pumped requires a designed and you do not currently have one.

How do hidden costs get into the quote?

Five items account for most of the overrun in this category and all five are visible in advance if you ask.

• Contractor signature count. A quote written against your current pumping contractor is not a quote for your archive. Ask how many distinct acquisition system versions are in scope and what happens to the price when an unknown one appears.
• Historical backfill. Frequently assumed to ride along with the new build. It does not. It is a separate workstream with its own validation.
• Real time streaming. Priced as a feature, delivered as an architecture. Ask specifically about pad connectivity, buffering, gap reporting and who is on call.
• Fiber optic and microseismic data. Large enough to change your storage design. If it is in scope, it belongs in the architecture conversation, not in a later phase.
• Design of record structuring. If your stage sheets are not already structured data, converting them is a real task and it is often left out of both sides of the estimate.

The fix is a fixed price discovery. Pull three job folders from three different contractors, plus one stage sheet and one field ticket, and pay for a short scoping exercise against real files before anyone commits to a delivery number. The questions the team asks during that exercise tell you more than the proposal will.

What separates a build that works from one that fails here?

The successful builds share four habits. They store the raw file untouched forever and derive everything else, so a mapping error found in month nine is a reprocess rather than a loss. They never let a human type a summary number, because a typed number cannot be recomputed and becomes the thing everyone argues about. They quarantine unrecognised files rather than guessing, and they treat the quarantine queue as a normal part of operations with a named owner. And they measure segmentation and mapping accuracy against a labelled set on every build, so drift is caught by a test rather than by an engineer who happened to notice.

The failures share one habit: they trust output nobody has validated against something known to be true. A pipeline producing confident unchecked numbers is more dangerous than the spreadsheet it replaced.

One last point on ownership. Your stage archive is the training set for anything you build later, including the pressure signature classification that flags screenouts and poor breakdowns from your own engineers' historic calls. Own the repository, the cloud accounts and the raw data from the first commit, with the written right to hire anyone else to continue the work. At Digital Heroes that is the default, and in this category it matters more than in most.