Geotechnical Data Management Software Problems: The 7 That Cost You Boreholes, and How to Avoid Them

Why does the historical archive swallow the whole project?

Every geotechnical data project starts with logging and ends up talking about the archive, because the archive is the emotive part. Twenty years of investigations across a region you keep working is genuinely the firm's most valuable asset, and once a technical director sees it framed that way the scope becomes extract everything, verify everything, index everything.

That scope does not survive contact with a scanned 2004 factual report. The extraction cost per report is roughly flat, so the total is a straight multiple of report count, and firms consistently underestimate their own count by a factor of two once someone opens the network drive properly. Meanwhile the part that pays for itself, which is knowing that boreholes exist at a location and roughly what they found, needs only location, depth, stratum descriptions and headline results.

The fix is to scope the archive by value rather than by volume, and to say so explicitly in the contract. Extract the corridors and cities where you bid most often, not the whole collection chronologically. Accept that the engineer will open the original report anyway, so the index only has to be good enough to find it. One regional consultancy we worked with found the payback came almost entirely from three urban areas.

Sequence the build the same way. A first release covering the project and borehole data model, tablet field logging with offline capture, laboratory ingestion profiles and automated log and section production runs $60,000 to $140,000 and ships in 12 to 16 weeks in our delivery experience. The archive, client specific deliverables, laboratory scheduling and parameter workflow belong to the phased $160,000 to $420,000 platform.

What goes wrong migrating twenty years of borehole records?

Three problems recur and none of them are technical difficulty in the usual sense.

The first is identity. Holes get renumbered between phases, redrilled and reused, and clients frequently impose their own numbering on a later phase. A borehole called BH04 in 2011 and BH12 in 2017 may be the same location. Any migration that treats the hole name as the identifier will produce duplicate locations at the same coordinates and a section that plots two different logs at one chainage. Location and coordinate must anchor identity, with names as attributes that carry a project and a date.

The second is retired terminology. Test method references change, descriptive standards change, and a stratum description written under a superseded convention does not mean quite the same thing as the current one. Do not silently normalise old descriptions into current terms. Keep the original text and record the standard it was written under, so an engineer reading a 2006 log knows what they are looking at.

The third is units and precision. Historic spreadsheets carry units that changed, values recorded to different precisions, and the occasional depth entered in feet on a project that was otherwise metric. Load with validation that rejects implausible values rather than accepting anything numeric, and route the rejections to someone with site knowledge. Rushing that step contaminates every trend and every correlation you build afterwards.

Why do lab feeds and data exchange exports break after launch?

The laboratory interface is the part that works on the day it is tested and drifts afterwards. Labs change instruments and the method code changes with them. A technician reorders columns in an export template. A lab acquires another lab and the reporting format follows the acquirer. None of this is announced to you.

The answer is a per laboratory ingestion profile that is configuration rather than code, so a changed column mapping is an afternoon rather than a release. Every ingest should produce an exception report before anything is committed: samples dispatched but not returned, results returned for samples never sent, method codes not in your dictionary, and values that contradict the field description. That last check is the one that earns its keep, because a plasticity index that argues with a logged soil description is machine detectable and invisible to a person proofreading a spreadsheet.

The AGS data exchange format helps enormously where your market uses it, and it should be a first class import and export rather than an afterthought. It also breaks in its own ways. Different versions of the format are in circulation, clients and labs will send you the version they use rather than the one you prefer, and a strict parser that rejects a slightly off file leaves an engineer emailing a lab at 6pm. Accept leniently, validate loudly, and report what you had to assume.

What happens when log control and parameter traceability are not covered?

Two disciplines get skipped in the first release and both cost more later than they would have cost to include.

The first is presentation control. If a log can be exported and then edited, it will be, and the moment it is edited you have two versions of the truth and one of them gets issued to a client. Firms allow this because a client's specification requires a column the template cannot express and the deadline is Friday. The correct response is that an unusual client requirement is a template change, not a manual edit, and that every rendering is reproducible from the data. Software is what makes that rule enforceable rather than aspirational, so build the rule in rather than adding it as an instruction nobody follows.

The second is parameter traceability, and it is the one that carries professional liability. A characteristic value in a report comes from a set of results filtered by an engineer's judgement: a suspect specimen discarded, a correlation weighted, a design line drawn through scatter. That reasoning almost always lives in a spreadsheet on one machine. Two years later, when the design is questioned, nobody can reconstruct which results were included or why.

Make parameter selection a recorded step: the dataset chosen, the filters applied and captured, the design line, the justification, and a permanent link from the parameter set back to the exact test results behind it. Reissue the report after four more boreholes and the system shows what changed. No packaged logging tool attempts this, and it is usually the feature a technical director sponsors once they have sat through a liability review.

Should you build custom or configure what you already own?

If you run under about 15 investigations a year, use one or two laboratories and produce logs to a single common standard, do not build. gINT, Bentley OpenGround, HoleBASE SI and the Datgel extensions cover that well, licence cost is modest against a build, and there are engineers in the market who already know them. Buy also if nobody in the firm will own the system, because a data platform without an internal owner degrades within two years regardless of how it was built.

Configure before concluding. Many teams that feel constrained have never built the templates properly and are hand editing logs out of habit rather than necessity. Fix that and see whether the constraint survives a project cycle.

Build when two or more of these are true. Laboratory data entry is a recognisable cost line. You absorb results from more than three laboratories in incompatible formats. You hold an archive in a region you keep working and cannot search it. A significant client requires deliverables your current tool cannot produce without manual editing. Or a liability review has made undocumented parameter selection uncomfortable.

How do hidden costs get into the quote?

• Presentation standards, counted individually. Each distinct log and section style you must support is genuine drafting work. A quote that says client templates without naming a number has priced one of them.
• Archive condition. Digital text based reports extract far more cheaply than scans. Sample fifty reports across your worst decade before anyone quotes, because the ratio of scans to digital moves the archive line materially.
• Instrument level integration. Reading cone penetration test output or a piezometer logger directly, rather than importing a file somebody exported, is device by device work and it is usually assumed rather than scoped.
• Offline field capture. Tablets in genuinely remote conditions with sync, conflict handling and preserved timestamps is architecture, not a toggle, and it is where field focused releases run late.
• Running costs and the internal owner. Hosting, storage that grows permanently as the archive loads, mapping services, and a named person with time protected to run the system. The last one is not a software cost and it is the one that decides whether the investment survives.

What separates a build that works from one that fails here?

You can filter developers in three questions on the first call.

Ask whether they know what the AGS data exchange format is. If your market uses it and they have not heard of it, they will invent an interchange format and you will spend two years explaining to clients why you cannot send them a standard file.

Ask how a sample is identified. The right answer is a composite of project, hole, depth range and sample reference, because laboratory references are not stable and clients renumber holes between phases. A developer who proposes a single system issued sample identifier has never dealt with a lab returning results under their own numbering.

Ask how they would approach the archive, and listen for whether they scope by value or by volume. Anyone proposing complete extraction of every value from every report is going to overrun, and they will overrun on the part of the project that had the least urgency.

Then make one internal decision that matters more than any of the above: name the person who owns the system after launch and protect their time. Systems in this category fail from neglect far more often than from engineering. Finally, settle in writing before kickoff that you own the repository, the cloud accounts, the database and the right to hire anyone else to continue. At Digital Heroes the client owns the code from the first commit, and a borehole archive is the firm's accumulated knowledge of the ground under a region. It should never sit behind somebody else's keys.