Geotechnical Investigation Data Software: Why Does Every Borehole Log Arrive in a Different Spreadsheet?

The ground investigation is over before anyone checks the data

A ground investigation for a rail overbridge finishes on a Friday. Three drilling subcontractors worked the site over five weeks. Two laboratories ran the testing. On the following Wednesday a graduate engineer starts assembling the factual report and finds the usual archaeology. One driller reported SPT results with blow counts in separate columns per increment, another reported only the final N value. Sample references from rig 2 do not match the laboratory schedule, because the driller renumbered after redrilling BH07. Three water strike depths are recorded in a comments field as text. One laboratory returned moisture content as a percentage and the other as a decimal fraction.

None of these is exotic. Every geotechnical team has a version of this week. It takes another two to three weeks to reconcile, and the reconciliation happens after every rig has demobilised, which means anything genuinely missing is now either a return visit or a gap in the report with a caveat attached.

Unforeseen ground conditions are one of the classic sources of dispute on civil projects, and the borehole record is the evidence base on which those arguments are settled. A dataset that was cleaned up under time pressure by whoever was available, with judgement calls buried in a spreadsheet, is a weak position to argue from years later. That is the real reason this deserves a system rather than better spreadsheet hygiene.

Problem 1: validation happens weeks after it could have changed anything

The single highest value change in this whole category is moving validation to the moment of data creation. If a driller's daily return is validated on upload the same evening, a missing water strike, an out of range value or a duplicated sample reference can be fixed while the rig is still on site. A day later it is a phone call. A month later it is a caveat in a report.

That means a schema first pipeline. Every incoming file, whether it is an AGS transfer, a laboratory return or a spreadsheet from a subcontractor who has never heard of AGS, is parsed against a defined structure and rejected with specific, human readable errors. Not a warning log nobody reads. A rejection that names the row, the field and the rule, and goes back to the person who created it.

The hard part is not the validator, it is the culture around it. So the design decision that matters is making it easy to comply: give small drilling subcontractors a simple upload form or a template that produces valid data, rather than demanding they buy software. A build can do that. Products that assume everyone in the supply chain owns the same licence cannot.

Problem 2: AGS is treated as an export, not a schema

AGS in the UK and Commonwealth markets, and DIGGS where it is adopted, exist precisely so ground data can move between organisations without reformatting. Most consultancies treat the standard as a deliverable produced at the end of a project by exporting from whatever internal mess they accumulated. That inverts the value. The standard is most useful as the internal schema, applied from the first daily return, because then the deliverable is a byproduct and every project's data is comparable with every other project's.

There is a real complication. Clients amend. A major infrastructure client will specify additional groups and fields, mandate particular abbreviation lists, or require their own naming for locations. A build handles that with a base schema plus project profiles that extend it, so a client specific requirement is configuration rather than a parallel process run by one person who knows the client.

Problem 3: the sample chain breaks between the rig and the laboratory

A sample is taken at a depth in a hole on a date by a person. It gets a reference. It travels. A laboratory schedules tests against it, sometimes tests that were changed by phone after the schedule was issued. Results come back weeks later. Somewhere in that chain the identity of the sample has to survive, and in spreadsheet based workflows it frequently does not.

What a build enforces is a chain of custody with the sample as the persistent object. Logged at the rig with location, depth, type and condition, transferred with a manifest, received by the laboratory with a confirmation, scheduled against a test suite, and results returned against the original reference rather than a laboratory internal number. Turnaround tracking falls out of this for free, which matters because laboratory delay is one of the most common causes of a late factual report and currently nobody can prove where the time went.

Problem 4: the factual report and the ground model are built from different copies

The logs are drafted by one team, usually in gINT or OpenGround. The interpretative ground model is built by another engineer in a modelling package from a spreadsheet extract taken at some point in the process. The two then diverge. A stratum boundary revised after a review meeting gets corrected in the log and not in the model, or the model gets refined and the log is never updated to match.

The correct architecture is a single data store with multiple outputs. Logs, sections, laboratory summary tables, exceedance tables and model input files all render from the same records at the moment they are produced, with a version stamp. Bentley gINT and OpenGround are genuinely good at log drafting and it would be foolish to rebuild that. The pragmatic build treats them as an output channel, pushing validated data into them rather than replacing them, and pushing the same records into your modelling environment.

Problem 5: contaminated land and geotechnical data live in separate worlds

On brownfield sites, the same holes produce both geotechnical and environmental samples. The environmental side needs screening against project specific assessment criteria that vary by land use, exceedance tables and reporting that geotechnical tools do not attempt. So most consultancies run two systems and one shared set of holes, and the location register is maintained twice.

A build makes the location and sample registers common and applies discipline specific processing on top: geotechnical test suites on one side, contaminant suites with screening criteria on the other, both reporting from the same holes. When the criteria change, which they do when a scheme's proposed end use changes, rescreening is a recalculation instead of a fortnight of spreadsheet work.

Problem 6: your firm has drilled that site before and cannot find the data

Ten years of investigations across a city sits in project folders on a file server, organised by job number, in a format nobody can query. So each new bid is priced as though the ground is unknown, and each investigation is scoped without reference to what the firm already knows two streets away.

Once data is schema first and validated, a regional database is nearly free. Spatially query every hole within 500 metres of a proposed alignment, see what strata were encountered and what testing exists, and scope a new investigation against real prior knowledge. For a consultancy that works repeatedly in the same geography, this is the part of the build that changes commercial behaviour rather than just efficiency.

What this costs and how long it takes

Across the 2,000 plus projects Digital Heroes has delivered, the shape here is as follows. A first release covering schema first ingestion with AGS or DIGGS validation, project profiles for client specific requirements, sample chain of custody, laboratory scheduling and turnaround, and factual output runs $60,000 to $140,000 and ships in 12 to 18 weeks. A full platform adding an offline field application for rig returns, contaminant screening, ground model export, and the firm wide spatial database runs $150,000 to $400,000 over 6 to 12 months.

What drives the number up: the number of laboratories and their result formats, since each interface is real work. A field application, particularly offline on remote sites with photographs and geolocation. Direct integration into gINT or OpenGround, which is worthwhile and is not trivial. CPT and other continuous instrument data, because the volumes and the processing are different from discrete sampling. And historic data migration, which is where budgets disappear, so decide honestly how far back is worth converting.

What keeps it down: one client's schema, one investigation type, and validation at ingest. That alone removes most of the three week reconciliation.

Build versus buy, and when buying is right

Buy if you are a small or mid sized consultancy producing logs on a modest number of projects a year with conventional client requirements. gINT remains widely used, OpenGround is the current path, and Datgel adds genuinely useful tooling on top of that ecosystem. Log drafting represents years of accumulated detail and you should not attempt to reproduce it.

Build when two or more of these are true. You run large programmes with many subcontract drillers and laboratories, so data arrives from parties who do not share your software. Clients impose bespoke schemas and deliverables that your team currently satisfies by hand. You want validation at source with a field application, which changes the economics of the whole investigation. Your work spans geotechnical and contaminated land on the same holes. Or you work repeatedly in one geography and the accumulated ground dataset should be an asset you can query and price against.

How to choose a developer for geotechnical data software

Ask them what happens when a laboratory returns a result for a sample reference that does not exist. The right answer involves a quarantine queue and a defined resolution, not a silent insert or a failed import that somebody discovers a fortnight later.

Ask how project specific schema extensions are handled without forking the codebase. If every new client requirement is a development ticket, the system will be behind within a year and your team will go back to spreadsheets for the awkward jobs.

Ask whether they intend to replace gINT or feed it. Anyone proposing to rebuild log drafting from scratch as part of a first release has not understood what they are quoting.

Ask about units and about how they represent a value that is below a laboratory detection limit, because handling that wrongly corrupts every summary statistic downstream. And settle code ownership before kickoff: you should hold the repository, the infrastructure accounts and the right to bring in another firm. At Digital Heroes the client owns the code from the first commit, which matters for data that may be re examined in a dispute a decade later.