Humanitarian Field Data Collection Problems: The 7 That Cost Real Money, and How to Avoid Them

Why does the scope collapse into rebuilding the form engine so often?

Because the form is the part everyone can see. The programme manager has used the tablet and the questionnaire is the artefact that gets argued about in workshops, so the requirement that reaches a developer is a better form, and a developer who has not worked in this sector will happily quote for one.

What that misses is that offline form capture is a solved problem with an open standard behind it. XLSForm is portable, KoboToolbox costs nothing, and SurveyCTO already captures text and audio audits that show how an interview progressed. Rebuilding any of that spends the budget on the one part you can get for free, and it introduces a class of bugs, offline sync and conflict resolution, that took those projects years to settle.

The fix is a scoping rule you can state before you speak to anyone: we are not building a form renderer. Write the first release around the submission rather than before it. Enumerator records with training scores, teams and supervisors. Work packages pushed to devices so a tablet only ever holds its own caseload. Paradata captured on ingest. Automated back check sampling weighted toward flagged enumerators. In our delivery experience, keeping an existing capture tool rather than replacing it takes roughly a third off the budget and removes the highest risk engineering entirely.

What goes wrong when you migrate historic rounds and mid round form versions?

Two things, and both are discovered late. The first is that historic exports were flattened for analysis, not for storage. Repeat groups became wide columns with numbered suffixes, select multiple answers became concatenated strings, and column meaning depends on which questionnaire version produced it. Importing that is a reconstruction rather than a load job, and it fails quietly because nothing errors, it just produces a row that looks plausible.

The second is form versioning. Assessment questionnaires get corrected mid round, because a translation was wrong or a skip rule fired incorrectly, and the correction is applied to the live form. Unless the form version is stamped on every submission, you now have one dataset with two definitions of the same variable and no way to separate them. That is the failure that contaminates an indicator rather than just losing a district.

The concrete fix is to treat the form version as a first class fact from day one, stamped on every submission and stored alongside the definition it refers to, so a mid round correction becomes analysable rather than fatal. For history, migrate selectively: bring in the rounds you genuinely reanalyse, with a documented mapping per version, and archive the rest as files with a manifest. Reconciling counts per district per round against the original export, before anyone trusts the new system, is the step teams skip and regret.

Why do the integrations that matter here break after launch?

Three integrations carry the value in this category and each fails in a different way. The capture tool connection is the first. If you have kept Kobo or SurveyCTO, your platform pulls submissions from it, and that link breaks when the questionnaire changes shape. A new repeat group appears, a question is renamed, and the ingest either drops fields silently or rejects the batch. Nobody notices for a week because the field teams are still submitting successfully.

Mobile money for enumerator payment is the second. Payment integrations in the countries where this work happens are country specific, they change without much notice, and they fail in ways that leave a transaction in an unknown state. Paying two hundred people per interview means reconciliation is not optional. Every disbursement needs an idempotency key and a status check, so a retry cannot double pay and a timeout does not become a dispute at the next round.

Donor reporting is the third, and it fails not technically but definitionally. A donor changes what counts inside indicator 2.3 mid award, your export mapping still runs, and the number it produces is now measuring something else. The fix is that indicator definitions are versioned configuration with numerator, denominator, disaggregation dimensions and source questions held explicitly, so a definition change produces a new version and both figures stay reproducible. Add a small ingest contract test that runs against the live form after every questionnaire change, and an alert on submission volume dropping to zero for any team. Silent integration failure is the norm here, not the exception.

What happens when personal data protection and consent are not covered?

It is not covered in most first releases, because it is invisible in a demonstration. Household surveys collect names, locations, household composition, income, and in protection work, incident detail and health status. That data sits on tablets that travel, in a database, and in exports that get emailed to a consultant who needed a quick file.

The failure is rarely a breach in the dramatic sense. It is that an analyst downloads an identified extract because that is the only export available, a partner receives it, and nobody can say afterwards who held what. Consent was taken verbally at a door, translated on the spot by the enumerator, and recorded as a tick. When a donor or a protection adviser asks what people actually agreed to, there is no answer.

The fix is structural and it belongs in the first release, not a later phase, because retrofitting separation across a live dataset is genuinely expensive. Split identifiers from analytical data at rest, restrict the identified layer to named roles with access logging, and make de-identified export the default so the quick file does not create the incident. Record consent as a structured fact carrying the language used and the version of the consent text read out. Set retention as a configured period with automatic disposal rather than an intention nobody owns. Devices carry only the assigned caseload, encrypted, with remote revocation that has been tested rather than assumed.

Should you build custom or configure what you already own?

Configure, for a large share of organisations, and we would say so on a call. If you run two rounds a year with around 25 or 30 enumerators in one or two languages, and a supervisor can genuinely review incoming submissions daily, KoboToolbox costs nothing and does the job. SurveyCTO earns its licence when data quality is your main worry and your rounds are modest, because its audit capture is real and most teams never use what they already have. CommCare is the right choice when the work is longitudinal case management with returning visits rather than a cross sectional assessment. Spending on enumerator training will beat spending on software at that scale, reliably.

The build case starts when two or more of these are true. You run rounds above roughly 80 enumerators and assignment lives in a spreadsheet. You pay per interview and payment disputes have already cost you time. You have found fabricated data after demobilisation. The same dataset feeds three or more donor templates with conflicting definitions. Or you hold protection sensitive records and your honest answer to who can see identified data is that the analysts can. Even then, the recommendation stands: configure the capture layer, build the workforce, quality and indicator layers around it.

How do hidden costs get into the quote?

Four ways, and all of them are avoidable if you ask early. Right to left and non Latin script rendering is the first. It is treated as translation and it is not. Numeric input inside a right to left form with skip logic breaks on specific tablet models, response option order changes meaning when translated, and Eastern and Western Arabic numerals arrive mixed because enumerators use whichever keyboard they have. Device level rendering tests belong in the release checklist and they are real work that nobody quotes.

Donor templates are the second. A quote that says donor reporting is a line item, not a count. Ask how many templates, and whether their indicator definitions conflict, because they usually do, and each conflicting definition is a separate measurement path rather than a formatting option.

Payment is the third, particularly mobile money in the field, and it is a subsystem rather than a feature. Multi country deployment is the fourth, because the personal data position changes with jurisdiction and so does the hosting decision.

Ask for the estimate broken out by these four before you sign anything.

What separates a build that works from one that fails here?

The working builds get detection to a supervisor while the team is still in the district. That is the whole difference. Flagged submissions enter the next day's back check sample automatically, the short verification form is on the supervisor's device, mismatches roll into an enumerator quality score, and rejected work does not count toward payment. That last rule changes behaviour more than any training session, and it only works if the pieces are connected.

The failing builds treat quality as reporting. They flag outliers into a dashboard that a data manager reads after demobilisation, which is documentation rather than quality control. They also tend to over automate the judgement. Ranking suspicious submissions so a supervisor listens to twenty recordings instead of a random hundred is worth building. Scoring an enumerator's honesty automatically and acting on the score is a liability, and any developer who offers it has not thought about what happens when a field worker disputes it.

Two practical tests before you commit. Ask a developer to describe what happens when a device has been offline for five days, the battery died mid sync, and the same submission arrives twice. Vague reassurance about offline support means data loss. Then ask them to walk through how identifiers are separated from analytical data and who can see each layer. If personal data protection comes back as a permissions checkbox, they should not be holding your respondents' records. Get code ownership in writing before kickoff, covering the repository, the infrastructure accounts and the right to hire someone else. In a sector funded round to round, a vendor holding your pipeline is why organisations re-procure the same system every four years.