Communications Surveillance and Archiving Software: How Do You Produce Every Message in a Defensible Form When the Request Arrives?

Why capture is solved and supervision is not

The request arrives on a Tuesday. All communications involving four named individuals across an eleven month window, any channel, produced in original form with metadata intact. Your archive holds email properly. Chat is in a different vendor. Voice is in the turret recorder with a separate retention schedule. Teams messages started being captured in March and the two months before that are gone. One of the four people was a contractor whose messages were captured under a different identity. Producing this is now a six week project involving three vendors and a law firm, and the gaps will be found by someone other than you.

Enforcement over off channel communications has been the clearest compliance signal of recent years, with actions across dozens of firms and penalties large enough that capture programmes became board level obligations rather than compliance projects. The industry responded by buying capture, which was the right first move. Smarsh, Global Relay, Theta Lake, Shield and Behavox all capture well, and rebuilding connectors to WhatsApp, Bloomberg chat, Microsoft Teams, Zoom and a voice turret would be an expensive way to reinvent something mature.

Capture, however, is the easy half. What regulators actually examine is supervision: whether a qualified person reviewed the right things, on a defined basis, with documented reasoning, and whether the firm can prove it. That part is where packaged tools hand you a generic lexicon, a queue and a checkbox, and where firms discover that their review programme is a monthly ritual nobody believes in.

Problem 1: lexicon review generates volume, not supervision

A keyword list flags every message containing guarantee, and most of them are somebody guaranteeing to call back after lunch. Reviewers learn the pattern within a week and start closing in bulk. The queue is worked, the sign off is recorded, and the programme has quietly become a formality.

What changes it is treating review population as a policy decision rather than a search result. A build lets compliance define populations explicitly: everything from this desk during the quiet period, all external messages from staff on the restricted list, a random sample of a stated percentage from every registered person, plus risk triggered items. Each population has a rationale, a reviewer role, a frequency and an evidentiary record. When an examiner asks how you decided what to review, you show the policy and its version history rather than describing a keyword list.

This is where language models earn their place, and it is the one part of communications compliance where they clearly outperform the incumbent approach. Keyword matching cannot tell frustration from a complaint, or a joke about a market from a discussion of a specific order. A classifier scoring messages for the behaviours your policy actually cares about, running alongside the lexicon rather than replacing it, reduces reviewed volume substantially while raising the proportion of reviewed items that are genuinely worth reading. It must remain advisory: the model prioritises, the qualified principal decides and signs.

Problem 2: identity is fragmented and that is where the gaps live

The same person is an email address, a Bloomberg UUID, a Teams object identifier, a mobile number, a WhatsApp account and a turret extension. Some of those changed when they married, moved desks or came back as a contractor. Reconstructing a person's full communication history therefore depends on an identity map that in most firms is a spreadsheet maintained by whoever remembers.

A build makes identity a governed record: every channel identifier bound to a person with effective dates, sourced from the human resources (HR) system and the registration records so joiners and leavers propagate automatically. That single structure is what makes a production request answerable in hours instead of weeks, and it is also what makes supervision honest, because a reviewer covering a desk actually covers everyone on it including the person who joined last Thursday. The failure mode is always the same: the message existed, the capture worked, and nobody knew that identifier belonged to that person.

Problem 3: retention and legal hold rules collide across jurisdictions

Books and records obligations under SEC Rule 17a-4 sit alongside adviser record keeping requirements, market abuse obligations in the EU and UK, and privacy regimes that push in the opposite direction. A message from a trader in Frankfurt to a client in Singapore, held on a US archive, is subject to several regimes at once, and the answer to how long you keep it and who may read it is not uniform. Meanwhile a legal hold must freeze deletion for a specific matter without breaking the general schedule for everything else.

Packaged archives implement a retention policy. What they handle less gracefully is a matrix of overlapping policies where the longest applicable period wins, holds override schedules, and the reasoning must be reconstructable years later. A build expresses retention as rules over message attributes with effective dating and a full audit of every disposition, so that when a message is finally deleted you can show precisely which rule permitted it and confirm no hold applied. That is the difference between defensible disposal and hoping.

Problem 4: production is judged on completeness you cannot currently prove

The reason production requests are frightening is not effort, it is uncertainty. Was every channel included. Was the contractor period covered. Did the WhatsApp connector fail for eleven days in June and did anyone notice.

The feature that addresses this is unglamorous and is the one we would build first: continuous capture assurance. Every channel reports expected versus received volumes on a schedule, gaps raise an alert with an owner, and each remediation is recorded. Then a production package can carry a completeness statement covering the exact window, listing the channels in scope, the identities included and any known gaps with their explanations. Volunteering a documented, explained gap is survivable. Having a gap found by the person reviewing your production is not.

What this costs and how long it takes

A focused first release, meaning ingestion of your existing capture feeds into a unified message store, identity resolution across channels, policy defined review populations with lexicon and classifier scoring, reviewer workflow with documented sign off, and defensible search and export, runs $95,000 to $210,000 and ships in 14 to 20 weeks. A full platform adding voice transcription and review, cross channel risk scoring, legal hold and matter management, jurisdictional retention rules, capture assurance monitoring and linkage to trade surveillance cases runs $260,000 to $700,000 phased across 9 to 16 months.

What drives cost up here specifically: channel count and the fact that each capture vendor exports in its own shape; voice, which brings transcription quality, speaker separation and multilingual handling and is reliably the most expensive channel to do properly; multilingual review, since a classifier tuned on English does not transfer to Mandarin trading chat; message volume, because storing and searching years of messages with attachments at production speed is a real engineering problem; and jurisdictional spread, as each additional regime adds retention and access rules that must coexist rather than replace one another.

What holds it down: keeping your existing capture vendors and building only the supervision, identity and production layer above them. Very few firms should be writing connectors.

Build versus buy, and when the vendor is enough

Buy end to end if you are a single jurisdiction firm on email plus one chat platform, with a headcount where a principal can genuinely review a meaningful sample, and no voice obligation. Smarsh or Global Relay will cover capture, retention and review, and building would be an expensive way to arrive at the same place.

Build the layer above capture when two or more of these are true. You run more than about five channels across more than one capture vendor. Your reviewers close flagged items in bulk and everyone privately knows the review is a formality. You cannot produce a complete communication history for a named individual across all channels in under a day. You have jurisdictional retention conflicts resolved by someone's judgement rather than by a rule. Or you already run trade surveillance and cannot link a message to a trading case without manual work.

The strategic point is simple. Capture is a commodity and should be bought. Supervision is your policy, your risk appetite and your accountability, and every one of those is firm specific. A programme that outsources supervision to a vendor lexicon has outsourced the part that carries personal responsibility for a named principal, which is not a transfer any vendor contract can actually effect.

How to choose a developer for communications supervision software

Ask them how they model identity across channels over time. Effective dated identifiers bound to a person record sourced from human resources should come up unprompted. If identity is a user table, production requests will keep missing things and you will not know until someone else finds out.

Ask what they will do about immutability and disposition evidence. You want an append only store, cryptographic integrity on stored content, and a complete audit trail of every access and every deletion with the rule that authorised it. Record keeping rules allow for approaches beyond traditional write once media, but whichever route you take you have to be able to demonstrate the controls, not assert them.

Ask specifically about voice. Transcription accuracy on a noisy trading floor, speaker separation on a turret line and handling of code words and tickers are hard problems, and a team that has not done it will underestimate it by a factor you will feel.

Ask who owns the code, the classifier training data and the cloud accounts, and get it in writing before kickoff. At Digital Heroes the client owns all of it from the first commit. Your review policy expressed as code is a supervisory artefact, and if a vendor owns it you cannot fully explain your own programme when it matters most.