Market Data Entitlement Software Problems: The 7 That Fail an Exchange Audit, and How to Avoid Them

Why does the seat-shaped entitlement model keep getting built?

Because it is the model every engineer has seen before. Software asset management has trained a generation to think a licence belongs to a person or a machine, and the first data model somebody sketches for market data reproduces that assumption without anyone questioning it. It survives the first review because it describes terminals accurately, and terminals are the part of your estate everyone can see.

Then the exceptions arrive. One user is entitled to full depth on one venue and top of book on another. A single physical server carries an entitlement counted per application rather than per device. A risk engine restarts nightly and each instance counts. A price arrives from an exchange, passes through your own model, and reaches a client as derived data with its own rules and its own fee. None of that fits a seat, so it gets handled in a spreadsheet beside the system, which is exactly where you started.

Fix it by refusing to start development until the model expresses user based, device based, application based and non display counting as first class alternatives, with the counting basis carried on the entitlement rather than assumed globally. Ask any developer to model one specific case on a whiteboard before you sign: a single internal pricing service, consumed by four downstream applications, running three instances in production and one in disaster recovery. If they draw a user table, stop. That case is not an edge case in your estate. It is most of your exposure.

What goes wrong when the existing subscription spreadsheet is loaded as truth?

Loading it takes an afternoon. Believing it costs you the project. The workbook that currently drives your declarations carries forward last month's numbers with a few manual edits, was last fully reconciled during a reorganisation, and contains rows that nobody living can explain. Import it unchallenged and you have built a fast, confident system that produces wrong answers, which is materially worse than the spreadsheet, because people will now trust it.

The specific damage in this domain is that errors compound into declarations. A stale row means a user who left eighteen months ago is still being declared and paid for, or worse, a consumer who was added last year has never been declared at all. The first is money leaking. The second is the one that hurts in a review.

Treat migration as a reconciliation, scoped as weeks with your market data team rather than as a data load. Rebuild the population from the systems that hold live truth, meaning the vendor administration exports, the data access control system and your identity directory, then compare that population against the spreadsheet and work the differences one by one. Every difference gets a decision, an owner and a date. The output you want before go live is not a clean import. It is a list of every row the spreadsheet asserted that the source systems do not support, with a named person's judgement recorded against each.

That list is uncomfortable reading. It is also the single most valuable artefact the project produces, because it is the first honest picture of the estate anyone has had.

Why do the vendor, permissioning and identity feeds break after launch?

Because they are not really feeds. They are exports, produced by systems owned by other people, in formats that change when those people change something for their own reasons. A vendor administration export gains a column. The permissioning system's export changes how it represents a group. Human resources (HR) moves to a new identity platform and the leaver attribute you keyed on is now expressed differently. Nothing errors. The file parses. The counts drift.

The identity side is the one that quietly breaks declarations. Joiners, movers and leavers determine which cost centre carries a subscription and whether a person should be counted at all, and a mover who changed desks last quarter but kept their entitlement is both a recharge dispute and a declaration error at once.

Build three defences. First, validate on shape and volume rather than on parse success, so a source that returns eleven per cent fewer records than its own trailing average halts the run and raises a ticket instead of producing a smaller declaration. Second, keep the raw source file for every collection, immutably, so when a count is challenged in two years you can show what the source actually said on the day. Third, give every source a named owner on both sides and a documented expectation of what it contains, because the failures are organisational as often as technical.

Then reconcile continuously rather than monthly. A daily comparison between the identity directory population and the entitled population turns a quarterly surprise into a two line exception queue.

What happens when non display and derived data use is never instrumented?

This is the gap that makes audits expensive, because it is the half of the estate nobody can see. A developer built a price cache in 2019 and four other systems now read from it. A client facing calculator has been republishing a vendor field into a portal for two years. A model transforms an exchange price and the output reaches customers, which may make it derived data with its own policy and its own fee. None of that appears in any subscription inventory, because no subscription was ever taken out for it.

The usual substitute is a spreadsheet of applications maintained by hand from architecture diagrams. That records intent, not consumption, and it goes stale the first time a team deploys something without telling you.

Instrument at the point of use. Each internal service that reads a price emits an event carrying the application identifier, the entitlement it is consuming under, the venue and the instance, and those events roll into the monthly declaration. This is an engineering programme with named owners in other teams, not a data collection task, and any developer who describes it as easy has not done it. Budget the negotiation, not just the code.

For derived data, map at field level rather than product level. The exposure is almost never a product you know about. It is a specific field in a specific output that somebody classified informally years ago, and field level mapping with the classification, its author and its date attached is what turns that from a discovery into a record.

Should you build custom or configure what you already own?

If your estate is a few dozen terminals from one or two vendors, with no internal redistribution, no derived data and no non display declarations beyond the obvious, do not build. A disciplined spreadsheet reviewed quarterly is honest, cheap and proportionate, and TRG Screen or MDSL will cover you properly when you outgrow it.

If you already own one of those products, check what you have configured before concluding it cannot help. Both hold contracts, subscriptions, cost lines and invoice reconciliation, and firms routinely run them as a passive inventory with notice period tracking switched off and invoice matching never set up. Getting your contract dates and renewal notices actually loaded is weeks of work inside a tool you already pay for, and it captures the fastest saving available in this discipline, which is cancelling something unused before the notice window closes.

The line is instrumentation. A packaged inventory knows what you told it. Nobody can sell you a connector for a system your own developers wrote, and that is where non display usage and internal redistribution live. Most large firms end up running a packaged inventory alongside a custom instrumentation and declaration layer, not replacing one with the other, and proposing a full replacement of a working inventory is usually a sign a developer wants a bigger project.

How do hidden costs get into the quote?

The largest omission is per exchange work. Each declaration format, each policy interpretation and each reconciliation is separate effort, so a quote priced around your largest exchange is not a quote for the other eleven. Ask for the price of exchange two and exchange three explicitly, in writing, at the start.

The second is the negotiation cost of instrumentation. Every internal application that consumes prices has an owning team with its own roadmap, and getting an emitter into their release is a conversation before it is a code change. Firms that budget engineering time and no relationship time run three months late on that workstream alone.

Then four that are routinely left out. Data quality remediation on your identity directory, which is frequently weeks before anything else works. Evidence storage sized against the longest audit look back you can be subject to, which is a multi year commitment rather than a launch cost. Parallel running through at least two declaration cycles, where your team produces both the old and the new numbers and explains any difference. And the validation package if compliance requires the evidence store to be write once, which changes the storage design rather than adding a setting.

What separates a build that survives an audit from one that does not?

Reproducibility. The artefact that makes an audit survivable is not a number, it is the reconciliation: the raw population you collected, the rules you applied, the exclusions you made, and the declared figure that resulted, preserved as an immutable monthly snapshot. You are not trying to prove you were right. You are showing your working, and a system that recalculates historical months against current parameters cannot do that.

Versioned policy interpretations. When your reading of a rule changes, last month's declared numbers must not move, but the reasoning behind them has to remain retrievable alongside the new one. A developer who has done this raises immutable monthly snapshots before you ask.

Notice periods treated as first class dates with escalating alerts to named people. This is the least interesting feature in the system and it produces the fastest payback, because the recurring failure in market data is discovering an unused service one month after its notice window closed.

An allocation basis a desk head can argue with line by line. Direct attribution where a subscription ties to a named user from the identity directory, and a documented, published basis where it does not. The political requirement is not accuracy, it is that the recharge cannot be dismissed as arbitrary.

And ownership settled before kickoff: the repository, the cloud accounts and the unrestricted right to hire another firm. For a system holding audit evidence that is a control requirement rather than a commercial preference, because you may need to produce and explain records long after the build relationship has ended.