Pharmaceutical Serialization Software Problems: The 5 That Hold Shipments, and How to Avoid Them

Why does building the happy path first hold shipments later?

Every serialization project starts the same way. Commission a serial number, print it, verify it with the vision camera, aggregate the unit into a case and the case onto a pallet, send the file. That path is demonstrable, it is what the packaging line vendor shows in a factory acceptance test, and it works.

Then production happens. A case is dropped and rebuilt. A unit is rejected by vision after the case was aggregated. A pallet is broken down for a partial shipment. Quality pulls a unit for a stability sample. Each is routine, each changes the true hierarchy, and each needs a matching event. On most lines the exception path is whatever the integrator configured years ago, frequently a supervisor override that fixes the line and leaves the repository believing something untrue.

The fix is a sequencing decision made before the first sprint. Build unpack, repack, decommission and sample as first class events available to the operator at the point of handling, on a scanner, in the seconds when the handling actually happens. Treat aggregation as an event stream that can only be corrected by further events, never by editing state, so current hierarchy is computed from history. A reconciliation against a physical scan then becomes instant and explainable. Teams that build exceptions first ship a system that works. Teams that leave them until phase two ship a system that produces holds.

What goes wrong with master data and migrating from an existing provider?

Master data alignment is the quiet reason a first partner exchange fails, and it has nothing to do with your code. Your product identifier, your location identifiers and your trading partner's expectations of both have to agree before a single event is accepted. A location identifier that was registered years ago for a site that has since been renumbered, or a product identifier that differs by a single digit between your enterprise resource planning (ERP) system and the data you send, will reject every event in the file with an error message that describes the symptom rather than the cause.

Migration adds a second layer. If you are moving off a hosted repository, you need the historical events, not a summary, because a partner investigation can reach back into product still in the channel. What providers will export varies, and the export is rarely in the form you want. Serial number ranges already issued to lines have to move without any possibility of reissue, which means the number pool state is a cutover artefact that has to be exact.

What works: run master data reconciliation as a distinct workstream before integration testing, comparing your identifiers against what each partner expects and what your enterprise system holds, and fix the source rather than mapping around it. Treat number pool cutover as a controlled change with a documented freeze, a verified final issued range and a reconciliation both sides sign, because a reissued serial number is a defect you cannot recall from the channel.

Why do line and partner integrations break after launch?

Because both sit on someone else's change schedule and neither will tell you in advance.

On the line, a printer is replaced, a vision system gets a firmware update, or a controller is reconfigured during a maintenance window. Line integration is not like connecting to a web service: handshakes are often undocumented, older controllers are proprietary, and the integrator who built it may have moved on. The work also has to happen around production windows rather than whenever engineering is ready, which means a change that would take an afternoon takes three weeks of scheduling.

On the partner side, GS1 EPCIS is a standard implemented with local dialects. One wholesaler expects specific business step and disposition values, another requires particular extension fields, one wants files over AS2, another over SFTP, another over a web interface with its own authentication. Partners change requirements, add validations and rotate credentials, and you find out when a file is rejected.

The architecture that survives this separates the internal event model from the partner profile entirely. Events are stored once in canonical form. Each partner is a profile defining the transformation, the transport, the required fields, acknowledgement handling and retry behaviour, so onboarding or updating a partner is configuration and a test cycle rather than a code release. Retain every outbound file exactly as sent alongside its acknowledgement or rejection, because the first question in any dispute is what you actually transmitted and when. On the line side, budget integration per line, sequence it by difficulty, and treat vintage as the main variable rather than count.

What happens when verification and investigation requests are not covered?

These arrive on somebody else's clock and they are easy to leave out of scope because nothing in your own operation generates them.

When a wholesaler receives a saleable return, they need to verify the product identifier before that unit can go back into commerce. When a partner suspects an illegitimate product, an investigation follows with obligations on both sides. Under the enhanced requirements of the Drug Supply Chain Security Act, transaction information travels electronically at package level and trading partners must be able to respond to verification and investigation requests, which is a fundamentally different posture from the lot level world it replaced. The questions asked are about individual packages, not shipments.

What to build: a verification service that answers automatically from your repository, logs every request with its requester and response, and escalates to a person only when the answer is anything other than a clean confirmation. Give every exception a named workflow with the correct events built in, exposed to the people who encounter it. A warehouse operator scanning a damaged case needs a decommission path that takes ten seconds and produces correct data, because the alternative is that they set it aside and tell someone on Monday. And treat repeated verification attempts against the same identifier from unexpected sources as an alert, since that pattern is exactly what these systems exist to surface.

Should you build custom or configure what you already own?

If you are a single site manufacturer with one or two packaging lines and a small number of trading partners, buy. TraceLink, Antares Vision rfxcel and Systech will get you exchanging data faster and cheaper than a build, and the network connectivity they bring is genuinely valuable because trading partner exchange only works if the other party is connected too. Rebuilding a partner network from scratch is not a sensible use of capital and we will say so.

Build when the exception paths are where your money goes. The signals are specific: hierarchy mismatches regularly hold shipments and the packaged system's exception handling does not match how your warehouse actually works; you operate across several markets and maintain parallel systems that disagree; you are a repackager or wholesaler whose business model creates aggregation events that manufacturer oriented products handle badly; you need serialization data joined to your own batch, quality and logistics data for analysis a hosted repository will not give you; or per transaction pricing has grown enough that the arithmetic of ownership has changed.

A hybrid is frequently the right answer and is under used. Keep a commercial service for partner connectivity, where the network is the value, and build the site and enterprise layer where your operations live. That combination gives you exception handling you control without rebuilding pipes that already work.

How do hidden costs get into the quote?

Five items, all of which are real and none of which shows up as a feature.

• Validation. A GxP system whose records support product release and regulatory reporting, so requirements, risk assessment, traceability, executed evidence, audit trails and controlled change are a workstream rather than paperwork. Audit trail design deserves specific attention, because serialization event volumes will defeat an approach borrowed from a document system.
• Line integration by vintage, not by count. A modern line with a documented controller interface is days. An older line with a proprietary controller and no original integrator is weeks, in production windows you do not control.
• Partner test cycles. Each profile needs a test cycle with the partner, whose availability is not yours. Two profiles can take longer than five depending on who answers email.
• National system certification. Each regulated market is a separate integration with its own certification, and some require codes to be requested from a state system before printing, which changes the number pool design.
• Contract manufacturer alignment. A contract manufacturer producing on your behalf must send events in a format you can accept. Getting several aligned is a programme, not a feature.

What separates a serialization build that works from one that fails?

Four things, and they are visible in the first design conversation.

The first is an append only event stream with current hierarchy computed from history. If a developer proposes updating a parent identifier on a child record, they have built an inventory application and will produce exactly the mismatches that stop your shipments. Ask them to model a repack before you sign anything.

The second is partner profiles as configuration. If onboarding a new wholesaler requires a code release, you have bought a permanent consulting engagement rather than a system.

The third is exception handling designed around the person who encounters the exception. Warehouse operators and line technicians will not walk to a workstation to record a decommission. Put it on the scanner they already carry, make it take seconds, and the data stays true. Make it inconvenient and the physical world and the data separate again, which returns you to the original problem with more software.

The fourth is market obligation as configuration attached to product and destination, so a packaging order knows what it must do before it runs. Serial number sourcing differs by market, so the number pool has to support both internally generated ranges and externally issued codes without special casing the line.

Then settle ownership before kickoff: the repository, the infrastructure accounts and the right to hire anyone else. At Digital Heroes the client owns the code from the first commit. Serialization records support regulatory reporting and partner investigations for years, which is longer than most vendor relationships last.