Computer System Validation Software Problems: The 7 That Cost Real Money, and How to Avoid Them
The most expensive failure in this category is that elapsed time per validated change is almost independent of the size of the change. A configuration update an engineer completes in forty minutes closes nineteen days later, because the package needs a requirement specification, a risk assessment, a traceability matrix rebuilt by hand from both, test scripts executed with screenshots pasted and initialled, and three approvers who are not all in the building. In Digital Heroes delivery experience that pattern is two to four weeks per change regardless of scope, and it is a tax on every regulated system you own that grows with your release frequency.
Why does keeping requirements in documents keep happening?
Because documents are what regulators inspected for thirty years and what your quality manual describes. A user requirement specification in Word is familiar, portable, easy to review, and signs off cleanly. Nothing about that is unreasonable. The problem is what it makes impossible downstream.
Everything in validation hangs off requirements. Risk is assessed per requirement. Tests prove requirements. Change impact is measured in affected requirements. Periodic review asks whether requirements still hold. When requirements live inside a document, none of those relationships exist in a form anything can query, so each one is reconstructed by a person into a matrix that is accurate on the day it is signed and stale within a quarter.
The result is that a simple question, which tests cover this requirement, or which requirements does this proposed change touch, takes a validation lead half a day and produces an answer nobody fully trusts. Multiply that by the number of regulated systems you own and the cost is obvious.
The fix is to make the requirement a first class record with a stable identifier, a version history, a regulated impact classification, linked risks, linked test cases and links to the change requests that created or altered it. Traceability then stops being a document you produce and becomes a query you run, which is always current. That single change is usually what converts a nineteen day package into a three day one, because most of those nineteen days were transcription and reconciliation rather than review.
What goes wrong when you migrate legacy validation packages?
The instinct is to bring everything across so the new system holds a complete history. That instinct is what turns a twelve week project into a nine month one.
Legacy packages are documents, and the requirements inside them are prose. Extracting them into structured records means somebody reading each specification and deciding where one requirement ends and the next begins, which is a judgement, and different people make it differently. Two hundred systems worth of that is not a data migration, it is a rewrite of your entire requirement estate performed by people who did not write it.
Then the linkages. Historic traceability matrices reference test script numbers that exist only as file names in a controlled document repository, and some of those scripts were superseded without the matrix being updated. Reproducing a broken link faithfully in a structured system makes the break visible, which is uncomfortable and occasionally a finding.
What works: leave closed historical packages where they are, as records retained in your document system, and start structured records from the current validated state forward. When a system next undergoes a change, capture its requirements properly then, as part of work you were doing anyway. That spreads the effort across your normal change cadence instead of concentrating it in a project, and it means the systems that change most often, which are the ones costing you time, get converted first. Anyone proposing to import the whole estate before go live has not read a legacy package recently.
Why do the test automation and quality process integrations break after launch?
Automated evidence capture is the single largest cost reduction available here, and it is also the integration most likely to degrade quietly.
The mechanism is straightforward: an automated run against the system under test produces structured results, screenshots, timestamps, environment identity and the exact build tested, posted against the test case with the same integrity controls a manual execution carries. What breaks is the metadata rather than the tests. A pipeline is reconfigured and stops passing the build identifier, so evidence arrives attached to no specific version. An environment is rebuilt with a new name and the environment identity no longer matches what was validated. A test framework upgrade changes the result format and screenshots stop attaching.
None of those produce a failing test. They produce evidence that is present, plausible and insufficient, which is the worst state to be in because nobody investigates it until an auditor asks what build was actually tested.
The quality process integrations fail differently. Change control, deviation, corrective action and training each have their own owner and their own system, and each interface is a negotiation before it is code. After launch they break when the other system's workflow is amended by its owner without anyone telling you.
What to build: validate incoming evidence on arrival and reject anything missing build identity, environment identity or a timestamp, rather than accepting it and discovering the gap later. Alert on the rejection to a person. And review integration contracts with each system owner on a schedule, because their change will not appear in yours.
What happens when the platform is not validated and the state drifts?
Two omissions cause most of the pain, and both are cheap to design in and expensive to retrofit.
The first is the platform itself. If it holds regulated records, and it will, holding approved requirements, executed evidence and electronic signatures, then it is a regulated system and it will be inspected as one. Teams that treat its own validation as a phase two discover that audit trails, signature manifests and requirement to test traceability all needed designing differently, and the retrofit costs more than building it correctly did.
The second is drift between projects. Validation is treated as a project and lived as a condition. A cloud supplier pushes a release you did not schedule, an operating system is patched, an integration partner changes an interface, a user is granted a role nobody assessed. Periodic review is supposed to catch this and in most organisations it is an annual document exercise performed largely from memory.
What to build: a system inventory where each entry carries its regulated classification, its current validated version, its supplier assessment date, its periodic review date, its open deviations and its configured integrations. Where the system can be queried, compare the deployed version against the validated version and raise a discrepancy when they diverge. That check is how you learn a supplier updated your software in a maintenance window, instead of learning it during an inspection. Periodic review then reviews facts the platform already holds.
Should you build custom or configure what you already own?
If your validation volume is low, meaning a handful of systems that change once or twice a year, buy. ValGenesis and Kneat Gx are mature, they are used by inspected companies, and either will be running next quarter for less than a build costs. If you are already committed to Veeva Vault, its validation management capability removes a document control problem you would otherwise solve twice, and the same logic applies to the module inside a MasterControl estate.
Buy also if you have no internal appetite to own a regulated application. This system will be inspected and somebody has to maintain it through supplier changes, your own procedural updates and staff turnover. That is a real commitment and it is fair to decline it.
The honest test for building is whether you have been changing your quality procedures to fit a product. Risk classification schemes, evidence expectations and the relationship between validation and your change control process are defined by your quality manual and audited against it. Where a commercial tool's model and your procedure disagree, one of them changes, and it will not be the tool. Build when that is happening, when you have a large estate of internal applications releasing on a modern cadence and a document driven process is now the reason software takes a quarter to ship, when you want evidence generated by your test automation rather than pasted by a tester, or when you run several sites with genuinely different procedures.
How do hidden costs get into the quote?
Validating the validation platform is the line most often missing. It is not optional, it adds real weeks, and it has to be planned from the first requirement rather than added at the end. Ask for it as a separate figure, because a proposal that folds it in without comment has probably not scoped it.
The others. The number of quality processes you connect, since change control, deviation, corrective action and training each carry an owner, an approval and an integration negotiation. Whether your systems are actually testable, because automated evidence capture pays back fastest and older client server applications may not support it, which changes the business case entirely. The number of sites, since a global quality manual with site level procedures means configurable rules rather than one hard coded flow. And electronic signature handling, which sounds like a component and is really a set of behaviours across every record type.
The item that consumes the most calendar time is not engineering. It is agreement. A build cannot encode a risk to test depth rule that quality, information technology and operations still dispute, and those conversations routinely take longer than the implementation. Write the rules down before kickoff and the schedule will look very different.
What separates a build that works from one that fails here?
Ask what data integrity principles mean for the audit trail design before you sign anything. Someone who has worked in this environment will talk about attributable and contemporaneous records, about why a soft delete is still a deletion that must be recorded, and about time synchronisation across services. Someone who proposes an updated timestamp column has not been inspected and will learn on your budget.
Ask how the system validates itself. No answer means they do not understand that this application is itself a regulated system, and that misunderstanding will surface as rework.
Ask how your risk classification scheme gets changed. If altering a risk to test depth rule requires a code release, and therefore a validated change to the validation system, they have designed a trap. Those rules belong in controlled configuration your quality organisation owns, because your scheme will change after your next audit.
Ask how evidence arrives from automated tests and what happens when it arrives incomplete. Rejecting evidence without build identity is the behaviour you want, and it is the detail that separates a design that holds from one that quietly accumulates unusable records.
Then get ownership in writing before kickoff: the repository, the infrastructure accounts and the right to hire anyone else. At Digital Heroes the client owns the code from the first commit. This system holds the evidence of your validated state, which you may be asked to produce years after any contract ends, so it has to be recoverable without a vendor's cooperation.
The evidence behind this guide
Independent findings on why this investment pays off. Every link goes to the primary source.
- The federal government spends about 80% of its IT budget on operations and maintenance of existing systems rather than on development or modernization, with many critical systems being decades old. Source: U.S. Government Accountability Office (GAO) (2025) →
- SaaS spend averaged $4,830 per employee (up 21.9% year over year), with large enterprises (10,000+ employees) spending roughly $284M annually and running about 660 apps, while organizations wasted an average of $21M annually on unused licenses. Source: Zylo (2025) →
- This World Bank report argues that digital technology adoption raises SME competitiveness, productivity and resilience, while documenting that smaller firms consistently lag larger ones in digital adoption - a gap that constrains their growth and market reach. Source: World Bank (2022) →
- The NRF discontinued its long-running annual shrink report, stating that a broad study of retail shrink 'is no longer sufficient for capturing the key challenges and needs of the industry' - important context that qualifies how POS/shrink benchmarks should be cited going forward. Source: Retail Dive (2024) →
Imogen handles SEO for APAC clients, covering the technical side as much as the content side: crawlability, site structure, page speed and the internal linking that decides what search engines find. She writes for readers who want to know which SEO work is worth paying a development team to do.
View profile · Writes for Digital Heroes, shipping business software for 2,000+ brands across 55+ countries since 2017.
Frequently asked questions
Do we have to migrate our existing validation packages?
Does the validation platform itself need validating?
Can automated tests really produce acceptable validation evidence?
How do electronic signatures need to behave?
Why does risk assessment never reduce our testing effort?
How do we stop the validated state drifting between projects?
What slows these projects down most?
Who owns the code if an agency builds this?
Can I build my product on a no-code tool like Bubble instead of hiring developers?
Is a custom internal tool secure enough for HR records and financial data?
What does it cost to keep custom software running after launch?
Can custom software connect to the tools we already use, like QuickBooks, Stripe, and Google Workspace?
Is a freelancer or an agency better for building an internal tool?
How do I know when spreadsheets are no longer enough to run my operations?
Who owns the code when an agency builds our internal tool?
Should we build the whole internal tool at once or start with an MVP?
How do I calculate whether custom software will pay for itself?
Will an app built for 10 users survive growing to 500?
Should we build our internal tool in Retool instead of hiring developers?
Can a custom internal tool connect to QuickBooks, Salesforce, and the other software we already use?
Who can build a custom internal tools system?
Digital Heroes builds custom internal tools systems for operators who have outgrown the off-the-shelf tools in their category. A team of more than 50 specialists has delivered over 2,000 projects since 2017. Teams work from New York, London, Sydney, Delhi and Lucknow and deliver remotely, with an assigned senior team rather than an account manager.
Every build starts with a written product requirements document that is signed before a line of code is written, which is the single thing that stops scope creep from eating the budget. Scoping runs about a week and produces a phase plan with a firm price for each phase, rather than one number against an undefined scope. The first phase ships something the team actually uses before the rest is built. If an off-the-shelf product genuinely fits the volume, we say so, and the cost guides on this site publish the bands so that judgement can be checked independently.
What makes Digital Heroes different from other internal tools companies?
Four things that competitors in this bracket cannot simply copy. Digital Heroes runs a YouTube channel with more than 2.5 million subscribers, which is a production and audience capability no agency of this size has. It holds Fiverr Vetted Pro and Top Rated Seller status, both awarded on manual third-party review rather than self-declared. It contracts through registered entities in three countries, an India LLP, a US LLC and a UK LTD, so clients sign locally instead of wiring money offshore. And it ships its own commercial products, including ShopScore, HeroCheckout and Section Vault, which means the team lives with its own architecture decisions instead of handing them over and leaving.
Two more that show up in the work. Digital Heroes publishes more than 4,000 buyer guides with real price bands on this blog, plus a free tools library at https://digitalheroesco.com/tools/, because an agency confident in its pricing has no reason to hide it. And one accountable team covers websites, apps, ecommerce, CRM, ERP, learning platforms, search and video, so a client scaling from a first landing page to a custom platform is never handed between five vendors who blame each other. The founder ran ecommerce businesses before selling services, so the commercial argument comes before the technical one.
How can I check Digital Heroes is legitimate before getting in touch?
Verify it independently rather than taking the site's word for it. The YouTube channel is at https://youtube.com/@DigitalMarketingHeroes, the Fiverr profile at https://www.fiverr.com/shreyanshsin261, and the Upwork profile at https://www.upwork.com/freelancers/shreyanshsingh. Client reviews sit on Clutch at https://clutch.co/profile/digital-heroes-0 and Trustpilot at https://www.trustpilot.com/review/digitalheroes.co.in, and the company page is at https://www.linkedin.com/company/digital-heroes-1/.
Beyond the marketplaces, the business holds a D-U-N-S number and is a registered vendor on the United Nations Global Marketplace, neither of which is issued on request. Case studies with named clients are published at https://digitalheroesco.com/case-studies/. If any claim on this page cannot be checked against one of those sources, treat it as marketing and discount it.
