Penetration Testing Delivery Software Problems: The 7 That Cost Consultant Days, and How to Avoid Them

Why does report template fidelity get underestimated in almost every quote?

Because it looks like formatting. A developer sees a Word document, sees a templating library that produces Word documents, and prices two weeks. What is actually in that document is a decade of accumulated client expectation: numbered heading styles that continue across appendices, a findings table that repeats its header row across page breaks, a severity chip whose colour matches the firm's brand, an executive summary that renders differently for a retest than for an initial engagement, and evidence images that must not push a finding onto an orphan page.

The consequence is specific and predictable. Consultants do not complain in a meeting. They export from the platform once, spend forty minutes fixing it in Word, and never mention it. Adoption dies silently, and the practice lead finds out at the quarterly review when the finding library still has eleven entries in it.

The fix is to invert the build order. Before any workflow is designed, take a real, recent, signed off report and reproduce it from structured data end to end, including the appendix and the retest variant. If the team cannot do that in the first three weeks, the rest of the project is at risk regardless of how good the feature list looks. Ask for that as a paid discovery deliverable if necessary, because it is cheaper than finding out in month five.

What goes wrong when you try to import historical findings from old reports?

Every firm wants the back catalogue in the new system, because trend reporting across engagements is the commercially valuable output and it needs history. The problem is that the history is prose. Five years of Word documents contain findings written by twenty different consultants in twenty different voices, with severities assigned by feel, evidence embedded as cropped images with no reproduction step, and no stable identifier linking the same issue across two engagements for the same client.

Firms attempt this in one of two ways and both go wrong. Manual re-keying gets scoped as a few weeks of junior time, then stalls at the point where somebody has to decide whether two differently worded findings are the same issue. Automated extraction produces a database full of plausible looking records that nobody trusts, which is worse, because the first trend report built on it will be wrong in a client meeting.

What works is a deliberate cut. Import structured metadata only for old engagements: client, date, scope, finding count by severity as recorded at the time, and a link to the original document. Do not attempt to normalise old finding text into the new taxonomy. Then start the real library from the current quarter forward, and accept that meaningful cross engagement trend reporting begins twelve to eighteen months after launch. Firms that accept that timeline get a clean data set. Firms that insist on backfilling get a dirty one and lose six weeks doing it.

Why do scanner imports and ticketing integrations break after launch?

Scanner output is the classic case of an integration that works in the demo and degrades quietly in production. Each tool exports its own shape, versions change those shapes without announcement, and the same underlying issue is described differently by two tools on the same host. A parser written against one export of one version will keep running after an upgrade and simply produce fewer findings, which nobody notices because nobody is counting what did not appear.

Ticketing is worse in a different way, because it involves someone else's system. Pushing findings into a client's issue tracker is a genuinely useful feature and it is the one most likely to break, because the client changes their project configuration, renames a field, tightens permissions on the service account, or moves to a different instance, and none of that is under your control. The failure is not an error message, it is a finding that silently never arrived, discovered when the client asks why remediation was never raised.

The fixes are unglamorous. Version detect on import and fail loudly on an unrecognised format rather than parsing what you can. Reconcile counts: if the file contains 214 findings and you created 190 records, that difference gets surfaced to the consultant before they proceed. For ticketing, treat every push as a job with a persistent status and an alert on stale queues, and give the client a reconciliation view listing findings that have no corresponding ticket. Assume the integration will break, and design so that its breaking is visible within a day.

What happens when tenant isolation and evidence retention are not covered?

These are the two requirements where a shortcut is not a technical debt, it is a business ending event, and both are routinely left until the client portal ships.

Isolation has to be enforced at the data layer, with tests that prove a query executed in one tenant's context cannot return another tenant's rows. Filtering in application queries works right up until one developer writes one query without the condition, and the failure mode is a client seeing another client's vulnerabilities. There is no recovery from that in a testing practice. If a developer describes it as an ordinary access control problem rather than a data layer boundary designed on day one, that is disqualifying.

Evidence retention is the one your own clients will ask about before you think of it. Screenshots and request captures from an engagement frequently contain live customer data, session tokens and internal hostnames. Once that material is in your platform rather than on a consultant's laptop, you have taken on custody of it, which means you need per client retention rules, deletion that actually deletes including from backups, and automated detection of common sensitive patterns before a report renders. Regulated clients will put all of this in a supplier assessment, and a build that has no answer will cost you the account it was meant to serve.

Should you build custom or configure what you already own?

Most testing firms should buy, and we say so before quoting. PlexTrac is a mature product built for exactly this workflow, with a reusable finding library, evidence handling and reporting that a firm of fifteen consultants will not beat with a build. AttackForge is strong on workflow and automation. Dradis has done the finding library well in the open source world for years and is worth a look if you have engineering appetite and want to avoid per seat costs.

Configure rather than build when your dissatisfaction is about report formatting or a missing field, because both are configuration problems in the mature tools and neither justifies a platform. Configure when your finding library is inconsistent, because that is a content and editorial problem, not a software one, and no build fixes a taxonomy nobody has written down.

The build case is arithmetic. It appears when your methodology and finding taxonomy are what you sell against and a generic library dilutes them, when per consultant and per portal user licensing has become a real line item, when enterprise clients need a portal authenticated against their own identity provider, when you have a commercial model such as retainer day drawdown that your tooling cannot express, or when data residency obligations for regulated clients rule out a hosted platform. Two or more of those, at twenty or more consultants, and the numbers work. One of them, at ten consultants, and they do not.

How do hidden costs get into the quote?

Five items, and each is usually a single line in a proposal.

The report template. Discussed above, and consistently the largest underestimate in this category. Price it as its own workstream against your real document.

Per client single sign on. Every enterprise client wants their own identity provider, and each one is an onboarding exercise with the client's own IT team, not a setting. Ask how many the quote assumes.

Scanner parsers. Each tool is its own parser with its own maintenance. A quote saying scanner ingestion without naming the tools is quoting an unknown.

Multi tenancy done properly. Adding a hard data layer boundary after the fact is a rebuild of the data access layer, so it either exists from day one or it costs several times more later.

Evidence storage and residency. Storing evidence for regulated clients under their residency terms can mean a second deployment region, which affects hosting, backups and operations for the life of the platform.

What separates a build that works from one that fails here?

Consultant adoption first, portal second. A client portal sitting on top of a platform your own people avoid is money spent on a facade. Ship the finding library, structured evidence and report generation, watch three real engagements run through it end to end, and only then build the client facing side.

Restraint about where AI belongs. Drafting a finding narrative from structured evidence in your house voice, rewriting technical detail into a business impact paragraph, and clustering duplicate findings across a large scope are all useful and safe because a consultant reviews before anything ships. Generating findings, inventing impact or assigning severity without a human is not acceptable, because your signature on that report is the product. A developer who offers the second set has not understood what you sell.

Scoring that encodes your own matrix rather than a base score. A vulnerability score describes the vulnerability, not the client's exposure, and the reason your seniors argue in review is that the reasoning lives in prose. Encode asset exposure, data classification, exploit complexity and compensating controls as inputs, let the rating fall out, and have the report explain the reasoning in your language.

Ownership settled before kickoff: the repository, the infrastructure accounts and the right to hire another firm. At Digital Heroes the client owns the code from the first commit, and for a platform holding client vulnerability evidence we would expect your own team to test it before go live, which is exactly what we would demand if we were the client.