Abstract and Peer Review Software: Why Blinded Routing and the Program Build Collapse in the Same Six Weeks

Why the annual meeting breaks a general event platform

Submissions close at 11:59pm on a Tuesday in November. By Wednesday your meetings coordinator has 2,800 abstracts in one system, a list of 600 volunteer reviewers with self declared expertise keywords, and six weeks to hand the education committee a scored, ranked, session assembled program. She is matching abstracts to reviewers with a spreadsheet formula that compares the first keyword on each side. It does not know that reviewer 214 trained under the senior author, or that two reviewers on the same panel are co investigators on the same grant.

The stack around this is usually an abstract tool such as Oxford Abstracts, Ex Ordo or ConfTool, a registration platform, a speaker inbox, a CME office spreadsheet for disclosures, and a meeting app vendor who wants a final export eight weeks out. Each holds a piece. None of them holds the object the meeting actually runs on: a submission that knows its authors and their institutions, the reviewers who must not see it, the scores it received and how harsh those particular reviewers are, the session it lands in, the room and time slot that session occupies, the presenter who cannot be in two rooms at once, and the embargo date attached to the press release.

The cost of that gap is not abstract. In the societies we have worked with, the recurring pattern is four to seven weeks of one senior staff member consumed entirely by the review and build cycle, a handful of conflicted reviews that have to be thrown out and redone after someone notices, and at least one scheduling collision discovered by a presenter rather than by staff. The meeting is often the single largest revenue line the society has, and the process that produces it exists mostly in one person's method.

Conflict of interest is a graph, not a checkbox

Every abstract platform on the market lets a reviewer click a button that says I have a conflict. That is self reporting after the fact, and it fires only when the reviewer already has the abstract open. What societies actually need is a rule set applied before assignment: no reviewer from the same institution, no co author within the last three years, no current grant co investigator, no thesis advisor relationship, no one from the same department at a different campus of the same system, and for some societies no reviewer employed by a company named in the abstract's funding statement.

Encoding that means holding a relationship graph and running the assignment against it, which is the thing off the shelf tools do not do. Oxford Abstracts and Ex Ordo will let you upload reviewer expertise and set a maximum load; ConfTool has genuinely capable bidding. None of them models a co authorship history pulled from the submissions of the previous three meetings, which is where most real conflicts hide. A custom build imports past submissions, derives the co authorship edges automatically, and treats a violated edge as a hard constraint the assignment engine cannot break rather than a warning someone will dismiss at 10pm.

A mean score is not a ranking

Two reviewers score the same abstract 6 and 9 on a ten point scale. The mean is 7.5, which is meaningless if the first reviewer has never given anything above a 7 and the second has never given below an 8. Sorting by raw mean rewards abstracts that happened to draw generous reviewers, and every program chair knows this and compensates with gut feel at the ranking meeting, which is the part they trust least.

What a custom build does here is not exotic statistics. It computes each reviewer's own distribution across everything they scored, normalizes within reviewer, and surfaces discordance explicitly: any abstract where the spread between reviewers exceeds a threshold you set goes to a third reader automatically rather than sitting in the middle of the list. It also separates the score from the decision, because most societies do not just take the top N. They take the top N per track, with a floor for underrepresented topics, plus committee picks, plus everything that meets a quality bar going to poster. Encoding your actual decision rules is a two week piece of work and it removes the annual argument about whether the ranking is fair.

The program build is a constraint solver you are running by hand

This is the part that no abstract tool touches and the part that costs the most staff time. You have 400 accepted oral presentations, 26 rooms of different capacities, four days, a plenary that blocks everything, sponsor sessions with contractual time slots, presenters who arrive Thursday and leave Saturday noon, session chairs who must not be double booked, tracks that should not run against each other because the same audience wants both, and a poster hall with a fixed board count over two sessions.

Staff solve this with a wall of sticky notes or a color coded grid, then spend a fortnight absorbing changes: a withdrawn presenter, a room swapped by the venue, a sponsor who buys a slot late. Each change ripples and nobody can see the ripple. A custom build states the constraints once and re solves in seconds, which changes the whole shape of the work. Room capacity against expected attendance becomes a solvable assignment rather than a guess. Presenter availability windows become hard constraints. Topic adjacency becomes a soft penalty the solver minimizes. When a withdrawal lands in week five, staff press a button instead of redrawing a grid.

Disclosures are evidence for credit, not a form

If your meeting offers continuing education credit, the ACCME Standards for Integrity and Independence require you to collect financial relationships from everyone in a position to control content and to mitigate relevant relationships before the activity. That is an evidence trail, not a checkbox. It has to attach to the presentation, be renewed if the presenter changes, be shown on the slide and in the app, and survive an audit two years later.

Most societies run this in a spreadsheet fed by an email chase, and the chase is brutal because the people least likely to answer email are the senior clinicians giving the keynotes. A custom build makes disclosure a record attached to a person with a validity window, propagates it to every role that person holds across the meeting, blocks the program export until the mandatory ones are complete, and produces the audit pack as a query rather than a fortnight of assembling PDFs. Document extraction is worth building here for one narrow job: reading the disclosure text people paste in from their institution's own system and normalizing company names so the same pharmaceutical firm does not appear under five spellings.

Late breakers, embargo and publication

Late breaking submissions are a different category with their own window, their own review path, usually a smaller and faster reviewer pool, and often a different acceptance bar. Off the shelf tools model this as a second submission form, which means the review rules, the numbering and the program insert are all manual again.

Embargo is the sharper risk. If your abstracts publish in a journal supplement with DOIs, and if press releases go out under an embargo date, then a single abstract visible in the app before the embargo lifts is a real problem with a journal and a sponsor. That needs publication state on the abstract itself, with the app and website reading a feed that respects it, not a staff member remembering to switch a folder to public at the right hour.

What a custom build has to include

• A submission object that versions cleanly, because authors will edit after the deadline and you need to know which version was reviewed
• A relationship graph derived from prior meetings, driving conflict rules as hard constraints on assignment
• Reviewer normalization, discordance detection, and automatic third reader routing
• Decision rules per track, including committee picks and oral to poster demotion, applied as a rule set rather than a sort
• A constraint based session and room builder that re solves after every change
• Disclosure records with validity windows, blocking export until complete
• Publication and embargo state on each abstract, driving app, website and supplement feeds from one source
• An export that the meeting app and the print vendor accept without a human reformatting it

What this costs and how long it takes

Across the 2,000 plus projects Digital Heroes has delivered, the honest shape here is a first release at $55,000 to $110,000 in 10 to 14 weeks, covering submission, conflict aware routing, scoring and decisions. That is the half that has a hard deadline, because your submission window opens on a fixed date and cannot slip. The session builder, disclosure management, embargo control and publication feeds bring the full platform to $140,000 to $320,000 phased across 5 to 10 months, and the sensible sequence is to ship review before one meeting and the builder before the next.

What pushes the number up: multiple meetings a year on the same platform with different rules, which sounds like reuse and is actually configuration depth. Integration with your AMS for member pricing and reviewer records, because iMIS, Fonteva and Personify are three different integration projects. A journal supplement pipeline with DOI registration. Anything involving the venue's own room management system. What keeps it down: running the first year with your existing decision rules unchanged, rather than redesigning the review process and the software simultaneously.

Build versus buy, honestly

Buy if you run one meeting a year, take under about 600 abstracts, use a single review round without a formal conflict policy, and build a program of under 60 sessions. Ex Ordo and Oxford Abstracts are good products for exactly that and cost a fraction of a build. Cvent Abstract Management makes sense if you are already deep in Cvent for registration and housing and your review process is simple, because the integration you avoid rebuilding is worth real money.

Build when two or more of these hold. Your conflict policy has rules a self declaration checkbox cannot express. Your program build takes more than three weeks of senior staff time. You award credit and the disclosure trail is assembled by hand each year. You run late breaking and regular submissions with different rules. Or your meeting revenue is large enough that a scheduling error is measured in refunded registrations rather than embarrassment.

How to choose a developer for abstract and program software

Ask them to whiteboard the model before you sign. A developer who has done this draws submission, submission version, author with affiliation, person, disclosure with a validity window, reviewer assignment, score, decision, session, slot and room, and they will immediately ask whether a person can hold several roles at one meeting. A developer who draws users and events has built a ticketing site.

Ask specifically how they will implement conflict rules, and listen for whether the answer is a graph traversal applied before assignment or a flag the reviewer sets afterwards. Ask how the session builder handles a withdrawal in week five. If the answer is that staff edit the grid, they have built you a nicer spreadsheet.

Ask what they have integrated. An AMS such as iMIS or Fonteva, a meeting app such as EventPilot or Whova, and a journal supplement with DOI registration are three distinct problems and you want the specific names. Finally, get code ownership in writing before kickoff. You should hold the repository and the cloud accounts, and be free to hire anyone else next year. At Digital Heroes the society owns the code from the first commit, and we would tell you to walk from a vendor who hedges on that, because a meeting platform you cannot move is a renewal negotiation you always lose.