Hire AI Engineers: Rates, Vetting and Engagement Models

What an AI engineer does, and where hiring goes wrong

An AI engineer is not a data scientist and is not a machine learning researcher. They are a software engineer whose job is to make probabilistic components behave predictably inside a normal production system. On a real project that means designing retrieval so the model sees the right context, writing an eval suite that catches quality regressions before your users do, handling the failure modes that only exist with language models (hallucinated citations, tool calls with malformed arguments, context windows blowing past limits, prompt injection through user-supplied documents), and keeping token spend from quietly becoming your second largest cloud line item.

We have walked into this exact situation more than once. A client comes to us with a support assistant built by a contractor over six weeks. The demo was excellent. Then the contractor left. Now the answers are wrong maybe one time in eight, and nobody can tell whether that is new or whether it was always that bad, because there is no eval set. The retrieval is a single call to a vector store with top_k set to 5 and no reranking, so a question about the refund policy pulls three chunks about shipping. The prompt is a 900 line f-string with no versioning, so nobody can safely change a word of it. Every request goes to the largest available model because the contractor never measured whether a smaller one would do, and the monthly bill is $11,000 for a feature with 400 daily users. None of this is exotic. It is what happens when you hire someone who can build a demo and assume that is the same skill as building a system.

The gap between those two skills is the whole hiring problem. Demos reward prompt intuition. Production rewards measurement discipline. Plenty of people have the first, and almost nobody advertises the absence of the second.

Engagement models and the trade-offs

In-house hire. Right when the AI system is the product, not a feature on top of it. Someone who lives with your evals for two years builds an intuition about your domain that no contractor gets. The cost is real: this is one of the harder roles to fill right now, and good candidates have options. Expect a long search and expect to lose candidates to companies paying more.

Freelancer. Good for a bounded, well-specified piece: build the ingestion pipeline, get the eval harness running, port a working prototype to a production API. Risky for open-ended work. The failure mode is not that freelancers are bad, it is that AI systems degrade silently and a freelancer is not there in month four when your document corpus changes shape and retrieval quality quietly craters. If you go freelancer, get the eval suite and the prompt versioning as explicit deliverables, not as a nice-to-have.

Agency. Fits when you want the outcome and not the management overhead: a scoped system, delivered, with a tech lead reviewing the work. You pay a premium over a raw freelance rate and you get review, continuity when someone is sick, and someone who has already made the mistakes on other people's projects. The trade-off is less day-to-day control and a harder time changing direction mid-build if the scope is fixed.

Staff augmentation. Our most common arrangement for this role and usually the right default. An engineer or a small pod works inside your team, your standups, your repo, your ticket board, but they come with an agency behind them for review and replacement. You get in-house-like integration without the recruiting cycle or the severance risk, and you can size up or down as the project shifts. It fits the reality of AI work, where scope changes every time a new model release moves the goalposts.

Rates and what it really costs

These are the bands we see in our own delivery and in the market we hire from. Rates move with region and seniority, and both matter more here than in most roles.

• India, South Asia, Eastern Europe: roughly $45 to $95 per hour. The top of that band is strong. The bottom is often someone who has shipped chatbot demos and not production systems.
• Latin America, Southern and Western Europe: roughly $80 to $160 per hour, with timezone overlap being a real part of what you are paying for.
• US and UK senior specialists: roughly $120 to $220 per hour, higher for people with a track record on systems at scale.

For an in-house hire, the salary is not the cost. Benefits, payroll taxes, equipment, software and workspace overhead push the base up meaningfully, and when we model this with clients we plan on a loaded figure well above the offer letter number. Then add recruiting, whether that is an agency fee at a percentage of first-year salary or your own team's hours running a search that will realistically take two to four months for this role in the markets we hire from. Then add ramp: even a strong AI engineer needs six to ten weeks before they understand your domain well enough to write a useful eval set, and evals are where their value comes from. And add the tooling: an observability layer for LLM traces, an eval runner, a vector store, and the model API spend itself, which for a system under real load is not a rounding error.

The comparison people get wrong is hourly rate versus salary. Compare fully loaded annual cost against a contract that delivers a working, measured system in twelve weeks, and the arithmetic often looks different than expected.

How to vet an AI engineer

Skip anything about transformer internals. Nobody is training a base model on your project. Vet for production judgment.

Ask: walk me through your eval setup on the last thing you shipped. This is the most diagnostic question in the interview. A weak candidate says they tested it manually and it worked well. A strong one talks about a golden set of a few hundred labeled examples, what the pass criteria were, why they chose LLM-as-judge for some checks and exact string or structural assertions for others, how they calibrated the judge against human labels, which runner they used (Ragas, promptfoo, DeepEval, Braintrust), and how the suite ran in CI so a prompt change could not merge without a score. If they cannot describe an eval loop, they build demos.

Ask: your RAG system returns the wrong chunk. Debug it out loud. Listen for whether they separate retrieval failure from generation failure before touching anything. Good answers reach for retrieval metrics first, then chunking strategy (fixed-size versus semantic versus structure-aware splitting, and why), then whether embedding-only search is the problem and a reranker such as a cross encoder or Cohere Rerank would fix it, then hybrid search combining BM25 with vector scores, then query rewriting for multi-hop questions. A weak answer jumps straight to changing the prompt or swapping the embedding model.

Ask: how did you get token cost down on a real system. Real answers are specific: routed easy requests to a smaller model and kept the big one for hard ones, restructured prompts so the static instruction block sits at the front and hits the cache, trimmed retrieved context because more chunks were not improving accuracy, batched offline work. Generic advice about efficiency is noise.

Ask: how do you make a model call return reliable structured output. They should reach for structured output modes or tool schemas rather than parsing JSON out of prose, plus schema validation with Pydantic or Zod, plus a defined behaviour on validation failure. If they say they regex the response, that is your answer.

Ask: a user pastes a document that contains instructions to the model. What happens. This tests whether they think about the security boundary at all. You want to hear that retrieved and user content is data, not instruction, plus how they keep that boundary in the prompt structure and in what tools the model is allowed to call.

The take-home. Give them a small, messy corpus, thirty of your own real questions with correct answers, and about four hours. Ask for a system that answers them plus an eval report. Do not grade the answers. Grade the report. The best submissions come back saying the system scores 71 percent, here are the eight it failed, six are retrieval misses on multi-hop questions and two are the chunker splitting a table across boundaries, here is what I would fix first. That candidate is worth hiring. The one who submits a polished demo and no numbers is the contractor from the opening scene.

Portfolio review. Ask to see the repo, not the demo video. Look for a tests or evals directory. Look for prompts stored as versioned files rather than inline strings. Look for tracing with something like LangSmith or Langfuse around model calls. Look at the git history to see whether quality was measured over time or whether the whole thing appeared in three commits the week before the deadline.

Red flags

• "I fine-tuned a model for it." Fine-tuning is right for narrow, high-volume, stable tasks, and wrong for most of what companies want. If it is their first instinct instead of better retrieval or better prompting, they are reaching for the expensive tool. Ask instead: what did you try before fine-tuning, and what did the evals say about each.
• Everything is a LangChain or LlamaIndex abstraction and they cannot explain the layer underneath. Frameworks are fine. Not knowing what the actual API request looks like means they cannot debug it when the abstraction leaks, which it will. Ask instead: show me the raw request your chain produces, and what happens when the tool call comes back malformed.
• No opinion on cost. Anyone who has run an AI feature under real traffic has a scar from a bill. Ask instead: what did your last system cost per thousand requests, and what did you do about it.
• They talk about accuracy without an eval set. "It works really well" is not a measurement. Ask instead: what is your baseline number, and how would I know if a prompt change made it worse.
• They cannot name a failure mode of their own system. Every deployed AI system has known weak spots. Someone who describes theirs as reliable across the board has either not shipped it or has not looked. Ask instead: what does it still get wrong, and why have you not fixed it.

When to hire this role, and how Digital Heroes staffs it

Do not hire an AI engineer to build an internal tool that a strong backend engineer could build with an API call and good prompt hygiene. Plenty of AI features are one model call inside otherwise ordinary CRUD, and a good full stack developer will ship that faster and cheaper. Do not hire one when your data is the actual problem: if your documents are inconsistent, unlabeled and scattered across four systems, no retrieval architecture saves you, and a data engineer comes first. Do not hire one if what you need is a model trained on tabular data to predict churn, because that is a machine learning engineer and a different person.

Hire an AI engineer when the quality of the model output is the product experience, when you need retrieval or agentic tool use rather than a single call, or when something already built has quietly stopped working and nobody can measure why.

At Digital Heroes we staff this as a small pod rather than a lone engineer, because the work splits naturally: an AI engineer on retrieval, prompting and evals, a backend engineer on the surrounding service, data pipeline and API surface, and a tech lead reviewing architecture and cost decisions. The first deliverable on almost every engagement is the eval harness and a baseline number, before anyone touches the prompt, because without that every subsequent change is guesswork. Most engagements run as staff augmentation inside the client's repo and standups, which keeps the team building alongside yours rather than tossing a black box over the wall. Code, prompts, evals and infrastructure are yours from the first commit.