Hire LLM and Generative AI Engineers

What an LLM engineer does, and where hiring goes wrong

A fintech client came to us with a support assistant that a contractor had built over roughly four thousand help center articles and policy PDFs. The demo was flawless. Six weeks after launch, support agents had quietly stopped using it, because it kept citing the wrong version of the refund policy. The cause was not the model. The documents had been split with a naive 512 token character splitter, so policy tables broke mid row and lost their headers. There was no reranker, so the top five chunks by cosine similarity were near duplicates from three different policy years. There was no metadata filter on effective date, so a superseded 2023 policy outranked the live one. And there was no evaluation set at all, which is why nobody caught any of it until humans complained.

The missing evaluation set is the part that matters most. An LLM engineer is not someone who calls an API. They are someone who turns a probabilistic model into a system whose behavior you can measure and defend. In practice that means: designing the retrieval layer (chunking strategy, embedding model choice, hybrid BM25 plus vector search, reranking, metadata filters), assembling context under a token budget, enforcing structured output contracts with JSON schema or constrained decoding, designing tool calling and agent control flow so it terminates instead of looping, guarding against prompt injection and PII leakage, building an evaluation harness against a labeled set, wiring tracing so you can see what the model was actually sent, and engineering cost and latency through prompt caching, model routing and streaming.

Hiring goes wrong when you buy demo skill instead of evaluation skill. Wiring LangChain or the Vercel AI SDK into a chat UI over a weekend is a solved problem, and a large share of candidates have done exactly that and nothing more. The engineer worth the money is the one who, before writing a prompt, asks what your accuracy target is, what a wrong answer costs you, and where the ground truth labels are going to come from.

Engagement models: in house, freelancer, agency, staff augmentation

In house hire. Right when the AI feature is the product, not a feature of the product, and when the model layer will be under continuous change for years. An in house engineer accumulates the thing that matters most and transfers worst: knowledge of your data, your users' weird queries, and why the last four prompt changes regressed. The trade offs are cost, and a hiring market where you are competing with companies paying research level compensation. You will also interview a lot of people who have read every paper and shipped nothing.

Freelancer. Right for a bounded, well specified piece of work: an evaluation harness, a retrieval rebuild, a fine tune on a dataset you already have, an inference cost reduction pass. The trade off is depth of accountability. LLM systems degrade silently when your data or the underlying model version shifts, and a freelancer who has moved on is not going to notice your recall dropping. If you go freelance, buy a deliverable, not a body.

Agency. Right when you need the whole system, not just the model layer: ingestion pipelines, vector store operations, the application around it, auth, billing, and the evaluation and monitoring that keeps it honest. Very few AI features fail on the AI. They fail on the data plumbing and on the absence of a feedback loop. An agency gives you a team with those roles already in it, and someone who owns the outcome. The trade off is a higher hourly rate than a lone contractor and the need to define scope properly up front.

Staff augmentation. Right when you have engineering leadership and a working product, and you need one or two people with specific LLM experience embedded in your team on your board and your standups. It is the cheapest way to buy senior judgment without a permanent hire. The trade off is that you have to manage them, and if your own team has no opinion about evaluation, the augmented engineer's good habits will not survive their departure.

Rates and what it really costs

These ranges come from Digital Heroes delivery work and from what we see when we bid against other shops. They are not a survey. Rates move with region, seniority and how much of the problem you are handing over.

• India and South Asia: roughly $25 to $60 per hour. Strong applied engineers exist here in volume. So do people who have completed one RAG tutorial.
• Eastern Europe and Latin America: roughly $45 to $95 per hour.
• United States, United Kingdom, Western Europe: roughly $100 to $200 per hour for a solid applied engineer, and $200 to $250 plus for people doing serious inference optimization, training work or safety critical evaluation.
• Marketplace listings under about $20 per hour: almost always wrapper level work. You will get a chat UI over an API key and no evaluation of any kind.

For a US in house hire, the salary is the smaller half of the story. Benefits, payroll taxes, equipment, software and general overhead push the fully loaded cost well above base, and when we model this with clients we plan on roughly 1.25 to 1.4 times base. Add a recruiting fee if you use an agency, commonly a fifth to a quarter of first year base. Then add ramp: for this stack we consistently see eight to twelve weeks before a new hire is making confident changes, because they have to learn your domain data and your users' query distribution before their instincts are worth anything. Prompt tuning without domain knowledge is guessing.

The line most buyers forget entirely is the running cost of the system itself. Inference spend, embedding regeneration when you change models, vector database hosting, evaluation runs (which are themselves model calls, often the expensive kind), and observability tooling. On a mid sized RAG product, that bill can rival a junior salary. An engineer worth hiring will bring this up before you do.

How to vet an LLM and generative AI engineer

Ignore the model name dropping. Look for these signals.

They lead with evaluation. Ask: "How did you know your last system was working?" A weak answer is "we tested it" or "the outputs looked good." A strong answer names a golden set they built by hand, the metrics they tracked (retrieval recall at k, faithfulness, answer correctness, refusal rate), the tooling (Ragas, promptfoo, DeepEval, LangSmith, Langfuse, Braintrust), and the regression suite they ran on every prompt change. Follow up: "How did you calibrate your LLM as a judge?" If they have never questioned whether the judge agrees with humans, they have been grading their own homework.

They can debug retrieval, not just prompts. Ask: "Users say the answer is wrong. Walk me through your diagnosis." You want to hear them separate retrieval failure from generation failure before touching the prompt: check whether the correct chunk was even in the context, inspect the top k, test hybrid BM25 plus vector against pure vector, try a reranker such as a cross encoder or Cohere Rerank, check chunk boundaries against the real document structure, check metadata filtering. Someone who jumps straight to "I would improve the prompt" has never run a real RAG system.

They have opinions about when not to use an LLM. Ask: "When would you refuse to build this with RAG?" Good answers: when the query is really a SQL aggregation, when the corpus is small enough to fit in context with prompt caching, when the task is classification and a fine tuned small model or a classifier is cheaper and better.

Fine tuning judgment. Ask: "When do you fine tune versus prompt versus retrieve?" The answer should be that fine tuning teaches form and behavior, not facts, and that retrieval is for facts. Ask what they used, LoRA through PEFT, TRL, Unsloth, Axolotl, and how many labeled examples they had. If someone proposes fine tuning to fix hallucinations, that is the whole interview.

Production and cost engineering. Ask what their p95 latency was and how they got it there. Listen for time to first token versus total, streaming, prompt caching, semantic caching, routing cheap queries to a small model, batching. If they self host, ask about vLLM or SGLang, continuous batching, KV cache, and quantization tradeoffs with AWQ or GGUF.

Security. Ask how they handled prompt injection in a tool calling agent. You want to hear about untrusted content never being given authority, tool permission scoping, human approval on side effecting actions, and output validation. Anyone who says "we told the system prompt to ignore injections" is dangerous with a tool loop.

The take home. Do not ask them to build a chatbot. Give them two hundred real documents from your domain, twenty five questions with correct answers you wrote yourself, and a small budget. Ask for a system plus an evaluation report. Score the report, not the demo. The best candidates come back and tell you that four of your twenty five questions are unanswerable from the corpus. That is the hire. For a portfolio review, ask to see a trace, an eval dashboard, or a before and after accuracy number on something real.

Red flags

• No evaluation set, ever. They describe quality in adjectives. Ask instead: "Show me the last regression your eval suite caught."
• Fine tuning as the default answer. Proposing a fine tune before retrieval and prompt work is usually resume building on your budget. Ask instead: "What would you try first, and what would prove it insufficient?"
• Framework fluency with no system underneath. They can recite LangChain, LlamaIndex, LangGraph and CrewAI but cannot explain what their agent does when a tool returns an error, or how it terminates. Ask instead: "What was your maximum loop depth and why?"
• Cosine similarity as the only retrieval tool. No BM25, no hybrid search, no reranker, no metadata. Ask instead: "How did you handle a query where the right document does not share vocabulary with the question?"
• Benchmark talk instead of product talk. Deep on public leaderboards, empty on what their users actually asked. Ask instead: "What was the most common failure your real users hit, and what did you change?"

When to hire this role, and how Digital Heroes staffs it

Do not hire an LLM engineer to add a chat box to a product that has no data problem. If your need is "summarize this" or "draft that" against a small, clean input, a competent full stack engineer with API access will get you 90 percent of the way, and you should hire that instead. If your need is search over a messy corpus, a good backend engineer plus pgvector will beat a mediocre AI specialist. If your need is a classifier or an extraction pipeline over structured documents, you may want a machine learning engineer, not a generative AI one. Hire an LLM engineer when correctness is contested, when retrieval quality is the product, when an agent takes actions with consequences, or when inference cost is becoming a line item you are worried about.

Digital Heroes does not sell you a lone prompt wizard. An engagement is an applied LLM engineer paired with a backend engineer who owns the ingestion and data layer, because that is where these projects fail, plus a delivery lead who makes sure an evaluation set exists in week one rather than after your users find the bugs. The first milestone is almost never the feature. It is a labeled question set and a harness that produces a number, so that every change after that is an argument you can win with data. Code, prompts, evaluation data and any trained weights are yours from the first commit, in your repository. If the answer is that you do not need this role yet, we would rather tell you that than bill you for it.