Prompt, RAG, fine-tune, distill: the 2026 sequence and the RAFT tool nobody built

Two write-ups that crossed my desk on July 19 and 20, a BigData Boutique piece and the Winder.ai adaptation framework, say the same thing from different angles: production teams in 2026 have stopped arguing about RAG versus fine-tuning. The answer is a sequence. Prompt first, then retrieval, then a fine-tune, then distillation, each step taken only when the previous one stops paying. The number that matters in those write-ups is the cost ratio: a thin LoRA or QLoRA adapter on a strong base model, paired with retrieval, comes in 5-10x cheaper than a full fine-tune and is the highest-ROI move most teams have available.

Prompt, then RAG, then fine-tune, then distill

The order is not arbitrary. Each step fixes a different failure. Prompting fixes instruction and format problems and costs nothing but iteration. Retrieval fixes knowledge problems: the model does not know your contracts, your schema, your incident history. Fine-tuning fixes behaviour problems: tone, structured output that has to be right every time, domain vocabulary the retriever cannot paper over. Distillation fixes cost and latency once the behaviour is stable enough to compress into a smaller student. I covered the decision boundaries between the first three in fine-tune vs RAG vs prompt; what changed this year is that nobody serious treats them as alternatives anymore.

The same framing explains the hybrid pattern both write-ups call standard for legal, medical and finance: RAG plus a fine-tune, together. Retrieval supplies the citable facts; the adapter supplies the judgment about how to read them.

Where the ROI actually sits

The 5-10x figure is the argument I use with clients who want to jump straight to a full fine-tune. A LoRA adapter is a few hundred megabytes, trains in hours on one GPU or a large-memory Mac, and can be versioned, rolled back and A/B tested like any other artefact. A full fine-tune of a 70B model is a multi-day, multi-node job that produces new weights you now have to serve and re-validate. The adapter also keeps you on an open base: the July briefs name GLM-5.2 and Qwen 3.6 as the obvious MIT and Apache-licensed candidates, and both run locally on M-series hardware.

Retrieval quality is the other half of the ROI, and it is the half that gets neglected. A good adapter over a bad retriever learns to hallucinate confidently. Get hybrid search right before you train anything, because the adapter's training data is going to come out of that retriever.

Fine-tuning is not a replacement for retrieval; it is what you do once retrieval has shown you exactly which behaviours the model still gets wrong.

RAFT: the step nobody tooled

Retrieval-Augmented Fine-Tuning is the obvious next move: train the adapter on examples that include retrieved context, distractor documents included, so the model learns to use retrieval rather than merely tolerate it. Both July write-ups call it promising and both name the same blocker: there is no tool that takes a vector store plus query logs and produces a RAFT training set. Everything exists in pieces. The vector store has the chunks, the query logs have the real questions, the eval harness has the rated answers. Nobody has wired them into a dataset generator that a team without an ML engineer can run.

That is a product gap. The manual version: sample real queries from logs, retrieve top-k for each, label which chunks were relevant, generate a gold answer citing them, then inject one or two irrelevant chunks as distractors. Painful but mechanical, which is exactly what should be a tool.

What I would do this quarter

• Log queries and retrieved chunk IDs from day one; RAFT is impossible without them.
• Pick an MIT or Apache base (GLM-5.2, Qwen 3.6) so the adapter stays portable.
• Train a QLoRA adapter only after retrieval evals plateau; measure the delta, not the vibe.
• Keep adapters small and versioned; treat them like prompts, not like models.
• Distill only once the adapter has been stable in production for a month.

The honest gap

The 5-10x cost claim comes from the frameworks themselves, not from a benchmark I can reproduce, and it will vary with base size and hardware. RAFT's benefit over plain LoRA plus retrieval is a research result, not something I have measured on a client workload. Until someone ships that dataset tool, most teams will stop at step three, and that is fine.