Fine-tuning is continued training for a specific domain, and prompting is usually the right place to start.
2
SFT imitates exact examples, DPO learns preferences between paired outputs, and RFT trains reasoning through a grader.
3
Fine-tuning works best when the task, data, and evaluation signal are clear, while noisy or subjective labels can make results worse.
Summary
Ilan Bigio explains how to choose between prompting and OpenAI's fine-tuning methods. Supervised fine-tuning, or SFT, teaches a model to imitate specific outputs and works well for classification, formatting, structured extraction, and distillation. Direct preference optimization, or DPO, learns the direction of a preference between two responses, which makes it useful for tone and style. Reinforcement fine-tuning, or RFT, trains reasoning models against a grader and fits hard problems with unambiguous answers. Bigio stresses that fine-tuning does not add large bodies of knowledge efficiently, so retrieval and tools are usually better for that. His examples include distilling GPT-4 function calling into GPT-3.5, tuning joke quality with DPO, and a failed email classifier caused by subjective, noisy labels. The practical advice is to begin with prompt engineering and evals, then fine-tune once prompting has reached its limits and the data is ready.
Fine-tuning changes model behavior, while prompting changes the instructions around it
Bigio describes every language-model application as three parts: the input, the model, and the surrounding system of tools and scaffolding. The input includes the prompt and context. The model contains its pretrained weights. The application adds tools and orchestration around it. Fine-tuning focuses on the model weights and model choice, while prompt engineering works on the input and scaffolding. Fine-tuning is continued training for a domain. Prompting is quick and cheap to change, and it is enough for most jobs. Fine-tuning requires clean data and longer iteration cycles, but it can automate a repeated behavior and reach results prompting cannot achieve.
SFT, DPO, and RFT learn different kinds of signals
Supervised fine-tuning uses input-output examples and teaches imitation. Direct preference optimization receives an input, a preferred response, and a non-preferred response, then learns the difference between them. Reinforcement fine-tuning receives inputs and a grader, optionally with reference answers, and trains a reasoning model toward higher scores. Bigio describes SFT as learning a soft mapping from inputs to outputs. DPO learns the delta between two outputs. RFT changes the model's chain of thought through reinforcement from successful solutions.
SFT fits constrained outputs and small, repeatable tasks
Bigio recommends SFT for classification, formatting, and structured data extraction. It is useful when the desired output is tightly constrained and the model needs to imitate a known pattern. He also presents it as a route for distillation, such as teaching a smaller model to perform a structured task when prompting is not enough. SFT can use around 50 to 100 examples before signs of improvement appear, with a few hundred examples often more suitable for production. He warns that models can regress on tasks absent from the fine-tuning data, so data diversity matters.
Fine-tuning is poor at teaching large amounts of new knowledge
OpenAI's fine-tuning approaches update only a subset of the model through low-rank adaptation. Bigio explains that the model weights are decomposed into two matrices, producing a smaller component to update. The base model keeps its earlier behavior while the fine-tune adds some new behavior. This makes fine-tuning useful for methods and patterns, but it is not a good way to load a large knowledge base into a model. Bigio recommends retrieval-augmented generation and agentic retrieval when the goal is to provide new information.
Synthetic data can distill a large model into a faster smaller model
Bigio describes a function-calling project with roughly 100 to 120 functions and a strict latency requirement. The team lacked a large set of labeled examples, so they generated function-call permutations from the customer schemas with a Python script. They then asked GPT-4 to produce natural-language commands that would lead to those calls. The team also ran unlabeled customer inputs through GPT-4, filtered the results, and combined both data sources. They fine-tuned GPT-3.5 and reached GPT-4-level performance for the constrained function-calling task at much lower latency. Bigio says the approach improved both function selection and parameter filling.
DPO learns preference direction and needs meaningful comparisons
Bigio uses joke generation to explain DPO. He collected better jokes, extracted their topics, and asked GPT-4 to generate weaker jokes for the same topics. Each training item therefore had an input topic, a preferred joke, and a non-preferred joke. This pairing tries to isolate the difference between a funny response and a mediocre one. DPO is more forgiving about diversity than SFT because it learns a preference direction rather than a specific output format. However, synthetic preference data is only as good as the judgments behind it. Real human comparisons or another natural signal give a cleaner training signal.
RFT depends on an unambiguous grader and low-noise data
RFT generates a chain of thought and an answer for each input, then uses a grader to assign reward. Successful reasoning patterns become more likely on later attempts. Bigio says the task needs unambiguous grading, very low-noise data, and problems where the model can reach a verifiable solution. With 40 to 80 high-quality examples, RFT can learn to reason over related examples. Graders can be string checks, text-similarity measures, sandboxed Python code, model-based scores or labels, and combinations of these. The grader should express what good performance actually means.
Subjective email labels produced a poor RFT result
Bigio labels about 600 of his own emails into actions such as glance, ignore, archive, store, respond, or take action later. His RFT email classifier performs poorly, with results around the mid-50 percent range. He attributes this to subjective decisions, inconsistent labels, a few mistakes, and a prompt that did not explain his preferences well. He admits that he might not assign the same category to similar emails a second time. The example shows why RFT amplifies noisy reward signals. A reasoning model cannot reliably learn a preference that its creator has not defined consistently.
Prompt engineering and evals should come before fine-tuning
Bigio's practical recommendation is to avoid fine-tuning until it is needed. Start with prompt engineering and measure the result with evals. If changing the prompt still improves performance, there is no reason to fine-tune yet. Consider fine-tuning when prompting has reached its limit, the task has enough clean data, and the team can afford the longer training cycle. He also recommends keeping the fine-tuning prompt at inference time. Changing it can put the model out of distribution, so a materially different prompt may require another fine-tuning run.
"You really want to evaluate, right? If your evals are showing that you are reaching the limits of what you can accomplish through prompting, but you know it's possible to do better, then that's when you can pull out fine-tuning if you have the data."1:02:38
Who should watch
You are deciding whether to spend time building a fine-tuning pipeline or continue improving prompts and retrieval.
Your task has clean examples and a constrained output, such as classification, structured extraction, or function calling.
You want to try reinforcement fine-tuning and need to understand why grader quality and label consistency matter.
