LLM applications need explicit tests because teams cannot safely change prompts, configurations, or model providers without knowing what good output means.
2
llmeval is a configurable command-line tool that creates folders for prompts and tests, runs repeated samples, and produces reports for individual runs and the application overall.
3
Model-based grading can produce richer feedback than pass or fail, but it has biases, so human feedback can be modeled to help review new completions.
Summary
Niklas Nielsen presents llmeval, a command-line tool for testing and reporting on large language model applications. He starts with a simple arithmetic prompt and shows how small prompt changes can make GPT-3.5 and Claude return extra text instead of the exact answer. llmeval supports configurable tests, custom Python metrics, repeated samples, and reports that separate individual runs from the overall test history. Nielsen also demonstrates testing generated Python code by checking whether the result can be parsed, then shows model-based evaluation for grading Mermaid diagrams on a scale from one to five with a written reason. He is candid about the weaknesses of model judges, including a tendency to prefer their own outputs and difficulty giving reliable numeric scores. Log10's approach also combines prior human feedback with model-generated suggestions, leaving a person involved in reviewing incoming completions.
LLM products need a definition of good output before teams change them
Nielsen says teams have reached a "now what" point after shipping many intelligence features based on GPT. Without knowing what good means for a generative application, changing prompts and configurations becomes risky. The same problem appears when moving between model providers or adopting more advanced techniques such as self-hosting and fine-tuning. His proposed answer is to turn expected behavior into tests that can be run repeatedly as the application changes.
llmeval creates a local structure for prompts, tests, and reports
llmeval is introduced as a command-line tool that can run locally. Initialization creates folders and suggested practices for storing prompts and tests. The system uses Meta's Hydra for configuration, so teams can extend its settings. Metrics are written in Python and can contain any logic, including calls to language models. After evaluation runs, the tool generates reports that give an overview of the application and its tests while allowing flexible criteria for fuzzy model outputs.
Repeated samples give a better view of unstable tests
Nielsen explains that a test does not always need to pass on every single sample. For a fuzzy model behavior, a team might decide that three out of five cases should pass. llmeval runs five samples by default, which gives a better read on the stability of a test, although users can change that setting. The command line can run the full suite or select a prompt such as the math example, and configuration values can be overridden for a particular run.
Small prompt changes can turn an exact-output test into a failure
The demo uses a prompt asking for the answer to an addition problem without an explanation. The test compares the model output with the expected answer and strips leading and trailing spaces, since some models add them. Nielsen removes the instruction to return only the answer, then runs the test across Claude, GPT-4, and GPT-3.5. GPT-3.5 becomes chatty and returns text such as "4.5 equals 9," while Claude also writes out the equation. Restoring the prompt makes the most recent run pass.
Reports preserve both the latest run and the application's history
The reporting command shows which test failed and what the model actually returned. A report is generated for each run, and the overall report also says whether anything has failed across those runs. This gives the team a way to distinguish a passing recent change from a suite that still contains failures in its history. The report connects a test result to the prompt behavior that caused it.
Nielsen moves to a tool-use example that asks the model to generate Python code. The prompt needs several clauses because the model tends to surround the code with explanations. The test checks whether the returned text is an actual Python program that can be parsed. In the demonstrated report, the tool-use tests pass. The example shows how a metric can inspect a structural property of the output rather than compare it only with one exact string.
Model judges provide richer scores, with serious sources of bias
Model-based evaluation uses one language model to judge another model's output. It can produce pass or fail results, a score from one to five, a preference between options, and reasoning. Nielsen demonstrates grading Mermaid diagrams with criteria and a one-to-five score. He also describes problems with this method. Models can favor their own output when different providers are compared, and they are poor at assigning dependable point scores, especially across wider ranges such as zero to one hundred.
Human feedback can guide suggested reviews without removing people
Log10's approach combines model-based evaluation with earlier human feedback. Feedback from a person such as John can be modeled into an automated version of that person's review style. New completions can then receive suggested feedback for review. Nielsen shows two existing human judgments, one simple score and one more detailed comment, alongside a pending completion with an AI-suggested answer. The person remains part of the review process rather than being removed from it.
"Instead of removing the human completely from the feedback you start taking in all feedback that might have been given prior and start modeling it."08:16
Who should watch
You are changing prompts or model providers and need repeatable checks for regressions in output format or behavior.
Your application generates code or other structured artifacts that need tests beyond exact string matching.
You are considering an LLM judge and need to account for self-preference, weak numeric scoring, and the value of human review.
