WHITEBOXX / GLOSSARY
LLM evaluation: testing AI
before you rely on it.
LLM evaluation is the testing of a language model, or a system built on one, against defined cases and criteria. It measures whether the system gives correct, safe and useful results for your task, and whether that stays true when the model, prompts or data change.
What is LLM evaluation?
Public benchmarks compare models on general tasks. They say little about how a model will perform on your documents, your data and your definitions. Enterprise evaluation uses a test set built from your own work: real inputs, with the results an expert would expect.
For agents, evaluation covers the whole run, not only the final answer. A correct answer reached by calling the wrong tool, or at ten times the expected cost, is still a failure.
METHODS
Evaluation methods.
| Method | How it works | Best for |
|---|---|---|
| Reference answers | Compare output with a known correct answer | Extraction, calculations and classification |
| Rubric scoring | A person or a model scores output against written criteria | Summaries, reports and other open-ended work |
| Expert review | Experts judge a sample of results | Setting the standard and checking automated scores |
| Production monitoring | Score a sample of live runs over time | Catching changes after release |
TEST SET
How do you build a test set?
- 01
Collect real cases
Use past inputs from the task, including awkward ones.
- 02
Write the expected result
Ask the people who do the work today what a correct answer looks like.
- 03
Add cases that should fail
Include requests the system should refuse or pass to a person.
- 04
Run it on every change
Test again when you change the model, prompts, tools or data.
- 05
Track cost and time too
Record what each case costs and how long it takes.
Why does evaluation matter for risk?
The NIST AI Risk Management Framework, published in January 2023, makes measurement one of its four core functions, alongside govern, map and manage. If you have not tested a system against your own cases, you do not know its error rate.
How we test Evidence OS extensions.
Each extension project is tested against the user’s task before release: the data path, the analytical result and what happens when inputs are missing.
See delivery and operations