WHITEBOXX / GLOSSARY

Human in the loop: where people
check the work of AI.

Human in the loop (HITL) means a person takes part in an AI system’s work at defined points: approving an action before it happens, reviewing results before they are used, or correcting the system when it is unsure. The aim is to catch errors where they matter most.

By Gautam Mehra · Published · Updated

What does human in the loop mean?

The term comes from engineering and simulation, where a person stays part of a control loop. For AI agents, it means deciding in advance which steps a person must see. A person might approve every payment an agent prepares, or review only the cases the agent marks as uncertain.

Human in the loop is often confused with human on the loop. On the loop, a person monitors the system and can step in, but does not approve each action.

PATTERNS

Four ways to involve people.

PATTERNS
PatternHow it worksUse it when
Approve before actingThe agent prepares an action and a person approves itActions are costly or hard to undo, such as payments or messages to customers
Review before useThe agent completes the work and a person checks it before it is sharedResults inform decisions, such as research or reports
Review exceptionsA person sees only the cases the agent flags as uncertain or out of policyVolume is high and most cases are routine
Sample and auditA person checks a random sample of completed runsErrors are cheap to fix and you need to track quality over time

What does the EU AI Act require?

The EU AI Act, which came into force in August 2024, requires high-risk AI systems to be designed so that people can oversee them effectively. Under Article 14, the people overseeing a system must be able to understand its limits, interpret its output and override or stop it. Many companies apply the same idea to lower-risk systems.

Review only works if the reviewer can see what the system did. A person asked to approve a result without the sources and steps behind it can only guess.

What does human review cost?

Reviewer time is often the largest running cost of an AI workflow. Five minutes of review on each of 2,000 monthly runs is about 167 hours a month. Choosing the right pattern for each step matters as much as choosing the model.

Estimate it with the cost calculator

How Evidence OS supports review.

Each Evidence OS run keeps the sources, steps and calculations behind its results, so a reviewer can check a finding without rerunning the work. Your team decides which workflows need approval and who gives it.

See the audit trail