6 min
Evaluate agents the way you evaluate landing pages
Growth teams already know how to judge a system they cannot fully explain. Agentic evaluation is the same discipline with a different unit of work.
- agents
- evaluation
- growth
Nobody asks a landing page to explain itself. You define the outcome, you run variants, you measure, you keep what wins and you kill the rest. That mindset transfers almost perfectly to agents.
The unit of work
For a page, the unit is a session. For an agent, it is a run: an input, a plan, a set of tool calls, an output, and a consequence. If you record the whole run, you can score it. If you only record the output, you are guessing.
type AgentRun = {
id: string
input: string
steps: { tool: string; args: unknown; ok: boolean }[]
output: string
// the part everyone skips
outcome: { metric: string; value: number; observedAt: string } | null
}Three scores, not one
- Correctness: did it resolve against the semantic layer, or invent a column?
- Cost: tokens, latency, tool calls. A right answer at ten seconds is a different product than at one.
- Consequence: did the action it took move the metric it claimed it would?
Most teams measure the first, complain about the second and never instrument the third. The third is the only one a business cares about.
Evaluation is not a test suite you run before launch. It is the telemetry you keep running forever.