What design should steal from how AI labs test their models

Every serious AI lab runs on evals. Before a model ships, it's scored against hundreds of tests: maths problems, coding tasks, reading comprehension, refusals, safety cases. When a new version comes out, the first thing anyone looks at is the table showing which scores went up and which went down.

Design teams don't have anything like that, and I think we should. Not because design is like machine learning, but because the problem those labs solved is now our problem: the thing being built changes faster than people can review it.

Here are the habits I think are worth stealing, and one trap that comes with them.

1. Write the definition of good before you build

The most important thing an eval does happens before anything runs. Someone has to write down what a correct answer looks like. That act forces decisions that would otherwise stay vague until the review.

In design, the equivalent is writing rules before the prototype exists. Every form field has a visible label. The headline names the product and the audience. No more than one primary button per screen. Writing these down is uncomfortable, because it exposes the places where the team doesn't actually agree. That discomfort is the value. It's much cheaper to argue about a rule once than about every screen.

2. Separate the test from the test-taker

Labs are careful to keep the thing being evaluated away from the evaluation. A model that has seen the test answers during training will score well and prove nothing.

The design equivalent is subtler. If the same person who generated a prototype also reviews it, they will see what they intended rather than what's there. If the same agent that built a page is asked does this look good?, it will usually say yes.

So the judge should be separate from the builder. In Glitterball the builder is whatever coding agent you use, and the judge is a different model, Jev, built to give calibrated verdicts on specific questions rather than to write. The builder gets the rules up front. It doesn't get to grade itself.

3. Score per rule, not per screen

A single quality score for a model is nearly useless. What matters is the breakdown: better at code, worse at maths, same at reading. The breakdown tells you what to do next.

Design reviews tend to collapse everything into one verdict: ship it, or don't. A per-rule result is far more useful. It tells you that variant B fixed the contrast and broke the button labels. It tells you which rules fail most often across the whole team's work, which is a direct pointer to what your design system is missing.

4. Treat confidence as part of the answer

Good evaluation reports how sure it is. A grader that says fail with 55% confidence is telling you something different from one that says fail with 98%.

This one I learned the hard way. My first version reported every failure it suspected, and the result was a wall of findings that people stopped reading. Now, a judged rule only counts as failed when the judge is at least 60% sure. Below that, it's marked uncertain and kept out of the fix list. Fewer findings, all of them worth acting on. I'll write about that more in the next piece.

5. Run it every time, and compare

The real power of an eval isn't any single run. It's the comparison between runs. Labs run the same suite on every candidate model, so a regression shows up the day it's introduced rather than the day a customer finds it.

Design rarely has a regression check. A fix to the checkout flow quietly breaks the focus states. A refreshed marketing page reintroduces the generic button copy the team removed last quarter. If the same rules run on every version, those regressions become visible as a score that went down, on a specific rule, between two specific runs.

The trap: optimising for the test

There's a well-known failure mode in model evaluation. Once a benchmark becomes the target, teams start optimising for the benchmark rather than the ability it was meant to measure. Scores go up and the model doesn't really get better. It's Goodhart's law: when a measure becomes a target, it stops being a good measure.

Design evals will have the same problem. If every button names its outcome is a rule, an agent can learn to write long, awkward button labels that technically pass. A page can pass every accessibility check and still be confusing to use.

I don't think the answer is to avoid evals. I think it's to hold three things in mind:

• Evals are a floor, not a ceiling. They catch the things that should never ship. They don't tell you whether the work is great. That's still a human call.
• The rule set is a living document. When a page passes and is still bad, that's a missing rule. Write it. When a rule keeps failing good work, it's a bad rule. Change it.
• Measure what you can, and be honest about what's judged. A contrast ratio is a fact. The copy is specific is an opinion, even when a model gives it. The tool should show you which is which.

Where the analogy breaks

Model evals have one big advantage: lots of tasks have a known right answer. A maths problem is solved or it isn't. Most design questions don't work like that. There are many good answers to design the onboarding for this product, and the eval can't tell you which one to pick.

What evals can do is rule out the bad ones quickly, consistently and without a meeting. They turn the long list of things a good reviewer would catch into a check that runs in seconds. That leaves the people in the room with the questions that actually need them: is this the right idea, is it distinctive, would our customers trust it.

That's a better use of a design team's attention than counting pixels in a crit. In the next piece, I'll get into how the checks themselves work, and why mixing up measurement and judgement made my first version almost useless.