The design leader's job is now to write the eval

For most of my career, a big part of leading design meant being in the room. Crits, reviews, sign-offs. The team made the work, and I helped decide whether it was good enough, one screen at a time. My taste was applied by hand.

That model is already breaking. When agents can produce more variants in a day than a team can review in a week, a leader who judges one screen at a time becomes the slowest part of the system. Either the leader becomes a bottleneck, or the work starts shipping without the judgement, and the product drifts toward the average.

There's a third option, and I think it's where the job is going. The leader writes the eval.

Taste as an artifact

Every good design leader has a set of things they say over and over. What's the one thing you want them to do here? That copy could be about any product. Why is this the primary button? Where does this go when it's empty?

Those repeated comments are the most valuable thing a design leader brings. They're also the least scalable, because they live in one person's head and get delivered one meeting at a time.

Writing them down as rules changes that. Each screen has one clear primary action. Copy names concrete features, numbers or examples. Every list has a designed empty state. Once written, those rules apply to every piece of work the team and its agents produce, whether the leader is in the room or not. Taste stops being a performance and becomes an artifact: something you can read, share, version and improve.

What the job looks like

I think the design leader of the next few years spends their time differently.

Less time reviewing screens, more time reviewing results. Instead of looking at every variant, you look at what the evals say about them. Which rules fail most often across the team? Which rules did this release regress on? Where are the uncertain verdicts that need a human?

Writing and pruning the rules. The rule set is the team's standard. When something bad gets through, you write the rule that would have caught it. When a rule keeps failing good work, you fix or remove it. This is editing, and it's continuous.

Deciding what not to automate. Some questions should stay human: is this the right idea, is it distinctive, will it build trust. The leader decides where the line sits, and protects the time the team spends on the work above it.

Teaching the standard. Rules come with reasons. A team that knows why one primary action matters will apply it in places the rule never anticipated. The rule set is a teaching tool as much as a test.

What it asks of us

This shift isn't free, and I don't want to pretend otherwise.

Writing a rule is harder than giving feedback. In a crit, you can react to what's in front of you. A rule has to work for screens you haven't seen yet. It forces precision that a lot of design culture has avoided, because precision exposes disagreement.

It also asks leaders to make their judgement inspectable. A rule set can be read, argued with and found wrong. For some leaders that's uncomfortable. I think it's healthy. A standard that can't be written down was always harder to learn, harder to apply fairly, and easier to use as a matter of status rather than quality.

And it asks us to be honest about what evals can't do. They're a floor. They catch the things that should never ship. They won't tell you whether the work is great. That part is still the job, and arguably it's now the whole job, because everything below the floor is handled.

Why this is the moment

The AI industry spent the last few years learning that you can't improve what you can't evaluate. Labs that invested in their evals moved faster and more safely than labs that relied on impressions. The same lesson is now arriving in product design, because the same conditions have arrived: the thing being built changes faster than people can look at it.

Design teams that write their standards down will get agents that build to those standards. Teams that don't will get the average, faster.

I've been building Glitterball to make that practical: rule packs for accessibility, usability and the tells of generated design, your own rules in plain words, checks that run in the browser, and fixes that go straight back to the agent. It's an experiment, and I'm building it in the open.

If you lead a design team and you've started writing your judgement down, or tried and found it hard, I'd like to compare notes. That conversation is the reason I'm writing this series.