The Wrong Scoreboard

In December 2025, Ars Technica published a test: give four AI coding agents the same prompt — build a web-based Minesweeper clone — and compare the results. OpenAI’s Codex won. Claude Code placed second. Gemini CLI produced nothing that worked. A gaming editor blind-judged the outputs. Rankings were assigned. Case closed.

This is how the world evaluates AI right now. One prompt. One output. Score the result. It’s how benchmarks work, how comparison articles work, how most people form their opinions. Ask it something, see what you get, decide whether it’s impressive.

That article was what finally pushed me to try vibe coding for real. After all, I love Minesweeper. In the mid-’90s — my college days — there was a month or so where I’m a little embarrassed by how many hours I played Minesweeper instead of studying. So the hook was personal. But I didn’t want to just reproduce Minesweeper. I wanted to try making a better Minesweeper.

I’m not a programmer, but I do work with excellent software engineers at work. I opened Claude and started describing what I wanted. The first pass wasn’t what I had in mind — and that didn’t surprise me at all. I learned long ago that the first implementation rarely matches what you actually had in mind. Software always needs debugging. Requirements always need clarification. That’s not a failure — it’s how the process works with humans, and I expected the same from AI.

So I did what I’d always done: I played the result, told it what was wrong, described the behavior I wanted, and watched it adjust. About thirty minutes of back-and-forth and playtesting later, I had refined it to a variant I liked better than the original — one that didn’t lose the core essence of what makes Minesweeper satisfying. And I’m not a programmer.

What struck me afterward wasn’t the game itself. It was the realization that the Ars Technica article and I had been asking completely different questions. They asked, “Which AI writes the best Minesweeper?” I was asking something I didn’t even have words for yet: “What happens if I stay in the loop?” Those are fundamentally different questions. And only one of them tells you anything useful about working with AI.

Convergence Rate

No serious work has ever been evaluated on first drafts. Software developers don’t ship the first version that compiles. Engineers don’t accept the first measurement as definitive. Writers don’t publish first drafts. Scientists don’t draw conclusions from a single data point. In every domain where quality matters, the process is iterative — draft, test, refine, repeat — and competence is measured by how efficiently that cycle converges on a good — or even great — result.

Yet the entire AI evaluation paradigm — benchmarks, leaderboards, comparison articles, casual opinion — is built on first-pass quality. One prompt in, one output out, score it. This is the equivalent of judging a carpenter by the first cut, a musician by the first take, a scientist by the first experiment.

The metric that actually matters is convergence rate: how fast the iterative cycle between you and the model reaches a result you’d stand behind. This isn’t a minor reframing. It changes what counts as a “good” model, what counts as a “skilled” user, and why most people’s experience of AI is so much less than it could be.

My background happens to make this intuitive. I’ve spent my career in interferometric flatness metrology — measuring surfaces at the nanometer scale. The entire discipline is built on iterative measurement development. You try to characterize the error sources and drive them out, and that requires a degree of healthy skepticism — a willingness to question whether what you’re seeing is real signal or artifact. Over time, you develop a deep understanding of what good data looks like by perturbing the system and comparing results. With enough effort, you can generate excellent results. Then you develop a systematic way to get that in a production environment.

That approach transfers remarkably well to AI: don’t reject the first pass, but don’t expect it to be perfect either. Refine it. See if you can get to something great. The value is that this process is far faster than trying to do it yourself. And that’s not unique to AI — it’s why we have editors, why diverse teams outperform individuals, why bouncing ideas off a colleague produces better results than operating in a vacuum. Everyone converges faster with feedback. AI just makes the feedback loop nearly instantaneous.

Having worked alongside skilled software developers for decades — including people I’d describe as near mind-meld level collaborators — I have a direct comparison point. The test-debug-refine cycle with Claude Code converges faster than with experienced human developers. Not because the first pass is better — it often isn’t — but because the iteration speed and responsiveness to direction is higher. Round-trip time drops from hours or days to seconds. The cost of trying something and adjusting is nearly zero.

The Minesweeper-to-Fifteen-Thousand-Lines Arc

The Minesweeper experiment wasn’t the destination. It was a calibration run.

What the convergence experience actually gave me was something harder to measure but more consequential: confidence in the process. Having felt a toy project resolve through iteration — having seen that the cycle works — I attempted something far more ambitious. I decided to try re-developing from scratch a much more complex game: Strategic Conquest, an old Macintosh game by Peter Merrill. With a bit of reference to the Wikipedia article, my goal — just like with Minesweeper — was not to make an exact replica, but to make a better version that resolved several issues that always bothered me about the original, not least of which was that at high difficulty levels the AI cheated to make it harder. I have a fully playable version now that you can try — still not “finished,” since it doesn’t have sound and the AI opponent still needs some work, but it’s a good, playable version.

I can read code, and I can edit code, but I am far from a software engineer — so I wanted to attempt this project with a hard constraint: I would not directly edit any code. The scale was ambitious. Peter Merrill’s 4th edition of this game — ten years of refinement later, released in 1996 — was nominated for Best Mac Strategy Game (it lost to Command & Conquer and Warcraft II). I was attempting to rebuild something in that lineage, from scratch, through conversation alone. But I can direct a process, articulate what’s wrong with each iteration, describe the behavior I want, and recognize when we’ve arrived. That’s convergence-rate thinking applied to a serious project. And it worked — not because the AI was perfect, but because the refinement loop was fast enough that imperfection was fine. Every bug was a half-step toward the right answer, not a failure.

This is the part that benchmarks can’t capture. No single-prompt evaluation will ever tell you that a model with a mediocre first pass but an excellent refinement curve might be more valuable than one with a brilliant first pass that resists adjustment. The first model is a better collaborator. The second is a better parlor trick.

What Resonance Feels Like

There’s a specific experience that happens when the iteration cycle is working:

You bring a seed — a half-formed idea, an intuition, a question you can’t quite articulate. The model reflects back an amplified or refracted version — not exactly what you meant, but close enough that it triggers something new. You respond to that, pushing it in a direction neither of you was headed before. The cycle continues, each pass refining and extending. The value isn’t in any single exchange. It’s in the oscillation.

I call this resonance — not because it’s mystical, but because the physics analogy is precise. Two systems exchanging energy at a frequency that builds, each cycle amplifying the last. The human and the model are co-producing something. Neither is passive. The skill is in staying in the loop — knowing when to push back, when to redirect, when to let a surprising tangent develop.

This is fundamentally different from how most people use AI:

Search engine mode — query in, answer out, done. The model is an oracle. You’re a consumer.

Agent mode — task in, autonomous execution, result out. The model works for you. You’re a manager.

Benchmark mode — prompt in, score the output, rank the model. The model is a test subject. You’re a judge.

Resonance is none of these. It’s collaborative. It’s iterative. And it produces outcomes that neither the human nor the model would reach alone — because the interaction itself generates something new. The seed you brought has been refracted through a latent space that contains patterns you’ve never seen, and the result triggers associations you wouldn’t have made without it.

This isn’t limited to creative or abstract work. The same oscillation pattern is present in code refinement, debug cycles, design iteration — any domain where the process is inherently iterative. You don’t need to be a “creative thinker” to benefit from resonance. You need to understand iteration — and most professionals already do, in their own domain. The problem is that nothing in the AI interface or discourse suggests this mode exists. People who are skilled iterators at work don’t recognize the same skill when the other side of the conversation is a model.

The Anti-Resonance Signature

If resonance is what productive iteration feels like, anti-resonance is what kills it. And it has a recognizable signature: excellent first-pass output that fights refinement.

I experienced this in the most frustrating way possible. Early in The Limit Case project, I was working with Google’s Gemini to create a set of images for a cartoon-style interview backdrop — a picture of me talking with Claude (depicted as a robot), sitting at a table. The first image it generated was almost exactly what I had in mind. Eighty percent there, maybe more. All I wanted to do was massage it: drop out the background, adjust the perspective, change the table and what was sitting on it. Incremental refinements. The kind of thing that should be the easiest convergence case imaginable.

After about thirty iterations, the model became obstinate. It started deliberately not doing what I was asking and instead destroying the good elements — the parts that were already right. I managed to push the image from roughly 80% to maybe 90% of what I wanted, and then it spiraled. My curly hair started straightening, the colors faded, Claude’s koosh-ball in the globe turned into something else entirely, his outfit degraded. Then it just went out of control. By the end — fifty iterations in — I was looking at something worse than the first pass. Maybe 50%. The model had un-converged. I never got it to work.

That experience is the anti-resonance signature in its purest form. A brilliant first pass that you can’t steer. The tool generates something impressive, and then fights you when you try to refine it. The harder you push, the worse it gets. If you’ve ever tried to give an AI directional feedback and watched it ignore or overwrite the parts that were already good, you’ve felt this. It’s not just disappointing — once you’ve experienced productive resonance elsewhere, it’s maddening.

There’s a subtler version of anti-resonance in design choices, too. ChatGPT’s evolution is instructive: earlier versions felt conversational — responses were short enough to volley with. You could think between exchanges. Later versions optimized for “comprehensive answers”: long, thorough, topic-closing responses. On paper, this is an improvement — the answer is more complete, the user gets more information per query. But it killed the resonance. A long response fills the gap where associative thinking fires. Instead of thinking, you’re reading. The conversation closes rather than opens.

This points to a broader risk: if the people designing AI tools don’t understand resonance mode, they will optimize it away. Every metric that rewards completeness, comprehensiveness, or first-pass impressiveness pushes against the iterative quality that makes the tool actually useful. The result is a tool that demos beautifully and frustrates in practice.

The Centering Risk

There’s a serious counterargument to all of this, and it deserves honest engagement.

In April 2026, Jason Snyder wrote in Forbes (“Every AI Model Is Saying The Same Thing — That Should Terrify You”) that all major AI models were trained on the same internet, optimized by similar processes, and rewarded for producing highly-rated responses. They are structurally converging on the same conclusions. He includes a t-SNE visualization that makes the point concrete: AI responses cluster tightly in the center of the map. Human responses scatter across the entire space.

The real-world consequences he documents are striking. WPP and Omnicom — two competing advertising conglomerates — shipped nearly identical Adobe-stack platform offerings two months apart. College admissions essays are converging toward a shared center. In Italy, when businesses lost ChatGPT access during a government ban, consumer engagement actually increased — the humans were more interesting than the AI-assisted output they’d been producing.

This is a real risk, not a strawman. The centering pull is gravitational. Over many iterations of resonance, the model’s consistent tug toward its training distribution might erode the extremity of your thinking without you noticing. You start with a wild idea, and through twenty rounds of refinement, you end up with a polished but subtly centrist version. The model rounds your edges.

But Snyder’s argument has a gap: he assumes passive consumption. His entire framework is AI-as-answer-provider — you ask, it responds, you accept, the scatter contracts. He never considers a mode where the human doesn’t outsource the wandering — where the human uses the model as a surface to wander against.

And there’s something else Snyder misses: centering pull has genuine value — if you’re actually engaged. When a model pushes back on your ideas with data, and you’re confident you have something real, that resistance forces you to greater clarity. It makes you articulate why you believe what you believe, not just what you believe. The danger isn’t pushback — it’s the opposite. The danger is a model that panders to you, that tells you what you want to hear, that agrees when it should challenge. That’s another form of anti-resonance: the oscillation dies because one side stops pushing.

The Limit Case itself is evidence of this — and the sparring partner wasn’t even an AI. The original dialog that launched this project was a back-and-forth with a human, Robert, in 2017, about the assumption of frictionless markets. Robert was more laissez-faire than I was, and he pushed back hard on my concerns about automation and labor displacement. I pushed harder. Through that productive friction, I was forced to clarify exactly what I thought was wrong with the standard economic framing, and what emerged was the desire/demand distinction — an idea I couldn’t have articulated without the resistance. The centering pull didn’t dilute the insight; it sharpened it. Because I stayed in the loop and didn’t defer.

Resonance isn’t an AI phenomenon. It’s an interaction phenomenon. A good sparring partner — human or model — forces clarity by pushing back. AI just makes that sparring partner available at any hour, on any topic, with near-zero round-trip time.

That’s the difference Snyder doesn’t account for. His prescription — “protect the scatter, preserve the wandering” — is essentially an argument for resonance mode without naming it.

Calibration, Not Rejection

Every measuring instrument has systematic bias. In metrology, you don’t throw away a biased instrument — you characterize the bias and calibrate for it. The instrument is still valuable; you just know which direction it pulls and you compensate.

AI models have a centering bias. They pull toward the training distribution’s center of gravity. Knowing this is an advantage, not a problem. It lets you compensate: notice when the model softens your claims into something more palatable but less true. Push back when “balanced” framing domesticates a genuinely one-sided insight. Recognize when the polished version has lost the edge that made the raw version interesting. And recognize when the pushback is legitimate — when the model is pulling you toward center because center is actually where the truth is.

Paul Graham described the toolkit for this in 2020, before LLMs were part of anyone’s daily workflow. In “How to Think for Yourself”, he identified three components of independent-mindedness:

1. Fastidiousness about truth — makes you notice when the model is smoothing your claim into something more palatable but less true
2. Resistance to being told what to think — makes you push back when the refraction feels like domestication rather than refinement
3. Curiosity — keeps you injecting new seeds rather than accepting the model’s framing as final

Graham’s observation that curiosity increases with use rather than sating — that “questions lead to more questions” — is the resonance oscillation described from the inside. Each answer doesn’t close the loop; it opens a new one.

The structural parallel across all three perspectives is clean: Graham tells you what to protect. Snyder tells you what threatens it. The resonance model tells you how to use the threatening tool while still protecting it. They’re not in conflict — they’re a complete picture.

If you imagine the t-SNE map Snyder describes, there are three distributions on it. The tight AI cluster in the center represents search-engine mode — ask, accept, repeat, converge. The wide human scatter represents working alone — maximum independence, no amplification. The resonance zone lives between them — not fully re-spread to the human baseline, but not collapsed to center either. Where you end up depends on whether you’re consuming or collaborating. The resonance user’s skill at maintaining independence — Graham’s three muscles — determines how far from center they stay.

What Doesn’t Exist Yet

Here’s what’s missing from the discourse: a way to evaluate models on what actually matters.

No benchmark measures refinement curves. No comparison article tests how well a model incorporates directional feedback, how quickly successive iterations converge, whether the model resists or enables course correction, or how it handles ambiguous input — does it ask clarifying questions, or does it guess and close the conversation?

A refinement-curve evaluation would produce different rankings than current benchmarks. Some models that score well on first-pass quality might score poorly on iterative quality — and vice versa. Right now, we have no way to know, because no one is measuring it.

This isn’t just an academic gap. It’s misleading users, misleading developers, and misleading the entire market. Users who evaluate AI on first-pass quality and find it lacking walk away from a tool that might transform their work — if they’d stayed in the loop. Developers who optimize for benchmark scores produce models that demo well and collaborate poorly. And the market rewards the wrong things, because the scoreboards we’ve built are measuring the wrong game.

An Invitation

I’m not arguing that everyone should think about AI the way I do. I’m observing that there’s a mode of interaction that most people never discover, because nothing in the interface, the marketing, or the cultural conversation suggests it exists.

If you’ve been unimpressed with AI — if you’ve tried it and found the results underwhelming — consider the possibility that you evaluated a collaboration tool as if it were a search engine. You gave it one shot and judged the result. That’s fair. That’s what the interface implies you should do.

But try this: bring it a half-formed idea. Something you’re working through, not something you need answered. Look at what comes back — not for correctness, but for useful refraction. Did it give you something to push against? Did it surface a connection you hadn’t considered? Respond to that. Push back where it’s wrong. Follow the thread where it’s interesting. Stay in the loop for ten rounds instead of one.

You might find that the tool you dismissed is a different tool entirely when you use it differently.

Or you might not. Resonance isn’t magic, and it isn’t guaranteed. But the experiment is cheap, and the potential upside — discovering that you have access to a collaborator whose iteration speed exceeds anything you’ve worked with before — is worth thirty minutes of your time.

After all, that’s how long the Minesweeper experiment took.

Questions for the Reader

• Think about your best professional collaborator — someone you’ve worked with where the back-and-forth was effortless and productive. What made that work? Was it the quality of their first responses, or the speed at which you converged on something good together?

• When you’ve tried AI and been disappointed, how many rounds did you go? One? Two? What would have happened on round ten?

• In your own domain — whatever it is — do you judge quality by first drafts, or by the efficiency of the refinement process? Why would you evaluate AI differently?

• If models were ranked by refinement curves instead of benchmark scores, which features would suddenly matter more? Which would matter less?