16 posts tagged with "Perspective"

When I saw the Hacker News post titled Benchmarking Opus 5 on SlopCodeBench, the number that caught my eye was 4/17. My first instinct was the usual one: scan the score, file the model somewhere in my mental ranking, move on. Four passes out of seventeen checkpoints. Not great. Not catastrophic. Just another data point on the leaderboard treadmill.

Then I made the mistake of opening the paper.

The longer I use coding agents, the less I trust a model ranking on its own.

Rankings are useful, but they only tell me so much. I run Claude Code and Codex on real projects, and over time one thing has become hard to ignore: the same model can produce very different results under different harnesses. How context enters the run, which tools are available, and where verification happens all change the final output. Those choices can matter as much as the model itself, and sometimes more.

That is why my first reaction to a new model release is no longer to scan the leaderboard. I want to know what Agent, what harness, and what effort setting produced the new number.

Claude Opus 5 was released on July 24. This time, the public material gives us enough detail to ask that question more clearly.

The diagnosis was correct. The turn completed. Codex CLI returned the smallest-fix explanation for a ZeroDivisionError, and the final status read completed. If I had stopped there — closed the terminal, pasted the answer, moved on — I would have called the run a success and never questioned what happened in between.

Then I read the event stream.

On 2026-07-23 I watched a Codex CLI 0.145.0 turn complete correctly. Five tool calls, the right diagnosis, a final status of completed. I had preserved the bounded JSONL event stream from the observability work I'd been doing — the recorded tool calls, their results, and the two failed attempts along the way before the Agent reached the correct diagnosis. It was a satisfying run to look at.

And then I realized I had exactly zero evidence about tomorrow.

That trace explains one path through one turn on one day. It cannot tell me whether swapping the model version will regress that behavior, whether adding a new Skill to the loadout will cause a misfired tool, whether a prompt tweak to the harness will change how the agent approaches similar tasks. One successful run is not a regression test. It is evidence from one trial.

That gap is where Eval-Driven Development begins.

Something happened on Juejin on July 23 that is worth paying attention to, not because any single post broke new ground, but because of what the posts, taken together, reveal about where the conversation has moved.

A Juejin hot list checked on July 23 included a Top 10 Skills roundup, a guide to designing better Skills, and a piece asking, plainly, why installing more Skills can make an AI feel slower and less reliable. On Hacker News in the same window, projects surfaced for diagnosing loadout health, gating Skill changes with real usage evidence, and mapping which installed capabilities rarely fire. I am not citing these as verified market signals. Autocomplete suggestions and hot-list rankings show intent, not volume. But the intent itself is the signal: developers are no longer asking mainly "where can I get more Skills?" They are asking which Skills actually trigger, which ones collide, which ones sit unused, and whether any of them are worth their context cost.

That shift is worth understanding from first principles.

The observation did not start with a term. It started with work.

I was building something with multiple Agents—or what I was calling Agents at the time—and a problem kept surfacing: one overall goal did not map cleanly onto one loop of action and feedback. Different units of work needed different local objectives. They needed different signals to know whether they were converging. They needed different stopping conditions and different answers to the question: who is allowed to change the goal here?

The loops were not independent. Some depended on what another produced. Some could block another from proceeding. One unit's partial result could make a previous unit's objective obsolete. What I was actually working with was a graph—I just had not made it explicit. When I later came across the terms Loop Engineering and Graph Engineering in developer discussion and on X, they named a structure I had already been trying to manage.

That is the order I want to keep: observation first, terminology second.