The Review Paradox

Developers who fully delegated to AI finished fastest and scored worst. The skills you need to supervise the model are the first ones it erodes, and you cannot feel it happening.

Share
Bloom 행사 현장

Developers who fully delegated to AI finished fastest and scored worst on evaluation.

The paradox

One observation has been circulating widely in developer communities: engineers who handed their tasks entirely to AI finished fastest and scored worst on evaluation. And the novices who gain the most from AI productivity are precisely the ones who need debugging skills to supervise it, which is the skill AI erodes first.

Call it the review paradox, because the more you use AI, the less qualified you are to review it, and the two are not separable.

Recognizing good work is not something you get from reading about good work. You get it from doing it badly, being corrected, and accumulating judgment over years.

It applies well outside engineering

This is discussed mostly among developers, but it lands the same way in every knowledge job.

Once AI handles most of the execution, what remains for us is review, management, planning and strategy, and that sounds appealing.

But consider how anyone learned those things, by doing the unglamorous work directly and by making mistakes and getting corrected. The judgment required to review output exists because you did the task hundreds of times yourself.

Take that away and the arrangement breaks, with AI executing, you reviewing, and you having no basis for knowing what good looks like. The erosion is bad enough, and what makes it worse is that you cannot feel it happening.

Review the spec instead

Developer communities have already started solving this, and the principle is simple: review the spec and the architecture, not the code.

Write a real specification before any code exists: define the problem precisely, understand the tradeoffs, translate business language into product requirements and product requirements into technical architecture. A human reads and reviews the spec, the architecture, and the verification plan, which means actually understanding what is being built and why.

Then AI writes the code, and the check is whether the code follows the spec, which is something models do well. Judging whether the spec is right is the human job.

Some teams now mandate this, and without a mandate nobody does it, because the path of least resistance is to work in flow and ship whatever comes out.

Typing was never the job

One line captures the whole of it:

"Software engineering was never just about typing code. It's defining the problem well, understanding the problem, translating the language from business to product to code, clarifying ambiguity, making tradeoffs, understanding what breaks when you change something."

Swap "software engineering" for any knowledge work and the sentence still holds.

Do not just take the answer

Claude has a Learning setting that trades back and forth with questions instead of handing over an answer. A few months ago that would have felt like friction, and it reads differently now. If the thing you are protecting is judgment, then receiving finished answers is the worst possible input.

Intervening at the spec level does not slow you down, because it moves your time to the part that decides the outcome.

Keep reading


Join Bloom

Bloom builds offline rooms where people and technology meet. We run them in Seoul, and now beyond it.

Read more

일론 머스크와 샘 올트먼의 흑백 사진

한국에선 2030년 대담을 나눌 자리가 없다

같은 2030년, 다른 체감 오후에 만난 개발자 친구는 AI와 상관없는 일을 하는 친구들을 만나면 늘 이렇게 말한다고 했다. 2030년쯤 되면 우리는 AI에게 대체되느냐 마느냐가 아니라, AI를 사람으로 볼지 말지를 이야기하고 있을 거야. 개발자 친구는 지금의 AI 논의가 상당히 피상적이라고 봤다. 많은 전문가도 삶이 바빠 시야가 얕고, 최근 AI와 전혀 상관없는

By Bloom
“AI SLOP” 글자와 녹아내리는 캐릭터들이 그려진 일러스트

AI 시대에는 신뢰가 병목이다

AI가 만든 것들에 점점 피로감을 느낀다는 미국에서 막 돌아온 창업자와 점심을 먹었다. 요즘은 스타트업들이 PR 자료를 AI로 대충 만들어 뿌리는 경우가 많아졌는데, 그런 자료는 결국 효과가 형편없을 거라는 이야기였다. 살짝 AI slop 느낌 나는 순간 저 그냥 안 보거든요. 그는 AI가 쓴 티가 나는 글은 신뢰가 확 떨어진다고 했다. 나는

By Bloom