The Expert Trap of Managing AI

Lately the team has been working out how we want to collaborate with AI on writing software.

For the past two weeks, we’ve been on a path of fine-grained management — making sure the AI doesn’t make a mistake at any step. We’ve also seen people who don’t write code at all, never look at the code, only check the functions — and somehow build large systems that just work. It just works. We tried that other path ourselves, but didn’t quite pull it off.

By the end of the discussion, everyone was thinking, but no one had a conclusion.

I kept turning it over afterward. My take: as long as we see through this particular 知障 — the obstruction of what we already know — we may not move fastest at first, but we’ll go further.

The mental trap I think we’re in: because we know the work (we can write code), we instinctively reach in to micromanage the AI worker’s specific moves. We override the AI step by step — and in doing so, we ignore the real question: how to use management discipline and methodology to make AI ship real work.

People who can’t code at all can’t micromanage. They’re forced to find other ways.

It’s a familiar pattern from traditional management: the boss who doesn’t know the trade often becomes very good at people and organization (Liu Bei, Sam Altman). The boss who does know the trade, if they realize they can’t do everything themselves and learn to delegate and build org structure, can be even more formidable (Zhang Yiming, Lei Jun, Pony Ma).

People who can’t code have already proven by their results that you don’t need line-by-line micromanagement to get AI to ship. Other methods work. We just have to find out what they are.

On this question, we’ve moved from don’t know what we don’t know to know what we don’t know. Once we figure it out, the toolbox gains a tool, and the mental model gains an option.

The Expert Trap (Competency Trap)

People who know the work most easily over-intervene at the execution layer — and that blocks the system from scaling.

• Experts are highly sensitive to correctness.
• Every step the AI takes “could be wrong,” and that naturally triggers us to correct it.
• Result: local optimum + global suboptimum.

This is exactly the Expert Trap, applied to the AI era.

Why people who can’t code can ship

The people who can’t write code, because they can’t control line by line, are forced to:

• constrain by goal,
• enforce by acceptance criteria,
• design systems where components correct each other,
• tolerate failure.

They’re being pushed to operate at the second or third level of control.

When we “manage AI line by line as it codes,” we’re using human bandwidth to suppress a highly parallel system. The ceiling comes up fast.

Knowing the work vs. not knowing the work

Bosses who don’t know the trade usually have to be excellent at people and organization (Liu Bei, Sam Altman, Jack Welch).

Bosses who do know the trade, if they grab every detail, get pulled down by the details.

The truly formidable ones are the ones who know the work and discipline themselves not to use that knowledge — they spend their attention on:

• mechanism,
• structure,
• cadence,
• boundary conditions.

This isn’t denying the value of “knowing the technology.” It’s accepting that knowing the technology makes it easier to get stuck at Tier One — but once you make the jump, the ceiling is much higher. Zhang Yiming / Lei Jun / Pony Ma are the textbook knowing-the-work + restraint archetype.

The mindset shift

For AI collaboration, we need to move from:

Make sure the AI doesn’t make a mistake at any step.

to:

Accept that the AI will make mistakes, and design a system where, even when individual AI agents make mistakes, the system still runs.

What that means:

• “Correctness” upgrades from a process requirement to a result constraint.
• “Sense of safety” moves from humans watching to mechanism catching.
• “Capability” shifts from human + AI co-operation to humans designing rules, AI executing and exploring.

Once we see this, our current path is fine.

Tier-One management, where conditions allow, is a phase you can’t skip. Only by going through Tier One do you build a fine-grained, hands-on sense of what AI can do — what’s risk, what’s the boundary, what’s quality, what’s non-negotiable. That earned sense becomes the confidence that lets us hold a high standard later, when we’re delegating to AI.

Awareness

Once you recognize this trap, your professional skill stops being a 知障 and becomes basic equipment for the next level. The real path is upgrade: walk Tier One solidly, then climb step by step to Tier Two, then Three.

1. Tier One — manage behavior (necessary, valuable)

   • Focus: how it gets done, whether it’s right
   • Output: high-quality, explainable implementation experience and engineering instinct
   • Value: laying foundations, setting standards, identifying risk

2. Tier Two — manage rules (the core of the next phase)

   • Focus: input constraints, acceptance criteria, tests, boundary conditions
   • Output: reproducible methods, scalable processes
   • Value: output no longer depends on someone watching the details

3. Tier Three — manage system self-evolution (long-term direction)

   • Focus: module autonomy, self-healing, continuous refactoring, alignment between organizational structure and system structure
   • Output: a system that gets faster the more you do it, more stable the more it runs
   • Value: scale and long-term compounding

What I want to emphasize: Tier One → Two → Three isn’t an either/or. It’s a growth path.

Tier One isn’t to be denied. It’s to be inherited, abstracted, and upgraded.

The people who go far aren’t usually the ones who skipped Tier One. They’re the ones willing to keep upgrading the level.

What’s next

Directions worth trying:

1. Identify which modules / tasks must stay at Tier One (high risk, core infrastructure, the parts that can’t be rolled back).
2. Move more attention from step-by-step correction to defining acceptance and automated verification, and multi-agent division, collaboration, supervision, mutual correction.
3. Allow the individual / process to be imperfect — but with explicit goal constraints and acceptance criteria, and verifiable, reversible, iterable.
4. Crystallize reproducible collaboration patterns: prompt structures, skills, interface contracts, test templates, review checklists.

The goal isn’t look at less code. It’s: gradually free humans from the details, and use higher-level mechanisms to hold the line on quality.

Closing

Our own ability and our current way of working has been a very valuable stretch — it’s given us standards and a floor.

I feel lucky that within the first two weeks of the team starting to operate, we noticed this and discussed it openly. That’s a real step on one of Causally’s two core missions — exploring an AI-native form of organization.

As long as we stay open and don’t let I already know how to do this hold us in place, walking from Tier One to Tier Two to Tier Three, we’ll go further and on firmer footing.