From Prompt to Loop — Hongkai He

Most people’s working model of AI stops at the prompt: you ask, it answers, you ask again. That’s fine for a chat window. It falls apart the moment you try to turn a real business workflow into something an AI system can run reliably — reading context, using tools, making bounded decisions, asking for human review, logging what happened, improving through evals, and running again tomorrow without you.

To build that, you need five words that get used interchangeably but shouldn’t be: workflow, agent, orchestration, harness, loop. Each names a different layer of the system. Confuse them, and you end up debating “should we use agents?” when the real question is “who owns the control flow?”

Here’s the mental model I keep coming back to.

Workflow: the process

A workflow is a structured sequence of steps that turns an input into a useful output:

input → judgment → action → output → feedback → audit

A workflow can include AI without becoming an agent. If the sequence is mostly predefined by code — user uploads a document, system extracts text, LLM summarizes, evaluator checks format, output ships — that’s still a workflow. The LLM is doing a step; it isn’t deciding what the steps are.

The workflow defines the job: what needs to happen, in what broad order, toward what output.

Agent: the actor

An agent is a system where the LLM has some control over how the task proceeds. It can decide what information it needs, call tools, inspect the results, choose the next step, stop when the task is done, and ask for human review when it isn’t sure.

The simplest way to tell them apart:

An LLM call produces an answer. An agent advances a task.

A model responding to an instruction is a conversation. An agent is a participant in the execution of work.

Orchestration: the control flow

Orchestration decides which step runs next, which tools are available, when to call another agent, when to retry, when to stop, and when to escalate to a human.

There are two styles. Code-driven: the developer defines the flow — classify, retrieve, generate, evaluate, return. LLM-driven: the model decides the next step within defined boundaries — given a goal, it chooses what to search, read, call, compare, or ask.

Most production systems mix both, and the mix is the design:

Code defines the rails. The LLM handles ambiguity inside the rails.

Note what orchestration is not about: the number of agents. Multi-agent design is control-flow design. The key question is never “how many agents?” — it’s “who owns the task, the state, the tools, and the final output?”

Harness: the production wrapper

A model plus a prompt is not a system. The harness is everything you wrap around the agent to make it reliable, testable, observable, and reusable: instructions, tools, memory and context, orchestration, permissions, guardrails, evals, logs and traces, human review, error handling, handover documentation.

LangChain’s framing is the cleanest I’ve seen: Agent = Model + Harness. A raw model cannot maintain durable state, execute code, manage its own context, run in a sandbox, verify its work, or enforce constraints. Those are harness-level responsibilities.

The model provides intelligence. The harness turns “AI can do this once” into “the system can do this repeatedly and safely.” That sentence is most of what “production-ready” means.

Loop: the feedback cycle

A loop is a recurring agentic system that keeps working toward a goal over time. Instead of prompting an agent one step at a time, you design the system that prompts it:

goal → discovery → task → agent action → verification → state update → next task

Concretely: every morning, scan open bugs and failed tests → pick one issue → create an isolated branch → an agent drafts the fix → a verifier runs the tests → update the tracker → record what remains. Nobody typed a prompt.

A loop that actually works needs a schedule or trigger, persistent state, a clear stopping condition, an isolated work environment, project instructions, tools and connectors, a verifier, and human review for the decisions that matter.

That last item is the whole game. Bad loops produce unattended mistakes, token waste, and comprehension debt — a system nobody understands anymore, still running. A loop should not remove human judgment. It should move the human role upward: from repeatedly prompting the agent to designing the loop, reviewing the outputs that matter, and improving the system.

The five layers, side by side

Layer	One line
Prompt	You tell the agent what to do, once
Workflow	The process — a defined path from input to output
Agent	The model-driven actor inside the process
Orchestration	The control flow that decides what happens next
Harness	The runtime and control system around the agent
Loop	The recurring system that keeps working, checking, improving

A complete AI-native system usually contains all of them: the workflow defines the job, an agent performs the flexible reasoning inside it, orchestration controls what happens next, the harness makes it safe and observable, and the loop lets it repeat and improve.

Which reduces the design work to one practical question:

What should be deterministic, what should be model-driven, what should be reviewed by humans, and what should improve through loops?

Answer that well and the architecture mostly writes itself.