OpenAI Codex’s Agent Loop: A Counterintuitive Engineering Choice

I stayed up last night reading OpenAI engineer Michael Bolin’s article, and I was shocked by their counterintuitive engineering choice. Not because of how powerful the model is, but because they deliberately refuse to use the optimization interface.

When using Claude Code or other AI programming tools nowadays, it’s easy to think of them as chatbots that write code. But Codex is positioned completely differently.

You want to fix a bug in your project, so you ask Codex to check why the login keeps failing. It won’t just give suggestions - it actually looks at the project structure, executes commands, reads file contents, runs tests, checks error messages, then decides the next step based on these real results. The whole process is like someone operating on your computer until the problem is solved.

This is the agent loop.

Codex isn’t a single model, but a complete system. CLI tools, cloud services, VS Code plugins - they all share the same core execution framework.

You send a command, Codex assembles a detailed prompt containing your message, working directory, shell type, tool list, project documentation. Then it calls the model, which determines whether to respond directly or call a tool.

If a tool is needed, it executes ls, cat, npm test commands locally on your machine. After execution, it stuffs the output back into context and calls the model again.

It loops repeatedly until the model says no more tools are needed, then gives you final feedback.

A simple task might involve dozens of model calls. This agent loop mechanism is Codex’s core competitive advantage.

The Responses API has a previous_response_id parameter that can optimize context transmission, avoiding sending the complete conversation each time and reducing quadratic JSON expansion.

Codex deliberately doesn’t use it.

I was stunned - why make things harder for yourself? After hearing their reasons, I was convinced.

Requests are completely stateless, each request independent without relying on server state. Supporting Zero Data Retention means the server stores no user conversation data at all.

So even if OpenAI’s servers get hacked or subpoenaed by court, your code and conversations are inaccessible to them.

For security and compliance, Codex would rather sacrifice performance than use that seemingly obvious optimization.

So what about performance? The focus is all on prompt caching.

The article mentions the pitfalls they encountered. MCP tools in different order - all cache misses. Changing sandbox configuration mid-way - cache invalidation. Switching working directories requires appending new messages, not modifying old prompts, otherwise cache is wasted.

These are details we encounter frequently when coding.

There’s also conversation compaction.

When context approaches the window limit, Codex automatically triggers compression. It’s not a simple summary, but a compressed representation with encrypted state returned via the Responses API.

Users can’t see the complete conversation, but the model remembers previous content. Like the human brain - you might not remember what you ate for every meal, but you definitely remember whether you like spicy food.

The compression and snapshot mechanism works like this:

When calling the model for the first time, Codex constructs the prompt and processes inference results.

After executing the tool call, results are stuffed back into context for the next round of inference.

When the model decides no more tools are needed, it gives the final answer and the loop ends.

Codex CLI is completely open source on GitHub.

Many AI developers are now reverse-engineering closed-source tools like Claude Code. OpenAI directly revealing their internal implementation and writing about engineering details - this is genuinely admirable.

Someone on HackerNews said OpenAI is definitely the best at context compression, better than Claude Code.

After reading the article, my feeling is: the competition in AI programming tools has moved from who has the stronger model to who has better engineering. Model capability is foundational, but what really determines user experience are these invisible engineering details. Agent loop design, tool invocation management, context compression and caching - these are the keys.

By revealing Codex’s internal mechanisms, OpenAI is also showing the industry how to properly build software agents.

GitHub repository: openai/codex.

If you found this article interesting or learned something, give it a like, forward it to friends, and let them know about Codex’s black technology. Follow me, and next time OpenAI drops more news, I’ll be the first to tell you.

See you next time.