Loop Engineering: The DevOps Engineer's CI/CD Guide for AI Agents

The DevOps Engineer's Guide to Loop Engineering: CI/CD for AI Agents

Last month, Boris Cherny (creator of Claude Code) and Peter Steinberger posted nearly identical lines on X: "I don't prompt agents anymore. I have loops running that prompt the agent." Within hours, Andrew Ng devoted an entire Batch letter to the concept. The term "loop engineering" went from niche to essential overnight.

But while most coverage focuses on what loop engineering is, it barely scratches the surface of the DevOps and infrastructure challenges that decide whether these loops ship value or quietly burn through your token budget producing slop. That's the gap this article fills.

The Three-Layer Loop Architecture

Andrew Ng's framework maps three nested loops running at different cadences:

The inner loop is where the model writes code, runs tests, iterates until passing. The middle loop is where a human reviews output, steers direction, updates specs. The outer loop feeds real-world performance back into requirements. Here's the critical insight: the verifier is always the bottleneck, never the generator.

For DevOps engineers, this reframes everything about CI/CD. Your existing pipelines are designed for humans writing code and machines verifying it. Loop engineering inverts that: machines write code, and both machines and humans verify it at multiple stages simultaneously.

Open Loops vs Closed Loops: A DevOps Decision

Every loop sits on a spectrum between two modes, and choosing the right one is an architectural decision:

• Closed loops pin success criteria in advance. Every iteration is evaluated against hard checks with an explicit stop condition. They run predictably within budget but won't surprise you.
• Open loops give the agent a goal and loose conditions, letting it explore broadly. They can produce genuinely novel output but degrade into expensive slop machines without strong verification.

The practical implication: you need different CI/CD pipelines for each mode. Closed loops map cleanly to standard PR-based workflows — run tests, check coverage, gate on quality gates. Open loops require what I call "convergence pipelines": loops that explore freely but must pass a floor of verifiable checks before output is considered shippable.

The Verifier Problem: Why Your CI Pipeline Needs a Rewrite

In traditional DevOps, the CI pipeline is the verifier. It runs tests, lints code, checks coverage. In loop engineering, two verifiers are needed:

1. Internal verifier (machine): Tests, linters, type checks — the same CI pipeline you already have. The agent's self-check before declaring success.
2. External verifier (human): Code review, architectural validation, "does this actually solve the problem?" — the loop that Ng places in your hands.

The failure mode is well-known: a model grading its own homework always gives itself an A. The moment you see your CI pipeline passing the same category of checks repeatedly without catching regressions, it's time to add a second verifier layer — either a different model with different instructions (sub-agent verification), or an actual human review step before merging.

Operational Patterns: Making Loops Production-Ready

Addy Osmani identified five primitives that every loop needs. Translating those to DevOps infrastructure:

• Scheduled automations become cron jobs, GitHub Actions workflows, or CI/CD triggers — the heartbeat of your loop. OpenAI runs internal automations for daily issue triage and CI failure summarization.
• Worktree isolation maps to branch-per-agent strategies in Git. Two agents writing to the same file is the same collision problem as two developers committing conflicting changes.
• Skill codification becomes documented runbooks, AGENTS.md files, and project-specific CI/CD configurations that persist across agent sessions.
• Connectors and plugins are your MCP servers — the bridge between the agent loop and your issue trackers, databases, staging environments, and Slack channels.
• Sub-agents for verification map to separate CI jobs or reviewer agents with distinct instructions and models. The implementer never grades its own work.

Cost Control: The Uncomfortable Truth About Loop Engineering

Here's what nobody wants to hear: loops burn tokens, and the cost scales non-linearly with loop complexity. A simple closed loop might cost a few dollars per run. An open loop exploring broadly across a large codebase can easily spend hundreds of dollars before converging.

The DevOps engineer's job is to build observability into these loops from day one. Track token consumption per loop iteration, set hard budget caps, log verifier decisions (not just pass/fail but why it passed or failed). Without telemetry, you're flying blind into a cost explosion.

OpenAI's internal guidance is explicit: usage patterns can vary wildly, and teams should be "token rich or token poor" about what they invest in. If your loop runs 40 agents in the time it takes to make coffee, you need dashboards showing where that spend went before you ship.

The Verdict

Loop engineering isn't replacing your CI/CD pipeline — it's layering a new verification paradigm on top of it. The core shift is this: your existing pipelines verify human-written code. Loop engineering requires verifying machine-written code, with machines doing the first pass and humans doing the second. The verifier is the bottleneck in both cases, but in loop engineering the gap between "passes CI" and "actually good" widens significantly.

The engineers who thrive here aren't the ones who write the best prompts. They're the ones who design the most robust verification architectures — the ones who build loops that converge instead of wander, that respect budget constraints, and that produce work a human reviewer can actually trust without re-deriving everything from scratch.

In short: prompt engineering was about getting the model to do the right thing. Loop engineering is about building the system that ensures "the right thing" keeps happening over and over again.