Google engineer Addy Osmani recently proposed Loop Engineering: stop having manual conversations with AI, and instead design automated loops where AI agents run, fix, verify, and iterate until the job is done.
The concept is compelling. But after reading it, I kept getting stuck on one question: the loop finishes, AI says âdone,â how do you actually know itâs done?
Osmaniâs answer is sub-agent review, have another AI act as a manager to check the work. But AI reviewing AI is ultimately probabilistic. Both models agreeing something is correct doesnât make it correct. This post is about a building block he didnât mention, and why some environments are naturally better suited for AI agent loops than others.
Loop Engineering in 30 Seconds
Osmaniâs architecture has five key components plus a memory layer:
- Automations: Scheduled triggers, no manual kickoff needed
- Worktrees: Isolated parallel workspaces, multiple agents donât interfere with each other
- Skills: Project conventions written as docs, agent reads through them before acting
- Plugins/Connectors: Let agents access more data through real-world tools (Slack, DB, CI)
- Sub-agents: Separate the doer from the checker to avoid bias or blind spots
- External Memory: Progress files so the system can pick up where it left off tomorrow
The core idea is: upgrade from âmanual conversationâ to âsystem design.â But for quality assurance, all he gives us is sub-agent review.
But hereâs where the problem comes in: what if both AIs share the same blind spot?
The Missing Sixth Block: Deterministic Verification
Sub-agent cross-review has a clear ceiling: itâs still one model evaluating another modelâs output. If the output looks plausible and the logic holds up, the reviewer agent waves it through. But âlooks plausibleâ and âactually correctâ arenât the same thing.
Deterministic verification is different. It doesnât estimate, doesnât evaluate, doesnât use confidence scores. It gives you binary pass/fail, right is right, wrong is wrong, no gray area.
| Verification method | Nature | Signal on failure |
|---|---|---|
| Sub-agent review | Probabilistic | âI think there might be an issue hereâ |
| Unit test | Semi-deterministic | âThis assertion failedâ |
| Compilation + boot + CTS | Fully deterministic | âSystem refuses to start / hardware wonât cooperateâ |
Unit tests are half-deterministic, theyâre binary when they exist, but coverage is a human decision. Cases you didnât write wonât be caught.
But compilation failure, SELinux denial, boot loop, these arenât tests you wrote. Theyâre physical constraints of the system itself. You canât avoid these constraints.
Why AOSP / Embedded Naturally Has the Advantage
In web Loop Engineering, the hardest verification is usually CI test pass. But tests are human-written, and coverage always has gaps. An AI agent can absolutely produce code that âruns, passes tests, but is logically wrong.â
The AOSP environment is different. Verification isnât just your tests, itâs hard gates the system enforces at every level:
- Compilation fails = the type system rejects you. API doesnât exist? Canât even generate a binary.
- SELinux deny = the kernel rejects you. Permission not granted? Process blocked outright, not a warning, a denial.
- Boot loop =
PermissionManagerServicerejects you. State inconsistent? System refuses to start. - CTS fail = Googleâs compliance framework rejects you. Behavior out of spec? You donât ship.
These verification layers werenât designed to manage AI. They exist to protect system integrity. AI agents just happen to land in an environment where the defenses are already deployed.
A web appâs definition of âworkingâ is fuzzy, page renders, no 500s, users arenât complaining, probably fine. An Android deviceâs definition of âworkingâ is precise: boots up, passes verification, all services alive under SELinux enforcing mode. Binary standard. Thereâs no âprobably works.â
What the Loop Actually Looks Like
Mapping Osmaniâs building blocks to our existing AOSP workflow:
| Loop Engineering block | AOSP implementation |
|---|---|
| Automations | Jenkins CI / scheduled builds |
| Worktrees | git worktree (isolated per feature) |
| Skills | ~/.claude/skills/ + knowledge docs |
| Plugins/Connectors | AOSP MCP (module deps, sepolicy, HAL interfaces, init services) |
| Sub-agents | Gerrit code review + AI reviewer (fresh sub-agent, no author bias) |
| External Memory | Plan docs + active-projects.md + TaskList |
| Deterministic Verification | Compilation / boot / CTS / VTS, comes free |
A concrete loop in action:
Jira ticket arrives
â Agent reads source tree (MCP grounding, not guessing from memory)
â Agent generates CL
â Sub-agent review (fresh context, no author bias)
â Jenkins build (compilation gate)
â Device boot (PermissionManager gate)
â CTS subset (compliance gate)
â Human +2 (final checkpoint)
â Merge
Seven steps, three of which are deterministic gates. The agent wants to bypass them? Not possible. The compiler doesnât accept âlooks plausibleâ as an argument.
What About Web?
This isnât to say web canât do Loop Engineering, of course it can. But if you want to move toward deterministic verification, youâll need to add some things yourself:
- TypeScript strict mode = approximation of a compilation layer (type errors block execution)
- Contract testing / schema validation = approximation of boot-time checks (API contract mismatch = fail)
- Property-based testing = closer to deterministic than unit tests (auto-generates edge cases)
But honestly, these are all approximations. The web runtime is too forgiving, undefined is not a function wonât make your server refuse to start, itâll just blow up for one user on one request. Embedded runtimes arenât that polite: wrong means the entire device doesnât move.
No Hard Gate Is Omnipotent
Deterministic verification has blind spots too. Correct logic but terrible UX, performance regressions, race conditions, none of these will fail compilation or trip CTS. You still need human judgment.
But the bottom line is clear: at minimum, âwill the system blow up?â is a question that AI agents canât break in an AOSP loop. They can make mistakes, but those mistakes canât escape into production. Achieving this bottom line in web environments takes some effort.
Loop Engineering is the architecture, hallucination defenses provide the theoretical foundation, AOSP MCP is the implementation. Stack all three and the conclusion is: not every codebase is equally suited for AI agent loops. Environments with deterministic verification are naturally a safer playground.
References
- Addy Osmani, Loop Engineering, 2026
- Why AI Confidently Makes Things Up: From Root Causes to Practical Engineering Strategies, companion piece
- S-LoRA / vLLM multi-LoRA serving infrastructure
- Android CTS (Compatibility Test Suite) documentation