← All writing

AI Engineering

Why Agent Infrastructure Is Moving to Rust

Mani G 3 min read

Macro photo of a rusted iron gear meshing with a polished steel gear, sparks drifting in a dim workshop.

I have a theory that every agent tool eventually gets rewritten in Rust, and the only question is whether the team admits it in a blog post. In the last eighteen months, two very big teams admitted it. Their reasons are more interesting than "Rust is fast", and one of them came with a token bill worth putting your tea down for.

Codex CLI went first

In May 2025, OpenAI announced it was rewriting Codex CLI, its terminal coding agent, from TypeScript into Rust. The engineer's post was refreshingly practical. Users needed Node 22 or newer just to install it, which annoyed people. The Linux sandboxing already leaned on Rust because the bindings existed. No garbage collector meant lower memory use. And a native core made it easier to build a wire protocol so others could extend the agent in other languages. Today npm i -g @openai/codex quietly installs the native build.

Notice what's missing: a startup-time benchmark. You may have seen the claim that Codex dropped to under 300 milliseconds after the rewrite. OpenAI didn't say that. That number belongs to a different project.

The Copilot runtime's Rust numbers

The number comes from GitHub. In September 2026, Stephen Toub wrote up how GitHub ported the Copilot agent runtime, the engine behind Copilot CLI, the Copilot SDK and agent features across VS Code, Visual Studio and Office apps, from TypeScript on Node.js to more than 800,000 lines of Rust. AI agents wrote most of the code across 128 pull requests, mostly driven by one developer over a few months.

The pain point was embedding. Every SDK user in C#, Python, Go or Java had to spawn a Node process and talk to it over JSON-RPC, roughly 100 MB of extra working set per client for a runtime the host didn't want. Now the runtime loads straight into the host process.

The benchmark: creating a client, a session and running one turn went from 5.25 seconds to 1.33 seconds with Rust in a separate process, and 292 milliseconds in-process. That's 18 times faster. Memory for a ten-client batch fell from about 1,383 MB above baseline to 126 MB. The fine print matters, and Toub is upfront about it: the test used a canned model server on localhost, so it measures the harness, not how quickly the AI answers you.

A quick example

Picture a service that spins up an agent for every pull request. If each agent costs a Node process and five seconds of setup, 50 PRs landing at once means a small traffic jam and a big memory graph. At under 300 milliseconds and a fraction of the memory, the same machine absorbs the burst without anyone noticing. That's the real argument for Rust here. It's density and embedding, not bragging rights.

It's showing up everywhere else

Look around the agent stack and the pattern repeats. Firecracker, the microVM behind many agent sandboxes, is Rust. Obscura, the headless browser for agents, is Rust. Cloudflare's new Kitesurf browser team wrote "use Rust when possible" into their design rules.

I won't pretend Rust is free. Toub lists regressions from stricter libraries, and he says plainly that this is no claim that every big TypeScript program should become Rust. Python still owns training and notebooks, and it should. But for the runtime that sits under thousands of concurrent sessions, the trade looks settled to me.

Now the tea moment. Toub reports the port consumed about 136 billion tokens, mostly cached reads, for a bill of roughly $120,000. That's cheap for a rewrite this size and terrifying for a personal credit card. Would your team spend that to make its agent harness 18 times faster to start?