At Netflix, an engineer who suspects a service is burning too much CPU has a slow way to find out. First, trigger a profiler against a single production instance. A profiler is a tool that samples a running program to record which functions are using CPU time, without touching the program’s own source code. Next, download the output. The raw data is a JSON dump of a call stack, the chain of function calls active at the moment the profiler took its sample, and nobody reads that dump by hand. So it goes into a visualizer. Then the real work starts. Rajat Shah, a staff software engineer on Netflix’s AI platform team, called it “this treasure hunt” [1]: many minutes spent searching mostly ordinary code for the handful of lines that are actually the problem.
That hunt happens rarely. It’s slow, and reading a call stack well takes a learning curve of its own [1]. Shah’s team asked whether an AI agent could read the same profiler output and find the same bottlenecks faster. Shah presented a proposal for how to do exactly that at AI Engineer World’s Fair 2026. The more interesting part is how he proposes to make one service’s finding available to every other service Netflix runs [1].
We follow the talk in Shah’s order: why the manual process stayed rare, why an agent reading a profiler on its own only gets partway there, and why the fix he proposes is architectural [1].
Figure 1 - The Loop Shah Proposes: A profiler’s raw call stack goes to an agent, which proposes a fix. A human approves it before anything reaches production. Only then does the confirmed pattern get written into a markdown catalog held in Git, for a future agent run on any service to read before it starts hunting from scratch.
Why the profiler mostly gets read at 2 a.m.
A performance engineer doing this the old way triggers profiling on one instance out of the fleet, meaning the many identical copies of a service running in production. The engineer downloads the raw call stack and opens it in a visualizer, since the JSON alone is unreadable. From there, in Shah’s words, the engineer has “to spend… many, many minutes to identify the bottlenecks,” a process with “a learning curve” of its own before anyone gets good at it [1]. Once a suspect method turns up, the engineer searches for it across the company’s code repositories, confirms it’s actually the cause, writes a fix, and sends it out for review. Then the cycle starts over on the next service.
That cycle is so slow and manual that, in Shah’s words, it “is done very rarely. People typically end up looking at profiling data only when something is going wrong at 2:00 a.m.” [1]. In practice, profiling happens during incidents.
Netflix’s engineers have done a version of this by hand for a decade. A flame graph turns thousands of profiler samples into one picture. It stacks them so the widest bars mark the code paths burning the most CPU [2]. Netflix’s own engineering blog was already writing about the practice in 2016, in a post titled “Saving 13 Million Computational Minutes per Day with Flame Graphs” [3]. The visualizers got better over that decade. The work still needed a person with time to read the output. That’s why it stayed rare, and we think it’s the real constraint Shah’s talk responds to.
The pressure grew because the code arriving in front of performance engineers changed [1]. With coding agents, Shah says, engineers are “authoring code now at a 10x faster speed,” and compute cost is “increasing at a similar pace,” which he calls a “slight exaggeration” [1]. The cause is simple. An agent doesn’t know a specific platform’s internal conventions, so it copies patterns it has seen elsewhere or invents new ones nobody at Netflix expected [1].
Figure 2 - The Cycle This Replaces: Every step in the old loop is manual: trigger the profiler, download the call stack, open a visualizer, hunt for the bottleneck, fix one service, and start over on the next one. Shah says this happens rarely, usually only once something is already failing in production.
An agent reads the call stack and skips the hunt
Shah’s team started from two assumptions and tested both against real services [1]. The first is that every profiler speaks roughly the same language, whatever the service is written in. Java, Python, and Go all produce a call stack sampled at high frequency, with self CPU (time spent inside a function’s own code) and inclusive CPU (that time plus everything the function calls). The second is that a coding agent, trained on a large volume of public code, already recognizes common performance anti-patterns. An O(n^2) loop, also called a quadratic algorithm, is one example. Its work grows with the square of the input size, so it’s fine at 10 rows and ruinous at 10 million. A value recomputed on every pass through a loop, when it could be computed once outside it, is another. So is an object allocated over and over on a path that runs constantly.
Both assumptions held. Given a profiler’s raw output and no access to the source code, an agent identified a case where two rows of a call stack represented the same call path. The culprit was an immutable map copy inside a merge method. An immutable map is a key-value lookup that can’t be changed, so every update builds a whole new copy [1]. In Shah’s words, the agent “can actually identify that this is a quadratic… algorithm and not a linear algorithm,” and “this is not by looking at the code base. This is purely by looking at the call stack” [1].
So the agent found a real inefficiency from the shape of a profiler’s output alone, without reading a line of source code. But a flagged call stack is only a lead. A human engineer needs a change to review and merge. Something still has to find where that code lives, confirm what’s actually running in production, and turn the finding into a diff.
Figure 3 - Finding a Pattern Without Opening a File: The agent recognized a quadratic algorithm from two matching rows in the raw call stack alone. No source file was open at this point in the process. The pattern is visible in the shape of the profiler output itself.
From a flagged call stack to a change a human can approve
Turning a flagged call stack into something reviewable takes four more steps. Shah says an agent with the right skill or prompt can do all four in the same run [1]:
- Code-search the repositories to find where the flagged method lives.
- Check out the exact commit currently running in production, the one the profiler actually sampled, not whatever is newest on the main branch.
- Trace the full call path inside that exact code.
- Produce a code review.
For the O(n^2) finding above, Shah says the full sequence, from a raw call stack to a code review a human could act on, “could be done… in a very large code base with powerful enough code agents in less than 5 minutes” [1]. That one method was “consuming 8.8% of the CPU time during that… period of profiling” [1].
A second finding showed what the same pipeline could do when it searched across repositories. This time the pattern was a metrics-counter object, a small object that counts events for monitoring. The code allocated a fresh one on every iteration of a hot path, the code that runs most often, where small waste multiplies. A cross-repo code search turned up the exact same pattern “implemented in seven different services” [1]. Shah estimates that fixing it in all seven would save “between 0.5 to 4.6% of CPU cycles” [1].
That second finding also shows the pipeline’s real limit. The cross-repo search worked because the agent still had the pattern in hand during that one run. When the run ended, the pattern went with it, since an agent has no memory between runs. The next time any service gets profiled, next week or next quarter, the agent starts the hunt from zero, even for a bug it has already found seven times. Shah’s point is that a later agent profiling a different service should be able to use the first agent’s finding “and not having to redo all the exercise that the first agent did” [1].
Figure 4 - Finding to Reviewable Fix in Four Steps: Code search locates the method, the agent checks out the exact commit running in production, traces the full call path in that code, and produces a code review. On one real finding, consuming 8.8% of profiled CPU time, Shah says the sequence can reach a code review in under 5 minutes.
Figure 5 - Found Seven Times in One Search, Remembered Zero Times After: One cross-repo search turned up the same counter-allocation pattern in 7 services. When the run ended, nothing kept the pattern for the next profiling run.
KEY INSIGHT: Each profiling run produces a confirmed pattern as well as a fix. If nobody writes the pattern down, the next run pays to find it again.
A stateful catalog for a model with no memory of its own
An LLM (large language model, the kind of model behind a coding agent) does carry some memory these days, Shah says, “but it’s very compact… it won’t have all the information… that you potentially as a performance engineer know when you’re trying to debug” [1]. His proposed fix changes the architecture around the model. Build a catalog of confirmed patterns and anti-patterns, keep the agent itself stateless (holding nothing between runs), and let the catalog do the remembering. In his words: “a stateful catalog and a stateless LLM can become a full fleet-wide memory” [1].
Shah’s reason for the format is practical. Both a human and a coding agent can read and write a Git repo, so an engineer building a new framework can add a pattern as easily as an agent can [1]. He’s specific that the catalog “is not very fancy vector search or a vector database… you can start with just markdown files in a centralized Git repo” [1]. Git adds more. A file there gives an agent exact matches on names, a history of who confirmed what, and a diff on every change. Similarity search, the approximate matching a vector database does, gives none of that.
A catalog can start empty and grow from confirmed findings. Shah also points to public material that can seed one. The first is a performance-hints guide from Jeff Dean [1][4], written with Sanjay Ghemawat. The second is PyTorch’s torchfix repository, which Shah describes as cataloging anti-patterns for kernels and model graphs [1]. Torchfix’s own repository describes itself differently, as a PyTorch linter with autofix, and it was archived in January 2026 [5].
The example entry Shah shows has four fields [1]. It lists the symbols (the function and class names the pattern involves) that an agent can search for quickly, and the services where the pattern was confirmed. It also carries a confidence level that rises as more services confirm it, and an anti-pattern next to its better version.
Figure 6 - What One Catalog Entry Holds: The example entry Shah shows carries four fields: symbols an agent can search for quickly, the services where the pattern was confirmed, a confidence level that climbs as more services confirm it, and a paired anti-pattern and good-pattern example.
Checking the same shape against Uber
A pattern from a single source invites an obvious objection: maybe this only works at Netflix’s scale, with Netflix’s engineering culture, on Netflix’s kind of service. We found a second data point in a separate talk. Two Uber engineers, Uday Kiran Medisetty and Adam Huda, co-presented a talk at AI Engineer World’s Fair 2026 describing a structurally similar idea that solves a different problem at a scale Uber measures in millions of entries [6]. Uber built a context graph, a database of things like services, teams, and design docs (the nodes), plus the links between them (the edges): “one context graph… 150 unique node and edge types. We have 40 million entries there right now,” Medisetty says [6]. The graph exists because agents kept burning tokens (the units of text a model reads and is billed for) hunting for basic context: which system a service lives in, what it depends on, who owns it. That information was scattered across “20 to 30 different systems” [6]. The content differs from Netflix’s, since it’s organizational knowledge instead of performance anti-patterns. The shape is the same. Write the answer down once, in a structure an agent can query, so nobody re-derives it from scratch on every run.
A reader might ask why Uber didn’t just add more live connectors instead of building a shared structure. Uber’s talk touches on that, though its MCP story is about a related but separate problem. MCP (Model Context Protocol) is a standard way for an agent to reach outside tools and data, and Uber’s problem is what happens once an organization already has a lot of those connectors. “Once you end up with enough MCPs,” Medisetty says, “they’ll all add up to… a massive token tax” [6]. Uber didn’t cut tools. It still runs more than a thousand MCP tools. It put them behind one gateway and one discovery entry point, and changed how tool responses reach the agent so they stop filling its context. Medisetty credits those optimizations with “more than 40% fleetwide savings” [6]. That number belongs to the connector work, not to the context graph. The graph is Uber’s separate answer to agents hunting over and over for the same basic context.
None of that changes where a team without Uber’s platform investment should start. Uber’s own graph started from traces showing agents “spending lot of time even trying to find basic context” [6]. The floor for this pattern is one markdown file in one Git repo. Uber’s numbers show how far the same structure can grow.
Figure 7 - Same Shape, Different Scale: Netflix’s markdown catalog and Uber’s context graph hold different content but a similar shape: a compiled structure an agent queries instead of re-deriving context from scratch. Uber separately reported more than 40% fleetwide savings from optimizing how its MCP tools are served, a result tied to its connector work rather than to the context graph.
KEY INSIGHT: Use a vector database when the job is approximate similarity across a large, fuzzy space. Use a plain file in Git when you have a specific, confirmed fact that needs exact search, a review history, and a diff.
Two checks before any human sees a pull request
Shah doesn’t let a confident pattern match skip the human reviewer. His reason is that changing code that is already correct and running in production is inherently risky, especially without full visibility into business context or test coverage. In his words: “I’m still keeping the confidence bar to just send a code review and not directly push it to production. That’s by intent” [1].
Shah recommends two automated gates before a code review reaches a human [1]. The first is the existing functional test suite. The agent runs it before proposing anything, to catch business-logic regressions the optimization might introduce. The second is a canary deployment, which sends a slice of live production traffic to a new version running next to the old one, to compare real behavior before a full rollout. Shah’s description: “two machines, one containing your old code, another containing your new code… the same traffic to both of them over a period of let’s say 10 minutes” [1]. The agent compares CPU, latency, and error rate between the two. An error-rate increase, in his words, should be “a red signal to not proceed” [1]. His summary of the whole arrangement: “Profiler gives the estimate, canary gives ground truth” [1]. We’d want that kind of check in place before letting any agent touch production code.
Figure 8 - What Shah’s Recommended Setup Checks First: In Shah’s recommended setup, the agent runs the existing test suite first, then a canary deployment sends identical traffic to the old code and the proposed fix side by side for roughly 10 minutes and compares CPU, latency, and error rate. An error-rate increase is meant to block the change before a human ever sees it.
KEY INSIGHT: A pattern match narrows where to look. Confirming the fix is safe still takes a separate check. Profiler-driven findings need an independent check against real traffic before anyone trusts them, the same as any performance change.
Moving the catch point earlier
Everything above describes a reactive workflow. The code is already running in production, and the agent investigates after the fact. Shah’s advice to teams: “Don’t think of… the reactive path as a bad approach… reactive path is where you want to start with” [1]. The reactive path is what builds the catalog in the first place, and nothing downstream of the catalog can work until it exists.
Two further stages exist only as a stated direction, not as something Netflix has shipped and measured. The first is review-time. Once the catalog has enough entries, a reviewer agent could consult it during code review and leave an inline comment flagging a newly introduced anti-pattern before it merges. The second is authoring-time. The coding agent would consult the catalog while it generates code, before it ever writes the inefficient version out. Shah is candid about the tradeoff that second stage would introduce. Checking the catalog mid-generation “could sometimes slow down” code generation and might end up “consuming more tokens” [1]. That’s why he says the catalog needs to be indexed hierarchically rather than flat. An agent should be able to jump to the one relevant entry without loading the whole catalog into its context window, the limited text it can hold at once [1].
Figure 9 - Reactive Today, Earlier By Design: The reactive stage, investigating code already running in production, is the only stage with measured results in the talk, and Shah names it as the place a catalog starts growing. Review-time and authoring-time guidance move the catch point earlier in the development cycle, but both remain the stated direction rather than a deployed system.
Where this stops working
Shah maps all of this onto a three-level autonomy ladder. Before the ladder comes the fully manual status quo: hours spent finding problems by hand, with no LLM involved at all. Level 1 brings in an LLM only to identify a potential fix. A human still triggers the profiler and still runs the canary [1]. Level 2, what this talk describes, is the fixed, predefined workflow above. The agent identifies and proposes fixes, a human approves every pull request, and once automated it runs on a weekly schedule. Level 3 would let the agent plan, reason, and act with less fixed structure. Shah defers it explicitly, pending stronger sandboxing (running the agent where it can’t damage real systems) and defenses against prompt injection (instructions hidden in data that hijack the agent) [1]. Netflix has not built Level 3.
Figure 10 - How Far Netflix Has Actually Gone: Before the ladder is the fully manual status quo. Level 1 brings in an LLM only to identify a potential fix, with a human still running the profiler and the canary by hand. Level 2, what this article describes, is a fixed workflow where a human approves every pull request, running weekly once automated. Level 3, an agent that plans and acts with less fixed structure, is explicitly deferred pending stronger sandboxing and prompt-injection defenses.
The other honest limits are about the evidence itself. Every number here, the 8.8%, the 7 services, the projected 0.5% to 4.6% range, the roughly 10-minute canary window, comes from Shah’s own account of Netflix’s internal system on a conference stage. None of it comes from a published benchmark or an outside audit. The talk also reports no measurement of the catalog itself, such as how often a later run reused an entry or what that reuse saved. Every measured result in this article comes from the profiler-to-code-review pipeline. Netflix has written publicly about the manual flame-graph practice this replaces [3]. We found no written Netflix source for the agent system itself. Everything here comes from Shah’s talk. The catalog’s seed sources age too. Torchfix, one of the two public seeds Shah points to, is archived and no longer maintained as of this writing [5]. A catalog meant to last for years has to plan for its sources going stale.
Start with one markdown file
The gap Netflix’s engineers describe is architectural. An agent that could already spot a quadratic loop from a call stack still forgot everything it learned the moment its run ended, so each new profiling run had to rediscover patterns from scratch. Shah proposes writing every confirmed finding into a markdown file in a Git repo, to turn that repeated rediscovery into a catalog that compounds. The talk doesn’t report a number for what that reuse would actually save. Uber’s context graph, a separate structure solving a separate problem, is evidence the same shape scales well past a single company’s use case.
The place to start is smaller than either example: one markdown file, in one Git repo, next to whichever service already has a profiler pointed at it. Write down the first confirmed pattern, the symbols an agent can search for, and the anti-pattern next to the fix that replaced it. Only add findings a human has actually confirmed.
Two companion pieces sit on either side of this one. For the loop that decides which services get profiled first, and how findings get scored against impact and risk, see Agentic Performance Audits: From Production Blind Spots to ROI-Scored Fixes [7]. Shah’s catalog is also one case of a wider shift in how these systems handle memory: build the expensive answer once instead of re-deriving it on every run. We mapped that shift in The Context Engineering Stack: Compression, Retrieval, and Decision Memory [8].
References
[1] R. Shah, “AI Agents for Performance: Ship Faster, Pay Less,” Netflix, AI Engineer World’s Fair 2026, YouTube, Jul. 28, 2026. https://www.youtube.com/watch?v=CgsWxRUY5Eo
[2] B. Gregg, “Flame Graphs.” https://www.brendangregg.com/flamegraphs.html
[3] Netflix Technology Blog, “Saving 13 Million Computational Minutes per Day with Flame Graphs,” Apr. 11, 2016. https://netflixtechblog.com/saving-13-million-computational-minutes-per-day-with-flame-graphs-d95633b6d01f
[4] J. Dean and S. Ghemawat, “Performance Hints,” Abseil, Google. https://abseil.io/fast/hints.html
[5] “meta-pytorch/torchfix,” GitHub (archived Jan. 28, 2026). https://github.com/meta-pytorch/torchfix
[6] U. K. Medisetty and A. Huda, “Agentic SDLC at Uber: Building Blocks for Uber’s Software Factory,” Uber, AI Engineer World’s Fair 2026, YouTube, Aug. 21, 2026. https://www.youtube.com/watch?v=17-YSUHo6Lk
[7] G. Dotzlaw, “Agentic Performance Audits: From Production Blind Spots to ROI-Scored Fixes,” Dotzlaw Consulting, Sep. 14, 2026. /insights/ai-40-agentic-performance-audits/
[8] G. Dotzlaw, K. Dotzlaw, and R. Dotzlaw, “The Context Engineering Stack: Compression, Retrieval, and Decision Memory,” Dotzlaw Consulting, Aug. 12, 2026. /insights/ai-16-context-engineering-stack/
Building production AI, or modernizing a legacy system?
That is the kind of work we do at Dotzlaw Consulting. Book a free 20-minute intro call and tell us what you are trying to build, or what is slowing you down.