· 6 min read
An agent for Android performance regressions
perfettoagent diagnoses Android performance regressions from two Perfetto traces and a commit range, under one rule: no citation, no claim.
A perf alert fires on an Android app. Startup got slower, or scrolling started to jank. You have a trace from before, a trace from after, and somewhere between 5 and 50 commits in between.
Finding the commit that did it means moving between trace queries, git log and git blame by hand. It takes hours. It’s also a good fit for an agent, because the steps are well defined and the answer can be checked.
That last part is what I cared about most. An agent that says “commit abc123 probably slowed startup” is only useful if you can check it. So I built perfettoagent around one rule: no citation, no claim.
What it does
You give it two Perfetto traces and a git range. It gives back the metric that regressed, by how much, the commit that caused it, and the trace rows that show it.
Here’s the top of a real diagnosis. In this eval case I planted a synchronous sleep in a tap handler, somewhere in a 12-commit range:
Metric:
main_thread_blocked_ms+706.25 ms (from 25.28 to 731.53 ms)
Culprit:c70fd55eea99(direct),FeedScreen.kt
Verified: 5 claims kept, 1 dropped; 9 of 11 citations passed
The claim behind it says the commit added Thread.sleep(TAP_HOLD_MS) to the tap handler and set that constant to 120 ms, and that git blame at the head of the range puts both lines on that commit. The run made 43 tool calls and cost $0.0242.
The verifier does the checking
A prompt can ask a model to cite its evidence. Only code can confirm that it did.
The agent works the problem the way an engineer would. It measures the metric delta, finds where it shows up in the trace, lines it up against the commit range, then blames. Its tools are plain functions: canned Perfetto metrics that return the SQL behind each number, read-only SQL on either trace, symbolization through R8 mappings, and read-only git log, diff, blame and grep on the target repo.
The model has to answer in a strict JSON schema. Then a deterministic verifier goes through the answer before anything is written:
- every cited query is re-run, and must return a row;
- every cited commit must sit inside the range and change the cited path;
- any claim with a failed citation is dropped;
- a culprit that no surviving claim cites is dropped;
- if no claim survives, the verdict becomes
inconclusive.
The model’s own confidence is recorded and never scored. A model saying “high confidence” tells you about its tone.
How I measured it
Numbers in a README are easy to write. I wanted numbers I could reproduce, so I planted the regressions myself.
I used two apps: the demo app from my Media3 playback library, superPlayer, and the JetNews sample app. Each got five planted regressions: main-thread I/O on startup, an allocation storm, a synchronous sleep, layout thrash and a listener leak. Each plant sits in a range of 10 to 12 commits alongside decoys, commits built to look guilty while changing nothing the scenario runs. Each app also got five clean pairs, two captures of the same build, to count false alarms.
Every case ran three times, at four reasoning-effort levels, with gpt-5.6-luna:
| Effort | Detection | Attribution | False positives | Citation validity | USD / trace | Wall time / trace |
|---|---|---|---|---|---|---|
| low | 73% (22/30; 70%–80%) | 91% (20/22) | 20% (6/30) | 96% (347/363) | $0.0074 | 84 s |
| medium | 80% (24/30; 70%–100%) | 92% (22/24) | 20% (6/30) | 92% (379/414) | $0.0134 | 145 s |
| high | 87% (26/30; 80%–90%) | 100% (26/26) | 27% (8/30) | 95% (525/552) | $0.0289 | 264 s |
| xhigh | 93% (28/30; 80%–100%) | 100% (28/28) | 10% (3/30) | 96% (603/627) | $0.0441 | 319 s |
Detection means a planted case was called a regression, judged by the metric the case expects. Attribution is the share of those detections that named the planted commit. Citation validity is the share of citations the verifier passed.
At the top setting it caught 28 of 30 planted runs, named the right commit in all 28, and cost about 4 cents a trace. It took 1.3 to 5 minutes per case on superPlayer. I compare that with 60 minutes of manual triage, and that 60 is my own assumption. I didn’t time it.
What it got wrong
This is the part I learned the most from.
Most misses named the right commit. 20 of the 120 planted-case runs weren’t detections. In 16 of those, the agent found the culprit and then judged it by a different metric than the one the case expected, a metric the plant really does move. A 120 ms sleep in a tap handler delays that frame. Startup I/O delays the full display as well as the first. When I named the expected metric up front, all 30 runs at high detected and attributed. The commit search was right. The metric choice was where it slipped.
False positives came from noise. 23 of the 120 clean-pair runs called a regression, and 21 of those came from just two pairs. One JetNews scenario draws few frames, so its p95 frame time moved from 19.9 ms to 36.0 ms between two captures of the same build. The agent read that as real and blamed the one commit in range that changed how something is drawn: a decoy on a screen the scenario never opens. With one capture per side, nothing tells the model how far a clean run moves.
A loose attribution type made those easy. 22 of the 23 false positives used correlated attribution, which lets a claim blame a commit with no trace row pointing into its diff. The verifier checks that the commit is in range and touches the cited path. It doesn’t check that the trace points into the change, and the decoys were built to fail exactly that check.
The verifier dropped true claims too. 102 of 1,956 citations failed. 82 of them were one repeated mistake: the model copied a long metric query into its citation and mangled the GROUP BY, so the SQL wouldn’t parse. The verifier dropped every one, which is its job. But those were correct claims, lost to a copying error.
One run per case would have hidden a lot. At each effort level, 7 to 10 of the 20 cases gave different answers across their three runs. With a single run, I’d have reported whichever answer came up first.
What I’d change next
- Measure the noise. Give the agent more than one capture per side, so it can judge a delta against how far a clean run moves.
- Let only
directattribution name a culprit, which needs a trace row pointing into the commit’s diff. - Let a citation reference a tool result, so the model never retypes SQL.
- Steer the metric choice toward the one that moved most against its noise.
Limits
Every capture comes from one API 36 emulator, with a single capture per side. It reads Android traces only. It doesn’t capture traces, has no UI and doesn’t fix code: it points at a commit and stops. I also ran claude-opus-5-5 once on the leak case, and it found the culprit with every claim kept, but one run isn’t a sweep.
Why I built it
I’ve spent a lot of my career on Android performance: startup time, jank, memory, crashes, and reading traces to find out why. The agent runs the loop I’d run. What I wanted to know was whether I could trust its answer, and how often.
The code, the eval cases, every result file and the full postmortem are on GitHub at github.com/ramesh130/perfettoagent. Every number in this post comes from a file in evals/results/.
If you work on Android performance and want to point it at a real regression, I’d like to hear how it does.
Also published on Medium.