We measured 1,610 production runs of Apex, our autonomous bug hunter. The return on compute follows one universal curve, and at the calibrated budget, it recovered 100% of the criticals and highs from independent human audits, plus live bugs that were missed.
When you give Apex more compute, you find more bugs. Across 1,610 production runs on real codebases, the same shape shows up every time, and it's sub-linear. Doubling your compute adds about 40% more findings, not 100%.
Each dot is one run. Both axes are rescaled so 1× equals a median-budget run, which puts runs on very different codebases on a comparable plot. The dark line is the cohort median; the shaded band is the middle 50% of runs. The dashed line is a fit (findings ∝ compute0.54). That's where the "40% per doubling" comes from.
"Validated" means a finding survived Apex's automated validator stage, same definition used in every chart below.
A curve through a cloud of dots could just be an average hiding a lot of noise. It isn't. When we fit the same power law to each run separately (1,033 runs with enough trajectory to fit), the exponent clusters tightly around the cohort value. The system doesn't scale efficiently on average; it scales the same way almost every run.
It goes further than a shared exponent. For the runs that go long enough to bend toward their ceiling, if you rescale each one by its own two fitted constants, the compute it takes to get halfway, and the findings it tops out at, every trajectory falls onto a single master curve. Different codebases, different budgets, one shape.
Some audits saturate: past a point, more compute stops yielding new findings. Among long audits, about one in four (24%) shows a clear ceiling, but the more striking number is the other side: 74% were still scaling at the budget they were given, with only a 2% sliver too ambiguous to call. Most audits, in other words, still had headroom when they stopped.
The bend is visible directly. Each line is one audit, with both axes normalised by that audit's own totals: every curve runs (0%, 0%) → (100%, 100%), so the only thing that varies is the shape. Saturating audits (red) collect most findings early and flatten; still-scaling audits (blue) track closer to the linear diagonal. We classify the shape with a standard model-selection test (Hill curve vs. power law); the caption gives the exact threshold.
This is the same saturation that produced the master curve in §1. The Hill fit that flags a ceiling here is exactly the fit that hands each run its two constants, half-saturation compute and finding ceiling, and once you rescale by those, these red trajectories are the points that collapse onto the universal curve. Saturation isn't a separate phenomenon from the scaling law; it is the scaling law, seen at the budgets where it starts to flatten.
The histogram below shows how the cohort splits on this score. The mass sits overwhelmingly to the right (still-scaling, 74%) with a clear-ceiling tail on the left (24%); the indistinguishable middle between −4 and +4 is only 2%, so this is close to a clean two-way split, not a three-way one.
If you split each audit's progress into 10 equal slices and look at what proportion of findings in each slice are critical, high, medium, low, and informational, the bands barely move. Critical hovers around 5–11% per decile, high around 23–35%, medium around 26–34%. No single decile is preferentially loaded with high-severity findings; any slice of an audit is roughly a representative sample of its severity mix.
The one mild trend: high-severity findings are slightly over-represented early (34% in the first decile, 23% in the last) and informational is slightly over-represented late. That's the only directional shift visible in the data; everything else is decile-to-decile noise.
If you re-run the same audit on the same codebase, the results will differ, and how much they differ depends on severity and budget. We measured it two ways: a stress test (44 runs on one codebase, deliberately varied across budgets and configurations) that bounds the worst case, and a controlled replicate (3 strictly identical re-runs) that shows what reproducibility looks like once that variation is removed.
The solid lines are the two launch-relevant severity classes, critical and high, across the varied-config family. The y-axis is the coefficient of variation in cumulative synth-valid findings at a given compute level, standard deviation divided by mean. Lower is more reproducible. Both drop roughly an order of magnitude as compute grows, from 200–400% CV at small budgets down into the 30–55% range by mid-budget.
The dotted lines are the controlled version of the same test: 3 strictly identical runs, same codebase, same configuration, no deliberate variation. They sit dramatically lower, and both converge to ~5–6% CV by 0.8× of their budget: critical 5%, high 6%, against ~137% and ~95% for the varied-config family at the same relative budget. Strip out config drift and the hunt is near-deterministic. Hover any point to read its CV.
§1 showed Apex's return-on-compute is predictable and universal. That's the foundation for a product decision: we engineered the audit scan to climb that curve efficiently, then calibrated how much compute it spends so it always reaches the point where it has matched an independent human audit's critical and high findings, including engagements that cost $500k+ and ran for months. Across the 8 codebases where we have a human audit to check against, it did: every critical, every high.
An audit scan is the configuration we run when coverage is the goal: higher per-task budget, deeper exploration, stricter validation. We picked the 8 rows below by availability: every codebase where both an Apex audit scan and an independent human audit exist, with no further filtering. Each cell is bugs Apex recovered / human-confirmed bugs the auditor posted (any non-withdrawn status). Green = 100%; n/a = no human bugs of that severity on that codebase.
| Audit | Critical | High | Medium | C + H + M |
|---|---|---|---|---|
| Audit 1 | 1 / 1 | 5 / 5 | 8 / 1362% | 14 / 1974% |
| Audit 2 | 1 / 1 | 4 / 4 | 6 / 875% | 11 / 1385% |
| Bug Bounty 1 | n/a | 2 / 2 | 7 / 888% | 9 / 1090% |
| Audit 3 | 2 / 2 | 13 / 13 | 18 / 18 | 33 / 33 |
| Audit 4 | 2 / 2 | 3 / 3 | 2 / 367% | 7 / 888% |
| Audit 5 | 2 / 2 | 4 / 4 | n/a | 6 / 6 |
| Bug Bounty 2 | n/a | 1 / 1 | 4 / 4 | 5 / 5 |
| Audit 6 | 1 / 1 | n/a | n/a | 1 / 1 |
| Pool · all 8 audits | 9 / 9100% | 32 / 32100% | 45 / 5483% | 86 / 9591% |
Pool stats across all 8: 9/9 critical, 32/32 high, 45/54 medium (83%), combined 86/95 = 91%. On 5 of the 8 codebases, medium coverage is below 100%; the misses are real and concentrated at that severity.
And this table understates what Apex did. It only credits Apex for matching the human auditor's findings. It gives no credit for what Apex found beyond them. On multiple of these engagements, the audit scan surfaced critical and high-severity bugs that the original human audit missed entirely, still live in the codebase, which we reported to the teams to fix. The 100% you see here is matched-to-human; the actual finding set was larger.
The calibration mechanism is the saturation stop from §2: the run climbs its own curve and halts once it reaches 95% of its estimated finding ceiling, far enough to clear the human-match point, without burning compute on a codebase that has already flattened. The figures below place each audit on its own scaling curve from §1. For each codebase, the dark bar is the AI compute at which Apex matched 100% of the human-confirmed crit + high set; the light grey bar is where the run actually stopped (its 95%-ceiling point). The green marker is where that codebase's actual human-audit budget would land on the same axis, if it were spent on AI compute at our blended provider rate. Everything is rescaled by the cohort-median run compute (1× = a median production run).
Point Apex at your repository and get critical and high coverage fast. Buying for a portfolio or running a firm? Talk to us about bundle pricing.