A Stronger Review Engine: Every Benchmark Bug Caught, 2.6x Faster on Large PRs

Optibot now runs on a newer frontier reasoning model. We tested it against the previous engine on the same pull requests. The new engine caught more bugs, blocked problems the old one let through, and finished faster. It's live now for every pull request and IDE review, with nothing to turn on.

Benchmark

Same pull requests. Two engines.

100%

of known bugs found

• Previous engine 87%
• New engine 100%

The previous engine missed about one in eight. The new one found them all.

2.6x

faster on large pull requests

• Previous engine
• New engine
  Done

Both start together. By the time the new engine finishes, the previous one is not even halfway.

+84

findings the previous engine never raised

1.4x

faster core review, across every pull request size

Improvements

Every benchmark bug caught

Our benchmark tests include known bugs, and we check every review against an answer key, so we know exactly what a good review should find. The new engine found every one of them. The previous engine missed about one in eight.

Blocks what used to slip through

On pull requests the previous engine had approved, the new engine raised merge-blocking issues. We checked each one in the code, and they were real bugs. It also added 84 findings the previous engine never raised, while still catching most of what the previous engine found.

Deeper thinking on bigger changes

Optibot matches its reasoning effort to the size of your change. A one-line fix gets a quick, focused pass. A large pull request gets the most thorough analysis Optibot has ever run.

2.6x faster on large pull requests

More depth usually means more waiting. Not this time: the core review pass on large pull requests finishes 2.6x faster than before, and about 1.4x faster across pull requests of all sizes.

Reviews keep running during outages

If an upstream service goes down in the middle of a review, Optibot moves the review to a backup automatically and finishes it. You get your review either way.

What you might notice

The new engine holds code to a higher bar. You may see more findings marked as blocking, including on pull requests that used to pass cleanly. We'd rather you hear about a real bug before merge than after. If a finding looks off, reply to the comment and let us know.