Reliability measurement for LLM inference

Served LLM inference fails under load in ways dashboards are worst at counting: the request that never returns, the stream that dies mid-answer, and the 200 with nothing in it.

Percentes is built to measure exactly those. Load is dispatched on a schedule fixed before the run, and latency is measured from each request's intended send time rather than its actual one, so a stalling backend cannot slow the load generator into hiding its own worst moments (the fix for coordinated omission). Every scheduled request is accounted for as completed, errored, or censored: still running when the pinned timeout expired, so known only to have taken at least that long. Completion-incidence curves (Aalen–Johansen: errors are competing terminal events, only timeouts are censored) are computed over every scheduled request, so the ones that never finish still count. And the client must pass four pinned self-checks before any number counts.

Current status, plainly: the instrument is certified against a mock serving stack, and the first run on real hardware is the 16 September 2026 single-replica calibration, published under Writing. No provider measurements are published.

Percentes is built and run by Varun Mahadkar. The instrument is open source, and the methodology was pre-registered before any measurement data was collected. A number is published only with the commit and configuration that produced it.

• Writing: measurement notes and teardowns, so far The failure your dashboard can't see, One event, five status codes and a class and What one vLLM replica on an L40 can carry and how it fails
• Methodology: what the numbers mean and what would invalidate them
• The instrument: Go, Apache-2.0