Understanding Latency: What p50, p95, and p99 Tell You About Your App

Your dashboard says the average request takes 120 milliseconds. A user says your app takes several seconds to respond.

Both can be true.

An average compresses many different experiences into one number. If most requests finish quickly, a smaller group of very slow requests can disappear behind an encouraging average.

Latency percentiles help you see that difference. You will often find them written as p50, p95, and p99. They describe different parts of the same distribution, from a typical request to the slower end of the experience.

In our previous post, we explored CPU, RAM, disk, and network bottlenecks. This time, we will look at how to measure the delays those bottlenecks can create.

First, define what you are timing

In application monitoring, latency usually means the elapsed time for a defined operation. That could be an HTTP request, a database query, or a background job.

The boundaries matter.

Your server might measure 80 milliseconds from receiving a request to completing its response. The browser can observe a longer duration because it also experiences network travel and may need DNS resolution, a new connection, or TLS negotiation. Rendering the page adds more work after the response arrives.

Before comparing numbers, ask:

• Where does the timer start and stop?
• Which operation or endpoint does it measure?
• Does it include failed requests and timeouts?
• Which users, regions, and time window does it cover?

A backend request duration and a browser page-load duration describe different parts of the journey.

What p50, p95, and p99 mean

Imagine sorting the measured request durations from fastest to slowest.

A percentile describes a position in that ordered set. With a common nearest-rank calculation, p95 is the value at the 95% position, rounded up to a whole observation. Some tools interpolate between observations, so exact results can differ slightly.

If your API has a p95 latency of 400 milliseconds, that does not mean every request takes 400 milliseconds. It also does not describe how slow the remaining requests become.

They might take 450 milliseconds. They might take ten seconds.

Percentiles are thresholds within the observations you measured, not guarantees for the next request.

How a good average can hide a bad experience

Consider this simplified example of 100 requests:

The average is:

(90 × 50 + 9 × 500 + 1 × 3,000) ÷ 100 = 120 ms

Using the nearest-rank method:

• p50 is 50 ms. The 50th request is in the fast group.
• p95 is 500 ms. The 95th request is in the slower group.
• p99 is 500 ms. The 99th request is also in that group.
• The maximum is 3,000 ms. The final request took three seconds.

That last detail matters: even p99 can leave an extreme request outside its threshold.

Look at several percentiles alongside request counts, errors, and representative slow traces. Each adds context the others cannot provide.

Tail latency: the requests at the slow end

The slower end of a latency distribution is often called the tail.

Tail latency can come from queueing, database locks, cache misses, runtime pauses, connection setup, retries, or a slow dependency. The metric identifies the symptom; investigation identifies the cause.

It matters even when the slow fraction seems small. At 100,000 requests in a measurement window, 1% represents 1,000 requests.

And requests are not users. A person who performs many actions can encounter several delays. A single page may also make many API calls, with one slow dependency holding up an important part of the page.

A p99 value therefore cannot be translated directly into “only 1% of users have a slow experience.”

Read the numbers together

Different patterns suggest different investigations.

p50 is stable, but p99 rises

Most requests are behaving much as before, while a smaller subset has become slower.

Compare slow traces with typical ones. Look for particular endpoints, larger payloads, cache misses, database locks, retries, or regions. Check whether the rise is sustained and whether there are enough observations to interpret it reliably.

p50, p95, and p99 all rise

The slowdown affects a broader part of the measured traffic.

Investigate shared dependencies, increased load, resource contention, recent deployments, and changes in traffic mix. A larger share of inherently expensive requests can also move the percentiles without every endpoint becoming slower.

Latency looks better, but errors rise

Fast failures can make a latency chart look healthy. If requests immediately return errors instead of completing useful work, lower durations are no victory.

Inspect success rate and timeouts alongside latency. Where possible, compare successful and failed request durations separately, while keeping an overall view of reliability.

These patterns help you choose where to look. They do not establish a root cause by themselves.

Keep comparisons fair

A percentile is only meaningful in context.

Compare similar operations. A health check, product search, file upload, and report export have different workloads. A single service-wide percentile can hide a slow endpoint behind a large volume of cheap requests.

Use the same time window. A five-minute p99 and a daily p99 describe different sets of requests. Short windows reveal bursts; longer windows provide context but can conceal brief incidents.

Check the sample size. With only a few dozen requests, p99 depends heavily on the slowest observations and the calculation method. A change may reflect a single request rather than a broad regression.

Understand sampling and exclusions. If your monitoring drops slow timeouts or samples only some requests, the displayed distribution may not represent all traffic.

Do not average percentiles. Averaging the p99 values of two replicas does not give you the combined p99. Combine the underlying observations or compatible histogram counts, then calculate the percentile for the combined distribution. Histogram-based percentiles are estimates whose precision depends on bucket resolution.

Turn a latency goal into something measurable

“Make the API fast” leaves too much undefined.

A more useful starting point might be:

Over a rolling seven-day window, at least 95% of successful product-search requests should complete within 400 milliseconds, measured at the API server.

This defines an operation, a population, a threshold, a time window, and a measurement boundary. The numbers are illustrative; choose targets that fit your users and workload.

Because this example measures successful requests, pair it with a separate availability or error-rate objective. You do not want a service to meet its speed target by failing difficult requests.

For threshold-based goals, measuring the fraction of requests within the threshold can be more direct than relying only on an estimated percentile. Be explicit about how timeouts are counted.

A practical investigation workflow

When users report slowness:

1. Choose the operation. Identify the endpoint or user action involved.
2. Check latency and reliability together. Review p50, p95, p99, request volume, errors, and timeouts over the same window.
3. Find the affected group. Break down by route, region, response status, or deployment version where useful. Avoid unnecessary high-cardinality labels such as individual user IDs.
4. Open slow traces. Locate time spent in application code, queues, database queries, and external calls. Traces are examples; check how they were sampled.
5. Make a targeted change. Address the observed wait, then compare performance under similar traffic.

The average tells you something about the total time spent across requests. p50 shows the middle of the distribution. p95 and p99 help expose the slower experience.

Use them together to connect a performance chart with the requests—and people—behind it.