Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 30 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 20:59:21.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.

Unlike its peers 3

stored 34h ago all 3median/MAD against the fleet

Services on production whose behaviour stands apart from the rest of the fleet over the last 30d — judged by the outliers system alert's settings: at least 3.5 MADs and 35% from the peer median, 5+ peers. ?Each service is compared against every other service, using the median and the median absolute deviation rather than a mean and a standard deviation. A mean is dragged around by the very outlier being hunted: one service at 40× the others inflates the deviation until nothing looks unusual, including itself. The robust version does not move, so the thing that stands out still does.

How to read this page — peer comparison

Every service is compared against every other service in the same cluster. Nothing is configured per service — no thresholds to maintain for eighteen services on four clusters — because the question is "why is this one different" rather than "is this over 400ms". The comparison's own settings (how far counts, which measures, who is never compared) are one system alert with system defaults an admin can change and reset.

how far out
How many median-absolute-deviations this service sits from the fleet median. Median and MAD rather than mean and standard deviation, because a mean is dragged around by the very outlier being hunted: one service at 40x the others inflates the deviation until nothing looks unusual, including itself.
fleet median
The middle value across the other services, so half are above and half below. Not an average.
worth a look
This service is on the bad side of the fleet for that measure. The other direction is shown too, quietly — being unusually fast is interesting, but not a problem.
Neo4j queries per request
A chatty service. This is the one that scales badly: twice the traffic means twice the queries per request as well.

What this page will not tell you

  • That being different is being wrong. rag-agent is slower than everything else because it does retrieval and generation; that is its job, and it will sit on this page forever.
  • Anything at all when fewer than five services have real traffic — "unlike its peers" needs peers, so the page stays empty rather than comparing three things.
  • A difference that is large in deviations but trivial in milliseconds. A fleet between 4 and 6ms has no meaningful outlier at 9ms, and reporting one is how these feeds become noise people mute.
Columns
Views
Export CSV
Services differing from their peers on production, most extreme first
Service Measure This one Fleet median How far out Reading
virtual-assistant p95 latency (ms) 5947.1 711.4 12.3× above the fleet — worth a look · vs 9 peers
dashql p95 latency (ms) 4378.9 711.4 8.6× above the fleet — worth a look · vs 9 peers
virtual-assistant p99 latency (ms) 18212.7 1907.2 7.8× above the fleet — worth a look · vs 9 peers