Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 30 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 16:55:43.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.

Findings 10

stored age unknown all 43detectors over stored rollups and live sources

What X-ray is prepared to defend about test27 right now — each one a claim with its numbers, the evidence to check it, and what normal looks like. Detectors last ran never. ?A finding is opened by a detector — a statistical test over the stored rollups, or a live check of a source — and stays ONE row while the condition persists: it updates, escalates, and resolves itself fifteen minutes after the detector stops seeing it. Acknowledge to say you have it; snooze to hide it for a while; nothing here is a threshold somebody configured.

How to read this page — how findings work

Every card is a claim X-ray can back with numbers you can click through to. Findings rank by severity first, then by an impact score (severity band + the subject's share of the cluster's traffic + how long it has been open), so the top of the page is the thing to look at first.

anomaly
Three of the last four five-minute buckets far outside what this hour of THIS weekday normally sees — median ± MAD over the last 2–8 weeks, not a threshold.
outlier
A service unlike the rest of the fleet on one measure, over six hours. Compares with peers, not with its own history — the two catch different things.
budget
Burn rate against an error objective derived from the service's own last week, and the date the 30-day budget runs out if the trend holds.
resolves itself
A finding a healthy detector run no longer observes resolves after fifteen minutes. A detector whose SOURCE fails resolves nothing — it says so on the Detectors page instead.

What this page will not tell you

  • That a finding is a diagnosis. It is a measured departure from normal with adjacent changes listed; the narrative says 'adjacency, not proof' and means it.
  • Anything about hours the ingester did not cover. A bucket with no rows is a gap, not a quiet period, and the baselines skip it.
  • Anything on the first day for the novel-error detector: its 90-day memory learns silently for 24 hours before it will call anything new.
show active resolved all severity everything warning + critical only clusters
test27
just test27 · nothing ticked means test27, the cluster selected up top
or just close the list
group all AnomaliesReliabilityNeo4jChangesJourneyX-rayWeaviateAIPodsJobsPlatformErrorsCustomCostCapacity + 4 fleet-wide 0 critical · 11 warning · 14 info detectors
warning Pod keeps restarting test27 resolved last seen 1d ago · since 1d ago · 1×

jss-server on test27 has restarted 9 times in 10 min — crash loop, image spreadsheet-agent-edits-d3202c0

jss-server in tilda on test27 has restarted in place 9 times since 2026-11-17 10:05 UTC (9 in the last 30 minutes, 9 in 24 hours), most recently at 2026-11-17 10:15 UTC. Kubernetes' account of it: 11 crash events, 3 sched events. Last event: BackOff — “Back-off restarting failed container jss-server in pod jss-server-5968bc86d8-pptmg_tilda(fda1a1d7-5f21-42bd-b73d-7d92a2cf2383)”. It is running image spreadsheet-agent-edits-d3202c0. Containers restarting: jss-server ×9. The workload carried 0.0% of the cluster's API calls in the last day. Restarts are counted from the container restart counter, so a rollout or a spot preemption does not count; a crash does. It resolves on its own after thirty quiet minutes.

warning Query shape degrading test27 Owner: Suresh Boddapati (backup: Madhava Mukkamala) resolved last seen 1d ago · since 1d ago · 4×

workplansvc shape 50B96E05: p95 1.45 s now vs 488 ms earlier (3.0×, 223 runs)

A workplansvc query shape on test27 has p95 1.4 s in the second half of the window against 488 ms in the first (3.0×, 223 executions). The shape reads: “MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO…”. Slower can mean more data, a lost index, or lock contention — the shape page has the CPU/wait split. Shapes whose p95 stays under a second are never reported: below that nobody notices, whatever the ratio.

warning Query shape degrading test27 Owner: Suresh Boddapati (backup: Madhava Mukkamala) resolved last seen 3d ago · since 3d ago · 1×

workplansvc shape 6947E55E: p95 1.20 s now vs 493 ms earlier (2.4×, 272 runs)

A workplansvc query shape on test27 has p95 1.2 s in the second half of the window against 493 ms in the first (2.4×, 272 executions). The shape reads: “MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO…”. Slower can mean more data, a lost index, or lock contention — the shape page has the CPU/wait split. Shapes whose p95 stays under a second are never reported: below that nobody notices, whatever the ratio.

info Latency anomaly test27 Owner: Suresh Boddapati (backup: Madhava Mukkamala) resolved last seen 1d ago · since 12d ago · 59×

studiosvc p95 7.35 s vs a normal 199 ms ± 157 ms for Tue 07:00 UTC (4 of the last 4 checks)

studiosvc's p95 is 7.4 s on test27, above a normal of 199 ms ± 157 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 45.6, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.

info Traffic anomaly test27 Owner: Suresh Boddapati (backup: Madhava Mukkamala) resolved last seen 1d ago · since 43d ago · 115×

studiosvc request rate 36.0 req/min vs a normal 0.0 req/min ± 5.0 req/min for Tue 07:00 UTC (4 of the last 4 checks)

studiosvc's request rate is 36.0 req/min on test27, above a normal of 0.0 req/min ± 5.0 req/min for this hour of the week (4 of the last 4 five-minute buckets, z = 7.2, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.

info Latency anomaly test27 resolved last seen 1d ago · since 84d ago · 348×

test27 (whole cluster) p95 5.10 s vs a normal 20 ms ± 15 ms for Tue 07:00 UTC (4 of the last 4 checks)

the whole test27 cluster's p95 is 5.1 s on test27, above a normal of 20 ms ± 15 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 341.3, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.

info Traffic anomaly test27 Owner: Prasad Limaye (backup: Sambasiva Bandarupalli) resolved last seen 1d ago · since 26d ago · 71×

onboardingsvc request rate 192.4 req/min vs a normal 0.0 req/min ± 5.0 req/min for Tue 07:00 UTC (4 of the last 4 checks)

onboardingsvc's request rate is 192.4 req/min on test27, above a normal of 0.0 req/min ± 5.0 req/min for this hour of the week (4 of the last 4 five-minute buckets, z = 38.5, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.

info Neo4j load anomaly test27 Owner: Prasad Limaye (backup: Sambasiva Bandarupalli) resolved last seen 1d ago · since 70d ago · 42×

onboardingsvc Neo4j load 575 q/min vs a normal 0 q/min ± 20 q/min for Tue 07:00 UTC (4 of the last 4 checks)

onboardingsvc's Neo4j load is 575 q/min on test27, above a normal of 0 q/min ± 20 q/min for this hour of the week (4 of the last 4 five-minute buckets, z = 28.8, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.

info Traffic anomaly test27 Owner: Madhava Mukkamala (backup: Shashank Agrawal) resolved last seen 1d ago · since 83d ago · 14×

execsvc request rate 143.8 req/min vs a normal 0.0 req/min ± 5.0 req/min for Tue 07:00 UTC (4 of the last 4 checks)

execsvc's request rate is 143.8 req/min on test27, above a normal of 0.0 req/min ± 5.0 req/min for this hour of the week (4 of the last 4 five-minute buckets, z = 28.8, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split.