Xray
daily rollups 1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 5 days▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 13:39:03.
› →|
updated 1d ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.

Incidents 0 happening now on test26

What is wrong right now, by what it means for people. ?critical: users are affected now, or data is at risk — about zero a week. high: degraded or getting worse — fewer than five a week. medium: worth a look today, grouped. low: context, in the daily digest. An incident is judged on its current state and closes itself when it recovers; one that comes back within a day at no higher level is the same incident.

Dry-run. Messages are composed and logged under Deliveries; nothing external is sent unless a channel was armed on its own.
0 now 0 critical 0 high 0 medium 0 low
Nothing happening now on test26. 18 older findings still open below.
Open, but not observed for an hour 18 nothing has resolved them yet; they send nothing new unless they come back at a higher level
LevelIncidentWhatForSeenGoes to
medium rag-agent (tilda) image pull failing for at least 119 min — 1 pod Pod cannot start 3d 1d ago in-app / digest only
medium finsvc (tilda) image pull failing for at least 119 min — 1 pod Pod cannot start 3d 1d ago in-app / digest only
medium studiosvc GET /api/studio/events: 4xx — 100 of 100 calls (100 %; other 4xx×100) in the last 15 min — it rejects nearly every call 4xx spike (403/401/404…) 7d 1d ago in-app / digest only
medium regsvc (tilda) image pull failing for at least 116 min — 1 pod Pod cannot start 10d 1d ago in-app / digest only
medium docsvc: a new error shape, 156× in the last hour, never seen in 90 days (template backfill: could not move document — document has no primary v) Error never seen before 2d 1d ago in-app / digest only
medium regsvc: a new error shape, 240× in the last hour, never seen in 90 days (failed to get organization — object Organization {deltimestamp:$a_delt) Error never seen before 2d 2d ago in-app / digest only
medium docsvc: a new error shape, 41× in the last hour, never seen in 90 days (Request completed — ? · ?/? documents promoted to TMF and ? failed) Error never seen before 2d 2d ago in-app / digest only
medium execsvc: a new error shape, 20× in the last hour, never seen in 90 days (could not provision managed-SMO org for site create — create doc seque) Error never seen before 2d 2d ago in-app / digest only
medium docsvc: a new error shape, 11× in the last hour, never seen in 90 days (Request completed — ? · document cannot be promoted to TMF: document m) Error never seen before 2d 2d ago in-app / digest only
medium studiosvc: a new error shape, ≥4357× in the last hour, never seen in 90 days (error on pubsub receive — rpc error: code = NotFound desc = Resource n) Error never seen before 3d 3d ago in-app / digest only
low feedsvc: Neo4j queries per request 162.4 — 22.3× the fleet's usual 7.3 Service outlier 21d 1d ago in-app / digest only
low enrollsvc: Neo4j queries per request 50.0 — 6.9× the fleet's usual 7.3 Service outlier 74d 1d ago in-app / digest only
low patientsvc: p99 latency (ms) 1.75 s — 3.3× the fleet's usual 530 ms Service outlier 74d 1d ago in-app / digest only
low execsvc: p99 latency (ms) 1.95 s — 3.7× the fleet's usual 530 ms (also p95 latency (ms) 1.48 s) Service outlier 84d 1d ago in-app / digest only
low docsvc: p99 latency (ms) 3.64 s — 9.9× the fleet's usual 368 ms Service outlier 83d 1d ago in-app / digest only
low virtual-assistant: p99 latency (ms) 1.45 s — 3.2× the fleet's usual 448 ms Service outlier 84d 1d ago in-app / digest only
low docproc: p99 latency (ms) 8.63 s — 17.9× the fleet's usual 482 ms (also p95 latency (ms) 4.85 s; Neo4j queries per request 590.8) Service outlier 82d 2d ago in-app / digest only
low chat-agent: p99 latency (ms) 15.00 s — 76.9× the fleet's usual 195 ms (also p95 latency (ms) 8.12 s) Service outlier 76d 3d ago in-app / digest only