onboardingsvc: 5xx rate (%) 0.34% — 6.3× the fleet's usual 0.05%
info
Service outlier
automation
open
impact 16
Owner: Prasad Limaye (backup: Sambasiva Bandarupalli)
opened 83d ago · last observed 2d ago · observed 1580× · #312 · c8799297d71069cc7d0f
What happened template
onboardingsvc's 5xx rate (%) is 0.34% against the 14 other services' median of 0.05% (3.6 robust MADs above, six-hour window). This compares the service with its peers, not with its own history — the seasonal detector does that.
A service unlike the rest of the fleet on one measure (robust z against its peers).
The facts
| measures | {'measure': '5xx rate (%)', 'value': 0.33536414957241073, 'value_fmt': '0.34%', 'median': 0.052914533016141824, 'median_fmt': '0.05%', 'mad': 0.07845108664973187, 'z': 3.60032765151296, 'direction': 'above', 'peers': 14} |
| measure | 5xx rate (%) |
| value | 0.34 |
| value_fmt | 0.34% |
| median | 0.05 |
| median_fmt | 0.05% |
| z | 3.60 |
| direction | above |
| peers | 14 |
| rule | {"min_score": 3.5, "min_relative_pct": 35, "warn_on": []} |
| discriminator | |
| window | 2026-11-16 18:58:16 UTC → 2026-11-17 00:58:16 UTC |
What is failing in onboardingsvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from onboardingsvc. In this finding's window (2026-11-16 18:58:16 UTC → 2026-11-17 00:58:16 UTC) there were 81; every line links to the grouped errors behind it.
GET /api/myprofile/visibility/study/:id/blindedScopes |
23 bad of 23 calls · 28% of the service's 5xx |
POST /api/organizations |
16 bad of 121 calls · 20% of the service's 5xx |
POST /api/sites/:id/rooms |
10 bad of 24 calls · 12% of the service's 5xx |
POST /api/esign |
6 bad of 38 calls · 7% of the service's 5xx |
POST /api/sites |
6 bad of 185 calls · 7% of the service's 5xx |
PUT /api/sites/:id |
6 bad of 16 calls · 7% of the service's 5xx |
| … | 7 more endpoints with fewer 5xx — all in the grouped view |
The error shapes behind them
| 840× | request failed — 500 since 2026-09-14 00:17:50 UTC |
| 170× | Request completed — 500 · failed to get study: objLabel Study: get with empty tildaId requires a struct with at least on… since 2026-09-26 01:48:22 UTC |
| 170× | could not get study for blinded-scope lookup — objLabel Study: get with empty tildaId requires a struct with at least on… since 2026-09-15 02:42:25 UTC |
| 165× | Error getting folder manager — FolderManager(StudyInstance) not found: <nil> since 2026-11-07 05:46:31 UTC |
| 88× | Request completed — 500 · Could not initialize an organization: organization name cannot be empty since 2026-09-25 09:01:42 UTC |
All onboardingsvc errors, grouped → · service page · its logs
History
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794876899 |
| age_seconds | 198 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | outliers |