studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1
warning
4xx spike (403/401/404…)
master
resolved
impact 47
Owner: Suresh Boddapati (backup: Madhava Mukkamala)
opened 7d ago · last observed 1d ago · observed 16× · #17065 · ff40fc298073211b4751
What happened template
4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 210, c4xx 37, share_pct 17.61904761904762, c401 0.
An endpoint answering far more 4xx than its own usual rate — a permissions change (403), expired or broken auth (401), a missing route or contract break (404). Its usual rate is its last seven days.
The facts
| endpoint | GET /api/studio/protocols/:id/visits/:id |
| service | studiosvc |
| calls | 210 |
| c4xx | 37 |
| share_pct | 17.62 |
| c401 | 0 |
| c403 | 0 |
| other_4xx | 37 |
| rejecting | no |
| usual_per_window | 1.10 |
| baseline_days | 7 |
| window_min | 15 |
| discriminator | GET /api/studio/protocols/:id/visits/:id |
| window | 2026-11-17 09:50:00 UTC → 2026-11-17 10:05:00 UTC |
Adjacent deploys — adjacency, not proof
| automated | studiosvc · 56 min before |
The story — No studiosvc deploy or hand change is recorded in the hour before this opened. Everything below is studiosvc only.
- 2026-11-11 07:53:18 UTCthis finding studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1
- 2026-11-14 00:00:00 UTClatency studiosvc p95 peaked at ≥ 2.0 min
- 2026-11-14 15:15:50 UTCdeploy automated deployed studiosvc
- 2026-11-15 00:00:00 UTCerrors studiosvc 5xx peaked at 59 in one five-minute bucket (median 0 in this window)
- 2026-11-15 01:01:27 UTCdeploy automated deployed studiosvc
- 2026-11-15 01:29:10 UTCdeploy automated deployed studiosvc
- 2026-11-16 01:19:27 UTCdeploy automated deployed studiosvc
- 2026-11-16 01:27:19 UTCdeploy automated deployed studiosvc
- 2026-11-16 05:19:14 UTCdeploy automated deployed studiosvc
- 2026-11-16 08:05:47 UTCdeploy automated deployed studiosvc
- 2026-11-16 10:19:38 UTCfinding · warning studiosvc shape 8D3802C7: p95 1.73 s now vs 746 ms earlier (2.3×, 232 runs)
- 2026-11-16 15:39:08 UTCdeploy automated deployed studiosvc
- 2026-11-16 18:19:54 UTCdeploy automated deployed studiosvc
- 2026-11-17 07:05:31 UTCdeploy automated deployed studiosvc
- 2026-11-17 09:09:49 UTCdeploy automated deployed studiosvc
- 2026-11-17 09:25:43 UTCdeploy automated deployed studiosvc
History
- 2026-11-15 08:04:18 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 250 calls (14 %; other 4xx×35) in the last 15 min — 48× its usual 0.7 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 250 calls (14 %; other 4xx×35) in the last 15 min — 48× its usual 0.7', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 250, 'c4xx': 35, 'share_pct': 14.0, 'c401': 0, 'c403': 0, 'other_4xx': 35, 'rejecting': False, 'usual_per_window': 0.73, 'baseline_days': 7, 'window': [1794728100, 1794729000], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794728100&to=1794729000&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794728100&to=1794729000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794729581, 'age_seconds': 283, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 42.7, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794729858, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 10, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 284 calls (12 %; other 4xx×35) in the last 15 min — 54× its usual 0.6. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 284, c4xx 35, share_pct 12.32394366197183, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794729858, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 346265, 'stale_s': 5}
- 2026-11-15 10:09:46 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 250 calls (14 %; other 4xx×35) in the last 15 min — 48× its usual 0.7', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 250, 'c4xx': 35, 'share_pct': 14.0, 'c401': 0, 'c403': 0, 'other_4xx': 35, 'rejecting': False, 'usual_per_window': 0.73, 'baseline_days': 7, 'window': [1794728100, 1794729000], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794728100&to=1794729000&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794728100&to=1794729000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794729581, 'age_seconds': 283, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 47.7, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794729858, 'resolved_at': 1794737386, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 10, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 250 calls (14 %; other 4xx×35) in the last 15 min — 48× its usual 0.7. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 250, c4xx 35, share_pct 14.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794729858, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 353792, 'stale_s': 7532}
- 2026-11-16 07:34:54 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 55 of 178 calls (31 %; 403×1, other 4xx×54) in the last 15 min — 66× its usual 0.8 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 55 of 178 calls (31 %; 403×1, other 4xx×54) in the last 15 min — 66× its usual 0.8', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 178, 'c4xx': 55, 'share_pct': 30.89887640449438, 'c401': 0, 'c403': 1, 'other_4xx': 54, 'rejecting': False, 'usual_per_window': 0.84, 'baseline_days': 7, 'window': [1794813000, 1794813900], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794813000&to=1794813900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794813000&to=1794813900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794814423, 'age_seconds': 78, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 36.0, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794814494, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 11, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 35 of 250 calls (14 %; other 4xx×35) in the last 15 min — 48× its usual 0.7. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 250, c4xx 35, share_pct 14.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794814494, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 430903, 'stale_s': 7}
- 2026-11-16 07:51:56 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 55 of 178 calls (31 %; 403×1, other 4xx×54) in the last 15 min — 66× its usual 0.8', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 178, 'c4xx': 55, 'share_pct': 30.89887640449438, 'c401': 0, 'c403': 1, 'other_4xx': 54, 'rejecting': False, 'usual_per_window': 0.84, 'baseline_days': 7, 'window': [1794813000, 1794813900], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794813000&to=1794813900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794813000&to=1794813900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794814423, 'age_seconds': 78, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 41.0, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794814494, 'resolved_at': 1794815516, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 11, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 55 of 178 calls (31 %; 403×1, other 4xx×54) in the last 15 min — 66× its usual 0.8. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 178, c4xx 55, share_pct 30.89887640449438, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794814494, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 431925, 'stale_s': 1029}
- 2026-11-16 13:13:39 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 132 calls (15 %; other 4xx×20) in the last 15 min — 22× its usual 0.9 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 132 calls (15 %; other 4xx×20) in the last 15 min — 22× its usual 0.9', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 132, 'c4xx': 20, 'share_pct': 15.151515151515152, 'c401': 0, 'c403': 0, 'other_4xx': 20, 'rejecting': False, 'usual_per_window': 0.93, 'baseline_days': 7, 'window': [1794832800, 1794833700], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794832800&to=1794833700&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794832800&to=1794833700'}], 'provenance': {'source': 'rollup', 'collected_at': 1794834347, 'age_seconds': 482, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 36.6, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794834819, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 12, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 55 of 178 calls (31 %; 403×1, other 4xx×54) in the last 15 min — 66× its usual 0.8. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 178, c4xx 55, share_pct 30.89887640449438, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794834819, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 451231, 'stale_s': 10}
- 2026-11-16 13:29:19 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 132 calls (15 %; other 4xx×20) in the last 15 min — 22× its usual 0.9', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 132, 'c4xx': 20, 'share_pct': 15.151515151515152, 'c401': 0, 'c403': 0, 'other_4xx': 20, 'rejecting': False, 'usual_per_window': 0.93, 'baseline_days': 7, 'window': [1794832800, 1794833700], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794832800&to=1794833700&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794832800&to=1794833700'}], 'provenance': {'source': 'rollup', 'collected_at': 1794834347, 'age_seconds': 482, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 41.6, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794834819, 'resolved_at': 1794835759, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 12, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 132 calls (15 %; other 4xx×20) in the last 15 min — 22× its usual 0.9. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 132, c4xx 20, share_pct 15.151515151515152, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794834819, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 452170, 'stale_s': 949}
- 2026-11-16 15:08:36 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 21 of 107 calls (20 %; other 4xx×21) in the last 15 min — 22× its usual 1.0 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 21 of 107 calls (20 %; other 4xx×21) in the last 15 min — 22× its usual 1.0', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 107, 'c4xx': 21, 'share_pct': 19.626168224299064, 'c401': 0, 'c403': 0, 'other_4xx': 21, 'rejecting': False, 'usual_per_window': 0.97, 'baseline_days': 7, 'window': [1794840000, 1794840900], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794840000&to=1794840900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794840000&to=1794840900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794841537, 'age_seconds': 188, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 35.5, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794841716, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 13, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 132 calls (15 %; other 4xx×20) in the last 15 min — 22× its usual 0.9. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 132, c4xx 20, share_pct 15.151515151515152, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794841716, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 458127, 'stale_s': 9}
- 2026-11-16 15:28:22 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 21 of 107 calls (20 %; other 4xx×21) in the last 15 min — 22× its usual 1.0', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 107, 'c4xx': 21, 'share_pct': 19.626168224299064, 'c401': 0, 'c403': 0, 'other_4xx': 21, 'rejecting': False, 'usual_per_window': 0.97, 'baseline_days': 7, 'window': [1794840000, 1794840900], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794840000&to=1794840900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794840000&to=1794840900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794841537, 'age_seconds': 188, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 40.5, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794841716, 'resolved_at': 1794842902, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 13, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 21 of 107 calls (20 %; other 4xx×21) in the last 15 min — 22× its usual 1.0. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 107, c4xx 21, share_pct 19.626168224299064, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794841716, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 459312, 'stale_s': 1194}
- 2026-11-16 23:50:57 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 60 calls (33 %; other 4xx×20) in the last 15 min — 20× its usual 1.0 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 20 of 60 calls (33 %; other 4xx×20) in the last 15 min — 20× its usual 1.0', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 60, 'c4xx': 20, 'share_pct': 33.333333333333336, 'c401': 0, 'c403': 0, 'other_4xx': 20, 'rejecting': False, 'usual_per_window': 1.02, 'baseline_days': 7, 'window': [1794871800, 1794872700], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794871800&to=1794872700&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794871800&to=1794872700'}], 'provenance': {'source': 'rollup', 'collected_at': 1794873017, 'age_seconds': 47, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 43.2, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794873057, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 14, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 21 of 107 calls (20 %; other 4xx×21) in the last 15 min — 22× its usual 1.0. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 107, c4xx 21, share_pct 19.626168224299064, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794873057, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 489466, 'stale_s': 7}
- 2026-11-17 01:15:27 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 36 of 81 calls (44 %; other 4xx×36) in the last 15 min — 34× its usual 1.0', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 81, 'c4xx': 36, 'share_pct': 44.44444444444444, 'c401': 0, 'c403': 0, 'other_4xx': 36, 'rejecting': False, 'usual_per_window': 1.05, 'baseline_days': 7, 'window': [1794875400, 1794876300], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794875400&to=1794876300&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794875400&to=1794876300'}], 'provenance': {'source': 'rollup', 'collected_at': 1794876896, 'age_seconds': 209, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 53.6, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794877096, 'resolved_at': 1794878127, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 15, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 36 of 81 calls (44 %; other 4xx×36) in the last 15 min — 34× its usual 1.0. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 81, c4xx 36, share_pct 44.44444444444444, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794873057, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 494537, 'stale_s': 1039}
- 2026-11-17 10:21:45 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1 level=medium was_level=medium finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 210, 'c4xx': 37, 'share_pct': 17.61904761904762, 'c401': 0, 'c403': 0, 'other_4xx': 37, 'rejecting': False, 'usual_per_window': 1.1, 'baseline_days': 7, 'window': [1794909000, 1794909900], 'window_min': 15, 'recent_deploys': [{'who': 'automated', 'resource': 'studiosvc', 'image': None, 'revision': 'e4eb245d2dfc686c6d55e4d5abf48687fcd3f7b3', 'ts': 1794907543, 'minutes_before': 56}], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794909000&to=1794909900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794909000&to=1794909900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794910455, 'age_seconds': 462, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 42.1, 'status': 'open', 'first_seen': 1794383598, 'last_seen': 1794910905, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 16, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 36 of 81 calls (44 %; other 4xx×36) in the last 15 min — 34× its usual 1.0. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 81, c4xx 36, share_pct 44.44444444444444, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794910905, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 527319, 'stale_s': 12}
- 2026-11-17 10:40:41 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17065, 'fingerprint': 'ff40fc298073211b4751', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'master', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1', 'facts': {'endpoint': 'GET /api/studio/protocols/:id/visits/:id', 'service': 'studiosvc', 'calls': 210, 'c4xx': 37, 'share_pct': 17.61904761904762, 'c401': 0, 'c403': 0, 'other_4xx': 37, 'rejecting': False, 'usual_per_window': 1.1, 'baseline_days': 7, 'window': [1794909000, 1794909900], 'window_min': 15, 'recent_deploys': [{'who': 'automated', 'resource': 'studiosvc', 'image': None, 'revision': 'e4eb245d2dfc686c6d55e4d5abf48687fcd3f7b3', 'ts': 1794907543, 'minutes_before': 56}], 'discriminator': 'GET /api/studio/protocols/:id/visits/:id'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=master&from=1794909000&to=1794909900&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=master&from=1794909000&to=1794909900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794910455, 'age_seconds': 462, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 47.1, 'status': 'resolved', 'first_seen': 1794383598, 'last_seen': 1794910905, 'resolved_at': 1794912041, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 16, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/protocols/:id/visits/:id: 4xx — 37 of 210 calls (18 %; other 4xx×37) in the last 15 min — 34× its usual 1.1. Facts: endpoint GET /api/studio/protocols/:id/visits/:id, service studiosvc, calls 210, c4xx 37, share_pct 17.61904761904762, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/protocols/:id/visits/:id', 'level': 'medium', 'told_level': None, 'episode_at': 1794910905, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 528454, 'stale_s': 1147}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794910455 |
| age_seconds | 462 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | endpoint_4xx_spike |