Xray
1h 6h 24h 48h 3d 5d 7d 14d 30d ‹
Last 1 hour▾
Ends now. Anything up to this cluster's 30-day retention.
Start and end times
Cancel
Interpreted in UTC (change it at the right of this bar). Earliest available: 2026-10-19 09:55:00.
› →|
updated 23h ago
Sample data. This database was generated by scripts/seed_dev_db.py — every service, deploy, person and figure on this page is synthetic. Names are sample.*@seed.invalid and image tags are prefixed SEED-. Nothing here happened.
← findings

studiosvc GET /api/studio/events: 4xx — 100 of 100 calls (100 %; other 4xx×100) in the last 15 min — it rejects nearly every call

warning 4xx spike (403/401/404…) test26 open impact 37 Owner: Suresh Boddapati (backup: Madhava Mukkamala) opened 7d ago · last observed 23h ago · observed 164× · #17063 · 001ce0a5dd0a7ac21dd0

What happened template

4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 100 of 100 calls (100 %; other 4xx×100) in the last 15 min — it rejects nearly every call. Facts: endpoint GET /api/studio/events, service studiosvc, calls 100, c4xx 100, share_pct 100.0, c401 0.

An endpoint answering far more 4xx than its own usual rate — a permissions change (403), expired or broken auth (401), a missing route or contract break (404). Its usual rate is its last seven days.

The facts

endpointGET /api/studio/events
servicestudiosvc
calls100
c4xx100
share_pct100.00
c4010
c4030
other_4xx100
rejectingyes
usual_per_window23.35
baseline_days7
window_min15
discriminatorGET /api/studio/events
window2026-11-17 10:15:00 UTC → 2026-11-17 10:30:00 UTC

The story — No studiosvc deploy or hand change is recorded in the hour before this opened. Everything below is studiosvc only.

History

  • 2026-11-15 03:00:37 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 43 of 43 calls (100 %; other 4xx×43) in the last 15 min — 688× its usual 0.1 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 43 of 43 calls (100 %; other 4xx×43) in the last 15 min — 688× its usual 0.1', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 43, 'c4xx': 43, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 43, 'rejecting': True, 'usual_per_window': 0.06, 'baseline_days': 7, 'window': [1794710100, 1794711000], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794710100&to=1794711000&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794710100&to=1794711000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794711636, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 22.5, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794711637, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 2, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: permission denied — 42 of 43 calls (98 %; 403×42) in the last 15 min — it usually has none. Facts: endpoint GET /api/studio/events, service studiosvc, calls 43, c4xx 42, share_pct 97.67441860465117, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794711637, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 332711, 'stale_s': 5}
  • 2026-11-15 08:32:27 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 49 of 49 calls (100 %; other 4xx×49) in the last 15 min — 64× its usual 0.8', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 49, 'c4xx': 49, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 49, 'rejecting': True, 'usual_per_window': 0.76, 'baseline_days': 7, 'window': [1794722700, 1794723600], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794722700&to=1794723600&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794722700&to=1794723600'}], 'provenance': {'source': 'rollup', 'collected_at': 1794724138, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 26.7, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794724138, 'resolved_at': 1794731547, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 18, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 49 of 49 calls (100 %; other 4xx×49) in the last 15 min — 64× its usual 0.8. Facts: endpoint GET /api/studio/events, service studiosvc, calls 49, c4xx 49, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794711637, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 352622, 'stale_s': 7415}
  • 2026-11-15 15:19:13 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 21 of 21 calls (100 %; other 4xx×21) in the last 15 min — 24× its usual 0.9 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 21 of 21 calls (100 %; other 4xx×21) in the last 15 min — 24× its usual 0.9', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 21, 'c4xx': 21, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 21, 'rejecting': True, 'usual_per_window': 0.88, 'baseline_days': 7, 'window': [1794754500, 1794755400], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794754500&to=1794755400&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794754500&to=1794755400'}], 'provenance': {'source': 'rollup', 'collected_at': 1794755953, 'age_seconds': 6, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 21.5, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794755953, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 19, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 49 of 49 calls (100 %; other 4xx×49) in the last 15 min — 64× its usual 0.8. Facts: endpoint GET /api/studio/events, service studiosvc, calls 49, c4xx 49, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794755953, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 377028, 'stale_s': 6}
  • 2026-11-15 18:14:30 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 40 of 40 calls (100 %; other 4xx×40) in the last 15 min — 46× its usual 0.9', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 40, 'c4xx': 40, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 40, 'rejecting': True, 'usual_per_window': 0.88, 'baseline_days': 7, 'window': [1794757500, 1794758400], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794757500&to=1794758400&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794757500&to=1794758400'}], 'provenance': {'source': 'rollup', 'collected_at': 1794758929, 'age_seconds': 6, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 36.9, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794758929, 'resolved_at': 1794766470, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 26, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 40 of 40 calls (100 %; other 4xx×40) in the last 15 min — 46× its usual 0.9. Facts: endpoint GET /api/studio/events, service studiosvc, calls 40, c4xx 40, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794755953, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 387544, 'stale_s': 7546}
  • 2026-11-16 00:11:50 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 29 of 29 calls (100 %; other 4xx×29) in the last 15 min — 25× its usual 1.2 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 29 of 29 calls (100 %; other 4xx×29) in the last 15 min — 25× its usual 1.2', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 29, 'c4xx': 29, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 29, 'rejecting': True, 'usual_per_window': 1.15, 'baseline_days': 7, 'window': [1794786600, 1794787500], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794786600&to=1794787500&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794786600&to=1794787500'}], 'provenance': {'source': 'rollup', 'collected_at': 1794787909, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 21.2, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794787910, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 27, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 40 of 40 calls (100 %; other 4xx×40) in the last 15 min — 46× its usual 0.9. Facts: endpoint GET /api/studio/events, service studiosvc, calls 40, c4xx 40, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794787910, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 408984, 'stale_s': 5}
  • 2026-11-16 01:29:10 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 42 of 42 calls (100 %; other 4xx×42) in the last 15 min — 36× its usual 1.2', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 42, 'c4xx': 42, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 42, 'rejecting': True, 'usual_per_window': 1.17, 'baseline_days': 7, 'window': [1794788400, 1794789300], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794788400&to=1794789300&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794788400&to=1794789300'}], 'provenance': {'source': 'rollup', 'collected_at': 1794789844, 'age_seconds': 5, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 26.6, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794789844, 'resolved_at': 1794792550, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 30, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 42 of 42 calls (100 %; other 4xx×42) in the last 15 min — 36× its usual 1.2. Facts: endpoint GET /api/studio/events, service studiosvc, calls 42, c4xx 42, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794787910, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 413624, 'stale_s': 2711}
  • 2026-11-16 01:36:14 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 38 of 38 calls (100 %; other 4xx×38) in the last 15 min — 30× its usual 1.3 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 38 of 38 calls (100 %; other 4xx×38) in the last 15 min — 30× its usual 1.3', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 38, 'c4xx': 38, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 38, 'rejecting': True, 'usual_per_window': 1.26, 'baseline_days': 7, 'window': [1794791700, 1794792600], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794791700&to=1794792600&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794791700&to=1794792600'}], 'provenance': {'source': 'rollup', 'collected_at': 1794792974, 'age_seconds': 6, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 21.3, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794792974, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 31, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 42 of 42 calls (100 %; other 4xx×42) in the last 15 min — 36× its usual 1.2. Facts: endpoint GET /api/studio/events, service studiosvc, calls 42, c4xx 42, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794792974, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 414048, 'stale_s': 5}
  • 2026-11-16 02:26:12 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 56 of 56 calls (100 %; other 4xx×56) in the last 15 min — 45× its usual 1.3', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 56, 'c4xx': 56, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 56, 'rejecting': True, 'usual_per_window': 1.26, 'baseline_days': 7, 'window': [1794793500, 1794794400], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794793500&to=1794794400&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794793500&to=1794794400'}], 'provenance': {'source': 'rollup', 'collected_at': 1794794999, 'age_seconds': 6, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 25.9, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794794999, 'resolved_at': 1794795972, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 34, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 56 of 56 calls (100 %; other 4xx×56) in the last 15 min — 45× its usual 1.3. Facts: endpoint GET /api/studio/events, service studiosvc, calls 56, c4xx 56, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794792974, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 417046, 'stale_s': 978}
  • 2026-11-16 02:51:52 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 25 of 25 calls (100 %; other 4xx×25) in the last 15 min — 18× its usual 1.4 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 25 of 25 calls (100 %; other 4xx×25) in the last 15 min — 18× its usual 1.4', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 25, 'c4xx': 25, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 25, 'rejecting': True, 'usual_per_window': 1.4, 'baseline_days': 7, 'window': [1794796200, 1794797100], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794796200&to=1794797100&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794796200&to=1794797100'}], 'provenance': {'source': 'rollup', 'collected_at': 1794797512, 'age_seconds': 6, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 20.7, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794797512, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 35, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 56 of 56 calls (100 %; other 4xx×56) in the last 15 min — 45× its usual 1.3. Facts: endpoint GET /api/studio/events, service studiosvc, calls 56, c4xx 56, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794797512, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 418586, 'stale_s': 5}
  • 2026-11-16 08:15:41 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 66 of 66 calls (100 %; other 4xx×66) in the last 15 min — 25× its usual 2.6', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 66, 'c4xx': 66, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 66, 'rejecting': True, 'usual_per_window': 2.62, 'baseline_days': 7, 'window': [1794814200, 1794815100], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794814200&to=1794815100&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794814200&to=1794815100'}], 'provenance': {'source': 'rollup', 'collected_at': 1794815516, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 36.3, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794815516, 'resolved_at': 1794816941, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 54, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 66 of 66 calls (100 %; other 4xx×66) in the last 15 min — 25× its usual 2.6. Facts: endpoint GET /api/studio/events, service studiosvc, calls 66, c4xx 66, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794797512, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 438018, 'stale_s': 1433}
  • 2026-11-16 08:28:23 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 20 of 20 calls (100 %; other 4xx×20) in the last 15 min — 7× its usual 2.9 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 20 of 20 calls (100 %; other 4xx×20) in the last 15 min — 7× its usual 2.9', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 20, 'c4xx': 20, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 20, 'rejecting': True, 'usual_per_window': 2.93, 'baseline_days': 7, 'window': [1794816300, 1794817200], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794816300&to=1794817200&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794816300&to=1794817200'}], 'provenance': {'source': 'rollup', 'collected_at': 1794817703, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 30.5, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794817703, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 55, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 66 of 66 calls (100 %; other 4xx×66) in the last 15 min — 25× its usual 2.6. Facts: endpoint GET /api/studio/events, service studiosvc, calls 66, c4xx 66, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794817703, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 438778, 'stale_s': 6}
  • 2026-11-16 08:44:24 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 20 of 20 calls (100 %; other 4xx×20) in the last 15 min — 7× its usual 2.9', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 20, 'c4xx': 20, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 20, 'rejecting': True, 'usual_per_window': 2.93, 'baseline_days': 7, 'window': [1794816300, 1794817200], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794816300&to=1794817200&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794816300&to=1794817200'}], 'provenance': {'source': 'rollup', 'collected_at': 1794817703, 'age_seconds': 7, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 35.5, 'status': 'resolved', 'first_seen': 1794378931, 'last_seen': 1794817703, 'resolved_at': 1794818664, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 55, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 20 of 20 calls (100 %; other 4xx×20) in the last 15 min — 7× its usual 2.9. Facts: endpoint GET /api/studio/events, service studiosvc, calls 20, c4xx 20, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794817703, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 439739, 'stale_s': 967}
  • 2026-11-16 12:49:12 UTCreopened x-ray · severity=warning was=warning title=studiosvc GET /api/studio/events: 4xx — 118 of 118 calls (100 %; 403×11, other 4xx×107) in the last 15 min — 39× its usual 3.0 level=medium was_level=medium finding={'id': 17063, 'fingerprint': '001ce0a5dd0a7ac21dd0', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'test26', 'subject': 'studiosvc', 'title': 'studiosvc GET /api/studio/events: 4xx — 118 of 118 calls (100 %; 403×11, other 4xx×107) in the last 15 min — 39× its usual 3.0', 'facts': {'endpoint': 'GET /api/studio/events', 'service': 'studiosvc', 'calls': 118, 'c4xx': 118, 'share_pct': 100.0, 'c401': 0, 'c403': 11, 'other_4xx': 107, 'rejecting': True, 'usual_per_window': 3.02, 'baseline_days': 7, 'window': [1794831900, 1794832800], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /api/studio/events'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=test26&from=1794831900&to=1794832800&service=studiosvc&status=4xx'}, {'label': 'studiosvc service page', 'href': '/service/studiosvc?cluster=test26&from=1794831900&to=1794832800'}], 'provenance': {'source': 'rollup', 'collected_at': 1794833351, 'age_seconds': 12, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 27.8, 'status': 'open', 'first_seen': 1794378931, 'last_seen': 1794833352, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 56, 'narrative': '4xx spike (403/401/404…): studiosvc GET /api/studio/events: 4xx — 20 of 20 calls (100 %; other 4xx×20) in the last 15 min — 7× its usual 2.9. Facts: endpoint GET /api/studio/events, service studiosvc, calls 20, c4xx 20, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /api/studio/events', 'level': 'medium', 'told_level': None, 'episode_at': 1794833352, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 454431, 'stale_s': 10}

Check it yourself

Every link carries the finding's own time window, so it shows tomorrow what it shows now.

Actions

Sign in (IAP) to acknowledge, snooze or resolve.

Where the numbers come from

sourcerollup
collected_at1794912040
age_seconds13
staleFalse
shownNone
totalNone
truncatedFalse
coverage_fromNone
retain_daysNone
noteNone
detectorendpoint_4xx_spike