studiosvc p95 7.35 s vs a normal 199 ms ± 157 ms for Tue 07:00 UTC (4 of the last 4 checks)
info
Latency anomaly
test27
resolved
impact 12
Owner: Suresh Boddapati (backup: Madhava Mukkamala)
opened 12d ago · last observed 1d ago · observed 59× · #16316 · 1ed1fd72aeacc504a8b8
What happened template
studiosvc's p95 is 7.4 s on test27, above a normal of 199 ms ± 157 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 45.6, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.
p95 far above its hour-of-week normal — and at least 1 s; under a second is noise.
The facts
| measure | api_p95_ms |
| unit | ms |
| value | 7,353 |
| expected_median | 199.26 |
| expected_mad | 156.81 |
| z | 45.62 |
| rel | 35.90 |
| buckets_hit | 4 |
| n | 21 |
| weeks | 3 |
| slot | Tue 07:00 UTC |
| bucket | 2026-11-17 07:40:00 UTC (1d ago) |
| floor_ms | 1,000 |
| discriminator | api_p95_ms |
| window | 2026-11-17 07:25:00 UTC → 2026-11-17 07:45:00 UTC |
What is failing in studiosvc — the bad events, decomposed
“Bad events” are HTTP 5xx answers from studiosvc. In this finding's window (2026-11-17 07:25:00 UTC → 2026-11-17 07:45:00 UTC) there were 1; every line links to the grouped errors behind it.
POST /api/studio/forms/:id/activationRuleChains |
1 bad of 47 calls · 100% of the service's 5xx |
The error shapes behind them
| 17× | http error response returned to client since 2026-10-06 01:20:44 UTC |
| 6× | Request completed — 500 · could not create activation rule chain:activation rule \ since 2026-10-02 11:58:33 UTC |
History
- 2026-11-16 14:48:32 UTCreopened x-ray · severity=info was=info title=studiosvc p95 1.7 min vs a normal 375 ms ± 141 ms for Mon 14:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 1.7 min vs a normal 375 ms ± 141 ms for Mon 14:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 101250.0, 'expected_median': 375.0, 'expected_mad': 140.847, 'z': 716.202688023174, 'rel': 269.0, 'buckets_hit': 3, 'n': 23, 'weeks': 3, 'slot': 'Mon 14:00 UTC', 'bucket': 1794839700, 'window': [1794838800, 1794840000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794817200&to=1794840000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794817200&to=1794840000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794840510, 'age_seconds': 20, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 8.1, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794840512, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 51, 'narrative': "studiosvc's p95 is 120.0 s on test27, above a normal of 432 ms ± 278 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 430.1, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 922891, 'stale_s': 18}
- 2026-11-16 15:08:36 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 1.7 min vs a normal 375 ms ± 141 ms for Mon 14:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 101250.0, 'expected_median': 375.0, 'expected_mad': 140.847, 'z': 716.202688023174, 'rel': 269.0, 'buckets_hit': 3, 'n': 23, 'weeks': 3, 'slot': 'Mon 14:00 UTC', 'bucket': 1794839700, 'window': [1794838800, 1794840000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794817200&to=1794840000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794817200&to=1794840000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794840510, 'age_seconds': 20, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 13.1, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794840512, 'resolved_at': 1794841716, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 51, 'narrative': "studiosvc's p95 is 101.2 s on test27, above a normal of 375 ms ± 141 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 716.2, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794840512, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 924094, 'stale_s': 1221}
- 2026-11-16 15:18:05 UTCreopened x-ray · severity=info was=info title=studiosvc p95 1.10 s vs a normal 461 ms ± 113 ms for Mon 15:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 1.10 s vs a normal 461 ms ± 113 ms for Mon 15:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 1099.9999999999943, 'expected_median': 461.0435, 'expected_mad': 112.94572, 'z': 5.657199759317966, 'rel': 1.3858920036829374, 'buckets_hit': 3, 'n': 30, 'weeks': 4, 'slot': 'Mon 15:00 UTC', 'bucket': 1794841500, 'window': [1794840600, 1794841800], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794819000&to=1794841800'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794819000&to=1794841800'}], 'provenance': {'source': 'rollup', 'collected_at': 1794842283, 'age_seconds': 18, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 8.1, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794842285, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 52, 'narrative': "studiosvc's p95 is 101.2 s on test27, above a normal of 375 ms ± 141 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 716.2, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794842285, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 924662, 'stale_s': 16}
- 2026-11-16 15:51:04 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 26.50 s vs a normal 461 ms ± 113 ms for Mon 15:00 UTC (4 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 26499.999999999985, 'expected_median': 461.0435, 'expected_mad': 112.94572, 'z': 230.54398608464302, 'rel': 56.47830736145285, 'buckets_hit': 4, 'n': 30, 'weeks': 4, 'slot': 'Mon 15:00 UTC', 'bucket': 1794842100, 'window': [1794841200, 1794842400], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794819600&to=1794842400'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794819600&to=1794842400'}], 'provenance': {'source': 'rollup', 'collected_at': 1794842900, 'age_seconds': 19, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 13.1, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794842902, 'resolved_at': 1794844264, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 53, 'narrative': "studiosvc's p95 is 26.5 s on test27, above a normal of 461 ms ± 113 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 230.5, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794842285, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 926638, 'stale_s': 1375}
- 2026-11-16 17:43:06 UTCreopened x-ray · severity=info was=info title=studiosvc p95 29.00 s vs a normal 452 ms ± 52 ms for Mon 17:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 29.00 s vs a normal 452 ms ± 52 ms for Mon 17:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 28999.999999999913, 'expected_median': 451.51785, 'expected_mad': 51.917477, 'z': 549.8819241543636, 'rel': 63.22780406134533, 'buckets_hit': 3, 'n': 20, 'weeks': 3, 'slot': 'Mon 17:00 UTC', 'bucket': 1794850200, 'window': [1794849300, 1794850500], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794827700&to=1794850500'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794827700&to=1794850500'}], 'provenance': {'source': 'rollup', 'collected_at': 1794850983, 'age_seconds': 19, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 7.8, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794850986, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 54, 'narrative': "studiosvc's p95 is 26.5 s on test27, above a normal of 461 ms ± 113 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 230.5, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794850986, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 933363, 'stale_s': 16}
- 2026-11-16 18:07:03 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 29.00 s vs a normal 452 ms ± 52 ms for Mon 17:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 28999.999999999913, 'expected_median': 451.51785, 'expected_mad': 51.917477, 'z': 549.8819241543636, 'rel': 63.22780406134533, 'buckets_hit': 3, 'n': 20, 'weeks': 3, 'slot': 'Mon 17:00 UTC', 'bucket': 1794850200, 'window': [1794849600, 1794850800], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794828000&to=1794850800'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794828000&to=1794850800'}], 'provenance': {'source': 'rollup', 'collected_at': 1794851431, 'age_seconds': 62, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 12.8, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794851475, 'resolved_at': 1794852423, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 55, 'narrative': "studiosvc's p95 is 29.0 s on test27, above a normal of 452 ms ± 52 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 549.9, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794850986, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 934800, 'stale_s': 964}
- 2026-11-16 20:37:48 UTCreopened x-ray · severity=info was=info title=studiosvc p95 2.0 min vs a normal 462 ms ± 101 ms for Mon 20:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 2.0 min vs a normal 462 ms ± 101 ms for Mon 20:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 120000.0, 'expected_median': 461.82895, 'expected_mad': 100.53881, 'z': 1188.9753921893446, 'rel': 258.83646109668956, 'buckets_hit': 3, 'n': 26, 'weeks': 5, 'slot': 'Mon 20:00 UTC', 'bucket': 1794860400, 'window': [1794859800, 1794861000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794838200&to=1794861000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794838200&to=1794861000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794861466, 'age_seconds': 17, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 7.6, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794861468, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 56, 'narrative': "studiosvc's p95 is 29.0 s on test27, above a normal of 452 ms ± 52 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 549.9, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794861468, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 943844, 'stale_s': 15}
- 2026-11-16 20:53:56 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 2.0 min vs a normal 462 ms ± 101 ms for Mon 20:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 120000.0, 'expected_median': 461.82895, 'expected_mad': 100.53881, 'z': 1188.9753921893446, 'rel': 258.83646109668956, 'buckets_hit': 3, 'n': 26, 'weeks': 5, 'slot': 'Mon 20:00 UTC', 'bucket': 1794860400, 'window': [1794859800, 1794861000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794838200&to=1794861000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794838200&to=1794861000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794861466, 'age_seconds': 17, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 12.6, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794861468, 'resolved_at': 1794862436, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 56, 'narrative': "studiosvc's p95 is 120.0 s on test27, above a normal of 462 ms ± 101 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 1189.0, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794861468, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 944811, 'stale_s': 982}
- 2026-11-16 23:44:07 UTCreopened x-ray · severity=info was=info title=studiosvc p95 55.50 s vs a normal 455 ms ± 63 ms for Mon 23:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 55.50 s vs a normal 455 ms ± 63 ms for Mon 23:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 55500.000000000044, 'expected_median': 455.0, 'expected_mad': 63.0105, 'z': 873.5845613032756, 'rel': 120.97802197802207, 'buckets_hit': 3, 'n': 21, 'weeks': 4, 'slot': 'Mon 23:00 UTC', 'bucket': 1794871800, 'window': [1794870900, 1794872100], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794849300&to=1794872100'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794849300&to=1794872100'}], 'provenance': {'source': 'rollup', 'collected_at': 1794872645, 'age_seconds': 16, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 7.8, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794872647, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 57, 'narrative': "studiosvc's p95 is 120.0 s on test27, above a normal of 462 ms ± 101 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 1189.0, 5 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794872647, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 955022, 'stale_s': 14}
- 2026-11-17 00:04:07 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 55.50 s vs a normal 455 ms ± 63 ms for Mon 23:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 55500.000000000044, 'expected_median': 455.0, 'expected_mad': 63.0105, 'z': 873.5845613032756, 'rel': 120.97802197802207, 'buckets_hit': 3, 'n': 21, 'weeks': 4, 'slot': 'Mon 23:00 UTC', 'bucket': 1794871800, 'window': [1794870900, 1794872100], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794849300&to=1794872100'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794849300&to=1794872100'}], 'provenance': {'source': 'rollup', 'collected_at': 1794872645, 'age_seconds': 16, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 12.8, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794872647, 'resolved_at': 1794873847, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 57, 'narrative': "studiosvc's p95 is 55.5 s on test27, above a normal of 455 ms ± 63 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 873.6, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794872647, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 956220, 'stale_s': 1212}
- 2026-11-17 01:43:17 UTCreopened x-ray · severity=info was=info title=studiosvc p95 49.50 s vs a normal 440 ms ± 149 ms for Tue 01:00 UTC (3 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 49.50 s vs a normal 440 ms ± 149 ms for Tue 01:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 49499.999999999956, 'expected_median': 440.0, 'expected_mad': 149.319, 'z': 328.55832144603136, 'rel': 111.4999999999999, 'buckets_hit': 3, 'n': 21, 'weeks': 4, 'slot': 'Tue 01:00 UTC', 'bucket': 1794878700, 'window': [1794877800, 1794879000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794856200&to=1794879000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794856200&to=1794879000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794879632, 'age_seconds': 179, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 8.2, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794879797, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 58, 'narrative': "studiosvc's p95 is 55.5 s on test27, above a normal of 455 ms ± 63 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 873.6, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794879797, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 962172, 'stale_s': 14}
- 2026-11-17 02:06:20 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 49.50 s vs a normal 440 ms ± 149 ms for Tue 01:00 UTC (3 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 49499.999999999956, 'expected_median': 440.0, 'expected_mad': 149.319, 'z': 328.55832144603136, 'rel': 111.4999999999999, 'buckets_hit': 3, 'n': 21, 'weeks': 4, 'slot': 'Tue 01:00 UTC', 'bucket': 1794878700, 'window': [1794877800, 1794879000], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794856200&to=1794879000'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794856200&to=1794879000'}], 'provenance': {'source': 'rollup', 'collected_at': 1794879632, 'age_seconds': 179, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 13.2, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794879797, 'resolved_at': 1794881180, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 58, 'narrative': "studiosvc's p95 is 49.5 s on test27, above a normal of 440 ms ± 149 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 328.6, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794879797, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 963554, 'stale_s': 1396}
- 2026-11-17 08:04:09 UTCreopened x-ray · severity=info was=info title=studiosvc p95 7.35 s vs a normal 199 ms ± 157 ms for Tue 07:00 UTC (4 of the last 4 checks) level=low was_level=low finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 7.35 s vs a normal 199 ms ± 157 ms for Tue 07:00 UTC (4 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 7352.941176470588, 'expected_median': 199.26471, 'expected_mad': 156.80675, 'z': 45.62097273536113, 'rel': 35.9003682411732, 'buckets_hit': 4, 'n': 21, 'weeks': 3, 'slot': 'Tue 07:00 UTC', 'bucket': 1794901200, 'window': [1794900300, 1794901500], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794878700&to=1794901500'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794878700&to=1794901500'}], 'provenance': {'source': 'rollup', 'collected_at': 1794902361, 'age_seconds': 306, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 7.5, 'status': 'open', 'first_seen': 1793917639, 'last_seen': 1794902649, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 59, 'narrative': "studiosvc's p95 is 49.5 s on test27, above a normal of 440 ms ± 149 ms for this hour of the week (3 of the last 4 five-minute buckets, z = 328.6, 4 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794902649, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 985028, 'stale_s': 18}
- 2026-11-17 08:26:41 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 16316, 'fingerprint': '1ed1fd72aeacc504a8b8', 'kind': 'anomaly.latency', 'severity': 'info', 'cluster': 'test27', 'subject': 'studiosvc', 'title': 'studiosvc p95 7.35 s vs a normal 199 ms ± 157 ms for Tue 07:00 UTC (4 of the last 4 checks)', 'facts': {'measure': 'api_p95_ms', 'unit': 'ms', 'value': 7352.941176470588, 'expected_median': 199.26471, 'expected_mad': 156.80675, 'z': 45.62097273536113, 'rel': 35.9003682411732, 'buckets_hit': 4, 'n': 21, 'weeks': 3, 'slot': 'Tue 07:00 UTC', 'bucket': 1794901200, 'window': [1794900300, 1794901500], 'floor_ms': 1000, 'recent_deploys': [], 'discriminator': 'api_p95_ms'}, 'evidence': [{'label': 'Service', 'href': '/service/studiosvc?cluster=test27&from=1794878700&to=1794901500'}, {'label': 'Fleet vs normal', 'href': '/fleet?cluster=test27&from=1794878700&to=1794901500'}], 'provenance': {'source': 'rollup', 'collected_at': 1794902361, 'age_seconds': 306, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 12.5, 'status': 'resolved', 'first_seen': 1793917639, 'last_seen': 1794902649, 'resolved_at': 1794904001, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 59, 'narrative': "studiosvc's p95 is 7.4 s on test27, above a normal of 199 ms ± 157 ms for this hour of the week (4 of the last 4 five-minute buckets, z = 45.6, 3 weeks of history). Whether users noticed depends on which endpoints carried it — the service page shows the split. A p95 that stays under a second is never reported — below that nobody notices, whatever the deviation.", 'narrative_kind': 'template', 'discriminator': 'api_p95_ms', 'level': 'low', 'told_level': None, 'episode_at': 1794902649, 'chat_thread': None, 'kind_label': 'Latency anomaly', 'group': 'Anomalies', 'age_s': 986382, 'stale_s': 1372}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794902361 |
| age_seconds | 306 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | seasonal |