feedsvc GET /studies/:id/feed/news: 4xx — 30 of 30 calls (100 %; other 4xx×30) in the last 15 min — 130× its usual 0.2
warning
4xx spike (403/401/404…)
automation
resolved
impact 31
opened 1d ago · last observed 1d ago · observed 6× · #17497 · a53207ef156d5030e292
What happened template
4xx spike (403/401/404…): feedsvc GET /studies/:id/feed/news: 4xx — 30 of 30 calls (100 %; other 4xx×30) in the last 15 min — 130× its usual 0.2. Facts: endpoint GET /studies/:id/feed/news, service feedsvc, calls 30, c4xx 30, share_pct 100.0, c401 0.
An endpoint answering far more 4xx than its own usual rate — a permissions change (403), expired or broken auth (401), a missing route or contract break (404). Its usual rate is its last seven days.
The facts
| endpoint | GET /studies/:id/feed/news |
| service | feedsvc |
| calls | 30 |
| c4xx | 30 |
| share_pct | 100.00 |
| c401 | 0 |
| c403 | 0 |
| other_4xx | 30 |
| rejecting | yes |
| usual_per_window | 0.23 |
| baseline_days | 7 |
| window_min | 15 |
| discriminator | GET /studies/:id/feed/news |
| window | 2026-11-17 07:10:00 UTC → 2026-11-17 07:25:00 UTC |
The story — No feedsvc deploy or hand change is recorded in the hour before this opened. Everything below is feedsvc only.
- 2026-11-17 02:44:54 UTCthis finding feedsvc GET /studies/:id/feed/news: 4xx — 30 of 30 calls (100 %; other 4xx×30) in the last 15 min — 130× its usual 0.2
- 2026-11-17 04:56:27 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 05:32:25 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 05:54:21 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 06:00:00 UTClatency feedsvc p95 peaked at 1.49 s
- 2026-11-17 06:47:38 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 07:20:37 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 07:37:12 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 07:51:06 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
- 2026-11-17 08:04:00 UTChand change [email protected] changed apps/v1/namespaces/matrix/deployments/feedsvc by hand (kubectl/v1.33.3 (darwin/arm64) kubernetes/80779bd,gzip(gfe))
History
- 2026-11-17 02:44:54 UTCopened x-ray · severity=warning title=feedsvc GET /studies/:id/feed/news: 4xx — 21 of 21 calls (100 %; other 4xx×21) in the last 15 min — it usually has none finding={'id': 17497, 'fingerprint': 'a53207ef156d5030e292', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'automation', 'subject': 'feedsvc', 'title': 'feedsvc GET /studies/:id/feed/news: 4xx — 21 of 21 calls (100 %; other 4xx×21) in the last 15 min — it usually has none', 'facts': {'endpoint': 'GET /studies/:id/feed/news', 'service': 'feedsvc', 'calls': 21, 'c4xx': 21, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 21, 'rejecting': True, 'usual_per_window': 0.0, 'baseline_days': 7, 'window': [1794882000, 1794882900], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /studies/:id/feed/news'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=automation&from=1794882000&to=1794882900&service=feedsvc&status=4xx'}, {'label': 'feedsvc service page', 'href': '/service/feedsvc?cluster=automation&from=1794882000&to=1794882900'}], 'provenance': {'source': 'rollup', 'collected_at': 1794883363, 'age_seconds': 138, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 26.4, 'status': 'open', 'first_seen': 1794883494, 'last_seen': 1794883494, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 1, 'narrative': None, 'narrative_kind': None, 'discriminator': 'GET /studies/:id/feed/news', 'level': 'medium', 'told_level': None, 'episode_at': 1794883494, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 7, 'stale_s': 7}
- 2026-11-17 03:35:58 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17497, 'fingerprint': 'a53207ef156d5030e292', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'automation', 'subject': 'feedsvc', 'title': 'feedsvc GET /studies/:id/feed/news: 4xx — 23 of 23 calls (100 %; other 4xx×23) in the last 15 min — it usually has none', 'facts': {'endpoint': 'GET /studies/:id/feed/news', 'service': 'feedsvc', 'calls': 23, 'c4xx': 23, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 23, 'rejecting': True, 'usual_per_window': 0.0, 'baseline_days': 7, 'window': [1794883500, 1794884400], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /studies/:id/feed/news'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=automation&from=1794883500&to=1794884400&service=feedsvc&status=4xx'}, {'label': 'feedsvc service page', 'href': '/service/feedsvc?cluster=automation&from=1794883500&to=1794884400'}], 'provenance': {'source': 'rollup', 'collected_at': 1794884967, 'age_seconds': 220, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 26.7, 'status': 'resolved', 'first_seen': 1794883494, 'last_seen': 1794885178, 'resolved_at': 1794886558, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 4, 'narrative': '4xx spike (403/401/404…): feedsvc GET /studies/:id/feed/news: 4xx — 23 of 23 calls (100 %; other 4xx×23) in the last 15 min — it usually has none. Facts: endpoint GET /studies/:id/feed/news, service feedsvc, calls 23, c4xx 23, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /studies/:id/feed/news', 'level': 'medium', 'told_level': None, 'episode_at': 1794883494, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 3073, 'stale_s': 1389}
- 2026-11-17 07:09:11 UTCreopened x-ray · severity=warning was=warning title=feedsvc GET /studies/:id/feed/news: 4xx — 33 of 33 calls (100 %; other 4xx×33) in the last 15 min — 191× its usual 0.2 level=medium was_level=medium finding={'id': 17497, 'fingerprint': 'a53207ef156d5030e292', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'automation', 'subject': 'feedsvc', 'title': 'feedsvc GET /studies/:id/feed/news: 4xx — 33 of 33 calls (100 %; other 4xx×33) in the last 15 min — 191× its usual 0.2', 'facts': {'endpoint': 'GET /studies/:id/feed/news', 'service': 'feedsvc', 'calls': 33, 'c4xx': 33, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 33, 'rejecting': True, 'usual_per_window': 0.17, 'baseline_days': 7, 'window': [1794897600, 1794898500], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /studies/:id/feed/news'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=automation&from=1794897600&to=1794898500&service=feedsvc&status=4xx'}, {'label': 'feedsvc service page', 'href': '/service/feedsvc?cluster=automation&from=1794897600&to=1794898500'}], 'provenance': {'source': 'rollup', 'collected_at': 1794898950, 'age_seconds': 410, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 26.3, 'status': 'open', 'first_seen': 1794883494, 'last_seen': 1794899351, 'resolved_at': None, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 5, 'narrative': '4xx spike (403/401/404…): feedsvc GET /studies/:id/feed/news: 4xx — 23 of 23 calls (100 %; other 4xx×23) in the last 15 min — it usually has none. Facts: endpoint GET /studies/:id/feed/news, service feedsvc, calls 23, c4xx 23, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /studies/:id/feed/news', 'level': 'medium', 'told_level': None, 'episode_at': 1794899351, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 15866, 'stale_s': 9}
- 2026-11-17 08:04:09 UTCresolved x-ray · reason=not observed grace_s=900 finding={'id': 17497, 'fingerprint': 'a53207ef156d5030e292', 'kind': 'endpoint.4xx_spike', 'severity': 'warning', 'cluster': 'automation', 'subject': 'feedsvc', 'title': 'feedsvc GET /studies/:id/feed/news: 4xx — 30 of 30 calls (100 %; other 4xx×30) in the last 15 min — 130× its usual 0.2', 'facts': {'endpoint': 'GET /studies/:id/feed/news', 'service': 'feedsvc', 'calls': 30, 'c4xx': 30, 'share_pct': 100.0, 'c401': 0, 'c403': 0, 'other_4xx': 30, 'rejecting': True, 'usual_per_window': 0.23, 'baseline_days': 7, 'window': [1794899400, 1794900300], 'window_min': 15, 'recent_deploys': [], 'discriminator': 'GET /studies/:id/feed/news'}, 'evidence': [{'label': 'The 4xx calls (status, caller, path)', 'href': '/api-calls?cluster=automation&from=1794899400&to=1794900300&service=feedsvc&status=4xx'}, {'label': 'feedsvc service page', 'href': '/service/feedsvc?cluster=automation&from=1794899400&to=1794900300'}], 'provenance': {'source': 'rollup', 'collected_at': 1794900779, 'age_seconds': 571, 'stale': False, 'shown': None, 'total': None, 'truncated': False, 'coverage_from': None, 'retain_days': None, 'note': None}, 'impact': 31.3, 'status': 'resolved', 'first_seen': 1794883494, 'last_seen': 1794901340, 'resolved_at': 1794902649, 'acked_by': None, 'acked_at': None, 'snoozed_until': None, 'occurrences': 6, 'narrative': '4xx spike (403/401/404…): feedsvc GET /studies/:id/feed/news: 4xx — 30 of 30 calls (100 %; other 4xx×30) in the last 15 min — 130× its usual 0.2. Facts: endpoint GET /studies/:id/feed/news, service feedsvc, calls 30, c4xx 30, share_pct 100.0, c401 0.', 'narrative_kind': 'template', 'discriminator': 'GET /studies/:id/feed/news', 'level': 'medium', 'told_level': None, 'episode_at': 1794899351, 'chat_thread': None, 'kind_label': '4xx spike (403/401/404…)', 'group': 'Errors', 'age_s': 19164, 'stale_s': 1318}
Check it yourself
Every link carries the finding's own time window, so it shows tomorrow what it shows now.
Actions
Sign in (IAP) to acknowledge, snooze or resolve.
Where the numbers come from
| source | rollup |
| collected_at | 1794900779 |
| age_seconds | 571 |
| stale | False |
| shown | None |
| total | None |
| truncated | False |
| coverage_from | None |
| retain_days | None |
| note | None |
| detector | endpoint_4xx_spike |